Cognitive Informatics, Computer Modelling, and Cognitive Science: Application to Neural Engineering, Robotics, and Stem: Volume 2: Application to Neural Engineering, Robotics, and STEM [2, 1 ed.] 0128194456, 9780128194454

Table of contents :
Cognitive Informatics, Computer Modeling, and Cognitive Science
Copyright
Dedication
Contents
List of contributors
Editors’ biographies
Authors’ biography
Preface
Acknowledgments
1 Approaches from cognitive neuroscience and comparative cognition
1.1 Introduction
1.2 Cognitive science
1.3 Neuroscience
1.4 Python
1.5 Review of literature
1.6 Cognitive neuroscience/physiology
1.7 Cognitive psychology
1.8 Conclusion
References
Further reading
2 Functional neuroanatomy and disorders of cognition
Abbreviations
2.1 Introduction
2.2 Neuroanatomy of memory encoding
2.2.1 Medial temporal lobe
2.2.2 Diencephalon
2.2.3 Basal forebrain
2.3 Mechanisms underlying memory formation
2.4 Neurotransmitters involved in cognition
2.4.1 Classical neurotransmitters
2.4.1.1 Acetylcholine
2.4.1.2 Glutamate
2.4.1.3 γ-Aminobutyric acid
2.4.1.4 Dopamine
2.4.1.5 Serotonin (5-hydroxytryptamine)
2.4.1.6 Agmatine
2.4.2 Neuropeptides
2.4.2.1 Cocaine- and amphetamine-regulated transcript
2.4.2.2 Neuropeptide Y
2.4.2.3 α-Melanocyte stimulating hormone
2.4.3 Neurosteroids
2.5 Cognition-related diseases
2.5.1 Alzheimer’s disease
2.5.1.1 Extraneuronal plaque deposition of β-amyloid
2.5.1.2 Intraneuronal accumulation of neurofibrillary tangles
2.5.2 Lewy body diseases
2.6 Conclusion
2.7 Acknowledgment
References
Further reading
3 A cognitive system of elderly exercise evaluation with sensors and robots
3.1 Introduction
3.2 System overview
3.3 Elderly exercise measurement
3.4 Exercise evaluation
3.5 Feedback by robot interface
3.6 Multiple Kinect application for occlusion problem
3.6.1 Frame synchronization
3.6.2 Sensing data integration without calibration
3.7 Conclusion
Acknowledgment
References
4 Models of making choice and control over thought for action
4.1 Outline of review
4.2 Introduction
4.3 Models of perceptual decision
4.3.1 Fast decision-making
4.3.2 Intuitive decision-making
4.4 Models of economic decision
4.5 Models of movement inhibition
4.5.1 Proactive control
4.5.2 Estimation of stopping efficacy
4.5.3 Trigger failures
4.5.4 Bayesian rational decision-making
4.5.5 Optimal Bayesian statistical inference
4.5.6 Decision process as optimal stochastic control
4.5.7 Linear approach to threshold explaining space and time model for decisions in space and time
4.6 Discussion
Conflict of interest
Acknowledgments
References
Further reading
5 Speech recognition technique for identification of raga
5.1 Introduction
5.2 Speech recognition
5.3 Applications of speech recognition
5.4 Speech analyses in music information retrieval
5.5 A brief history of Indian music
5.6 Mathematical structure of Carnatic music
5.7 Digital speech processing
5.8 Proposed methodology for classification of raga
5.9 A practical example using Praat
5.10 Conclusion
Reference
Further reading
6 Future of cognitive science
6.1 Introduction
6.2 Role of cognitive science in varied domains
6.2.1 Cognitive science for big data
6.2.2 Cognitive science for philosophy
6.2.3 Brain–machine interface
6.2.4 Cognition science for psychology
6.2.5 Cognition social science
6.2.6 Role of cognitive science in linguistics
6.2.7 Cognitive control
6.2.8 Cognitive image processing
6.3 Future of cognitive neuroscience and cognitive enhancement
6.3.1 Scope for neuroscience research and challenges
6.3.2 Cognitive enhancement
6.3.3 Ethical issues and concerns of cognitive enhancement
6.4 Conclusion
References
7 Application of virtual reality systems to psychology and cognitive neuroscience research
7.1 Introduction
7.1.1 Cognitive science
7.1.2 Virtual reality
7.2 Literary survey review
7.2.1 Cognitive neuroscience/physiology
7.2.2 Cognitive psychology
7.3 Conclusion
References
Further reading
8 Electrodermal activity and its effectiveness in cognitive research field
8.1 Introduction
8.2 History of electrodermal activity signal, psychophysiological, and physiological mechanism behind electrodermal activity
8.2.1 Application of electrodermal activity
8.2.2 Electrodermal activity as an indicator of general arousal
8.2.3 Electrodermal activity in different sleep stages
8.2.4 Electrodermal indices of emotion and stress
8.3 Experiment design—a good experiment design
8.3.1 Experimental design
8.3.1.1 Experiment design
8.3.1.2 Types of experiments
8.3.1.3 Hypothesis
8.3.1.4 Stimulus
8.3.1.5 Measure of performance
8.3.2 External and internal influences
8.3.3 Climatic conditions
8.3.4 Internal or physiological influences
8.3.5 Demographic characteristics
8.4 Electrodermal activity signal collection sites and pretreatment of sites
8.4.1 Electrodermal activity signal collection sites
8.4.2 Pretreatment of sites
8.5 Artifacts removal from the electrodermal activity signal
8.6 Analysis of electrodermal activity signal
8.6.1 Phasic electrodermal activity
8.6.1.1 Latency
8.6.1.2 Amplitude
8.6.1.3 Shape of electrodermal responses
8.6.2 Area measurements
8.6.3 Tonic electrodermal activity
8.7 End remarks
References
Further reading
9 Study of modern brain-imaging and -signaling techniques for brain–computer interface
9.1 Introduction
9.2 Brain-imagining techniques
9.2.1 Computer tomography
9.2.1.1 Computer tomography head
9.2.1.1.1 Benefits
9.2.1.1.2 Risk and limitation
9.2.2 Near-infrared spectroscopy–based imaging equipment
9.2.2.1 Functional near-infrared spectroscopy
9.2.2.2 Diffuse optical imaging or diffuse optical tomography
9.2.2.3 High-density diffuse optical tomography
9.2.2.3.1 Advantages and disadvantages of optical imaging
9.2.3 Magnetic resonance imaging
9.2.3.1 Magnetic resonance imaging head
9.2.3.2 Functional magnetic resonance imaging
9.2.3.2.1 Advantages of magnetic resonance imaging
9.2.3.2.2 Disadvantages of magnetic resonance imaging
9.2.4 Single-photon emission computed tomography
9.2.4.1 Advantages of single-photon emission computed tomography
9.2.4.2 Disadvantage of single-photon emission computed tomography
9.2.5 Cranial ultrasound
9.2.5.1 Advantages of cranial ultrasound
9.2.5.2 Limitations of cranial ultrasound
9.3 Brain-signaling techniques
9.3.1 Electroencephalography
9.3.1.1 Application of electroencephalography [28]
9.3.1.2 Advantages of electroencephalography
9.3.1.3 Disadvantages of electroencephalography
9.3.2 Magnetoencephalography
9.3.2.1 Advantages of magnetoencephalography
9.3.2.2 Limitations of magnetoencephalography
9.3.3 Electromyography
9.3.3.1 Applications of electromyography
9.3.3.2 Advantages of electromyography
9.3.3.3 Limitations of electromyography
9.4 Sleep-based disorder analysis using neurodiagnosis techniques
9.4.1 Polysomnography
9.4.1.1 Advantages of polysomnograhy
9.4.1.2 Limitation of polysomnograhy
9.5 Summary
References
Further reading
10 Reading an extremist mind through literary language: approaching cognitive literary hermeneutics to R.N. Tagore’s play T...
10.1 Introduction
10.1.1 Why transdisciplinary?
10.1.2 Tagore’s The Post Office: a cognitive neurology
10.2 Affecting factors to activate mirror neuron in R.N. Tagore
10.3 Hypothesis
10.4 Colonialism/nationalism or national extremism: symptoms psychoneurological disorders
10.5 The mind of extremist: a neurological observation
10.6 “Nation is the greatest evil for the Nation”?
10.7 Amal as a religion under control
References
Further Reading
Recommended Reading
11 REAH: Resolution Engine for Anaphora in Hindi dialogue
11.1 Introduction
11.1.1 Categorization of Hindi anaphora
11.1.2 Boundaries in anaphora resolution
11.1.2.1 Nonavailability of freeware Hindi discourse
11.1.2.2 Efficiency of linguistic preprocessor
11.1.2.3 No benchmark for POS tagging
11.1.2.4 Lack of efficient named entity recognizer
11.2 The state-of-the-art
11.2.1 Background of the authors
11.3 The resolution engine
11.3.1 The preprocessing phase
11.3.1.1 Data annotation
11.3.1.2 Defining the term patterns
11.3.1.3 Removal of irrelevant chunks and nonanaphoric
11.3.1.4 Identification of intermediate clause
11.3.1.5 Extraction of relevant noun phrases
11.3.1.6 Distance factors
11.3.1.7 Identifying inanimate entity
11.3.2 Anaphora resolution phase
11.3.2.1 Constraints
11.3.2.2 Identifying the equivalence class
11.3.2.2.1 Algorithm for resolving first-person pronouns
11.3.2.2.2 Algorithm for resolving second-person pronouns
11.3.2.2.3 Algorithm for resolving third-person pronouns
11.3.2.2.4 Algorithm for resolving reflexive pronouns
11.3.2.2.5 Algorithm for resolving locative pronouns
11.3.2.2.6 Algorithm for resolving demonstrative pronouns
11.4 Test datasets
11.5 Experiments and evaluations
11.6 Conclusion
References
12 Surveying various effective modes and research trends on cognitive Internet of Things over wireless sensor network
12.1 Introduction
12.2 Objects with computing devices and AI
12.2.1 Internet of Things
12.2.2 Objects with computing devices and computerized ones
12.2.3 Objects with computing devices is not AI
12.2.4 Need for AI in Internet of Things
12.3 Intellectual AI and Intellectual compute
12.3.1 Intellectual AI and cognition, AI
12.3.2 Intellectual computing
12.3.3 Further than mechanization
12.4 Objects with computing devices and Intellectual computing
12.4.1 The Intellectual Internet of Things
12.4.2 Ownership of Intellectual Internet of Things
12.4.3 The pillars of Intellectual Internet of Things
12.4.4 Challenge of Intellectual Internet of Things
12.5 Value of Intellectual Internet of Things
12.6 Areas where we used
12.6.1 Well turned-out livelihood
12.6.2 Elegant health
12.6.3 Household appliances
12.6.4 Smart cities
12.6.5 Wiki City
12.6.6 Synchronized analytics
12.7 Usecase
12.8 Conclusion
References
Further reading
13 Time and feature specific sentiment analysis of product reviews
13.1 Introduction
13.2 Related work
13.3 Proposed model
13.4 Need of feature specificity
13.5 The aging factor
13.6 Experimental setup
13.6.1 Collection and preparing of dataset
13.6.2 Define feature dictionary for product
13.6.3 Preprocess, tokenize, and vectorize the dataset
13.6.4 Classify the review tokens under the features in the feature dictionary
13.6.5 Find the sentiments of the review tokens for each feature
13.6.6 Multiply the polarity with the aging factor to get the sentiment score of the review term
13.6.7 Sum up the results for each feature
13.6.8 Visualize the results
13.7 Result and discussion
13.8 Conclusion and future work
References
14 Language learnability analysis of Hindi: a comparison with ideal and constrained learning approaches
Glossary
14.1 Introduction
14.2 Language acquisition theories
14.3 Evaluation models
14.3.1 Bayesian segmentation
14.3.2 Bayesian inference
14.4 Data preparation for learnability analysis
14.4.1 Transliteration
14.4.2 Syllabification
14.4.3 Phonemization
14.5 Results and discussions
14.6 Conclusion and future work
Acknowledgments
References
Further reading
15 A special report on changing trends in preventive stroke/cardiovascular risk assessment via B-mode ultrasonography
15.1 Introduction
15.1.1 Article search strategy
15.2 Risk assessment using traditional methods
15.3 Fundamentals of machine learning
15.3.1 Types of machine learning techniques
15.3.2 General framework of machine learning
15.3.2.1 Feature engineering: extraction and selection
15.3.2.2 Data partitioning
15.3.2.3 Training model design
15.3.2.4 Prediction or testing model
15.3.2.5 Performance evaluation of machine learning systems
15.3.3 Machine learning–based algorithms
15.4 Risk assessment in machine learning framework
15.4.1 Image-based stroke risk assessment using machine learning
15.4.2 Cardiovascular diseases risk assessment using machine learning
15.4.3 Cardiovscular disease/stroke risk assessment indices
15.5 Medical implications of machine learning–based risk assessment
15.6 Deep learning–based cardiovascular risk stratification
15.7 Challenges in machine learning design
15.8 Conclusion
Acknowledgments
Funding
Disclosure
References
Appendix: performance evaluation parameters
16 A healthcare text classification system and its performance evaluation: a source of better intelligence by characterizin...
16.1 Introduction
16.2 Brief literature survey and our proposed model
16.2.1 Our model
16.3 Data types
16.3.1 Data type 1: TwitterA dataset
16.3.2 Data type 2: WebKB4 dataset
16.3.3 Data type 3: Disease dataset
16.3.4 Data type 4: Reuters (R8) dataset
16.3.5 Data type 5: SMS dataset
16.4 Methodology
16.4.1 Brief discussion on classifiers
16.4.1.1 Support vector machine
16.4.1.2 Multilayer perceptron
16.4.1.3 AdaBoost
16.4.1.4 Stochastic gradient descent
16.4.1.5 Decision tree
16.5 Experiment protocol
16.5.1 Experimental protocol 1: system classifier accuracy computation over all parameters
16.5.2 Experimental protocol 2: effect of training data size on classification accuracy
16.5.3 Experimental protocol 3: overall mean performance using all parameters: D, C, K, and T
Sensitivity
Specificity
Positive predictive value
Accuracy
16.6 Results
16.6.1 Results of protocol #1: system accuracy computation over all parameters
16.6.2 Results of protocol #2: effect of the training data size on classification accuracy
16.6.3 Results for the protocol #3: overall mean performance over all D, C, K, and T
16.7 Hypothesis validation and performance evaluation
16.7.1 Hypothesis validation
16.7.1.1 System performance linking misrepresentation ratio with area under the curve of machine learning system
16.7.1.2 Effect of misrepresentation ratio on machine learning classification accuracy
16.7.1.3 Effect of misrepresentation ratio on mean area under the curve for all classifiers and all data types
16.7.2 Individual receiver operating characteristic plots for all K protocols, D data types, and C classifiers
16.7.3 Reliability and stability analysis
16.7.3.1 Reliability index
16.7.3.2 Stability index
16.8 Discussion
16.8.1 Benchmarking
16.8.2 A special note on classifier, ground truth labels and misrepresentation ratio
16.8.3 Strength weakness and extensions
16.9 Conclusion
Acknowledgment
Funding
Conflict of interest
References
Appendix A Types of dataset used in the study
A.1 TwitterA dataset
A.2 WebKB4 dataset
A.3 Disease dataset
A.4 Reuters (R8) dataset
A.5 SMS dataset
Appendix B Labels used in different text data types
Appendix C Receiver operating characteristic curves
C1 Receiver operating characteristic curves for K2 protocol using five classifiers
C2 Receiver operating characteristic curves for K4 protocol using five classifiers
C3 Receiver operating characteristic curves for K5 protocol using five classifiers
C4 Receiver operating characteristic curves for K10 protocol using five classifiers
C5 Receiver operating characteristic curves for JK protocol using five classifiers
Appendix D Area under the curve tables
Appendix E Postive predictive value tables
Appendix F Sensitivity tables
Appendix G Specificity tables
Appendix H List of abbreviations/symbols
Index

Citation preview

Cognitive Informatics, Computer Modeling, and Cognitive Science Application to Neural Engineering, Robotics, and STEM

Cognitive Informatics, Computer Modeling, and Cognitive Science Application to Neural Engineering, Robotics, and STEM Volume 2 Edited by

G. R. Sinha International Institute of Information Technology (IIIT) Bangalore, Bengaluru, India Myanmar Institute of Information Technology (MIIT), Mandalay, Myanmar

Jasjit S. Suri Stroke Monitoring and Diagnostic Division, AtheroPoint, Roseville, CA, United States Advanced Knowledge Engineering Center, Global Biomedical Technologies, Inc., Roseville, CA, United States

Academic Press is an imprint of Elsevier 125 London Wall, London EC2Y 5AS, United Kingdom 525 B Street, Suite 1650, San Diego, CA 92101, United States 50 Hampshire Street, 5th Floor, Cambridge, MA 02139, United States The Boulevard, Langford Lane, Kidlington, Oxford OX5 1GB, United Kingdom Copyright © 2020 Elsevier Inc. All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage and retrieval system, without permission in writing from the publisher. Details on how to seek permission, further information about the Publisher’s permissions policies and our arrangements with organizations such as the Copyright Clearance Center and the Copyright Licensing Agency, can be found at our website: www.elsevier.com/permissions. This book and the individual contributions contained in it are protected under copyright by the Publisher (other than as may be noted herein). Notices Knowledge and best practice in this field are constantly changing. As new research and experience broaden our understanding, changes in research methods, professional practices, or medical treatment may become necessary. Practitioners and researchers must always rely on their own experience and knowledge in evaluating and using any information, methods, compounds, or experiments described herein. In using such information or methods they should be mindful of their own safety and the safety of others, including parties for whom they have a professional responsibility. To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume any liability for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions, or ideas contained in the material herein. British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library Library of Congress Cataloging-in-Publication Data A catalog record for this book is available from the Library of Congress ISBN: 978-0-12-819445-4 For Information on all Academic Press publications visit our website at https://www.elsevier.com/books-and-journals

Publisher: Mara Conner Acquisition Editor: Chris Katsaropoulos Editorial Project Manager: Ana Claudia Garcia Production Project Manager: Surya Narayanan Jayachandran Cover Designer: Matthew Limbert Typeset by MPS Limited, Chennai, India

Dedication Dedicated to my Late Grand Parents, My Teachers, and Revered Swami Vivekananda G. R. Sinha Dedicated to my late loving parents, immediate family and children Jasjit S. Suri

Contents List of contributors ............................................................................................... xvii Editors’ biographies................................................................................................xxi Authors’ biography.............................................................................................. xxiii Preface .................................................................................................................... xli Acknowledgments ................................................................................................ xliii

CHAPTER 1 Approaches from cognitive neuroscience and comparative cognition .................................................. 1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8

Koushik CSN, Shruti Bhargava Choubey and Abhishek Choubey Introduction ....................................................................................1 Cognitive science ...........................................................................1 Neuroscience ..................................................................................3 Python.............................................................................................4 Review of literature........................................................................4 Cognitive neuroscience/physiology ...............................................5 Cognitive psychology.....................................................................6 Conclusion ....................................................................................12 References.................................................................................... 13 Further reading ............................................................................ 13

CHAPTER 2 Functional neuroanatomy and disorders of cognition ..................................................................... 21 2.1 2.2

2.3 2.4

2.5

Kartik Nakhate, Chandrashekhar Borkar and Ashish Bharne Abbreviations............................................................................... 21 Introduction ..................................................................................22 Neuroanatomy of memory encoding ...........................................22 2.2.1 Medial temporal lobe ........................................................ 23 2.2.2 Diencephalon..................................................................... 23 2.2.3 Basal forebrain .................................................................. 24 Mechanisms underlying memory formation................................24 Neurotransmitters involved in cognition .....................................25 2.4.1 Classical neurotransmitters ............................................... 25 2.4.2 Neuropeptides.................................................................... 29 2.4.3 Neurosteroids .................................................................... 31 Cognition-related diseases............................................................32

vii

viii

Contents

2.5.1 Alzheimer’s disease .......................................................... 32 2.5.2 Lewy body diseases .......................................................... 35 2.6 Conclusion ....................................................................................36 2.7 Acknowledgment..........................................................................36 References.................................................................................... 37 Further reading ............................................................................ 47

CHAPTER 3 A cognitive system of elderly exercise evaluation with sensors and robots ............................................. 49 Tatsuya Yamazaki Introduction ..................................................................................49 System overview ..........................................................................50 Elderly exercise measurement .....................................................52 Exercise evaluation ......................................................................53 Feedback by robot interface.........................................................57 Multiple Kinect application for occlusion problem ....................59 3.6.1 Frame synchronization...................................................... 60 3.6.2 Sensing data integration without calibration.................... 61 3.7 Conclusion ....................................................................................61 Acknowledgment ......................................................................... 62 References.................................................................................... 62 3.1 3.2 3.3 3.4 3.5 3.6

CHAPTER 4 Models of making choice and control over thought for action ....................................................... 65 4.1 4.2 4.3

4.4 4.5

Indrajeet Indrajeet, Shruti Goyal, Krishna P. Miyapuram and Supriya Ray Outline of review .........................................................................65 Introduction ..................................................................................66 Models of perceptual decision .....................................................67 4.3.1 Fast decision-making ........................................................ 72 4.3.2 Intuitive decision-making ................................................. 72 Models of economic decision ......................................................73 Models of movement inhibition...................................................75 4.5.1 Proactive control ............................................................... 78 4.5.2 Estimation of stopping efficacy........................................ 79 4.5.3 Trigger failures.................................................................. 80 4.5.4 Bayesian rational decision-making................................... 81 4.5.5 Optimal Bayesian statistical inference ............................. 83 4.5.6 Decision process as optimal stochastic control................ 84

Contents

4.5.7 Linear approach to threshold explaining space and time model for decisions in space and time ............................. 84 4.6 Discussion.....................................................................................86 Conflict of interest....................................................................... 89 Acknowledgments ....................................................................... 89 References.................................................................................... 89 Further reading ............................................................................ 99

CHAPTER 5 Speech recognition technique for identification of raga ....................................................................... 101 5.1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 5.9 5.10

Snehlata Barde and Veena Kaimal Introduction ................................................................................101 Speech recognition .....................................................................101 Applications of speech recognition............................................102 Speech analyses in music information retrieval ........................103 A brief history of Indian music .................................................103 Mathematical structure of Carnatic music.................................104 Digital speech processing...........................................................109 Proposed methodology for classification of raga ......................110 A practical example using Praat ................................................112 Conclusion ..................................................................................116 Reference ................................................................................... 116 Further reading .......................................................................... 117

CHAPTER 6 Future of cognitive science...................................... 119 Shankru Guggari, H. Nagendra, Santosh R. Desai and Umadevi V 6.1 Introduction ................................................................................119 6.2 Role of cognitive science in varied domains.............................120 6.2.1 Cognitive science for big data ........................................ 120 6.2.2 Cognitive science for philosophy ................................... 121 6.2.3 Brain machine interface ................................................ 121 6.2.4 Cognition science for psychology .................................. 122 6.2.5 Cognition social science ................................................. 123 6.2.6 Role of cognitive science in linguistics.......................... 123 6.2.7 Cognitive control ............................................................ 124 6.2.8 Cognitive image processing............................................ 124 6.3 Future of cognitive neuroscience and cognitive enhancement ...............................................................................125

ix

x

Contents

6.3.1 Scope for neuroscience research and challenges ........... 126 6.3.2 Cognitive enhancement................................................... 126 6.3.3 Ethical issues and concerns of cognitive enhancement.................................................................... 128 6.4 Conclusion ..................................................................................129 References.................................................................................. 129

CHAPTER 7 Application of virtual reality systems to psychology and cognitive neuroscience research ..................... 133 C.S.N. Koushik, Shruti Bhargava Choubey and Abhishek Choubey 7.1 Introduction ................................................................................133 7.1.1 Cognitive science ............................................................ 134 7.1.2 Virtual reality .................................................................. 135 7.2 Literary survey review ...............................................................136 7.2.1 Cognitive neuroscience/physiology ................................ 136 7.2.2 Cognitive psychology ..................................................... 137 7.3 Conclusion ..................................................................................141 References.................................................................................. 141 Further reading .......................................................................... 142

CHAPTER 8 Electrodermal activity and its effectiveness in cognitive research field ........................................... 149 Abdul Momin, Ambika Shahu, Sudip Sanyal and Pavan Chakraborty 8.1 Introduction ................................................................................149 8.2 History of electrodermal activity signal, psychophysiological, and physiological mechanism behind electrodermal activity ...149 8.2.1 Application of electrodermal activity............................. 156 8.2.2 Electrodermal activity as an indicator of general arousal ............................................................................. 157 8.2.3 Electrodermal activity in different sleep stages ............. 157 8.2.4 Electrodermal indices of emotion and stress.................. 158 8.3 Experiment design—a good experiment design........................158 8.3.1 Experimental design........................................................ 158 8.3.2 External and internal influences ..................................... 162 8.3.3 Climatic conditions ......................................................... 163 8.3.4 Internal or physiological influences ............................... 163 8.3.5 Demographic characteristics........................................... 163

Contents

8.4 Electrodermal activity signal collection sites and pretreatment of sites ...................................................................164 8.4.1 Electrodermal activity signal collection sites................. 164 8.4.2 Pretreatment of sites........................................................ 166 8.5 Artifacts removal from the electrodermal activity signal .........167 8.6 Analysis of electrodermal activity signal ..................................167 8.6.1 Phasic electrodermal activity.......................................... 168 8.6.2 Area measurements ......................................................... 170 8.6.3 Tonic electrodermal activity ........................................... 170 8.7 End remarks................................................................................171 References.................................................................................. 171 Further reading .......................................................................... 177

CHAPTER 9 Study of modern brain-imaging and -signaling techniques for brain computer interface ............... 179 Vikas Dilliwar and Mridu Sahu 9.1 Introduction ................................................................................179 9.2 Brain-imagining techniques .......................................................180 9.2.1 Computer tomography .................................................... 180 9.2.2 Near-infrared spectroscopy based imaging equipment ....182 9.2.3 Magnetic resonance imaging .......................................... 183 9.2.4 Single-photon emission computed tomography ............. 185 9.2.5 Cranial ultrasound ........................................................... 186 9.3 Brain-signaling techniques.........................................................187 9.3.1 Electroencephalography.................................................. 187 9.3.2 Magnetoencephalography ............................................... 189 9.3.3 Electromyography ........................................................... 189 9.4 Sleep-based disorder analysis using neurodiagnosis techniques ...................................................................................191 9.4.1 Polysomnography............................................................ 191 9.5 Summary.....................................................................................192 References.................................................................................. 193 Further reading .......................................................................... 195

CHAPTER 10 Reading an extremist mind through literary language: approaching cognitive literary hermeneutics to R.N. Tagore’s play The Post Office for neuro-computational predictions ....................... 197 Valiur Rahaman and Sanjiv Sharma 10.1 Introduction ................................................................................197

xi

xii

Contents

10.2 10.3 10.4 10.5 10.6 10.7

10.1.1 Why transdisciplinary? ................................................. 197 10.1.2 Tagore’s The Post Office: a cognitive neurology ........ 197 Affecting factors to activate mirror neuron in R.N. Tagore .....198 Hypothesis ..................................................................................199 Colonialism/nationalism or national extremism: symptoms psychoneurological disorders.....................................................200 The mind of extremist: a neurological observation ..................203 “Nation is the greatest evil for the Nation”?.............................205 Amal as a religion under control ...............................................207 References.................................................................................. 208 Further Reading ......................................................................... 209 Recommended Reading............................................................. 209

CHAPTER 11 REAH: Resolution Engine for Anaphora in Hindi dialogue..................................................................... 211 Seema Mahato and Ani Thomas 11.1 Introduction ................................................................................211 11.1.1 Categorization of Hindi anaphora ................................ 212 11.1.2 Boundaries in anaphora resolution ............................... 212 11.2 The state-of-the-art.....................................................................213 11.2.1 Background of the authors............................................ 214 11.3 The resolution engine.................................................................214 11.3.1 The preprocessing phase............................................... 214 11.3.2 Anaphora resolution phase............................................ 222 11.4 Test datasets ...............................................................................227 11.5 Experiments and evaluations .....................................................228 11.6 Conclusion ..................................................................................229 References.................................................................................. 230

CHAPTER 12 Surveying various effective modes and research trends on cognitive Internet of Things over wireless sensor network.......................................................... 233 Jaya Mishra, Siddhartha Choubey, Jaspal Bagga and Abha Choubey 12.1 Introduction ................................................................................233 12.2 Objects with computing devices and AI....................................234 12.2.1 Internet of Things.......................................................... 234 12.2.2 Objects with computing devices and computerized ones................................................................................ 234 12.2.3 Objects with computing devices is not AI ................... 235 12.2.4 Need for AI in Internet of Things ................................ 235

Contents

12.3 Intellectual AI and Intellectual compute ...................................237 12.3.1 Intellectual AI and cognition, AI.................................. 237 12.3.2 Intellectual computing .................................................. 237 12.3.3 Further than mechanization .......................................... 238 12.4 Objects with computing devices and Intellectual computing ...239 12.4.1 The Intellectual Internet of Things............................... 239 12.4.2 Ownership of Intellectual Internet of Things ............... 240 12.4.3 The pillars of Intellectual Internet of Things ............... 243 12.4.4 Challenge of Intellectual Internet of Things ................ 244 12.5 Value of Intellectual Internet of Things ....................................246 12.6 Areas where we used .................................................................249 12.6.1 Well turned-out livelihood............................................ 249 12.6.2 Elegant health................................................................ 249 12.6.3 Household appliances ................................................... 249 12.6.4 Smart cities.................................................................... 250 12.6.5 Wiki City....................................................................... 250 12.6.6 Synchronized analytics ................................................. 250 12.7 Usecase .......................................................................................250 12.8 Conclusion ..................................................................................251 References.................................................................................. 252 Further reading .......................................................................... 253

CHAPTER 13 Time and feature specific sentiment analysis of product reviews ........................................................ 255 13.1 13.2 13.3 13.4 13.5 13.6

Aakanksha Sharaff and Asma Soni Introduction ................................................................................255 Related work ..............................................................................256 Proposed model ..........................................................................259 Need of feature specificity .........................................................261 The aging factor .........................................................................262 Experimental setup.....................................................................264 13.6.1 Collection and preparing of dataset.............................. 264 13.6.2 Define feature dictionary for product........................... 264 13.6.3 Preprocess, tokenize, and vectorize the dataset ........... 265 13.6.4 Classify the review tokens under the features in the feature dictionary .......................................................... 265 13.6.5 Find the sentiments of the review tokens for each feature............................................................................ 265 13.6.6 Multiply the polarity with the aging factor to get the sentiment score of the review term .............................. 267 13.6.7 Sum up the results for each feature.............................. 268

xiii

xiv

Contents

13.6.8 Visualize the results ...................................................... 268 13.7 Result and discussion .................................................................269 13.8 Conclusion and future work.......................................................270 References.................................................................................. 270

CHAPTER 14 Language learnability analysis of Hindi: a comparison with ideal and constrained learning approaches................................................................ 273 14.1 14.2 14.3

14.4

14.5 14.6

Sandeep Saini and Vineet Sahula Glossary ..................................................................................... 273 Introduction ................................................................................273 Language acquisition theories....................................................274 Evaluation models ......................................................................277 14.3.1 Bayesian segmentation.................................................. 277 14.3.2 Bayesian inference ........................................................ 279 Data preparation for learnability analysis..................................280 14.4.1 Transliteration ............................................................... 282 14.4.2 Syllabification ............................................................... 282 14.4.3 Phonemization............................................................... 283 Results and discussions ..............................................................283 Conclusion and future work.......................................................287 Acknowledgments ..................................................................... 288 References.................................................................................. 288 Further reading .......................................................................... 290

CHAPTER 15 A special report on changing trends in preventive stroke/cardiovascular risk assessment via B-mode ultrasonography ........................................................ 291 Ankush Jamthikar, Deep Gupta, Narendra N. Khanna, Tadashi Araki, Luca Saba, Andrew Nicolaides, Aditya Sharma, Tomaz Omerzu, Harman S. Suri, Ajay Gupta, Sophie Mavrogeni, Monika Turk, John R. Laird, Athanasios Protogerou, Petros P. Sfikakis, George D. Kitas, Vijay Viswanathan, Gyan Pareek, Martin Miner and Jasjit S. Suri 15.1 Introduction ................................................................................292 15.1.1 Article search strategy .................................................. 295 15.2 Risk assessment using traditional methods ...............................295 15.3 Fundamentals of machine learning ............................................296 15.3.1 Types of machine learning techniques ......................... 296 15.3.2 General framework of machine learning...................... 297

Contents

15.3.3 Machine learning based algorithms............................ 300 15.4 Risk assessment in machine learning framework......................300 15.4.1 Image-based stroke risk assessment using machine learning.......................................................................... 300 15.4.2 Cardiovascular diseases risk assessment using machine learning ........................................................... 303 15.4.3 Cardiovscular disease/stroke risk assessment indices .. 305 15.5 Medical implications of machine learning based risk assessment ..................................................................................305 15.6 Deep learning based cardiovascular risk stratification............306 15.7 Challenges in machine learning design .....................................307 15.8 Conclusion ..................................................................................309 Acknowledgments ..................................................................... 309 Funding ...................................................................................... 309 Disclosure .................................................................................. 309 References.................................................................................. 309 Appendix: performance evaluation parameters ........................ 318

CHAPTER 16 A healthcare text classification system and its performance evaluation: a source of better intelligence by characterizing healthcare text ....... 319 16.1 16.2 16.3

16.4 16.5

16.6

Saurabh Kumar Srivastava, Sandeep Kumar Singh and Jasjit S. Suri Introduction ................................................................................319 Brief literature survey and our proposed model........................321 16.2.1 Our model ..................................................................... 322 Data types ...................................................................................323 16.3.1 Data type 1: TwitterA dataset....................................... 324 16.3.2 Data type 2: WebKB4 dataset ...................................... 324 16.3.3 Data type 3: Disease dataset......................................... 325 16.3.4 Data type 4: Reuters (R8) dataset................................. 325 16.3.5 Data type 5: SMS dataset ............................................. 325 Methodology...............................................................................326 16.4.1 Brief discussion on classifiers ...................................... 327 Experiment protocol...................................................................328 16.5.1 Experimental protocol 1: system classifier accuracy computation over all parameters .................................. 328 Results ........................................................................................330 16.6.1 Results of protocol #1: system accuracy computation over all parameters........................................................ 330

xv

xvi

Contents

16.6.2 Results of protocol #2: effect of the training data size on classification accuracy...................................... 330 16.6.3 Results for the protocol #3: overall mean performance over all D, C, K, and T ................................................. 332 16.7 Hypothesis validation and performance evaluation ..................332 16.7.1 Hypothesis validation.................................................... 334 16.7.2 Individual receiver operating characteristic plots for all K protocols, D data types, and C classifiers ..... 336 16.7.3 Reliability and stability analysis................................... 337 16.8 Discussion...................................................................................340 16.8.1 Benchmarking ............................................................... 340 16.8.2 A special note on classifier, ground truth labels and misrepresentation ratio.................................................. 343 16.8.3 Strength weakness and extensions................................ 343 16.9 Conclusion ..................................................................................343 Acknowledgment ....................................................................... 344 Funding ...................................................................................... 344 Conflict of interest..................................................................... 344 References.................................................................................. 344 Appendix A Appendix B Appendix C Appendix D Appendix E Appendix F Appendix G Appendix H

Types of dataset used in the study.............................................346 Labels used in different text data types.....................................349 Receiver operating characteristic curves ...................................350 Area under the curve tables .......................................................363 Postive predictive value tables...................................................364 Sensitivity tables ........................................................................365 Specificity tables ........................................................................367 List of abbreviations/symbols ....................................................369

Index ......................................................................................................................371

List of contributors Tadashi Araki Division of Cardiovascular Medicine, Toho University, Tokyo, Japan Jaspal Bagga SSTC-SSGI, Bhilai, India Snehlata Barde Department of MATS School of Information Technology, MATS University, Raipur, India Ashish Bharne Department of Pharmacology, University of Cambridge, Cambridge, United Kingdom Chandrashekhar Borkar Department of Psychology (Brain Institute), School of Science and Engineering, Tulane University, New Orleans, LA, United States Pavan Chakraborty Department of Information Technology, Indian Institute of Information Technology, Allahabad, India Abha Choubey SSTC-SSGI, Bhilai, India Abhishek Choubey Sreenidhi Institute of Science and Technology, Ghatkesar, Hyderabad, India Shruti Bhargava Choubey Sreenidhi Institute of Science and Technology, Ghatkesar, Hyderabad, India Siddhartha Choubey SSTC-SSGI, Bhilai, India Koushik CSN Sreenidhi Institute of Science and Technology, Ghatkesar, Hyderabad, India Santosh R. Desai Department of Electronics and instrumentation, B.M.S. College of Engineering, Bengaluru, India Vikas Dilliwar National Institute of Technology, Raipur, India Shruti Goyal Centre for Cognitive and Brain Sciences, Indian Institute of Technology, Gandhinagar, India Shankru Guggari Department of Computer Science and Engineering, B.M.S. College of Engineering, Bengaluru, India Ajay Gupta Department of Radiology, Cornell Medical Center, New York, NY, United States

xvii

xviii

List of contributors

Deep Gupta Department of ECE, Visvesvaraya National Institute of Technology, Nagpur, India Indrajeet Indrajeet Centre of Behavioural and Cognitive Sciences, University of Allahabad, Prayagraj India Ankush Jamthikar Department of ECE, Visvesvaraya National Institute of Technology, Nagpur, India Veena Kaimal Department of MATS School of Information Technology, MATS University, Raipur, India Narendra N. Khanna Department of Cardiology, Indraprastha APOLLO Hospitals, New Delhi, India George D. Kitas R&D Academic Affairs, Dudley Group NHS Foundation Trust, Dudley, United Kingdom C.S.N. Koushik Sreenidhi Institute of Science and Technology, Hyderabad, India John R. Laird Heart and Vascular Institute, Adventist Health St. Helena, St Helena, CA, United States Seema Mahato Dr. C.V. Raman University, Bilaspur, Chattisgarh, India Sophie Mavrogeni Cardiology Clinic, Onassis Cardiac Surgery Center, Athens, Greece Martin Miner Men’s Health Center, Miriam Hospital, Providence, RI, United States Jaya Mishra SSTC-SSGI, Bhilai, India Krishna P. Miyapuram Centre for Cognitive and Brain Sciences, Indian Institute of Technology, Gandhinagar, India Abdul Momin Department of Information Technology, Indian Institute of Information Technology, Allahabad, India H. Nagendra Department of Electronics & Communication, Poojya Doddappa Appa College of Engineering, Kalaburagi (Gulbarga), India Kartik Nakhate Department of Pharmacology, Rungta College of Pharmaceutical Sciences and Research, Rungta Educational Campus, Bhilai, India

List of contributors

Andrew Nicolaides Vascular Screening and Diagnostic Centre, University of Cyprus, Nicosia, Cyprus Tomaz Omerzu Department of Neurology, University Medical Centre Maribor, Maribor, Slovenia Gyan Pareek Minimally Invasive Urology Institute, Brown University, Providence, RI, United States Athanasios Protogerou Department of Cardiovascular Prevention & Research Unit Clinic & Laboratory of Pathophysiology, National and Kapodistrian University of Athens, Athens, Greece Valiur Rahaman Madhav Institute of Technology & Science, Gwalior, India Supriya Ray Centre of Behavioural and Cognitive Sciences, University of Allahabad, Prayagraj India Luca Saba Department of Radiology, University of Cagliari, Cagliari, Italy Mridu Sahu National Institute of Technology, Raipur, India Vineet Sahula Department of Electronics and Communication Engineering, Malaviya National Institute of Technology, Jaipur, India Sandeep Saini Department of Electronics and Communication Engineering, Myanmar Institute of Information Technology, Mandalay, Myanmar Sudip Sanyal Department of Computer Science, BML Munjal University, Gurugram, India Petros P. Sfikakis Rheumatology Unit, National Kapodistrian University of Athens, Athens, Greece Ambika Shahu Cognitive Science Lab, International Institute of Information Technology, Hyderabad, India Aakanksha Sharaff Department of Computer Science and Engineering, National Institute of Technology, Raipur, Chhattisgarh, India Aditya Sharma Cardiovascular Medicine, University of Virginia, Charlottesville, VA, United States

xix

xx

List of contributors

Sanjiv Sharma Madhav Institute of Technology & Science, Gwalior, India Sandeep Kumar Singh Department of Computer Science & Engineering, JIIT Noida, Noida, India Asma Soni Department of Computer Science and Engineering, National Institute of Technology, Raipur, Chhattisgarh, India Saurabh Kumar Srivastava Department of Computer Science & Engineering, JIIT Noida, Noida, India Harman S. Suri Brown University, Providence, RI, United States Jasjit S. Suri Advanced Knowledge Engineering Center, Global Biomedical Technologies, Inc., Roseville, CA, United States; Stroke Monitoring and Diagnostic Division, AtheroPointt, Roseville, CA, United States Ani Thomas Department of IT, Bhilai Institute of Technology, Durg, Chattisgarh, India Monika Turk Department of Neurology, University Medical Centre Maribor, Maribor, Slovenia V Umadevi Department of Computer Science and Engineering, B.M.S. College of Engineering, Bengaluru, India Vijay Viswanathan MV Hospital for Diabetes and Professor M Viswanathan Diabetes Research Centre, Chennai, India Tatsuya Yamazaki Niigata University, Niigata, Japan

Editors’ biographies Dr. G.R. Sinha is an adjunct professor at International Institute of Information Technology (IIIT) Bangalore and currently deputed as the professor at Myanmar Institute of Information Technology (MIIT) Mandalay, Myanmar. He obtained his B.E. (electronics engineering) and M. Tech. (computer technology) with Gold Medal from National Institute of Technology, Raipur, India. He received his PhD in electronics and telecommunication engineering from Chhattisgarh Swami Vivekanand Technical University (CSVTU) Bhilai, India. He has published 250 research papers in various international and national journals and conferences. He is an active reviewer and editorial member of more than 12 reputed international journals such as IEEE Transactions on Image Processing and Elsevier Computer Methods and Programs in Biomedicine. He has been dean of faculty and executive council member of CSVTU India and is currently a member of senate of MIIT. Dr. Sinha has been appointed as ACM Distinguished Speaker in the field of DSP for years (2017 20). He has also been appointed as expert member for Vocational Training Program by Tata Institute of Social Sciences (TISS) for 2 years (2017 19). He has been Chhattisgarh representative of IEEE MP Sub-Section Executive Council for the last 3 years. He has served as distinguished speaker in Digital Image Processing by Computer Society of India (2015). He has also served as distinguished IEEE lecturer in IEEE India Council for Bombay section. He has been the senior member of IEEE for last many years. He is the recipient of many awards such as TCS Award 2014 for Outstanding Contributions in Campus Commune of TCS; R B Patil ISTE National Award 2013 for Promising Teacher by ISTE New Delhi, Emerging Chhattisgarh Award 2013; Engineer of the Year Award 2011; Young Engineer Award 2008; Young Scientist Award 2005; IEI Expert Engineer Award 2007; ISCA Young Scientist Award 2006 Nomination; and awarded Deshbandhu Merit Scholarship for 5 years. He has authored six books, including Biometrics published by Wiley India, a subsidiary of John Wiley and Medical Image Processing published by Prentice Hall of India. He is consultant of various Skill Development initiatives of NSDC, Govt. of India. He is a regular referee of Project Grants under DST-EMR scheme and several other schemes of Govt. of India. He has delivered many Keynote/ Invited Talks and chaired many technical sessions in international conferences held in Singapore, Myanmar, Bangalore, Mumbai, Trivandrum, Hyderabad, Mysore, Allahabad, Nagercoil, Nagpur, Kolaghat, Yangon, Meikhtila, and many other places. His special session on “Deep Learning in Biometrics” was included in IEEE International Conference on Image Processing 2017. He is the fellow of IETE New Delhi and member of international professional societies such as IEEE

xxi

xxii

Editors’ biographies

and ACM and many other national professional bodies such as ISTE, CSI, ISCA, and IEI. He is a member of various committees of the university and has been Vice President of Computer Society of India for Bhilai Chapter for 2 consecutive years. He has guided eight PhD scholars and 15 M.Tech. scholars. His research interest includes image processing and computer vision, optimization methods, employability skills; outcome based education (OBE), etc. Jasjit S. Suri is an innovator, scientist, a visionary, an industrialist and an internationally known world leader in biomedical engineering. He has spent over 25 years in the field of biomedical engineering/devices and its management. He received his doctorate from University of Washington, Seattle and Business Management Sciences from Weatherhead, Case Western Reserve University, Cleveland, Ohio. Dr. Suri was crowned with President’s Gold Medal in 1980 and the fellow of American Institute of Medical and Biological Engineering for his outstanding contributions in 2004.

Authors’ biography C.S.N. Koushik is presently pursuing bachelors in Electronics and Communication Engineering in Sreenidhi Institute of Science and Technology currently in the seventh semester. He has already published a Scopus indexed book chapter and even attended workshops regarding the fields of communication and designing. He has participated in a few various competitions in the School and College. Dr. Shruti Bhargava Choubey received B.E. with honors (2007) from RGPV Bhopal and M.Tech degree in Digital Communication Engineering (2010) from RGPV Bhopal; subsequently, she carried out her research form Dr. K.N. Modi University Banasthali Rajasthan, and awarded PhD in 2015. Presently, she is working as an associate professor in the Department of Electronics and Communication at Sreenidhi Institute of Science and Technology, Hyderabad. She has published more than 50 papers (1 SCI, 13 Scopus) of national and international repute. Her research areas include signal processing Image processing and Biomedical Engineering. She is awarded by MP Council Fellowship in the years 2014 and 2015 for her contribution to Research. Currently, she is working as an associate professor in the Sreenidhi Institute of Science and Technology, Hyderabad, India. Abhishek Choubey received the PhD degree in 2017 from the Department of Electronics and Communication in the Jaypee University of engineering and technology Guna, India. Currently, he is working as an associate professor in the Sreenidhi Institute of Science and Technology, Hyderabad, India. His research interests include signal processing algorithms, VLSI architectures, and digital signal processing in general.

xxiii

xxiv

Authors’ biography

Dr. Kartik Nakhate received his B.Pharm., M. Pharm. (Pharmacology), and PhD (Faculty of Medicine) degrees from the Department of Pharmaceutical Sciences, R.T.M. Nagpur University, Nagpur, India in 2005, 2007, and 2013, respectively. He has authored/coauthored about 30 publications in various international journals and has presented about 25 papers in various national/international conferences. Dr. Nakhate is presently working as an associate professor at Rungta College of Pharmaceutical Sciences and Research, Chhattisgarh Swami Vivekanand Technical University, Bhilai, India. He received research grants from the funding agencies such as SERB, DST, and ICMR. His present research interests are on the evaluation of the role of neuropeptides in the regulation of feeding behavior. Dr. Chandrashekhar Borkar has pursued his B.Pharm., M. Pharm. (Pharmacology), and PhD (Faculty of Medicine) degrees from the Department of Pharmaceutical Sciences, R.T.M. Nagpur University, Nagpur, India in 2009, 2011, and 2019, respectively. He has authored/coauthored about 10 publications in various international journals and has presented about 15 papers in various national/international conferences. Dr. Borkar is presently working as Post-Doctoral Research Fellow at Department of Psychology, Tulane University, New Orleans, LA, United States. He has received the fellowships from DBT and CSIR for his doctoral research and travel funds from funding agencies such as IBRO and SERB to attend international conferences abroad. His present research interests are on the evaluation of molecularly defined subpopulation of amygdaloid neurons in the regulation of fear, memory, and anxiety behaviors. Dr. Ashish Bharne has pursued his B.Pharm., M. Pharm. (Pharmacology), and PhD (Faculty of Medicine) degrees from the Department of Pharmaceutical Sciences, R.T.M. Nagpur University, Nagpur, India in 2006, 2008, and 2019, respectively. He has authored/coauthored about 20 publications in various international journals and has presented 12 papers in various national/international conferences. Dr. Bharne is presently working as Post-Doctoral Research Fellow at Department of Psychology, Tulane University, New Orleans, LA, United States. He has received the travel funds from funding agencies such as IBRO and CICS to attend international conferences abroad. His present research interests are on the evaluation of role neuropeptides in cognition, reward, and neuroprotection.

Authors’ biography

Dr. Tatsuya Yamazaki received his B.E., M.E., and PhD degrees in information engineering from Niigata University, Niigata, Japan, in 1987, 1989, and 2002, respectively. He joined Communications Research Laboratory (at present, National Institute of Information and Communications Technology) as a researcher in 1989. Since August 2013 he has been with the Faculty of Engineering, Niigata University, Niigata, where he is at present a professor. Currently, he is also the director at the Big Data Activation Research Center of Niigata University. His research interests include pattern recognition, statistical image processing, sensing data analysis, and communication service quality management. Dr. Yamazaki has authored/coauthored 41 publications in various high impact factor, peer-reviewed, journals. Dr. Yamazaki has written 7 book chapters and has presented about 72 papers in international conferences. Dr. Yamazaki served as general cochair of IEEE Workshop on Knowledge Media Networking (KMN’02) and general chair of the 5th International Conference On Smart Homes and Health Telematics (ICOST 2007). He is a member of the IEEE, the Institute of Electronics, Information and Communication Engineers, the Information Processing Society of Japan, the Institute of Image Information and Television Engineers, and the Japanese Society for Artificial Intelligence. Indrajeet Indrajeet is a senior research fellow of UGC and D.Phil student at Centre of Behavioural and Cognitive Science (CBCS), University of Allahabad, Prayagraj, India. He earned B.A. (Hons.) in Psychology from Banaras Hindu University, Varanasi, India and M. Sc. in Cognitive Science from CBCS. He is investigating the influence of attention on the control of saccadic eye movement for his doctoral dissertation. His research interest includes executive control, eye movement, perceptual decision making, computational modeling, and cognitive neuroscience. He explores the research problems by using eye tracking, computational modeling, and EEG. Shruti Goyal received her master’s degree from CBCS Allahabad in 2014 and is currently a PhD student at IIT Gandhinagar. Her research is focused on understanding human decision-making process. Her research has specifically looked into the role of experience (as economic feedback and emotional experience) on descriptive choices under risk, predictors of choices using eye-tracking data and the effect of initial information on gain-loss asymmetry. She has also assessed the robustness of prospect theory on Indian population. Her research is based upon methodologies such as eye-tracking, electroencephalogram (EEG) and behavioral choice paradigms. She is a member of Society for Judgement and Decision Making and Association for Cognitive Science, India.

xxv

xxvi

Authors’ biography

Krishna P. Miyapuram received his M.Sc. in Electronics and M.Tech in Artificial Intelligence from University of Hyderabad, respectively, followed by his PhD degree from Department of Physiology, Development & Neuroscience, University of Cambridge, United Kingdom. He is a fellow of Cambridge Commonwealth Society. He has extensive experience in Computational and Cognitive Neuroscience. His postdoctoral work includes both industrial experience (Unilever R&D) and in academia at Centre for Mind/Brain Sciences, University of Trento. He is presently working as an assistant professor and coordinator for Centre for Cognitive and Brain Sciences at Indian Institute of Technology Gandhinagar. His present research interests are on Cognitive modeling, Decision making, Neural correlates of naturalistic Stimuli such as music and films using high-density EEG, Application of Machine learning methods for early detection of Alzheimer’s disease from MRI and resting state connectivity. Supriya Ray is an assistant professor at University of Allahabad and a fellow of the Wellcome Trust India Alliance. He received his M.Tech in Computer Science and Engineering from Calcutta University, and PhD in Neuroscience from National Brain Research Centre in India. During his postdoctoral study in the United States at Vanderbilt University, SKERI, and State University of New York, he studied neural mechanisms underlying perceptual decision making in nonhuman primate’s brain. His research team at CBCS tracks eye and manual movements, and record EEG signals in real time from healthy humans to understand the interplay between vision and action. Dr. Snehlata Barde is working as an associate professor in MAT’S University, Raipur (C.G.). She received her PhD in information technology and computer applications in 2015 from Dr. C.V. Raman University Bilaspur (C.G.). She obtained her MCA from Pt. Ravi Shankar Shukla University, Raipur (C.G.) and M.Sc. (Mathematics) from Devi Ahilya University Indore (M.P.). Her research interest includes Digital Image Processing and its Applications in Biometric Security, Forensic Science, Pattern Recognition, Segmentation, Simulation and Modulation, Multimodal Biometric, Soft Computing Techniques. She has published 40 research papers in various international and national journals and conferences.

Authors’ biography

Ms. Veena Kaimal is working as an assistant professor in MAT’S University, Raipur (C.G.). She is undergoing her PhD in information technology and computer application from MSIT Raipur (C.G.). She obtained her MCA from Sikkim Manipal University, Bhilai (C.G.). Her research interest includes Digital Speech/Signal Processing and Its Applications in Pattern Recognition, and Other Computing Techniques.

Mr. Shankru Guggari (PhD), M.Tech, is a research scholar in the Dept. of Computer Science and Engineering, B.M.S. College of Engineering, Bangalore. He is currently working in classification technique area for his PhD dissertation. Recently, he has won the best research paper award at the international conference. Pattern recognition, IOT, and Machine learning are the interested research area of him. He has published some of his research works in international conferences and a research paper in Elsevier publication journal. He has more than 4 years of industry experience and more than 3 years in academic research experience. Dr. H. Nagendra is with the Poojya Doddappa Appa College of Engineering, Kalaburagi-Karnataka (India). Presently, he is an associate professor in the Department of Electronics and Communication Engineering. He obtained his PhD degree from Indian Institute of Technology Roorkee in the year 2016. He has several publications both in international and national papers/conferences. His research area includes biomedical signal and image processing, cognitive enhancement, and neuroscience. He has received grants from All India Council for Technical Education (AICTE) for various activities. Dr. Santosh Desai obtained PhD degree from Indian Institute of Technology Roorkee in the year 2014. He is presently working as assistant professor in the Department of Electronics & Instrumentation Engineering, B.M.S. College of Engineering. He has several publications in leading journals, namely, Elsevier, Taylor and Francis, apart from national/international conferences. He has received grants from both state and central government agencies.

xxvii

xxviii

Authors’ biography

Dr. V. Umadevi obtained her PhD from IIT Madras and is currently working as an associate professor and head of Computer Science and Engineering Department at B.M.S. College of Engineering, Bengaluru. She has published her work in many reputed international conferences and also published many articles in leading journals with wellknown publishers (Elsevier etc.). She served as a resource person for many Workshops and Faculty development programs. Recently, she got international grants from Amoudi Scientific Research Foundation of Majmaah University, Kingdom of Saudi Arabia to conduct research in the area of Medical Thermography. Mr. Abdul Momin, PhD, is pursuing from Indian Institute of Information Technology, Allahabad, India. His PhD topic is “Creativity and Attention.” His research interests are Human Computer Interaction, Wearable Devices, Analysis of EEG, EDA, fMRI etc. signals.

Ms. Ambika Shahu, MS, is pursuing from IIITHyderabad, India. Her research topic is HCI, where she worked on identifying important display characteristics for complex environments in context of military operations. The thesis was in collaboration with defense research and development organization (DRDO, India). Post submission of her master’s dissertation for internal review, she is working with Google India as a UX researcher on contract. Other than HCI and UX research, she enjoys trekking and gardening. Dr. Pavan Chakraborty received the PhD degree from Astrophysics Indian Institute of Astrophysics, India in 2001. Currently, he is Professor at Indian Institute of Information Technology Allahabad. His research interests are Human Gait Analysis, Human Prosthetics, Biometrics, Image Processing, Graphics and Visual Computing, Graphical Projections, Robotics & Instrumentation, Real Time Simulation, High Performance Computing (HPC), Artificial Life Simulation and Intelligence, and Human Computer Interaction.

Authors’ biography

Prof. Sudip Sanyal received the PhD degree from the Banaras Hindu University, India in 1989. Today he works as the director at BMU, India. Prior to Joining BMU, Prof. Sanyal was Dean (Faculty Affairs), Member of Senate, Member of Board of Governors, Dean (R&D), and Chairman Grievances Cell with Indian Institute of Information & Technology, Allahabad, India. Prof. Sanyal has worked as faculty in leading universities such as Indian Institute of Information & Technology, Allahabad, Banaras Hindu University, and University of Roorkee. He is a PhD from Banaras Hindu University. Prof. S Sanyal has been selected as Consortium member for development of robust document analysis and recognition system, MCIT, 2006 14. Internationally, he is recognized for best software award CICLing 2012. He was Selection Committee Member for DRDO (2007). Selected as a resource person in the Course in Soft Computing Techniques and Applications IIIT Allahabad (2007), Technical Committee Member for UPRTOU (2006 onwards), Selection Committee Member for HRI (2006 07), Member of Advisory Committee— Conference on Wireless Communication and Sensor Network, IIIT Allahabad (2005). He has over 34 years of teaching and research experience. His areas of research are artificial intelligence, and machine learning and information retrieval with special interests in robust document analysis & recognition, OCR for noisy and degraded images, document summarization, and analysis of EEG and ECG signals. Vikas Dilliwar received his B.E. (Hons.) Degree in Information Technology from Pt Ravisankar Shukla University, Raipur, India in 2006, M.Tech degree in Computer Technology from National Institute of Technology, Raipur, India in 2011. He is an assistant professor of Information Technology Department in Chhattisgarh Institute of Technology, Rajnandgaon, India. He is currently pursuing PhD degree from National Institute of Technology Raipur (CG), India. His research interests include parallel processing, distributed computing, biomedical signal processing, and image processing and soft computing. He has published more than 25 research papers in various journals and conference proceedings.

xxix

xxx

Authors’ biography

Dr. Mridu Sahu has completed her graduation in Computer Science and Engineering in 2004 from Maulana Azad National Institute of Technology, Bhopal. She completed her Master of Technology in Computer Science and Engineering from RIT, Raipur in 2011 and completed the PhD in Computer Science and Engineering in 2018 from National Institute of Technology Raipur, India. She is having more than 10 year experiences in teaching; presently, she is working as assistant professor in Department of Information Technology, NIT, Raipur. She has published more than 15 research article in various journals and conferences in the field of Data Mining, Brain Computer Interface, Sensor devices, and Visual Mining Techniques. Valiur Rahaman, MA UGC-NET, PhD, is an assistant professor in the Department of Humanities, Madhav Institute of Technology & Science, Gwalior, India. Dr. Rahaman has been teaching English Language and Literature in reputed institutes of higher education in India since 2009. His doctoral work is based on the works of Jacques Derrida. He taught Literary Theory & Criticism, History and theory of Digital Humanities, Film Theory and Criticism, Design & Critical Thinking for UG, PG, and doctoral research students. His contribution is also noted as Content Writer to Digital Education Initiatives in India known as MHRD ePG Pathsala project for preparing five modules. Five books and a dozen research papers are published in his name. His well-received books include Astitvrasa: Aesthetics of Interrogation (2017), Acts of Literary Theory (2017), Introducing Digital Humanities (2016), Interpretation: Essays in Literary Theory (2011) and Liminality, Mimicry, Hybridity and Ambivalent in Literary Speculations of Homi K. Bhabha (2010). He also translated Jahanara Begam’s Munis ul Arwah from Urdu to English as The Master of Pure Souls (2015). He has delivered more than 50 expert lectures till date. He is the founding president of Indian Society of Digital Humanities. Currently, he is engaged in a TEQIP III-AICTE, India funded research project “Cognitive Literary Studies for the Progress of Computational Cognitive Modeling: A Humanities Inspired Approach to Technological Advances.”

Authors’ biography

Dr. Sanjiv Sharma, PhD (CSE), M.Tech (IT), B.E. (IT), received the PhD degree from Banasthali Jaipur Rajasthan in 2014. He works as an assistant professor in Department of Computer Science Engineering and Information Technology in Madhav Institute of Technology and Science, Gwalior. He have 13 year of teaching and research experience. He has more than 70 research publications in various reputed international journals and conferences. His area of research is big data analysis, machine learning, and data mining and social network analysis. Dr. Ani Thomas, PhD, did her MCA from Government Engineering College, Raipur, Chhattisgarh in 1998. Later, she received her PhD degree in Computer Applications from Chhattisgarh Swami Vivekanand Technical University, Bhilai, Chhattisgarh in 2013. She is presently working as the Professor and Head, Department of Information Technology, Bhilai Institute of Technology, Durg, Chhattisgarh, India. Dr. Thomas has authored 57 research papers in various international and national journals and conferences. She has been a reviewer in IEEE conferences and organized ACM SIGSOFT workshops. She also secured the second position in international Software Project contest organized by TAC-2013 conducted by NIST, Govt. of USA. Her present research interests are on the development of Text mining applications, data mining, image processing, machine learning, and NLP. Seema Mahato is a research scholar from Dr. C.V. Raman University, Bilaspur, Chhattisgarh, India. She received her MCA in 2005 from IGNOU. Later, she received her M.Phil. in Computer Applications from MATS University, Raipur, Chhattisgarh in 2009. She has authored/coauthored about six publications in various high impact factor international journals. Her research interests are on text mining applications, data mining, machine learning, and NLP.

xxxi

xxxii

Authors’ biography

Jaya Mishra is an associate professor and researcher in Electronics and Telecommunication at SSTC, Bhilai. She received her BE degree in ET&T in 2004. Later, she received her M.Tech degree in Electronics and Telecommunication from SSCET Bhilai. Jaya Mishra has around 10 publications in international and national journals and conferences. Her professional body activities involve memberships in IEI(I), ISTE, CSI. Her present portfolio is the President of the society and Vice Chairperson of the Board of Governors of SSTC. Her present research interests are wireless networking, Internet of Things, and machine learning. Dr. Siddhartha Choubey, M.Tech, PhD (Computer Science and Engineering), LMISTE, MCSI. He is working as a professor in Computer Science and Engineering in Shri Shankaracharya Technical Campus Bhilai, India. He has published more than 60 research papers in various international and national journals and conferences. His areas of interest includes networking, parallel processing, image processing, biomedical imaging, nanoimaging, neural network, fuzzy logic, pattern recognition, bioinformatics, AI, machine learning, deep learning, and IOT. Dr. Jaspal Bagga is a senior professor and researcher in Electronics and Telecommunication at SSTC, Bhilai. She received her BE degree in ET&T in 1993. Later, she received her M.Tech degree in Computer Technology from NIT Raipur. Dr. Bagga has around 40 publications in international and national journals and conferences. Her professional body activities involve memberships in IEI(I), ISTE, CSI, Internet Society and International Association of Engineer (IEANG). Her present portfolio is Head of Department of Information Technology, TEQIP III Coordinator and Convener IIC-MHRD amongst other responsibilities Her present research interests are automatic modulation recognition, wireless networking, and machine learning.

Authors’ biography

Dr. Abha Choubey, M.Tech, PhD (Computer Science and Engineering), LMISTE, MCSI, Life member ACM. She is working as a professor in Computer Science and Engineering in Shri Shankaracharya Technical Campus Bhilai, India. She has published more than 60 research papers in various international and national journals and conferences. Her areas of interest include networking, parallel processing, image processing, biomedical imaging, nanoimaging, neural network, fuzzy logic, pattern recognition, bioinformatics, AI, machine learning, and IOT. Aakanksha Sharaff has completed her graduation in Computer Science and Engineering in 2010 from Government Engineering College, Bilaspur (C.G.). She has completed her postgraduation Master of Technology in 2012 in Computer Science & Engineering (Specialization—Software Engineering) from National Institute of Technology, Rourkela and completed PhD degree in 2017 in Computer Science & Engineering from National Institute of Technology Raipur, India. Her area of interest is software engineering, data mining, text mining, and information retrieval. She is currently working as an assistant professor at National Institute of Technology Raipur India. Ms. Asma Soni has received her B.Tech degree from National Institute of Technology, Raipur in 2018. Her current interest is in time-specific and feature-specific sentiment analysis of textual data. She is currently working with a software company, Wipro.

Ankush Jamthikar has received M.Tech in Communication System and currently working as PhD research scholar at Visvesvaraya National Institute of Technology, Nagpur, India.

xxxiii

xxxiv

Authors’ biography

Narendra N. Khanna, MD, DM, FACC is Advisor to Apollo Group of Hospitals in India and is working as Senior Consultant in Cardiology & Coordinator of Vascular Services at Indraprastha Apollo Hospital, New Delhi.

Deep Gupta, PhD, is an assistant professor in the Electronics & Communication Engineering Department, VNIT, Nagpur (India). He received his PhD in Medical Image Processing from Indian Institute of Technology Roorkee, India.

Tadashi Araki received the MD degree from Toho University, Japan in 2003. His research topics include coronary intervention, intravascular ultrasound (IVUS), and peripheral intervention.

Luca Saba, MD, is with A.O.U. Cagliari, Italy. His research interests are in multi-detector-row computed tomography, magnetic resonance, ultrasound, neuroradiology, and diagnostic in vascular sciences.

Authors’ biography

Andrew Nicolaides, MS, FRCS, PhD (Hons.) is currently the Professor Emeritus at Imperial College, London. He is the coauthor of more than 500 original papers and editor of 14 books.

Aditya Sharma, MD, is a cardiologist at University of Virginia, where he directs the anticoagulation clinic and the medical optimization clinic, which helps patients with vascular disorders manage their risk factors with medication. His research interests are peripheral arterial disease, venous thromboembolism, and fibromuscular dysplasia.

Tomaz Omerzu, MD, is currently working at University Medical Centre Maribor, Slovenia. His research interests are radiology and cardiovascular medicine.

Harman S. Suri is currently pursuing his B.S. from Brown University, Providence, RI, United States. He worked in summers of 2015 in the area of telemedicinebased Autism industry at Behavioural Imaging, Boise, Idaho, United States and at Instituto Superior Te´cnico, Lisbon, Portugal in summers of 2018.

xxxv

xxxvi

Authors’ biography

Ajay Gupta, MD, MS, is currently working as an associate professor of radiology and neuroscience at Weil Cornell Medical College New York, United States. His research is focused on neuroradiology.

Monika Turk, MD, is currently working as a physician at Department of Neurology, University Medical Centre Maribor, Maribor, Slovenia.

John R. Laird, MD, FACC, is with St. Helena Hospital, CA, United States. Professor Laird is an internationally renowned interventional cardiologist and his expertise is innovative procedures for carotid artery disease.

Sophie Mavrogeni, MD, PhD, is currently working at Cardiology Clinic, Onassis, Athens, GREECE. Her research is focused on nonischemic cardiomyopathy, dystrophinopathies, myocarditis, and rheumatic diseases.

Authors’ biography xxxvii

Athanasios Protogerou, MD, PhD, is currently working at the Department of Cardiovascular Prevention & Research Unit Clinic, National and Kapodistrian Univ. of Athens, Greece. His research is focused on cardiovascular prevention.

Petros P. Sfikakis, MD, is the Dean of the School of Medicine at National and Kapodistrian University of Athens, Greece and a professor of Internal Medicine and Rheumatology. One of his main research interests focuses in the cardiovascular outcomes of patients with immunemediated diseases.

George D. Kitas, MD, PhD, FRCP, is the director of Research & Development-Academic Affairs, Dudley Group NHS Foundation Trust, Dudley, United Kingdom. He is an Honorary Professor of Rheumatology at the Arthritis Research UK Epidemiology Unit.

Vijay Viswanathan, MD, PhD, FRCP, is currently working as a chairman and director of MV Hospital for Diabetes and M Viswanathan Diabetes Research Centre, Chennai, India. His research interests are endocrinology diabetes and metabolism, diabetes, and foot research.

xxxviii Authors’ biography

Gyan Pareek, MD, is the Director of Minimally Invasive Urologic Surgery and professor of Urological Surgery at the Alpert Medical School of Brown University, Providence, United States. His areas of expertise include kidney stones, prostate cancer, and benign prostatic hyperplasia. Dr. Pareek is board certified in urology and is a fellow of the American College of Surgeons.

Martin Miner, MD, currently both practices as the codirector of the Men’s Health Center at the Miriam Hospital in Providence, United States and also serves as a Chief of Family and Community Medicine. He is the ex-president of the American Society of Men’s Health. Dr. Martin has published over 100 peer-reviewed papers and an expert in Cardiovascular risk evaluation of Erectile Dysfunction (ED) patients.

Jasjit S. Suri, PhD, MBA, is an innovator, visionary, scientist, and an internationally known world leader in the field of biomedical imaging and healthcare management. Dr. Suri is a recipient of President Gold Medal (1980), Fellow of American Institute of Medical and Biological Engineering by National Academy of Sciences, Washington DC (2004), and Marquis Life Time Achievement Award (2018). Dr. Suri is a board member with several organizations. Currently, he is the chairman of AtheroPoint, United States. Dr. Suri has published over 700 papers/patents/books/trademarks with an H-index of 55. Saurabh Kumar Srivastava, M.Tech, is a research scholar in the Department of Computer Science & Engineering at JIIT Noida, India. He obtained his M.Tech (Computer Engineering) from Shobhit University and B. Tech (CSE) from Uttar Pradesh Technical University, Lucknow (U.P.). He has been teaching for 12 years. During his teaching, he has coordinated several technical fests and international/national conferences at institute level. He has attended several seminars, workshops, and conferences at various levels. His area of research includes datamining, machine learning, artificial intelligence, and web technology.

Authors’ biography

Sandeep Kumar Singh, PhD, is an associate professor at JIIT in Noida, India. He has around 15 1 years of experience, which includes corporate training and teaching. His areas of interest are software engineering, requirements engineering, software testing, web application testing, internet and web technology, object-oriented technology, programing languages, information retrieval and data mining; model-based testing and applications of soft computing in software testing and databases. He is currently supervising four PhDs in Computer Science. He has around 28 published papers to his credit in different international journals and conferences. Vibha Pandey is an assistant professor and researcher in Computer Science and Engineering at SSTC, Bhilai. She received her degree in 2008. Later, she received her M. Tech degree in CSE from DIMAT, Raipur. She is working as an assistant professor in Computer Science and Engineering in Shri Shankaracharya Technical Campus Bhilai, India. She has around three publications in international and national journals and conferences. Her areas of interest include image processing, neural network, fuzzy logic, pattern recognition, bioinformatics, AI, machine learning, deep learning, and IOT.

Dr. Jyotiprakash Patra is a senior professor and researcher in Computer Science and Engineering at SSIPMT, Raipur. She received her BE degree in CSE in 2004. Later, she received her M.Tech degree in Computer Technology from SSTC-Bhilai. Dr. Jyoti Patra has around 40 publications in international and national journals and conferences. Her professional body activities involve memberships in CSI, EEE, and ISTE. Head of Department of Computer Science and Engineering is her present portfolio. Her areas of interest AI and soft computing.

xxxix

xl

Authors’ biography

Sandeep Saini received his B.Tech degree in Electronics and Communication Engineering from International Institute of Information Technology, Hyderabad, India in 2008. He completed his M.S. from the same institute in 2010. He is pursuing his PhD from Malaviya National Institute of Technology, Jaipur India. He has been working at Myanmar Institute of Information Technology from 2018. Before joining MIIT Mandalay, he had worked at LNM Institute of Information Technology, Jaipur as an assistant professor from 2011 onward. His research interests are in the areas of natural language processing, cognitive modeling of language learning models. Sandeep is a member of IEEE from 2009 and an active member of ACM as well. Vineet Sahula obtained his bachelors in Electronics (Hons.) from Malaviya National Institute of Technology, Jaipur, India in 1987 and masters in Integrated Electronics; Circuits from the Indian Institute of Technology, Delhi in 1989, and the PhD degree from Department of Electrical Engineering, Indian Institute of Technology, Delhi in 2001. In 1990 he joined as a faculty member at Malaviya National Institute of Technology, Jaipur, where he is currently the head of the Department of Electronics and Communications Engineering. He has 80 1 research papers in reputed journals and conference proceedings to his credit. His research interests are into system-level design, cognitive architectures, cognitive aspects in language processing, modeling and synthesis for analog and digital systems and computer-aided design for VLSI and MEMS. Dr. Sahula has served in the Technical Programme Committee of the VLSI Design and Test Symposium, India from 1998 to 2013. He has also served on organizing a committee of Embedded Systems Week, Oct. 2014 Delhi and as fellowship-chair of 22nd IEEE International Conference on VLSI Design, India in 2009. He is a senior member of IEEE, Life Fellow of IETE and IE, Life member of IMAPS and member of ACM SIGDA.

Preface The cognitive science is an emerging science as interdisciplinary field that covers philosophy, psychology, computer science, neuroscience, linguistics, etc. The science that has been ruling most of the modern world, the cognitive science, and therefore its intricacies, theory, and applications need to be highlighted and elaborated that could help numerous researchers, scientist, psychologist, philosophers, neuroscientists, and others working in the field of human brain and exploitation of its cognitive ability. This volume of the book highlights applications of cognitive science approaches in neural engineering, robotics, and STEM. This book covers cognitive neuroscience and comparative cognition approaches; functional neuroanatomy and disorders of cognition; a cognitive system of elderly exercise evaluation with sensors and robots; models of making choice and control over thought for action; speech recognition technique for identification of raga; future of cognitive science; application of virtual reality systems to psychology and cognitive neuroscience research; electrodermal activity and its effectiveness in various cognitive research; study of modern brain imaging & signaling techniques for brain computer interface (BCI); the mind of an extremist through literary language: approaching cognitive literary hermeneutics to R.N. Tagore’s play Post Office for neurocomputational predictions; REAH: resolution engine for anaphora in hindi dialogue; surveying various effective modes and research trends on cognitive internet of things over wireless sensor network; time specific and feature specific sentiment analysis of product reviews; language learnability analysis of hindi: a comparison with ideal and constrained learning approaches; a special report on changing trends in preventive stroke/cardiovascular risk assessment via B-mode ultrasonography; a healthcare text classification system and its performance evaluation: a source of better intelligence; reconnaissance of automated license plate detection using deep learning. G. R. Sinha and Jasjit S. Suri

xli

Acknowledgments Dr. Sinha expresses sincere thanks to his wife Shubhra, his daughter Samprati, and his great parents for their wonderful support and encouragement throughout the completion of this important book on Cognitive Informatics, Computer Modeling, and Cognitive Science - Volume 2 (Application to Neural Engineering, Robotics, and STEM). This book is an outcome of focused and sincere efforts that could be given to the book only due to great support of the family. Dr. Sinha is grateful to his teachers who have left no stones unturned in empowering and enlightening him, especially Shri Bhagwati Prasad Verma who is like Godfather for him. Dr. Sinha also extends his heartfelt thanks to Ramakrishna Mission order and Revered Swami Satyarupananda of Ramakrishna Mission, Raipur, India. Dr. Sinha would like to thank all his friends, well-wishers, and all those who keep him motivated in doing more and more; better and better. Dr. Sinha offers his reverence with folded hands to Swami Vivekananda who has been his source of inspiration for all his work and achievements. We sincerely thank all contributors for writing relevant theoretical background and real time applications of Cognitive Science and Informatics and entrusting upon us. Last but most important, we express my humble thanks to Chris Katsaropoulos, Senior Acquisitions Editor (Biomedical Engineering) of Elsevier Publications for great support, necessary help, appreciation, and quick responses. We also wish to thank Elsevier Publication for giving us this opportunity to contribute on some relevant topic with reputed publisher. G. R. Sinha and Jasjit S. Suri

xliii

CHAPTER

Approaches from cognitive neuroscience and comparative cognition

1

Koushik CSN, Shruti Bhargava Choubey and Abhishek Choubey Sreenidhi Institute of Science and Technology, Ghatkesar, Hyderabad, India

1.1 Introduction Through programing the current technology advancement has eventually made the use of present-day hardware and software easier for the advancement of the programing language that is used presently. Python, a very powerful tool, is used to perform the required task with an ease such that the programmer need not remember any syntaxes and codes or anything in normal human-understandable language. With the help of the python, one can develop high-level systems to process the data that have been collected by developing artificial intelligence (AI) system that can perform various tasks, thus showing the best alternative that one can proceed with [1].

1.2 Cognitive science Cognitive science is the study that usually depicts how the mind works in any scenario and analyzes it on the basis of the way the nervous system perceives or receives, in a particular condition, the data that are being given as inputs to it and does the corresponding actions. Through cognitive science one can enhance their language, perception, memory, attention, reasoning, and emotions and predicts, if required, the best next alternative. It plays a major role in the decision-making process of the individual, which can be represented as computational procedures for an effective understanding of the analysis that can be made. It could be understood from the topics like literature, psychology, AI, philosophy, neural science, and anthropology (Fig. 1.1) [1]. Psychology is that branch of science which basically deals with emotions and the perceptional ability of an individual that has effects on one’s behavior as per situation one pertains to be; in a simple way, it basically depicts how a person gets affected when he/she is subjected to the changes in the surroundings and the emotional state or any trauma, by studying how the person’s brain reacts to those variations in the sensory input. It can help various researchers to understand Cognitive Informatics, Computer Modelling, and Cognitive Science, Volume 2. DOI: https://doi.org/10.1016/B978-0-12-819445-4.00001-1 © 2020 Elsevier Inc. All rights reserved.

1

2

CHAPTER 1 Approaches from cognitive neuroscience and comparative

Cognitive science applications

Neural science, psychology and physiology

Literature

Anthropology

Artificial intelligence

FIGURE 1.1 Applications of cognitive science [1].

the difference between humans and animals as to how they tend to differ in terms of various subjected conditions. Human brain is mainly affected by depression or stress; when one undergoes that state, his/her behavior gets affected tremendously. The person’s mental state can also be easily affected by the hormonal levels in the body as well. For instance, the person can become angry or irritated or suddenly depressed, for the change in the hormonal levels, and it can even develop various sorts of behavioral complexes that can affect personality traits to a great extent. On the other hand, physiological science deals with how the organs or any part of the body works in an effective manner for the changes in inputs or emotions or chemical substances secreted in the body by it and the duration for which these are subjected to the body. The physiology may be considered only for the operational part/functionality of the part; but when studied further, it is understood that it can have an effect on the brain’s thinking process as to how it affects its functionality. If the body falls ill, it has a great impact on it; especially, if the person can have a trauma based on the intensity of the sickness, which develops as a result of the stress or depression and can even further develop suicidal tendencies in the individual, which can affect the concentration of the brain as well as of the individual from its respective activities. Linguistics or philosophy deals with the language or literature in its own respective manner that one reads or writes. While an individual reads or writes any literary text, it can leave a huge impact on the person who comes across it. Literary works have a great impact on the individual’s mind as a result of which the behavior of the individual can also vary based on how the individual gets obsessed with it. The person can be motivated or diverted; accordingly the behavioral patterns need to be analyzed in order to prevent any sort of behavioral disruptions that can be done in order for the individual to work. The individual can

1.3 Neuroscience

take it in an optimistic way or can take the entire work of the author in a pessimistic way, which can greatly do harm to the individual at some point of time in the mere future, and it solely depends upon the individual’s perceptional ability and even based on the past experiences. Hence in the field of cognitive neuroscience, it plays an important role in determining the behavioral pattern of an individual, when subjected to a wide range of emotional stimuli having a great impact on the individual’s mindset. It plays a major role in the AI system or neural network system wherein the implementation of the next step in the process occurs in an accurate sequence. As a result there would be an ease in the representation of the outcome and have a better understanding of the behavioral patterns of the network in the system. It can also be implemented with the help of the data analytic algorithms to analyze the future patterns in the behavior of the system model that is being depicted. It becomes an important work of the psychiatrists to do the understanding of the individual’s mind and diagnose if required. The analysis of the patterns needs to be achieved in an efficient way by the use of the AI algorithms, and the coding for the corresponding methods needs to be done via any language that can be highly apt for the coders and even for the end users to use it with high level of abstraction.

1.3 Neuroscience Neuroscience is a multidisciplinary branch of medical science, which is majorly concerned with the field of science of the nervous system. It basically deals with the study of the operational or functionality activities of the brain and even studies the structure of the brain and spinal cord in detail. Neural science generally helps one to find the solutions to various problems that arise to these important parts of the body via medicinal therapies that are available and feasible to the individual. It deals with, in particular, the cellular and molecular biology of the organs related to the nervous system as its physiology, anatomy, and required pharmacology of the nervous system. It also deals with the psychological behavior of the individual through computational/behavioral and cognitive neuroscience methods. It is highly complex in nature to understand, and many people are doing their best in their respective fields of research, so that they can find a way to solve all the problems pertaining to the central nervous system (CNS), especially to that of the brain and spinal cord. Due to a huge number of interconnectivities among the neurons, it becomes difficult to analyze and diagnose the problems related to them. If any of the interconnections is disturbed, the entire operationperforming cells of that particular area can be dead or paralyzed; hence, in order to proceed with further course of action, an efficient way has to be found for the treatment of the problems. The human brain consists of three main parts: cerebrum, cerebellum, and brain stem (medulla oblongata). It can be divided into many lobes and consists of

3

4

CHAPTER 1 Approaches from cognitive neuroscience and comparative

various parts filled by neuronal cells. These cells have the ability to do processing, analyzing, and reasoning. It has gray and white matter based on the density of the neurons. The brain is connected to the spinal cord. The transmission of the signals takes place based on the electrical pulses that are generated. They are transmitted with the help of the synaptic transmission that takes place at the synapse of the neurons. These neurons all together constitute a nerve that tends to be as single wire. Any damage to these nerves can cause lot of mental and physical damage to the body and to the individual as well. Each individual has their own perceptional abilities based on the development of the gray and white matter area in the brain and the spinal cord. Hence, there will be times wherein there will be clashes of opinion among the people. When the clashes arise, there will be an impact on the individual’s mentality. It can be achieved by developing an AI system; along with the deep-learning algorithms, it can help us to detect the problems, and we can even find the best way to diagnose the problem based on its prediction. People at present tend to develop a neural network in order to develop an artificial “brain” that can do the processing and analyze and predict the future course of action in the AI system. It consumes a tremendous amount of hardware and storage necessities in order to work like a human brain. Along with the machine-learning models, the outcomes of the actions can be enhanced. Each part of the network is termed a node or center. So the entire human or animal actions can be mimicked for the analysis of the behavioral pattern.

1.4 Python Python is a programing language—both object- and procedure-oriented programing languages. It has been developed from C11/Java wherein all the codes have been written as normal human-understandable language. There are a lesser number of syntaxes with a more number of packages. There are a more number of predefined functions such that any programmer can use those functions easily; hence, it is very easy to code in python programing language. It can be used to develop systems that can be of AI, machine learned, and can easily perform various tasks using image-processing techniques. It is highly efficient to code the requirement of the task. Hence it holds a good requirement of the programs to be written in it for the sake of the research. The number of lines of codes can be reduced to a greater extent and code redundancy can be reduced effectively. It can be used very easily even in the case of data analytics.

1.5 Review of literature Comparative cognition in the fields of neuroscience finds various applications in the fields of physiology, psychology, in various real-time applications in product

1.6 Cognitive neuroscience/physiology

development and gaming, which helps various scientists in their corresponding respective research activities. By using many data mining or data science algorithms on the data that has been collected, one can simulate the results in a three-dimensional virtual environment if required for the sake of the better understanding of the outcomes. The most famous algorithms used are K-means clustering and classification algorithm, which plays a major role in the unsupervised or semisupervised learning aspects for various users to understand the best possible alternative for the particular situation. It can even have various algorithms used along with principles such as AI and machine learning to enhance the output of the systems. There can be various visualization tools used for it to have the visualization to be done easily with the latest advancements in the technology. In the case of the comparitive coognition, when many people are chosen for the sake of the comparison at the time of the testing, subjected to various sensory stimuli, it becomes very crucial to record every aspect in detail so that it can generate the best possible outcomes.

1.6 Cognitive neuroscience/physiology Physiology is the branch of biology that deals with how the parts of the body tend to function. People, who suffer from both mental and physical or any other sicknesses, can be diagnosed very easily when they are subjected to various sensory stimulus for the sake of the comparative cognition; the symptoms of these diseased persons can be visualized either with the help of the visualization tools along with the help of graphs or by virtual reality and can be understood very easily. People suffering from various mental syndromes, stress disorders, and organ malfunctions can be diagnosed easily without any difficulty due to visualization achieved with the help of computerized models that are very accurate and can be understood very easily. It has the potential to enhance the effectiveness in the case of analysis of the results of the tests in the industry by increasing the chances of automation through programing and to reduce the risk of chances of loss of lives. The use of the robotics by the humans in order to treat the person with the help of the comparative cognition of the people or of the person with the simulation results of the outcomes, one can generate the outcomes in an effective way in the reduced amount of time. In context to the exposure theory, when the patients are subjected to the sensory stimuli, the outcomes are to be recorded and processed accordingly and show the indication of the level of impact, it is because with this, the diseases can be diagnosed very easily by comparative cognition, which helps in understanding the outcomes when various individuals of different mentality are compared. The data from these people are collected and then upon which a suitable analysis is conducted by finding out the standard errors or the outliers and are eliminated in the computerized simulated virtual environment. The dataset is collected or taken

5

6

CHAPTER 1 Approaches from cognitive neuroscience and comparative

from the patients that are the subject to be tested at present, which is either tested for or compared with the standard dataset, without having any abnormalities in it. The datasets of both the patients, from whom all the outliers are separated, are scanned effectively, and only the useful data for that sickness or disorder are recorded or collected; hence, it founds the basis for the unsupervised or semisupervised learning. For example, for the people suffering from various mental disorders or syndromes, all the aspects from different people are recorded on the basis of their behavior at various instants and the symptoms in those situations, thinking at that instant and that are analyzed particularly for the sake of diagnosis or research. It forms an unsupervised learning, which is analyzed from the data that are quite huge. These unsupervised data are collected and then given to the system that supports AI, which was developed by python along with various machine/deep-learning models. The mining algorithms such as the K-means clustering or the association rules (Apriori) algorithm can be applied in order to understand the causes and what sets of symptoms can be present at each instant of time and that particular location of different patients for those sensory stimuli given to them. The final processed data can be simulated in the virtual environment or visualized graphically. It can be even used to predict the amount of the medicines that need to be taken by the affected people depending on the intensity level of the sickness that each individual possesses and which can only be achieved based on the AI and machine-learning algorithms in a particular analyzing systems. The datasets can also be affected on the basis of the duration while they are affected; may be anyone of the patients. It can be even used to detect the cancer in the body based on comparison with a normal person who is healthy. The activities of the brain of each patient are recorded and then with the help of test signals such as the EEG signals, the functioning of the entire CNS can be understood/controlled. In the case of various diseases that an individual possess, it can also be detected in a person due to the simulation of all the inputs that the system receives. The outputs can be achieved much more efficiently upon comparative cognition. An AI system is required in order to detect the disorders wherein with the help of various neural networks all those disorders can be found at an ease. Various algorithms of the deep learning can be applied in order to detect the disorders and suggest the best suitable treatment for the disorders and even predict the further stages that can be found in the individual when the treatment is neglected and predict what must be done. Systems with these software programs tend to operate efficiently rather than the hardware part only. Its accuracy can be increased to a greater extent due to comparative cognition.

1.7 Cognitive psychology Psychology is a branch of medical science that is a study of understanding the human behavior with respect to time. For the people who are psychotic or act like

1.7 Cognitive psychology

maniacs or suffering from various mental problems where they tend to have problems in socially interacting with people properly due to their past traumatic experiences that have played a major role in it. The people who are psychotic tend to have no feelings or emotions; still it is even very difficult for people suffering with alexithymia to sense the emotions. The behavior of these various kinds of people are studied by recording every minute aspect carefully and in a safe manner as that sensory stimuli can affect them very badly, which can provoke them at times. Hence, it is essential to record their activities without offending and comparing them with the datasets for the worst sort of the psychotic behavior, from which the data can be actually retrieved based on the traits that they possess; the data can also be retrieved upon comparison with other psychotic people as well in order to compare their mental stages and find various reasons that can cause them. All the unwanted data are filtered from the datasets of the individual test subjects, that is, the outliers are detected upon the preprocessing stages itself and then the filtered data can be analyzed with the system that is intelligent enough in order to provide the best treatment that can be given to them. Their behavior can be simulated and predicted for their next future activities that are possible only when a neural network is present [2]. There are various sorts of the people suffering from different psychological disorders like paranoids etc, for whom the techniques can vary on the basis of the inclusion method, for which techniques such as green paranoid thoughts scale wherein the thought process of the different test subjected paranoids are understood, and based on the outcomes, they can be empathized in a respective manner and that can cure them; similarly there are SBQs (Safety-based Questionnaire), which is used at the public places upon the different test subjects in order to study where and how they behave at those places, etc. Based on the outcomes of the analysis, various sets of treatments (T) can be given [2,3]. The outcomes of these events can be generated only with the help of the machine/deep-learning models that are efficient enough to handle huge amounts of spatial and temporal data instantaneously. When people tend to socialize with others, it is the psychiatrist’s responsibility to consider the mental state of the people that tend to have difficulty in socializing and that can be cured only with the help of the comparative cognition (Figs. 1.2 and 1.3). Similarly in the case of animals, of the same or different species that can belong to different or same geographical places, or in the case of any person, behavior can be studied by understanding the behavioral pattern, with the help of the unsupervised learning only, due to randomness of the data that have been collected, the input of every movements or actions can be taken such that all the behavioral aspects of each organism at every time and location instants can be understood easily upon visualization. Based on the present emotional state aspect and present behavioral patterns, it is analyzed through the help of the data that can be recorded from the brain of the organism and its vitals based on which its future behavior and the movements can be predicted and future location instants can be predicted on the basis of the present trajectory. The dataset of the various aspects of each organism is taken in a separate table or database in which

7

8

CHAPTER 1 Approaches from cognitive neuroscience and comparative

Psychotic patient data set-1

Artificial intelligent system

Psychotic patient data set-2

Comparison for cognitive differentiation

Outcome check

Excluded from treatment

For treatment

FIGURE 1.2 Flowchart of comparative cognition for psychotic people [2].

the condition is recorded and the outcomes are generated accordingly upon which the data can be processed to generate the accurate simulation results. Simultaneously, the data of each organism can be recorded for various instances at that time and location into a separate database and the data can be processed for various aspects and its future outcomes as well that have every minute aspect time stamped [4]. The data can be collected from various sources based on the level of the hormones and their secretions based on their present body condition at that time. These aspects can even be used to understand what will be their future behavioral traits that can be used solely for research purposes [3]. Animals such as mice and rabbits are used for research purposes with utmost care; they can be used for the behavioral pattern analysis. As these animals tend to show similar behavioral traits to that of the humans and are less harmful, they have been selected even though they lack in brilliance when compared to humans. Even crossbreed cognitive comparison can also be done to have a better understanding of the behavioral traits among the animals or humans as well. All the aspects like when they are happy, sad, angry, concerned nature, tensed, and what times they are alert can be predicted and visualized with the help of the virtual reality. All the aspects can be predicted

1.7 Cognitive psychology

Sensory stimuli

Animals central nervous system ccomparison for the cognitive differentiation

Self analyze the outcomes

Corresponding response or reflex

FIGURE 1.3 Physiological behavioral gesture in animals or humans [4].

when they are subjected to those particular conditions that they can generate the various outputs in an emotional manner that is useful to predict in any research work. If the individuals tend to have any sort of phobias from spiders, snakes, dogs, dust, closed rooms, then the person is made to be subjected to these corresponding respective agents of the fear causing agents due to which, under the concept of the exposure therapy, the person’s behavior can be understood, which can play a major role in overcoming the fear causing factor; one can analyze what the body tries to do in those situations such as secretion of various chemical substances. Various physical test data can also be collected like that of the brain, which can even be useful in the person’s treatment. The person’s data for all the adrenaline rush and the emotional fear levels can be recorded from which the intensity of the fear can be analyzed from the processed data; upon comparison from the other people’s response for the same situation, it helps us to have a proper research regarding the same phobia. The processed data are taken or collected on the basis of their mean, variance, whisker ranges, etc. The data will be simulated on the display screen and the best treatment can be given [5,6]. The data visualization can be done even in box plots as well. The behavioral traits for every instant are recorded for the sake of the treatment or research that can generate the various possibilities of the individuals to do, in the upcoming mere future, based on the type of sensory inputs that are being given to the individual based

9

10

CHAPTER 1 Approaches from cognitive neuroscience and comparative

on the phobias. These phobias can increase the levels of stress and depression that need to be controlled in order to have diagnosis, and the understanding of phobia can be better by the explorer. These alone itself can be given to the individual as a simulated sensory input rather than a way to predict the future for the sake of treatment of individuals through virtual reality through a mere simulation. It can be even used for individuals who suffer badly from various syndromes as a part of their treatment. A particular fundus camera can be used in order to record the conditions and that can be given to an AI system, which, with the help of the neural networks, can predict the suitable next steps of action in either research or in treatment to provide medication and further course of preventive steps that are to be taken. The quality of the image pertaining to the patient needs to have a good resolution in order to process and analyze the outcomes of each individual. It is very useful to the psychiatrists for their patients to be diagnosed in an easier manner (Fig. 1.4).

Sensory stimuli

central nervous system comparison for the cognitive differentiation

Self analyze the outcomes

Corresponding response or reflex

FIGURE 1.4 Basic working process [3].

1.7 Cognitive psychology

Sensory stimuli

Cognitive check different cns

Ai or any learnable algorithms or display the outcomes

Reflexes of the motor system

Actions requires in gaming like mobility, etc.

FIGURE 1.5 Learning with virtual reality for gaming [7,8].

In the area of gaming, with the help of the virtual reality, which plays a major role in the enhancement of the user’s performance in the game, it increases the chances of winning the games due to simulation, helping the individual to understand the game much better and increase their chances of winning due to the competitive spirit that one possess. The mentality of the user needs to be understood in order to have a good product development and satisfy the user’s requirements. The player’s present movements are recorded with the help of a camera and then the input image dataset, which helps the computerized model to generate the outputs by predicting the next best move for the player in order to win the game. All the movements of the players of the games are recorded either physcially from the motion sensor or from the image dataset that has been captured and then by the use of the algorithms like Apriori algorithm, the best moves are associated and can be predicted for the user to enhance the chances of winning (Fig. 1.5).

11

12

CHAPTER 1 Approaches from cognitive neuroscience and comparative

When playing multiplayer games, it is quite essential to understand each player’s mentality and increase the toughness of the game. The output efficiency can be enhanced by the use of the AI. The analysis of the data can be achieved based on the behavioral and questionnaire basis to process the data from the individual players [9]. The moves can be affected by the present emotional state of the user and type of the games that the user wants to play. It helps one to have an empathy with the user and generate better results upon the comparison. Based on the level of interest of the gamer, the level of performance can be enhanced such that the developed idea or procedure can help both the gamer and the developer upon the prediction. It can even help the developer to enhance the interface of the individual and even the use of game in the market due to suggestions given by the users. It can be used in various malls in order to attract more customers, which can be used as a perfect marketing strategy. All the emotional aspects are recorded from various users that can be used to predict and enhance the market sales of the product in the future. Virtual reality can show all the scenario of the game in a detailed manner to the gamer to increase the chances of winning through virtual simulation and even predicting what must be done in the next step by an AI system. When a group of people read a book or novel or any scientific content, based on the reader’s understanding capabilities and the present emotional state of the people and the person’s thought process, the data that have been collected accordingly at times when the reader is happy or sad or tensed, which can be simulated either virtually or via graphically; and the level of imagination of the reader can even be depicted when inferred from the display. It can be useful in the field of education as well, which can help the students very much in understanding the contents of the subject and its respective future possible topics in advance, such that the teacher can enhance the method of teaching by the level of understanding of the students. Even in the field of business management for future prediction of the sales, it can be used for the sake of maximizing the profits. It can help very much in the research aspects for the scientists to simulate the alternative paths or ideas to their areas of research. So, whenever the readers reads a material, based on the intensity of the emotional connectivity with literature or the future sales of the book or future emotional state of the reader and continuity of the story can be predicted for the scientific processes that can be altered to have better optimal results in order to understand the mentality of the readers easily. It helps kids in understanding lessons very easily through the simulation and even analyzes the things with the help of AI and machine leaning for their predictions of their character or their behavior [7,8].

1.8 Conclusion Comparative cognition plays an important role in understanding the behavior of the human being through cognitive science of the individual human behavior.

Further reading

It enhances the outcomes of the task that is carried out by the each individual and decreases the load on the psychiatrist or an empathizer. Virtual reality also plays an important role in the field of neuroscience/brain’s physiology wherein the functionality of the brain of the individual can be studied and analyzed for better treatment and surgeries if required; due to the simulation, all the minute details can be observed. It even plays an important role in understanding the behavior of the individuals for their interests in literature, gaming, education, and their mental behavior in between others. Principles such as AI and machine learning can be implemented to enhance the outcomes of the situation and requirements. It can be the best way to tackle all the metal disorders easily and effectively.

References [1] S.J. Shettleworth, Modularity, comparative cognition and human uniqueness, Philos. Trans. R. Soc. London, Ser. B 367 (2012) 2794 2802. [2] T. van Gelder, Cognitive Science: Teaching Critical Thinking, Routledge, 2013. [3] F. Krause, O. Lindemann, Experiment: a Python library for cognitive and neuroscientific experiments, Behav. Res. Methods 46 (2014) 416 428. [4] M. Hanke, Y.O. Halchenko, P.B. Sederberg, S.J. Hanson, J.V. Haxby, S. Pollmann, PyMVPA: a Python toolbox for multivariate pattern analysis of fMRI data Pollmann, Neuroinform 7 (2009) 37 53. [5] W. Tecumseh Fitch, Physics of Life Reviews, 2014. https://doi.org/10.1016/j. plrev.2014.04.005. [6] S.D. Gosling, W. Mason, Internet research in psychology, Annu. Rev. Psychol. 66 (2015) 877 902. Available from: https://doi.org/10.1146/annurev-psych-010814015321. [7] K.J. Mullinix, et al., The generalizability of survey experiments, J. Exp. Polit. Psychol. 2 (2016) 109 138. [8] A. Kittur, et al., Crowdsourcing user studies with Mechanical Turk, in: M. Czerwinski (Ed.), Proceedings of the SIGCHI Conference on Human factors in Computing Systems, ACM, 2008, pp. 453 456. [9] A.J. Berinsky, et al., Evaluating online labor markets for experimental research: Amazon.com’s Mechanical Turk, Polit. Anal. 20 (2012) 351 368.

Further reading N. Stewart, J. Chandler, G. Paolacci, Crowdsourcing samples in cognitive science, Trends Cognit. Sci. 21 (2017) 736 748. Y.Y. Tang, B.K. Ho¨lzel, M.I. Posner, The neuroscience of mindfulness meditation, Nat. Rev. Neurosci. 16 (2015) 213 225. M. Buhrmester, et al., Amazon’s Mechanical Turk: a new source of inexpensive, yet highquality, data? Perspect. Psychol. Sci. 6 (2011) 3 5. J.J. Horton, et al., The online laboratory: conducting experiments in a real labor market, Exp. Econ. 14 (2011) 399 425.

13

14

CHAPTER 1 Approaches from cognitive neuroscience and comparative

D.B. Shank, Using crowdsourcing websites for sociological research: the case of Amazon Mechanical Turk, Am. Sociol. 47 (2016) 47 55. Available from: https://doi.org/ 10.1007/s12108-015-9266-9. D.N. Shapiro, et al., Using Mechanical Turk to study clinical populations, Clin. Psychol. Sci. 1 (2013) 213 220. Available from: https://doi.org/10.1177/2167702612469015. J.K. Goodman, G. Paolacci, Crowdsourcing consumer research, J. Consum. Res. 44 (2017) 196 210. Available from: https://doi.org/10.1093/jcr/ucx047. J.W. Bentley, Challenges With Amazon Mechanical Turk Research in Accounting, SSRN eLibrary, 2017. J.M. Stritch, et al., The opportunities and limitations of using Mechanical Turk (Mturk) in public administration and management scholarship, Int. Public Manage. J. (2017) Published online January 19, 2017. Available from: https://doi.org/10.1080/ 10967494.2016.1276493. J. Lutz, The validity of crowdsourcing data in studying anger and aggressive behavior a comparison of online and laboratory data, Soc. Psychol. 47 (2016) 38 51. Available from: https://doi.org/10.1027/1864-9335/a000256. Y. Majima, et al., Conducting online behavioral research using crowdsourcing services in Japan, Front. Psychol. 8 (2017) 378. Available from: https://doi.org/10.3389/ fpsyg.2017.00378. E. Peer, et al., Reputation as a sufficient condition for data quality on Amazon Mechanical Turk, Behav. Res. Methods 46 (2014) 1023 1031. Available from: https://doi.org/ 10.3758/s13428-013-0434-y. D.L. Crone, L.A. Williams, Crowdsourcing participants for psychological research in Australia: a test of micro-workers, Aust. J. Psychol. 69 (2016) 39 47. E. Peer, et al., Beyond the Turk: alternative platforms for crowdsourcing behavioral research, J. Exp. Soc. Psychol. 70 (2017) 153 163. Available from: https://doi.org/ 10.1016/j.jesp.2017.01.006. E. Estelle´s-Arolas, F. Gonza´lez-Ladrzo´n-De-Guevara, Towards an integrated crowdsourcing definition, J. Inf. Trends Cognit. Sci. 38 (2012) 189 200. Available from: https:// doi.org/10.1177/0165551512437638. F. Sulser, et al., Crowd-based semantic event detection and video annotation for sports videos, in: J. Redi, M. Lux (Eds.), Proceedings of the 2014 International ACM Workshop on Crowdsourcing for Multimedia, ACM, New York, 2014, pp. 63 68. K. Casler, et al., Separate but equal?. A comparison of participants and data gathered via Amazon’s MTurk, social media, and face-to-face behavioral testing, Comput. Hum. Behav. 29 (2013) 2156 2160. L. Casey, et al., Intertemporal differences among MTurk worker demographics, SAGE Open (2017), ,https://osf.io/preprints/psyarxiv/8352x.; ,https://doi.org/10.1177/ 2158244017712774.. K.E. Levay, et al., The demographic and political composition of Mechanical Turk samples, SAGE Open (2016). Published online March 15, 2016. Available from: https:// doi.org/10.1177/2158244016636433. T.S. Behrend, et al., The viability of crowdsourcing for survey research, Behav. Res. Methods 43 (2011) 800 813. Available from: https://doi.org/10.3758/s13428-011-0081-0. K.A. Arditte, et al., The importance of assessing clinical phenomena in Mechanical Turk research, Psychol. Assess. 28 (2016) 684. Available from: https://doi.org/10.1037/ pas0000217.

Further reading

J.K. Goodman, et al., Data collection in a flat world: the strengths and weaknesses of Mechanical Turk samples, J. Behav. Decis. Making 26 (2013) 213 224. Available from: https://doi.org/10.1002/bdm.1753. R. Kosara, C. Ziemkiewicz, et al., Do Mechanical Turks dream of square pie charts?, in: M. Sedlmair (Ed.), Proceedings of the 3rd BELIV’10 Workshop Beyond Time and Errors: Novel Evaluation Methods for Information Visualisation, ACM, New York, 2010, pp. 63 70. D.R. Johnson, L.A. Borden, Participants at your fingertips: using Amazon’s Mechanical Turk to increase student-faculty collaborative research, Teach. Psychol. 39 (2012) 245 251. Available from: https://doi.org/10.1177/0098628312456615. J.C. Veilleux, et al., Negative affect intensity influences drinking to cope through facets of emotion dysregulation, Pers. Indiv. Differ. 59 (2014) 96 101. Available from: https:// doi.org/10.1016/j.paid.2013.11.012. J. Chandler, D. Shapiro, Conducting clinical research using crowdsourced convenience samples, Annu. Rev. Clin. Psychol. 12 (2016) 53 81. Available from: https://doi.org/ 10.1146/annurev-clinpsy-021815-093623. A.A. Arechar, et al., Turking overtime: how participant characteristics and behavior vary over time and day on Amazon Mechanical Turk, J. Econ. Sci. Assoc. 3 (2016) 1 11. Available from: https://doi.org/10.2139/ssrn.2836946. X. Wang, et al., A community rather than a union: understanding self-organization phenomenon on Mturk and how it impacts Turkers and requesters, in: Association for Computing Machinery CHI’17 Conference, ACM, New York, 2017, pp. 2210 2216. N. Stewart, et al., The average laboratory samples a population of 7,300 Amazon Mechanical Turk workers, Judgm. Decis. Mak. 10 (2015) 479 491. In: ,http://journal.sjdm.org/14/14725/jdm14725.pdf.. J. Chandler, et al., Nonnaı¨vete´ among Amazon Mechanical Turk workers: consequences and solutions for behavioral researchers, Behav. Res. Methods 46 (2014) 112 130. ,https://doi.org/10.3758/s13428-013-0365-7.. J. Henrich, et al., Most people are not WEIRD, Nature 466 (2010). Available from: https:// doi.org/10.1038/466029a. J.R. de Leeuw, B.A. Motz, Psychophysics in a web browser? Comparing response times collected with JavaScript and psychophysics toolbox in a visual search task, Behav. Res. Methods 48 (2016) 1 12. Available from: https://doi.org/10.3758/s13428-015-0567-2. M.J. Crump, et al., Evaluating Amazon’s Mechanical Turk as a tool for experimental behavioral research, PLOS One 8 (2013) e57410. Available from: https://doi.org/ 10.1371/journal.pone.0057410. B.E. Hilbig, Reaction time effects in lab- versus web-based research: experimental evidence, Behav. Res. Methods 48 (2016) 1718 1724. Available from: https://doi.org/ 10.3758/s13428-015-0678-9. T. Simcox, J.A. Fiez, Collecting response times using Amazon Mechanical Turk and Adobe Flash, Behav. Res. Methods 46 (2014) 95 111. Available from: https://doi.org/ 10.3758/s13428-013-0345-y. R.A. Klein, et al., Investigating variation in replicability: a ‘many labs’ replication project, Soc. Psychol. 45 (2014) 142 152. Available from: https://doi.org/10.1027/1864-9335/ a000178. R.A. Zwaan, et al., Participant nonnaivete´ and the reproducibility of cognitive psychology, Psychon. Bull. Rev. (2017). ,https://osf.io/preprints/psyarxiv/rbz29..

15

16

CHAPTER 1 Approaches from cognitive neuroscience and comparative

S. Clifford, et al., Are samples drawn from Mechanical Turk valid for research on political ideology? Res. Polit. 2 (2015). Published online December 15, 2015. Available from: https://doi.org/10.1177/2053168015622072. M.R. Munafo, et al., A manifesto for reproducible science, Nat. Hum. Behav. 1 (2017). Available from: https://doi.org/10.1038/s41562-016-0021. R. Rosenthal, The file drawer problem and tolerance for null results, Psychol. Bull. 86 (1979) 638 641. Available from: https://doi.org/10.1037//0033-2909.86.3.638. J.P. Simmons, et al., False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant, Psychol. Sci. 22 (2011) 1359 1366. Available from: https://doi.org/10.1177/0956797611417632. R.W. Frick, A better stopping rule for conventional statistical tests, Behav. Res. Methods Instrum. Comput. 30 (1998) 690 697. J.K. Kruschke, Doing Bayesian Data Analysis: A Tutorial With R and BUGS, Academic Press, Burlington, MA, 2011. U. Simonsohn, Posterior-Hacking: Selective Reporting Invalidates Bayesian Results Also, SSRN eLibrary, 2014. Published online January 3, 2014. https://doi.org/10.2139/ ssrn.2374040. J. Cohen, Statistical Power Analysis for the Behavioral Sciences, second ed., Erlbaum, Hillsdale, NJ, 1988. K.S. Button, et al., Power failure: why small sample size undermines the reliability of neuroscience, Nat. Rev. Neurosci. 14 (2013) 365 376. Available from: https://doi.org/ 10.1038/nrn3475. Open Science Collaboration, Estimating the reproducibility of psychological science, Science 349 (2015) aac4716. Available from: https://doi.org/10.1126/science.aac4716. G. Cumming, The new statistics: why and how, Psychol. Sci. 25 (2014) 7 29. Available from: https://doi.org/10.1177/0956797613504966. U. Simonsohn, Small telescopes: detectability and the evaluation of replication results, Psychol. Sci. 26 (2015) 559 569. Available from: https://doi.org/10.1177/ 0956797614567341. Open Science Collaboration, An open, large-scale, collaborative effort to estimate the reproducibility of psychological science, Perspect. Psychol. Sci. 7 (2012) 657 660. Available from: https://doi.org/10.1177/1745691612462588. N. Schwarz, F. Strack, Does merely going through the same moves make for a ‘direct’ replication? Concepts, contexts, and operationalizations, Soc. Psychol. 45 (2014) 305 306. W. Stroebe, F. Strack, The alleged crisis and the illusion of exact replication, Perspect. Psychol. Sci. 9 (2014) 59 71. Available from: https://doi.org/10.1177/1745691613514450. S. Mor, et al., Identifying and training adaptive cross-cultural management skills: the crucial role of cultural metacognition, Acad. Manage. Learn. Educ. 12 (2013) 139 161. Available from: https://doi.org/10.5465/amle.2012.0202. M. Lease, et al., Mechanical Turk is Not Anonymous, SSRN eLibrary, 2013. Published online March 9, 2013. Available from: http://dx.doi.org/10.2139/ssrn.2228728. K. Fort, et al., Amazon Mechanical Turk: gold mine or coal mine? Comput. Ling. 37 (2011) 413 420. Available from: https://doi.org/10.1162/COLI_a_00057. W. Mason, D.J. Watts, Financial incentives and the performance of crowds, ACM SigKDD Explor. Newsl. 11 (2009) 100 108; 746 Trends in Cognitive Sciences, October 2017, vol. 21, No. 10.

Further reading

L. Litman, et al., The relationship between motivation, monetary compensation, and data quality among US- and India-based workers on Mechanical Turk, Behav. Res. Methods 47 (2009) 519 528. Available from: https://doi.org/10.3758/s13428-014-0483-x. A. Aker, et al., Assessing crowdsourcing quality through objective tasks, in: N. Calzolari (Ed.), Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), European Language Resources Association, 2012, pp. 1456 1461. C.-J. Ho, et al., Incentivizing high quality crowdwork, in: Proceedings of the 24th International Conference on World Wide Web, International World Wide Web Conferences Steering Committee, 2015, pp. 419 429. ,https://doi.org/10.1145/2736277.2741102.. J. Kees, et al., An analysis of data quality: professional panels, student subject pools, and Amazon’s Mechanical Turk, J. Advertising 46 (2017) 141 155. Available from: https://doi.org/10.1080/00913367.2016.1269304. J. Berg, Income security in the on-demand economy: findings and policy lessons from a survey of crowdworkers, Comp. Labor Law Pol. J 37 (2016). M. Yin, et al., The communication network within the crowd, in: Proceedings of the 25th International Conference on World Wide Web, International World Wide Web Conferences Steering Committee, 2016, pp. 1293 1303. S. Frederick, Cognitive reflection and decision making, J. Econ. Perspect. 19 (2005) 25 42. Available from: https://doi.org/10.1257/089533005775196732. K.S. Thompson, D.M. Oppenheimer, Investigating an alternate form of the cognitive reflection test, Judgm. Decis. Mak. 11 (2016) 99 113. M.L. Finucane, C.M. Gullion, Developing a tool for measuring the decision-making competence of older adults, Psychol. Aging 25 (2010) 271 288. Available from: https:// doi.org/10.1037/a0019106. D.G. Rand, et al., Social heuristics shape intuitive cooperation, Nat. Commun. 5 (2014) e3677. Available from: https://doi.org/10.1038/ncomms4677. W. Mason, et al., Long-run learning in games of cooperation, in: Proceedings of the Fifteenth ACM Conference on Economics and Computation, ACM, New York, 2014, pp. 821 838. J. Chandler, et al., Using non-naı¨ve participants can reduce effect sizes, Psychol. Sci. 26 (2015) 1131 1139. Available from: https://doi.org/10.1177/0956797615585115. S.E. DeVoe, J. House, Replications with MTurkers who are naı¨ve versus experienced with academic studies: A comment on Connors, Khamitov, Moroz, Campbell, and Henderson (2015), J. Exp. Soc. Psychol. 67 (2016) 65 67. ,https://doi.org/10.1016/j. jesp.2015.11.004.. D.J. Hauser, N. Schwarz, Attentive Turkers: Mturk participants perform better on online attention checks than subject pool participants, Behav. Res. Methods 48 (2016) 400 407. Available from: https://doi.org/10.3758/s13428-015-0578-z. J. Chandler, G. Paolacci, Lie for a dime: when most prescreening responses are honest but most study participants are imposters, Soc. Psychol. Person. Sci. (2017). Published online April 27, 2017. Available from: https://doi.org/10.1177/1948550617698203. R. Hertwig, A. Ortmann, Experimental practices in economics: a methodological challenge for psychologists? Behav. Brain. Sci. 24 (2001) 383 451. Available from: https://doi. org/10.1037/e683322011-032. Y. Krupnikov, A.S. Levin, Cross-sample comparisons and external validity, J. Exp. Polit. Psychol. 1 (2014) 59 80. ,https://doi.org/10.1017/xps.2014.7..

17

18

CHAPTER 1 Approaches from cognitive neuroscience and comparative

L. Litman, et al., TurkPrime.com: a versatile crowdsourcing data acquisition platform for the behavioral sciences, Behav. Res. Methods 49 (2016) 433 442. Available from: https://doi.org/10.3758/s13428-016-0727-z. K. Scott, L. Schulz, Lookit (Part 1): A new online platform for developmental research, Open Mind 1 (2017) 4 14. ,https://doi.org/10.1162/opmi_a_00002.. M. Tran, et al., Online recruitment and testing of infants with Mechanical Turk, J. Exp. Child Psychol. 156 (2017) 168 178. Available from: https://doi.org/10.1016/j. jecp.2016.12.003. A. Arechar, et al., Conducting interactive experiments online, Exp. Econ. (2017). Published online May 9, 2017. Available from: https://doi.org/10.1007/s10683-0179527-2. S. Balietti, nodeGame: real-time, synchronous, online experiments in the browser. Behav. Res. Methods 49 (5), 1696 1715. L. Yu, J.V. Nickerson, Cooks or cobblers? Crowd creativity through combination, in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, 2011, pp. 1393 1402. J. Kim, et al., Mechanical novel: crowdsourcing complex work through reflection and revision, Comput. Res. Repository (2016). Available from: https://doi.org/10.1145/ 2998181.2998196. R.R. Morris, R. Picard, Crowd-powered positive psychological interventions, J. Posit. Psychol. 9 (2014) 509 516. Available from: https://doi.org/10.1080/17439760.2014.913671. J.P. Bigham, et al., VizWiz: nearly real-time answers to visual questions, in: K. Perlin (Ed.), Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology, ACM, New York, 2010, pp. 333 342. A. Meier, et al., Usability of residential thermostats: preliminary investigations, Build. Environ. 46 (2011) 1891 1898. Available from: https://doi.org/10.1016/j.buildenv. 2011.03.009. M.H. Boynton, L.S. Richman, An online diary study of alcohol use using Amazon’s Mechanical Turk, Drug Alcohol Rev. 33 (2014) 456 461. Available from: https://doi. org/10.1111/dar.12163. J. Dorrian, et al., Morningness/eveningness and the synchrony effect for spatial attention, Accid. Anal. Prev. 99 (2017) 401 405. Available from: https://doi.org/10.1016/j. aap.2015.11.012. K. Benoit, et al., Crowd-sourced text analysis: reproducible and agile production of political data, Am. Polit. Sci. Rev. 110 (2016) 278 295. Available from: https://doi.org/ 10.1017/S0003055416000058. P. Mueller, J. Chandler, Emailing workers using Python, SSRN eLibrary, 2012. Published online July 5, 2012. ,https://ssrn.com/abstract 5 2100601.https://doi.org/10.2139/ ssrn.2100601.. S. Reimers, N. Stewart, Presentation and response timing accuracy in Adobe Flash and HTML5/JavaScript Web experiments, Behav. Res. Methods 47 (2015) 309 327. Available from: https://doi.org/10.3758/s13428-014-0471-1. S. Reimers, N. Stewart, Auditory presentation and synchronization in Adobe Flash and HTML5/JavaScript Web experiments, Behav. Res. Methods 48 (2016) 897 908. Available from: https://doi.org/10.3758/s13428-016-0758-5.

Further reading

J. de Leeuw, Jspsych: a JavaScript library for creating behavioral experiments in a web browser, Behav. Res. Methods 47 (2015) 1 12. Available from: https://doi.org/ 10.3758/s13428-014-0458-y. T.M. Gureckis, et al., Psiturk: an open-source framework for conducting replicable behavioral experiments online, Behav. Res. Methods 48 (2016) 829 842. Available from: https://doi.org/10.3758/s13428-015-0642-8. G. Stoet, PsyToolkit: a software package for programming psychological experiments using Linux, Behav. Res. Methods 42 (2010) 1096 1104. G. Stoet, PsyToolkit: a novel web-based method for running online questionnaires and reaction-time experiments, Teach. Psychol. 44 (2017) 24 31. Available from: https:// doi.org/10.1177/0098628316677643. T.W. Schubert, et al., ScriptingRT: a software library for collecting response latencies in online studies of cognition, PLoS One 8 (2013). Available from: https://doi.org/ 10.1371/journal.pone.0067769. I. Neath, et al., Response time accuracy in Apple Macintosh computers, Behav. Res. Methods 43 (2011) 353 362. Available from: https://doi.org/10.3758/s13428-0110069-9. R. Ulrich, M. Giray, Time resolution of clocks: Effects on reaction time measurement— good news for bad clocks, Br. J. Math. Stat. Psychol. 42 (1989) 1 12. Available from: https://doi.org/10.1111/j.2044-8317.1989.tb01111. xTrends in Cognitive Sciences, October 2017, vol. 21, No. 10 747.

19

CHAPTER

Functional neuroanatomy and disorders of cognition

2

Kartik Nakhate1, Chandrashekhar Borkar2 and Ashish Bharne3 1

Department of Pharmacology, Rungta College of Pharmaceutical Sciences and Research, Rungta Educational Campus, Bhilai, India 2 Department of Psychology (Brain Institute), School of Science and Engineering, Tulane University, New Orleans, LA, United States 3 Department of Pharmacology, University of Cambridge, Cambridge, United Kingdom

Abbreviations 5-HT ACh AD ALLO APP APPICD Aβ BF CA CART CNS CSF DA DHEA DLB FAD GABA GSK3β HD LBD LTM MC4R MSDB MTL MWM NFT NORT NOS

5-hydroxytryptamine acetylcholine Alzheimer’s disease allopregnanolone amyloid precursor protein APP intracellular domain β-amyloid basal forebrain cornu ammonis cocaine- and amphetamine-regulated transcript central nervous system cerebrospinal fluid dopamine dehydroepiandrosterone dementia with Lewy bodies familial AD γ-aminobutyric acid glycogen synthase kinase-3β Huntington’s disease Lewy body disease long-term memory melanocortin-4 receptor medial septum/horizontal diagonal band medial temporal lobe Morris water maze neurofibrillary tangle novel object recognition test nitric oxide synthase

Cognitive Informatics, Computer Modelling, and Cognitive Science, Volume 2. DOI: https://doi.org/10.1016/B978-0-12-819445-4.00002-3 © 2020 Elsevier Inc. All rights reserved.

21

22

CHAPTER 2 Functional neuroanatomy and disorders of cognition

NPY PD SAD sAPPβ STM α-MSH αS

neuropeptide Y Parkinson’s disease sporadic AD soluble amyloid precursor protein short-term memory α-melanocyte stimulating hormone α-synuclein

2.1 Introduction Cognition is a mental process, which involves a variety of functions such as attention, learning, memory, perception, planning, reasoning, analysis, problemsolving, decision-making, and executing actions [1]. Learning and memory are known as the fundamental brain functions to perform day-to-day life activities. Studies have conceptualized learning as the skill of acquiring and encoding information about an object, context, events, things, surrounding, etc., which leads to behavioral modifications. Memory is the persistence and recall of such behavioral modifications, in other words, experience-dependent internal representations of the world [2,3]. Memory function of the brain is a very complex system. Various brain regions are involved in the acquisition, consolidation, and storage of the information in long-term memory (LTM) and pathophysiology of diseases such as Alzheimer’s disease (AD), Parkinson’s disease (PD), Huntington’s disease (HD), schizophrenia, and epilepsy, wherein predominantly declarative memory is affected. Irreversible dementias especially AD and PD have become greater medical problems in the aging population. Currently available cognition-improving agents (smart drugs) such as tacrine, rivastigmine, galantamine, donepezil, and memantine have limited therapeutic value. Therefore there is an emerging need to find out novel targets for the development of future drugs by elucidating the underlying complex pathophysiological mechanisms and identifying the key mediators. In addition, the detailed study of memory trace formation and neuroanatomy of regions involved in cognitive regulation could help to understand better the pathophysiology of neurocognitive diseases and to develop the novel therapeutic agents. The present chapter emphasized on the process of memory trace formation, neuroanatomical regions involved in the cognitive processes especially in explicit memory, key neurotransmitters involved in the cognition, and some important disorders that hamper cognition.

2.2 Neuroanatomy of memory encoding Several brain regions are involved in acquisition, consolidation, and storage of the information as mentioned in Table 2.1 [4 7]. Some important neuroanatomical loci involved in memory processing are discussed below.

2.2 Neuroanatomy of memory encoding

Table 2.1 Brain regions involved in the different types of memory. Type of memory

Brain region

Emotional memory Recognition memory Working memory Motor skills Sensory memory Classical conditioning Habituation Spatial learning

Amygdala Hippocampus, temporal lobe Hippocampus, prefrontal cortex Striatum, cerebellum Various cortical areas, cerebellum Cerebellum Basal ganglia Hippocampus, parahippocampus, subiculum, cortex (temporal cortex, area 47, posterior parietal cortex)

2.2.1 Medial temporal lobe The medial temporal lobe (MTL) is one of the most important structures in the operation of declarative memory. The key elements of this brain region are the amygdala, hippocampus, and the entorhinal, perirhinal, and parahippocampal cortices [8]. Several classical studies have proved that amongst these regions hippocampus is a central domain and leads to severe and permanent explicit memory impairment on even a partial lesion. Hippocampal damage mostly affects the acquisition of new memories, while keeping the previously acquired memories intact, and unequivocally indicate that hippocampus is not work-alone brain region. Hippocampus establishes episodic, temporal, and spatial associations between two events [7,9]. Hippocampus is also implicated in the early processing of implicit memory. Later on, studies find that the neuronal circuits of hippocampus also activate the cortical areas such as parahippocampal, perirhinal, and entorhinal cortices in temporarily coordinated manner to drive the encoding of episodic memory [10]. However, amygdaloid complex largely regulates conditioned fear behavior, and its lesions do not cause explicit memory impairment [11 13].

2.2.2 Diencephalon The diencephalic regions such as mammillary bodies and mammillothalamic tract [14], intralaminar and midline thalamic nuclei [15], anterior thalamic nuclei [16], and mediodorsal thalamic nucleus [17] have thought to participate in memory functions. Diencephalon has many direct connections with the prefrontal cortex [18,19] and the MTL [20,21]. Previous studies have also confirmed the reciprocal connections from MTL to medial diencephalon through fornix and, interestingly, loss of memory was noted following fornix lesion. Moreover, equivocally, lesions of diencephalic brain nuclei result into severe memory loss in the rodents [22].

23

24

CHAPTER 2 Functional neuroanatomy and disorders of cognition

However, data show the involvement of globus pallidus [23,24] and basal ganglia in the several motor-learning processes [25].

2.2.3 Basal forebrain The hippocampus and various cortices (frontal, temporal, and parietal) receive the prominent cholinergic innervations from basal forebrain (BF) regions [26]. The BF is compartmentalized into the medial septum/horizontal diagonal band (MSDB), peripallidal regions, and the substantia innominata. The MSDB project to the hippocampus (septohippocampal pathway) to performed cognitive actions. The neocortical areas received projections from the horizontal limb of the diagonal band, peripallidal regions, and the substantia innominata, to regulate the attentions [27]. Studies noted the activation of sensory cortex prefrontal cortex BF sensory cortex loop signaling during cognitive output associated with specific task [28]. Particularly, septo-hippocampal pathway provides two anatomically and functionally distinct circuits. First, direct cholinergic circuit that increases hippocampal inhibitory interneurons firing along with reduced principal cells firing. Second, indirect circuit that synchronizes hippocampal theta wave mediated by noncholinergic cells located in the MSDB that are functions under cholinergic neurons. Studies involving immunotoxin 192 IgGsaporin-mediated cholinergic neuronal loss within BF [29,30] or scopolamine infusion into BF showed pronounced impairment in the cognition in rodents. Thus while these pathways are likely relevant for hippocampal cholinergic regulation of encoding versus retrieval modes [28,31], its massive loss is evident in the patients with AD and tau pathology [32,33].

2.3 Mechanisms underlying memory formation The perceived sensory information passes through three overlapping steps, namely, acquisition, consolidation, and retrieval. While acquisition associated with conscious perception and perseverance of sensory information to store it for short period, the consolidation process converts the registered memory into long term. However, retrieval is a process of reactivation, reconstruction, or similar presentation of the stored information, for its expression. Based on the duration of storage, memories can be classified as sensory, short-term (STM), and LTM. Sensory memory is momentarily (1 2 seconds) remembering about certain things such as smell, sound, look, feel, or taste. STM is referred to as primary memory or active memory that stores information for B10 12 seconds. The storage mechanism of STM includes changes in the excitation secretion coupling at the presynaptic level promoted by changes in channel conductance predominantly carried out in prefrontal cortex [34 36]. Following the conscious efforts, STM is converted to the most important LTM. This memory stores either explicit (declarative,

2.4 Neurotransmitters involved in cognition

conceptual knowledge, facts, events, etc.) or implicit (nondeclarative, habits, skills, simple conditioning, etc.) information, perhaps due to physical long-lasting changes in the synapses [2]. As described in above section, the crucial role of discrete brain regions such as MTL, diencephalon, and BF have already been reported in the acquisition and specific type information storage. MTL regions include the amygdala, hippocampus, and entorhinal, perirhinal, and parahippocampal cortices. The neuronal networks between entorhinal cortex and hippocampal are important for encoding memory. The sensory signal traveled through trisynaptic excitatory pathway from entorhinal cortex to dentate gyrus, cornu ammonis (CA)3, and CA1, which sends signals back to entorhinal cortex. Corroboration of scientific investigation on cognition in the last decade suggest that although hippocampus encodes the memories, their multimodal and multidomain representations are not exclusively stored in hippocampus [37,38]. It seems that hippocampus encodes the memories on the other neural substrate such as cortex [36]. It was noted that, hippocampus cells (place cells) encode internal cognitive maps [39] in coordinated manner with some cortical neurons (grid cells) that fire in spatially defined, periodic triangular array to positioning the animals with respect to the surrounding [40]. Following repetitive sensory imputes for specific information, physical modification takes place in synaptic assemblies that increases the strength of operational circuit and durability of the memory traces [41]. Two different, but not mutually exclusive, theories support the molecular mechanism of synaptic strengthening, namely, the induction of long-term potentiation (LTP) and/or activation of neural networks. The LTP, chiefly regulated by α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid (AMPA) and N-methyl-D-aspartate (NMDA) glutamic receptor, mediated the neurotransmitter release and dendritic spine plasticity, particularly in the cerebral cortex and the hippocampus. Other synaptic plasticity and thus memory regulating mechanism are the regulation of transmitter release, recycling of membrane receptors, persistent activation of kinase phosphatase system for example, calmodulin-dependent protein kinase II, mitogen-activated protein kinase (MAPK), serine/threonine and tyrosine kinase, and the molecular turnover at synapses [42,43].

2.4 Neurotransmitters involved in cognition 2.4.1 Classical neurotransmitters 2.4.1.1 Acetylcholine Acetylcholine (ACh) is a widely distributed neurotransmitter in the brain and involved in the ranges of physiological functions [44]. The initial studies have put forth the putative correlation between decline in cognitive functions in dementia and a decrease in cholinergic neurotransmission [45]. Later on, pharmacological studies using cholinergic antagonists showed the linking of learning and memory

25

26

CHAPTER 2 Functional neuroanatomy and disorders of cognition

processes to the cholinergic neurotransmission. For instance, intracranial infusion of scopolamine or carbopolin in the brain regions such as parahippocampus, perirhinal cortex, or dentate gyrus has resulted in impairment of learning and memory [46]. The neuroanatomical studies showed that, cholinergic neurons primarily originate from basal forebrain area and septum in the brain. BF structure provides the major cholinergic output of the cortex. Another cluster, the septohippocampal nucleus, is the key cholinergic output area projecting to the hippocampus and involved in different phases (e.g., learning and retrieval) of memory [47]. Microdialysis technique has also confirmed the increase in hippocampal ACh levels during hippocampal-dependent learning procedures [48]. Deficits in cholinergic transmission affect different aspects of cognition and behavior, including hippocampal and cortical processing of the information [49]. ACh is a key switch for the hippocampal states during memory encoding and consolidation, mostly facilitating the encoding of information from cortex to hippocampus [50]. Cholinergic cell loss in the basal forebrain is found during natural aging as well as in the condition such as AD [51 53]. In AD, β-amyloid (Aβ) oligomers affected cholinergic synapses [54] that lead to the major cognitive impairment [55]. Via the activation of both nicotinic and muscarinic receptors, ACh showed increase in the long-term potentiation. Notably, ACh is regulating only the hippocampus-dependent learning and memory such as declarative memory. Furthermore, the inhibition of acetylcholine esterase (the enzyme that breaks down ACh), to increase the ACh levels in the brain, is a primary treatment approach in AD patients [56], suggesting important role ACh in memory and attention.

2.4.1.2 Glutamate Glutamate is a major excitatory neurotransmitter in the central nervous system (CNS). Glutamate signaling follows the “Goldilocks principle” suggesting the both antagonist and agonist properties by the same neurotransmitter. While insufficient glutamate excitation is associated with concentration deficits or mental exhaustion, excessive stimulation leads to excitotoxicity and neuronal death. As described under the mechanism of memory formation, glutamatergic system plays a major role in memory formation and retrieval [57,58]. Glutamate mediates its functions through ionotropic and metabotropic receptors. The well-recognized ionotropic receptors are NMDA, AMPA, and kinate types, which are important for learning and memory [59]. In rodents, blockade of glutamate receptors impairs spatial working [60], recognition [61,62], associative [63], and episodic memory [64]. Further, the antagonism of these glutamate receptors impaired both the acquisition and retention of hippocampus-dependent memories [65]. Systemic administration of per se NMDA potentiated cognitive functions [66], and in combination with MK-801, this prevents amnesia induced by later [67]. Superior learning and memory was noticed by the genetic enhancement of NMDA receptor function [68]. In healthy human volunteers, antagonism of glutamatergic system by ketamine impair performance on tests of verbal and

2.4 Neurotransmitters involved in cognition

nonverbal declarative memory, verbal fluency, and problem-solving [69 71]. Acute administration of ketamine, phencyclidine, and MK-801 (NMDA receptor antagonists) impairs the executive function, cognitive flexibility, and attention processing [72]. Moreover, the expression of vesicular glutamate transporters (VGLUT)-1 and VGLUT-2 has a positive correlation with the capacity of learning and memory [73]. However, Groups II and III of metabotropic glutamate receptors are presynaptic, and thus its activation reduces the glutamate release. This strategy is currently under investigation to protect BF neurons from excitotoxicity [74]. Animal, human, and genetic studies clearly indicate that the glutamate receptor system in the brain is conceivably involved in the processes of learning and memory formation.

2.4.1.3 γ-Aminobutyric acid γ-Aminobutyric acid (GABA) is major inhibitory neurotransmitter in the CNS. The involvement of GABA in the regulation of vigilance, anxiety, and memory processes is well established [75,76]. Posttraining injections of GABAergic compounds modulate memory storage [77 79]. Intrahippocampal injection of the GABA-A receptor agonist muscimol impaired retrieval in the water maze task in rats [80]. It was noted that, while GABA administration significantly increased recognition index for the novel object, its depletion in prefrontal cortex causes deficits in delay task. Many evidence suggest that fear memory formation, reconsolidation, and extinction depend on reduced the activation of GABA-A receptors in different cerebral regions. In addition, studies noted the involvement of GABAergic interneurons in the encoding and maintenance of working memory information [81]. GABAergic system also plays a key role in the pathogenesis of AD, wherein moderate-to-significant reduction in GABA concentrations in various cortical areas was observed in tissue from patients with AD (postmortem) [82].

2.4.1.4 Dopamine Dopamine (DA) is a prominent neuromodulator that has wide-ranging effects on both cortical and subcortical brain regions. Pioneering studies revealed that the depletion of DA produces an impairment of working memory [83]. DA secretion in the hippocampus is activated by different stimuli that help to ascertain and stabilize hippocampus-dependent (plasticity) memories [84]. It has been reported that the intrahippocampal administration of D2 receptor agonists improves the spatial working memory in animals [85]. DAminergic system within the framework of hippocampus mediates the acquisition of novel information, which gets transformed into LTM, if it is biologically significant [86]. Within the framework of prefrontal cortex the dopaminergic modulation of neural activity plays an essential role for working memory [87]. DA projections to prefrontal cortex modulate attention and spatial working memory [88]. The intracortical application of D1 receptor antagonists produces an impaired performance of spatial working memory, both in monkeys [89] and rats [90]. The role of DA in the striatum is

27

28

CHAPTER 2 Functional neuroanatomy and disorders of cognition

related with the flexibility of the changes in response patterns characteristic of implicit learning and memory processes [91].

2.4.1.5 Serotonin (5-hydroxytryptamine) A high density of serotonergic projections is found in the hippocampus and prefrontal cortex. This underlines its anatomical and neurochemical linkage with learning and memory [92]. In the hippocampus 5-hydroxytryptamine (5-HT) regulates spatial navigation and social relationships [93,94]. In the prefrontal cortex, 5HT system plays a major role in working memory, attention, and reversal learning [95]. System also plays a role in decision-making. 5-HT makes a complex interactions with ACh, DA, GABA, and glutamate which is its role in the neurobiology of learning and memory might be attributed to [96]. Serotonergic system also plays a role in STM and LTM and cognitive performance, during aging [97] as well as in many psychiatric and neurological disorders [98]. Extensive serotonergic denervation in AD is also suggested [99]. In AD patients a decreased numbers and activity of 5-HTergic neurons, the concentration of 5-HT and its main metabolite 5-hydroxyindoleacetic acid was found in the postmortem brain [100], cerebrospinal fluid (CSF) [101], and blood platelets [102].

2.4.1.6 Agmatine Agmatine, an endogenous amine, is synthesized through decarboxylation of L-arginine. It is described as a novel neurotransmitter and/or neuromodulator in mammalian brain. Agmatine exhibits its biological effects in the CNS by interacting with NMDA, nicotinic and 5-HT3 receptors, and neuronal pathways. Number of studied reported cellular and subcellular localization of agmatine in the hippocampus that proposed its role in cognitive functions [103,104]. Systemic as well as intracerebroventricular administration of agmatine facilitated memory consolidation and improved the animal’s performance in the radial arm maze, fear conditioning, inhibitory avoidance, Morris water maze (MWM), and novel object recognition test (NORT) [105 107]. In aged rats, agmatine level was found to be decreased significantly in the CA1 and prefrontal cortex but increased in the CA2/3, dentate gyrus, and rhinal and temporal cortices [108]. In these rats, acute as well as chronic agmatine treatment significantly improved spatial working memory in the MWM and object-recognition memory in NORT. This action was found to be mediated by attenuation of age-related elevation in total nitric oxide synthase (NOS) activity and restoring endothelial NOS protein to its normal level [109]. In normal young rats, spatial learning in MWM and T-maze resulted in the region-specific elevation of agmatine levels [103,108,110]. These results proposed that endogenous agmatine plays a role in the initial phase of encoding and processing information retrieval. Furthermore, agmatine also prevented scopolamine-induced memory impairment [111]. The similar effect was observed in streptozotocin-induced memory deficits in diabetic rats [112] and in morphine or lipopolysaccharide-induced memory impairment in mice [113]. Agmatine also showed memory-enhancing effect in

2.4 Neurotransmitters involved in cognition

AD-like neurodegenerative diseases. It significantly decreased Aβ25 35-induced spatial learning and memory impairment [114]. Taken together, accumulating evidence demonstrated that endogenous agmatine is involved in the learning and memory process and administration of exogenous agmatine has considerable improving effects on cognitive impairment.

2.4.2 Neuropeptides Neuropeptides are widely distributed in cognition-regulating brain regions and induce morphological changes by facilitation of RNA and de novo protein synthesis required for the development of synaptic plasticity. Several neuropeptides improve cognitive performance, which include neuropeptide Y (NPY), cocaineand amphetamine-regulated transcript (CART) peptide, α-melanocyte stimulating hormone (α-MSH), corticotrophin-releasing hormone, thyrotrophin-releasing hormone, glucagon-like peptide 1, pituitary adenylate cyclase activating polypeptide, calcitonin gene-related peptide, vasopressin, somatostatin, and substance P. However, neuropeptides such as galanin, cortistatin, β-endorphin, and met-enkephalin have been reported to impair cognitive process. In addition, peptidergic systems form neuronal networks with each other. For example, melaninconcentrating hormone and NPY cells in the hypothalamus cocontained CART. Researchers suggest that such interactions may integrate and process cognitionrelated signals. Noteworthily, neuropeptides coexist and interact with classical neurotransmitters systems such as ACh, glutamate, DA, serotonin, and GABA to regulate cognitive functions. For example, NPY was present in the vicinity of cholinergic neurons at several brain sites. In nucleus accumbens shell, dopaminergic nerve terminals were seen contacting CART neurons. Few studies suggested the clinical importance of neuropeptides. Decreased NPY, CART, and α-MSH levels were observed in the CSF of patients of AD, suggesting the significance of these peptides in the pathogenesis of dementias. Moreover, thyrotrophin-releasing hormone attenuated the memory deficit in normal subjects and Alzheimer’s patients treated with an anticholinergic drug. Naloxone, an antagonist of endogenous opioids, improved cognitive performance in patients with AD. Intranasal administration of vasopressin improves the performance on memory tests in normal subjects. Similarly, α-MSH improves performance of mentally retarded subjects [115 119]. Roles of CART, NPY, and α-MSH have been extensively studied in the cognitive regulation and are discussed later.

2.4.2.1 Cocaine- and amphetamine-regulated transcript Neuropeptide CART is widely distributed in the cognition processing regions such as hippocampus and parahippocampal, including entorhinal cortex [120 122]. Yermolaieva et al. [123] reported L-type voltage-dependent intracellular Ca21 signaling in rat hippocampal neurons as a target for CART and first time proposed that CART in the hippocampus may play a role in learning and memory physiology. The intracranial self-stimulation mediated facilitation of

29

30

CHAPTER 2 Functional neuroanatomy and disorders of cognition

consolidation of implicit and explicit memories in rats is attributed to the expression of several hippocampal genes including that of CART [124]. A significant reduction in the level of CART in CSF was observed in dementia patients with Lewy bodies [125]. In the last decade, effect of exogenous treatment of CART on memory functions and underlying mechanism is studied in animals. Intracranial administration of CART seems to promote spatial learning and memory and reverse scopolamine-induced memory deficits [126]. In these rats, navigational experiences in MWM were found to upregulate the endogenous CART systems in the discrete regional of brain. CART infusion into hippocampus also improved the recognition memory in rats, and this may mediated by NMDA receptors-extracellular-signal-regulated kinase signaling in the ento- and perirhinal hippocampal circuit [127]. We also recently reported that CART is involved in schizophrenia-induced cognitive deficits via dopaminergic alterations and may contribute in preventing the memory loss attributed to the withdrawal of nicotine following chronic treatment in mice [128,129]. Recently, epigenetic modulation of CART gene is also reported in brain trauma mediated deficits in learning and memorizing [130]. They found persistent decrease in the levels of acetylated histone H3-Lys 9 (H3-K9ac) in CART gene with concurrent decline in CART messenger RNA and peptide levels. CART is also found to be effective in reversing symptoms in AD mice. Studies showed that CART decreases levels of soluble Aβ1 40 and Aβ1 42 in the hippocampus of amyloid precursor protein (APP)/PS1 mice and attenuates spatial memory deficits that may be associated with the MAPK and Akt pathways [131,132]. In Aβ1 42-induced AD model, intrahippocampal CART administration improved the spatial memory and locomotor ability [133]. In addition, CART decreased the Aβ1 42 and Aβ production associated enzyme β-secretase 1 levels, attenuated the oxidative stress damage with a concrete manifestation of increased malondialdehyde, as well as decreased glutathione, total superoxide dismutase, and ATP levels in the hippocampus which may be causatively implicated by activating the Nrf2/HO-1 signaling pathway.

2.4.2.2 Neuropeptide Y NPY is a 36-amino acid peptide with C-terminally amidated tyrosine (Y) [134]. NPY exerts its biological effects endorsed to the activation of several known and cloned pertussis-sensitive rhodopsin like receptors (Y1, Y2, Y4, Y5, and Y6) [134]. Wealth of studies has provided the evidence for NPY acting as a modulator of learning, memory, and neuroplasticity. Intracerebroventricular pretreatment with NPY or the Y1 receptor-preferring agonist [Leu31, Pro34]-NPY impaired acquisition of fear memory conditioning. In contrast, the Y2 receptor selective agonist NPY13-36 had no effect [135,136]. This is supported by studies in transgenic animals. While NPY or Y1 receptor knock-out mice exhibited enhanced acquisition of cued fear conditioning, Y2 receptor deficiency did not [137]. In passive avoidance paradigm, exogenous NPY infusion after reference memory training enhances memory consolidation, retention, and retrieval. In contrast,

2.4 Neurotransmitters involved in cognition

no effects on the acquisition after pretraining application of NPY were observed [138,139]. Effect of NPY on spatial memory was studied in MWM and transgenic rats overexpressing NPY in hippocampal CA1. This exhibited impaired acquisition and retention of spatial memories. It was speculated to be mediated via presynaptic Y2 receptor-mediated inhibition of glutamatergic transmission [140]. NPY overexpression delayed acquisition and impaired retention in the twoplatform discrimination MWM test. This was also correlated with the altered hippocampal short-term synaptic plasticity and partially impaired LTP induction [141]. NPY or NPY Y1 receptor agonist [Leu31, Pro34]-NPY decreased escape latency in MWM and also improved memory deficits in AD rats [142]. In a NORT, Y2 receptor knockout mice, the recognition ability was deteriorated at 6 hours intertrial interval but not at 1 hour, suggesting impaired memory retention but not normal acquisition [143]. Thus collectively it is suggested that NPY exert both inhibitory and stimulatory effects on memory, depending on memory type and phase, dose applied, brain region, and NPY receptor subtypes [144].

2.4.2.3 α-Melanocyte stimulating hormone α-MSH and MSH/adrenocorticotropic hormone analogs are mostly involved in the regulation of food intake, and energy metabolism also plays a role in the facilitation of learning and memory [145]. Both peptide and its receptors were detected in rodent hippocampus assuming the role for α-MSH in hippocampusdependent learning and memory [146]. This notion is further supported by Waltereit and Weller [147] who report that stimulation of melanocortin-4 receptor (MC4R) leads to the activation of Gs-cAMP-protein kinase A, a signaling cascade that is critical for specific forms of synaptic plasticity. Synaptic transmission and plasticity is also noted following the activation of postsynaptic MC4R in the hippocampus, may be via enhancement of AMPA receptor-mediated neurotransmission and LTP at the Schaffer collateral-CA1 pathway [148]. α-MSH is reduced in the brain, and CSF of AD patients and α-MSH autoantibody levels correlate with cognitive dysfunction [149,150]. A potent synthetic analog of α-MSH (DPα-MSH) also improved spatial memory in a 3 3 Tg (APPSwe/PS1M146V/ TauP301L) mouse model of AD [151]. α-MSH has emerged as a key regulator of excitatory inhibitory balance, perhaps via maintaining GABAergic inhibition, to improve the cognitive functions [152]. More recently, α-MSH was found to reverse the IL-1β induced reduction in GluA1 phosphorylation and its expression during memory reconsolidation [153]. Therefore studies project the α-MSH as a potential treatment option for memory deficit-like conditions.

2.4.3 Neurosteroids Progesterone metabolite allopregnanolone (ALLO) is reported to produce the cognitive deficits in rodents. It impairs spatial learning and memory performance in the MWM. Since ALLO is a positive allosteric modulator of GABA-A receptor, the effects may be mediated via these receptors present in the hippocampus. It may

31

32

CHAPTER 2 Functional neuroanatomy and disorders of cognition

be noted that benzodiazepines (GABA-A receptor agonists) impair memory [154]. However, dehydroepiandrosterone (DHEA) and its sulfate derivative DHEA-S (an allosteric antagonist of the GABA-A receptor) improves cognitive functions. DHEA also produced neuroprotective actions by promoting the expression of brain-derived neurotrophic factor and levels of ACh and catecholamines [155,156]. ALLO also impairs the encoding and consolidation of contextual fear memory and object memory by disrupting dorsal hippocampal function [157]. Moreover, continuous ALLO treatment for a period of 5 months with osmotic pumps causes memory decline and hippocampus shrinkage in mice. Chronic stress impairs the memory and increases ALLO levels in the brain. Therefore it is possible that ALLO involved in the pathogenesis of cognitive disturbances induced by stress [158]. Interestingly, intermittent administration of ALLO was found to be beneficial in AD-like condition. Once per week of ALLO in 3 3 TgAD mouse model of AD reversed learning and memory impairment, significantly promoted neurogenesis in the dentate gyrus, reduced Aβ, and microglial activation [159,160].

2.5 Cognition-related diseases Cognitive disorders primarily affect attention, learning, reasoning, planning, judgment, memory, and other thought processes. They are broadly categorized as dementia (loss of cognitive and mental abilities), amnesia (simple memory decline accompanied by no other cognitive impairments), and delirium (confusion and attention deficit). Dementias may be of metabolic (e.g., hyperthyroidism, hypothyroidism, Cushing’s syndrome, Addison’s disease, and diabetes), infectious (e.g., Treponema pallidum, Borrelia, human immunodeficiency virus, herpes virus, cytomegalovirus, Toxoplasma gondii, Cryptococcus, and Taenia solium), autoimmune (e.g., autoimmune encephalitis), vascular (e.g., hypertension and stroke), and neurodegenerative (e.g., AD, PD, and HD) origins. Although dementia is partially manageable, it is usually irreversible and over time gets worse. Dementia associated with AD and PD are most common.

2.5.1 Alzheimer’s disease AD is a progressive neurodegenerative disease characterized by the loss of mental ability and cognitive functions. It is the most common cause of dementia (60% 70% of cases), with worldwide prevalence of B30 million, making it one of the most important public health issues. It is classified into two types: early onset familial AD (FAD) and late-onset sporadic AD (SAD). Most of the AD cases ( . 99%) are SAD, which involves complex etio-pathophysiological mechanisms including genetic, metabolic, and environmental factors. The symptoms of SAD began after 65 years with progressive increase in the severity with age. FAD occurs within families in less than 1% cases and is inherited in an autosomal dominant fashion. The signs of FAD are

2.5 Cognition-related diseases

usually evident between the ages of 30 60 years. Hippocampal and entorhinal cortex atrophy is the most predominant structural imaging finding in AD along with dysfunction of the cholinergic system [161,162]. It is widely accepted that pathogenesis of AD is driven by the extraneuronal plaque deposition of the Aβ peptide and intraneuronal accumulation of neurofibrillary tangles (NFTs). These effects result in synaptic and neuronal loss, as well as learning-memory deficit.

2.5.1.1 Extraneuronal plaque deposition of β-amyloid Hydrolysis of an integral neuronal membrane APP occurs via two different pathways: nonamyloidogenic and amyloidogenic (Fig. 2.1). In the nonamyloidogenic pathway, cleavage of APP by α-secretase forms soluble APPα and C83. Subsequent cleavage of C83 by γ-secretase forms APP intracellular domain (APPICD) and a short fragment p3. In the amyloidogenic pathway the cleavage of APP by β-secretase forms a soluble APPβ and correspondingly longer C99. Subsequent cleavage of C99 by γ-secretase form APPICD and Aβ (40 42 amino acids) [163,164]. Aβ-degrading enzymes such as endothelin-converting enzyme, neprilysin, and insulin-degrading enzyme reduce the levels of Aβ [165]. Moreover, receptors for Aβ are present on microglia, which engulf and destroy Aβ. However, with time, either because of overwhelmingly increased Aβ formation as a result of mutation of APP gene [166] or due to declined functionality of microglia, Aβ slowly starts to

FIGURE 2.1 Processing of APP by nonamyloidogenic and amyloidogenic pathways. APP, Amyloid precursor protein; APPICD, amyloid precursor protein intracellular domain; Aβ, β-amyloid; sAPPβ, soluble amyloid precursor protein.

33

34

CHAPTER 2 Functional neuroanatomy and disorders of cognition

accumulate as plaques [167]. Cofactors such as copper, iron, and zinc bind Aβ peptide induce aggregation to form Aβ plaque. In fact, high contents of these cofactors have been observed in Aβ in AD brains [168]. Although the mechanism is unresolved, accumulation of Aβ is considered as a major factor contributing to synaptic dysfunction and loss in AD. In addition, increased levels of Aβ induce the formation of free radicals and proinflammatory cytokines, which lead to neuroinflammation and contribute to the destruction of neurons [167].

2.5.1.2 Intraneuronal accumulation of neurofibrillary tangles Extraneuronal Aβ triggers formation of NFTs in the neuron (Fig. 2.2). Aβ acts as an antagonist of neuronal insulin receptors and causes inhibition of phosphoinositide-3kinase (PI3K) and Akt. Normally, PI3K-Akt regulates the activity of glycogen synthase kinase-3β (GSK3β), an important tau kinase involved in the pathogenesis of AD. Inactivated form of Akt cannot inhibit GSK3β, increasing its activity [169,170]. Overactivation of GSK3β results in the aberrant phosphorylation of

FIGURE 2.2 Mechanism of the formation of NFTs in the neuron. Akt, Protein kinase B; Aβ, β-amyloid; GSK3β, glycogen synthase kinase-3β; NFTs, neurofibrillary tangles; PI3K, phosphoinositide-3-kinase.

2.5 Cognition-related diseases

microtubule-associated tau protein in neurons. Hyperphosphorylated tau proteins aggregate to form NFT, which leads to oxidative stress [171], breakdown of microtubules, and neurodegeneration [161]. It is worth noting a balance between activities of tau protein kinases, and tau protein phosphatases regulates tau phosphorylation [172]. Although tau kinase such as GSK3β has been recognized as an important pathophysiological factor in AD, tau phosphatases that reverse the actions of tau kinases by dephosphorylating hyperphosphorylated tau are also equally important. In fact, reduced activity of protein phosphatase 2 A appears to be a major factor in tau hyperphosphorylation and pathology of NFTs [173].

2.5.2 Lewy body diseases After AD, Lewy body diseases (LBDs) are the second most common cause of dementia. LBDs include dementia with Lewy bodies (DLB) and PD, which share many neurochemical, morphological, and clinical features. LBDs are characterized by the progressive accumulation of α-synuclein (αS) oligomers and toxic fibrils in Lewy bodies that trigger the degeneration of nigral striatal, neocortical, and limbic circuitries (Fig. 2.3). In fact, intracellular inclusions of cytotoxic Lewy bodies are

FIGURE 2.3 Mechanism of the formation of Lewy bodies in the neuron. DLB, Dementia with Lewy bodies; PD, Parkinson’s disease; αS, α-synuclein.

35

36

CHAPTER 2 Functional neuroanatomy and disorders of cognition

the pathologic hallmark of LBDs. Other proteins such as ubiquitin, neurofilament, and αB crystalline are also found in Lewy bodies [174]. αS exists as at least two structural isoforms: a disordered, cytosolic form and a helix-rich, membrane-bound form. In presynaptic terminals, αS is thought to be involved in clustering, trafficking, and refilling of synaptic vesicles to aid neurotransmitters release. Native αS exists as a monomer (predominant) and oligomer. Factors such as oxidative stress, defective posttranslational modifications, and high concentration of fatty acids can promote oligomerization of αS. Some familial forms of PD are found to be associated with the mutation of SNCA gene (encodes αS), which results in increased misfolded αS expression promoting oligomerization and accumulation. In sporadic forms of LBDs, failure of autophagy pathways that eliminate αS oligomers might enhance its toxicity. Interestingly, αS oligomers and fibrils get transferred between cells and spread disease to other brain regions by multiple mechanisms such as endocytosis, penetration, transynaptic transmission, or membrane receptors. In PD, Lewy bodies interact with DNA to cause nuclear degradation, while in DLB, Lewy bodies interact with mitochondria to induce the mitochondrial dysfunctioning [175 177].

2.6 Conclusion AD and LBD are the most common neurodegenerative disorders associated with cognitive dysfunctioning, and great clinical problems in aging population. Various neurotransmitters in the brain regions such as hippocampus, amygdala, and basal forebrain are involved in the processing of learning and memory trace formation. The presently available drugs target their pathways that control cognitive functions. However, the therapeutic potential is much limited. Therefore there is an emerging need to find out novel targets for the development of future drugs by elucidating the underlying complex pathophysiological mechanisms and identifying the key mediators. Even the development of drugs that symptomatically improve the cognitive deficits or slow down the progression of neurodegeneration would be breakthroughs in the field of AD and LBD. Extensive investigations using animal models of AD and LBD-like conditions would help to understand complex mechanisms and therapeutic potential of novel agents. Selective synthetic agonists and antagonists with excellent CNS penetration can be developed in this regard. It is possible to develop a strategy that would involve β-secretase inhibitors, positive regulators of tau protein phosphatases and Aβdegrading enzymes, and inhibitors of tau kinases and cofactors involved in the formation of Aβ plaque as a complementary approach for the management of AD.

2.7 Acknowledgment Author (KN) gratefully acknowledges the financial support received from Science and Engineering Research Board (SERB), Department of Science and Technology (DST),

References

Government of India, New Delhi under the Start-Up Research Grant (Young Scientists) scheme (File No. SB/YS/LS-106/2014).

References [1] M.A. Brandimonte, N. Bruno, S. Collina, Cognition, in: P. Pawlik, G. d’Ydewalle (Eds.), Psychological Concepts: An International Historical Perspective., Psychology Press, Hove, UK, 2006, pp. 1 22. [2] S. Deiana, B. Platt, G. Riedel, The cholinergic system and spatial learning, Behav. Brain Res. 221 (2) (2011) 389 411. [3] Y. Dudai, Molecular bases of long-term memories: a question of persistence, Curr. Opin. Neurobiol. 12 (2) (2002) 211 216. [4] M.M. Poo, M. Pignatelli, T.J. Ryan, S. Tonegawa, T. Bonhoeffer, K.C. Martin, et al., What is memory? The present state of the engram, BMC Biol. 14 (2016) 40. [5] R.F. Thompson, J.J. Kim, Memory systems in the brain and localization of a memory, Proc. Natl. Acad. Sci. U.S.A. 93 (24) (1996) 13438 13444. [6] R. D’Hooge, P.P. De Deyn, Applications of the Morris water maze in the study of learning and memory, Brain Res. Rev. 36 (1) (2001) 60 90. [7] H. Eichenbaum, A.P. Yonelinas, C. Ranganath, The medial temporal lobe and recognition memory, Annu. Rev. Neurosci. 30 (2007) 123 152. [8] A. Kafkas, D. Montaldi, How do memory systems detect and respond to novelty? Neurosci. Lett. 680 (2018) 60 68. [9] R.K. Olsen, S.N. Moses, L. Riggs, J.D. Ryan, The hippocampus supports multiple cognitive processes through relational binding and comparison, Front. Hum. Neurosci. 6 (2012) 146. [10] L.R. Squire, S. Zola-Morgan, The medial temporal lobe memory system, Science 253 (5026) (1991) 1380 1386. [11] M.G. Packard, L. Cahill, J.L. McGaugh, Amygdala modulation of hippocampaldependent and caudate nucleus-dependent memory processes, Proc. Natl. Acad. Sci. U.S.A. 91 (18) (1994) 8477 8481. [12] M. Davis, D. Rainnie, M. Cassell, Neurotransmission in the rat amygdala related to fear and anxiety, Trends Neurosci. 17 (5) (1994) 208 214. [13] S. Killcross, T.W. Robbins, B.J. Everitt, Different types of fear-conditioned behaviour mediated by separate nuclei within amygdala, Nature 388 (6640) (1997) 377 380. [14] J.J. Gold, L.R. Squire, The anatomy of amnesia: neurohistological analysis of three new cases, Learn. Mem. 13 (6) (2006) 699 710. [15] Y.D. Van der Werf, M.P. Witter, H.J. Groenewegen, The intralaminar and midline nuclei of the thalamus. Anatomical and functional evidence for participation in processes of arousal and awareness, Brain Res. Rev. 39 (2 3) (2002) 107 140. [16] E. Carrera, P. Michel, J. Bogousslavsky, Anteromedian, central, and posterolateral infarcts of the thalamus: three variant types, Stroke 35 (12) (2004) 2826 2831. [17] Y.D. Van der Werf, J. Jolles, M.P. Witter, H.B. Uylings, Contributions of thalamic nuclei to declarative memory functioning, Cortex 39 (4 5) (2003) 1047 1062. [18] J. Kievit, H.G. Kuypers, Organization of the thalamo-cortical connexions to the frontal lobe in the rhesus monkey, Exp. Brain Res. 29 (3 4) (1977) 299 322.

37

38

CHAPTER 2 Functional neuroanatomy and disorders of cognition

[19] S.T. Carmichael, J.L. Price, Sensory and premotor connections of the orbital and medial prefrontal cortex of macaque monkeys, J. Comp. Neurol. 363 (4) (1995) 642 664. [20] R.C. Saunders, M. Mishkin, J.P. Aggleton, Projections from the entorhinal cortex, perirhinal cortex, presubiculum, and parasubiculum to the medial thalamus in macaque monkeys: identifying different pathways using disconnection techniques, Exp. Brain Res. 167 (1) (2005) 1 16. [21] J.P. Aggleton, R.C. Saunders, The relationships between temporal lobe and diencephalic structures implicated in anterograde amnesia, Memory 5 (1 2) (1997) 49 71. [22] J.P. Aggleton, Multiple anatomical systems embedded within the primate medial temporal lobe: implications for hippocampal function, Neurosci. Biobehav. Rev. 36 (7) (2012) 1579 1596. [23] T.C. Napier, A.L. Mickiewicz, The role of the ventral pallidum in psychiatric disorders, Neuropsychopharmacology 35 (1) (2010) 337. [24] K.S. Smith, A.J. Tindell, J.W. Aldridge, K.C. Berridge, Ventral pallidum roles in reward and motivation, Behav. Brain Res. 196 (2) (2009) 155 167. [25] J.L. Evenden, H.M. Marston, G.H. Jones, V. Giardini, L. Lenard, B.J. Everitt, et al., Effects of excitotoxic lesions of the substantia innominata, ventral and dorsal globus pallidus on visual discrimination acquisition, performance and reversal in the rat, Behav. Brain Res. 32 (2) (1989) 129 149. [26] M.M. Mesulam, E.J. Mufson, A.I. Levey, B.H. Wainer, Cholinergic innervation of cortex by the basal forebrain: cytochemistry and cortical connections of the septal area, diagonal band nuclei, nucleus basalis (substantia innominata), and hypothalamus in the rhesus monkey, J. Comp. Neurol. 214 (2) (1983) 170 197. [27] L. Za´borszky, W.E. Cullinan, A. Braun, Afferents to basal forebrain cholinergic projection neurons: an update, Adv. Exp. Med. Biol. 295 (1991) 43 100. [28] M.G. Blake, M.M. Boccia, Basal forebrain cholinergic system and memory, Curr. Top. Behav. Neurosci. 37 (2018) 253 273. [29] E.M. Torres, T.A. Perry, A. Blockland, L.S. Wilkinson, R.G. Wiley, D.A. Lappi, et al., Behavioural, histochemical and biochemical consequences of selective immunolesions in discrete regions of the basal forebrain cholinergic system, Neuroscience 63 (1) (1994) 95 122. [30] G. Leanza, O.G. Nilsson, R.G. Wiley, A. Bjo¨rklund, Selective lesioning of the basal forebrain cholinergic system by intraventricular 192 IgG-saporin: behavioural, biochemical and stereological studies in the rat, Eur. J. Neurosci. 7 (2) (1995) 329 343. [31] L. Zaborszky, K. Pang, J. Somogyi, Z. Nadasdy, I. Kallo, The basal forebrain corticopetal system revisited, Ann. N. Y. Acad Sci. 877 (1999) 339 367. [32] M. Mesulam, The cholinergic lesion of Alzheimer’s disease: pivotal factor or side show? Learn. Mem. 11 (1) (2004) 43 49. [33] K. Belarbi, K. Schindowski, S. Burnouf, R. Caillierez, M.E. Grosjean, D. Demeyer, et al., Early Tau pathology involving the septo-hippocampal pathway in a Tau transgenic model: relevance to Alzheimer’s disease, Curr. Alzheimer. Res. 6 (2) (2009) 152 157. [34] K. Manabe, R.J. Dooling, E.F. Brittan-Powell, Vocal learning in Budgerigars (Melopsittacus undulatus): effects of an acoustic reference on vocal matching, J. Acoust. Soc. Am. 123 (3) (2008) 1729 1736.

References

[35] F. Benfenati, Synaptic plasticity and the neurobiology of learning and memory, Acta Biomed. 78 (1) (2007) 58 66. [36] K. Dharani, The Biology of Thought: A Neuronal Mechanism in the Generation of Thought A New Molecular Model., Academic Press, 2015, pp. 53 74. [37] Y. Dudai, A. Karni, J. Born, The consolidation and transformation of memory, Neuron 88 (1) (2015) 20 32. [38] S. Tonegawa, M. Pignatelli, D.S. Roy, T.J. Ryan, Memory engram storage and retrieval, Curr. Opin. Neurobiol. 35 (2015) 101 109. [39] J. O’Keefe, Place units in the hippocampus of the freely moving rat, Exp. Neurol. 51 (1) (1976) 78 109. [40] M.B. Moser, D.C. Rowland, E.I. Moser, Place cells, grid cells, and memory, Cold Spring Harb. Perspect. Biol. 7 (2) (2015) a021808. [41] R.M. Bruno, D.J. Simons, Feed forward mechanisms of excitatory and inhibitory cortical receptive fields, J. Neurosci. 22 (2002) 10966 10975. [42] A. Evstratova, K. To´th, Information processing and synaptic plasticity at hippocampal mossy fiber terminals, Front. Cell Neurosci. 8 (2014) 28. [43] S.S. Zakharenko, L. Zablow, S.A. Siegelbaum, Visualization of changes in presynaptic function during long-term synaptic plasticity, Nat. Neurosci. 4 (7) (2001) 711 717. [44] N.J. Woolf, Cholinergic systems in mammalian brain and spinal cord, Prog. Neurobiol. 37 (1991) 475 524. [45] P. Davies, A critical review of the role of the cholinergic system in human memory and cognition, Ann. N. Y. Acad. Sci. 444 (1985) 212 217. [46] M.E. Hasselmo, The role of acetylcholine in learning and memory, Curr. Opin. Neurobiol. 16 (6) (2006) 710 715. [47] F. Khakpai, M. Nasehi, A. Haeri-Rohani, A. Eidi, M.R. Zarrindast, Septo-hippocamposeptal loop and memory formation, Basic Clin. Neurosci. 4 (1) (2013) 5 23. [48] J.C. Pych, Q. Chang, C. Colon-Rivera, R. Haag, P.E. Gold, Acetylcholine release in the hippocampus and striatum during place and response training, Learn. Mem. 12 (2005) 564 572. [49] R.T. Bartus, On neurodegenerative diseases, models, and treatment strategies: lessons learned and lessons forgotten a generation following the cholinergic hypothesis, Exp. Neurol. 163 (2) (2000) 495 529. [50] J. Haam, J.L. Yakel, Cholinergic modulation of the hippocampal region and memory function, J. Neurochem. 142 (2) (2017) 111 121. [51] R. Schliebs, T. Arendt, The significance of the cholinergic system in the brain during aging and in Alzheimer’s disease, J. Neural Transm. 113 (2006) 1625 1644. [52] P.J. Whitehouse, D.L. Price, A.W. Clark, J.T. Coyle, M.R. DeLong, Alzheimer disease: evidence for selective loss of cholinergic neurons in the nucleus basalis, Ann. Neurol. 10 (2) (1981) 122 126. [53] R.T. Bartus, R.L. Dean, B. Beer, A.S. Lippa, The cholinergic hypothesis of geriatric memory dysfunction, Science 217 (1982) 408 414. [54] K.F. Bell, A. Ducatenzeiler, A. Ribeiro-da-Silva, K. Duff, D.A. Bennett, A.C. Cuello, The amyloid pathology progresses in a neurotransmitter-specific manner, Neurobiol. Aging 27 (11) (2006) 1644 1657. [55] D.J. Selkoe, Alzheimer’s disease is a synaptic failure, Science 298 (5594) (2002) 789 791.

39

40

CHAPTER 2 Functional neuroanatomy and disorders of cognition

[56] M.J. Ehret, K.W. Chamberlin, Current practices in the treatment of Alzheimer disease: where is the evidence after the phase III trials? Clin. Ther. 37 (8) (2015) 1604 1616. [57] J.Z. Tsien, Basic Neurochemistry, Elsevier, 2012, pp. 963 981. [58] B. Hassel, R. Dingledine, in: S.T. Brady, G.J. Siegel (Eds.), Basic Neurochemistry, Elsevier, 2012, pp. 342 366. [59] T.W. Robbins, E.R. Murphy, Behavioural pharmacology: 40 1 years of progress, with a focus on glutamate receptors and cognition, Trends Pharmacol. Sci. 27 (2006) 141 148. [60] R.G. Morris, NMDA receptors and memory encoding, Neuropharmacology 74 (2013) 32 40. [61] L. Rajagopal, J.S. Burgdorf, J.R. Moskal, H.Y. Meltzer, GLYX-13 (rapastinel) ameliorates subchronic phencyclidine- and ketamine-induced declarative memory deficits in mice, Behav. Brain Res. 299 (2016) 105 110. [62] B.D. Winters, T.J. Bussey, Glutamate receptors in perirhinal cortex mediate encoding, retrieval, and consolidation of object recognition memory, J. Neurosci. 25 (2005) 4243 4251. [63] G. Kumar, J. Olley, T. Steckler, J. Talpos, Dissociable effects of NR2A and NR2B NMDA receptor antagonism on cognitive flexibility but not pattern separation, Psychopharmacology (Berl.) 232 (21 22) (2015) 3991 4003. [64] T. Bast, B.M. da Silva, R.G. Morris, Distinct contributions of hippocampal NMDA and AMPA receptors to encoding and retrieval of one-trial place memory, J. Neurosci. 25 (25) (2005) 5845 5856. [65] K. Hayashi, T. Yoshihara, Y. Ichitani, Involvement of hippocampal metabotropic glutamate receptors in radial maze performance, NeuroReport 18 (7) (2007) 719 723. [66] Z. Hlina´k, I. Krejc´ı, N-Methyl-D-aspartate improved social recognition potency in rats, Neurosci. Lett. 330 (3) (2002) 227 230. [67] Z. Hlina´k, I. Krejc´ı, N-Methyl-D-aspartate prevented memory deficits induced by MK-801 in mice, Physiol. Res. 52 (2003) 809 812. [68] Y.P. Tang, H. Wang, R. Feng, M. Kyin, J.Z. Tsien, Differential effects of enrichment on learning and memory function in NR2B transgenic mice, Neuropharmacology 41 (2001) 779 790. [69] J.W. Newcomer, N.B. Farber, V. Jevtovic-Todorovic, G. Selke, A.K. Melson, T. Hershey, et al., Ketamine-induced NMDA receptor hypofunction as a model of memory impairment and psychosis, Neuropsychopharmacology 20 (1999) 106 118. [70] A. Parwani, M.A. Weiler, T.A. Blaxton, D. Warfel, M. Hardin, K. Frey, et al., The effects of a subanesthetic dose of ketamine on verbal memory in normal volunteers, Psychopharmacology (Berl.) 183 (2005) 265 274. [71] J.H. Krystal, D.C. D’Souza, L.P. Karper, A. Bennett, A. Abi-Dargham, D. Abi-Saab, et al., Interactive effects of subanesthetic ketamine and haloperidol in healthy humans, Psychopharmacology (Berl.) 145 (1999) 193 204. [72] M.R. Dauvermann, G. Lee, N. Dawson, Glutamatergic regulation of cognition and functional brain connectivity: insights from pharmacological, genetic and translational schizophrenia research, Br. J. Pharmacol. 174 (19) (2017) 3136 3160. [73] X.R. Cheng, Y. Yang, W.X. Zhou, Y.X. Zhang, Expression of VGLUTs contributes to degeneration and acquisition of learning and memory, Neurobiol. Learn. Mem. 95 (3) (2011) 361 375.

References

[74] Z. Gu, J. Cheng, P. Zhong, L. Qin, W. Liu, Z. Yan, Aβ selectively impairs mGluR7 modulation of NMDA signaling in basal forebrain cholinergic neurons: implication in Alzheimer’s disease, J. Neurosci. 34 (41) (2014) 13614 13628. [75] U. Rudolph, H. Mhler, Analysis of GABAa receptor function and dissection of the pharmacology of benzodiazepines and general anesthetics through mouse genetics, Annu. Rev. Pharmacol. Toxicol. l44 (2004) 475 498. [76] E.R. Korpi, G. Grunder, H. Luddens, Drug interactions at GABA(A) receptors, Prog. Neurobiol. 67 (2002) 113 159. [77] C. Castellano, J.L. McGaugh, Effects of post-training bicuculline and muscimol on retention: lack of state dependency, Behav. Neural Biol. 54 (2) (1990) 156 164. [78] M.E. Cha´vez, R. Salado-Castillo, M. Sa´nchez-Alavez, G.L. Quirarte, R.A. PradoAlcala´, Post-training injection of GABAergic antagonists into the striatum produces retrograde amnesia, Neurobiol. Learn. Mem. 63 (3) (1995) 296 300. [79] G. Chapouthier, P. Venault, GABA-A receptor complex and memory processes, Curr. Top. Med. Chem. 2 (8) (2002) 841 851. [80] M.B. Moser, E.I. Moser, Functional differentiation in the hippocampus, Hippocampus 8 (6) (1998) 608 619. [81] E.A. Calcagnotto, Interneurons: role in maintaining and restoring synaptic plasticity, Front. Psychiatry 7 (2016) 85. [82] K. Govindpani, B. Calvo-Flores Guzma´n, C. Vinnakota, H.J. Waldvogel, R.L. Faull, A. Kwakowsky, Towards a better understanding of GABAergic remodeling in Alzheimer’s disease, Int. J. Mol. Sci. 18 (8) (2017) 1813. [83] T. Brozoski, R.M. Brown, H.E. Rosvold, P.S. Goldman, Cognitive deficit caused by regional depletion of dopamine in prefrontal cortex of rhesus monkey, Science 205 (1979) 929 931. [84] E. Edelmann, V. Lessmann, Dopaminergic innervation and modulation of hippocampal networks, Cell Tissue Res. 373 (3) (2018) 711 727. [85] A. Wilkerson, E.D. Levin, Ventral hippocampal dopamine D1and D2 systems and spatial working memory in rats, Neuroscience 89 (1999) 743 749. [86] J. Lisman, A. Grace, The hippocampal-VTA loop: controlling the entry of information into long-term memory, Neuron 46 (2005) 703 713. [87] D. Durstewitz, M. Kelc, O. Gu¨ntu¨rku¨n, A neurocomputational theory of the dopaminergic modulation of working memory functions, J. Neurosci. 19 (7) (1999) 2807 2822. [88] J.A. Hosp, A.R. Luft, Dopaminergic meso-cortical projections tom1: role in motor learning and motor cortex plasticity, Front. Neurol. 4 (2013) 145. [89] T. Sawaguchi, P.S. Goldman-Rakic, D1 dopamine receptors in prefrontal cortex: involvement in working memory, Science 251 (1991) 947 950. [90] J.F. Seamans, S.B. Floresco, A.G. Phillips, Selective impairment on a delayed radial arm task following local administration of a selective D1, but not a D2, antagonist into the prefrontal cortex, Soc. Neurosci. Abstr. 21 (1995) 1942. [91] M. O’Neill, V.J. Brown, The effect of striatal dopamine depletion and the adenosine A2A antagonist KW-6002 on reversal learning in rats, Neurobiol. Learn. Mem. 88 (2007) 75 81. [92] Y. Charnay, L. Le´ger, Brain serotonergic circuitries, Dialogues Clin. Neurosci. 12 (2010) 471 487. [93] R.D. Rubin, P.D. Watson, M.C. Du, N.J. Cohen, The role of the hippocampus in flexible cognition and social behavior, Front. Hum. Neurosci. 8 (2014) 742.

41

42

CHAPTER 2 Functional neuroanatomy and disorders of cognition

[94] G. Buzsa´ki, E.I. Moser, Memory, navigation and theta rhythm in the hippocampalentorhinal system, Nat. Neurosci. 16 (2013) 130 138. [95] T.W. Robbins, From arousal to cognition: the integrative position of the prefrontal cortex, Prog. Brain Res. 126 (2000) 469 483. [96] M. Seyedabadi, G. Fakhfouri, V. Ramezani, S.E. Mehr, R. Rahimian, The role of serotonin in memory: interactions with neurotransmitters and downstream signaling, Exp. Brain Res. 232 (2014) 723 738. [97] J.J. Rodriguez, H.N. Noristani, A. Verkhratsky, The serotonergic system in ageing and Alzheimer’s disease, Prog. Neurobiol. 99 (2012) 15 41. [98] A.V. Terry Jr, J.J. Buccafusco, C. Wilson, Cognitive dysfunction in neuropsychiatric disorders: selected serotonin receptor subtypes as therapeutic targets, Behav. Brain Res. 195 (2008) 30 38. [99] J.R. Cirrito, B.M. Disabato, J.L. Restivo, D.K. Verges, W.D. Goebel, A. Sathyan, et al., Serotonin signaling is associated with lower amyloid-ß levels and plaques in transgenic mice and humans, Proc. Natl. Acad. Sci. U.S.A. 108 (36) (2011) 14968 14973. [100] M. Garcia-Alloza, F.J. Gil-Bea, M. Diez-Ariza, C.P. Chen, P.T. Francis, B. Lasheras, Cholinergic serotonergic imbalance contributes to cognitive and behavioural symptoms in Alzheimer’s disease, Neuropsychologia 43 (2005) 442 449. [101] H. Tohgi, T. Abe, S. Takahashi, M. Kimura, J. Takahashi, T. Kikuchi, Concentrations of serotonin and its related substances in the cerebrospinal fluid in patients with Alzheimer type dementia, Neurosci. Lett. 141 (1992) 9 12. ˇ [102] D. Mu¨ck-Seler, P. Presecki, N. Mimica, M. Mustapic, N. Pivac, A. Babic, et al., Platelet serotonin concentration and monoamine oxidase type B activity in female patients in early, middle and late phase of Alzheimer’s disease, Prog. Neuropsychopharmacol. Biol. Psychiatry 33 (2009) 1226 1231. [103] M. Rushaidhi, Y. Jing, H. Zhang, P. Liu, Participation of hippocampal agmatine in spatial learning: an in vivo microdialysis study, Neuropharmacology 65 (2013) 200 205. [104] P. Liu, N.D. Collie, S. Chary, Y. Jing, H. Zhang, Spatial learning results in elevated agmatine levels in the rat brain, Hippocampus 18 (11) (2008) 1094 1098. [105] B.E. McKay, W.E. Lado, L.J. Martin, M.A. Galic, N.M. Fournier, Learning and memory in agmatine-treated rats, Pharmacol. Biochem. Behav. 72 (3) (2002) 551 557. [106] N.S. Arteni, D. Lavinsky, A.L. Rodrigues, V.B. Frison, C.A. Netto, Agmatine facilitates memory of an inhibitory avoidance task in adult rats, Neurobiol. Learn. Mem. 78 (2) (2002) 465 469. [107] P. Liu, M. Rushaidhi, N.D. Collie, M.T. Leong, H. Zhang, Behavioral effects of intracerebroventricular microinfusion of agmatine in adult rats, Behav. Neurosci. 122 (3) (2008) 557 569. [108] P. Liu, S. Chary, R. Devaraj, Y. Jing, C.L. Darlington, P.F. Smith, et al., Effects of aging on agmatine levels in memory-associated brain structures, Hippocampus 18 (9) (2008) 853 856. [109] M. Rushaidhi, H. Zhang, P. Liu, Effects of prolonged agmatine treatment in aged male Sprague-Dawley rats, Neuroscience 234 (2013) 116 124. [110] P. Liu, Y. Jing, N.D. Collie, S. Chary, H. Zhang, Memory-related changes in L-citrulline and agmatine in the rat brain, Hippocampus 19 (7) (2009) 597 602.

References

[111] M. Moosavi, G.Y. Khales, L. Abbasi, A. Zarifkar, K. Rastegar, Agmatine protects against scopolamine-induced water maze performance impairment and hippocampal ERK and Akt inactivation, Neuropharmacology 62 (5 6) (2012) 2018 2023. [112] P. Bhutada, Y. Mundhada, V. Humane, A. Rahigude, P. Deshmukh, S. Latad, et al., Agmatine, an endogenous ligand of imidazoline receptor protects against memory impairment and biochemical alterations in streptozotocin-induced diabetic rats, Prog. Neuropsychopharmacol. Biol. Psychiatry 37 (2012) 96 105. [113] A. Zarifkar, S. Choopani, R. Ghasemi, N. Naghdi, A.H. Maghsoudi, N. Maghsoudi, et al., Agmatine prevents LPS-induced spatial memory impairment and hippocampal apoptosis, Eur. J. Pharmacol. 634 (1 3) (2010) 84 88. [114] D.H. Bergin, P. Liu, Agmatine protects against beta-amyloid 25 35-induced memory impairments in the rat, Neuroscience 169 (2) (2010) 794 811. [115] F. Agasse, L. Bernardino, H. Kristiansen, S.H. Christiansen, R. Ferreira, B. Silva, et al., Neuropeptide Y promotes neurogenesis in murine subventricular zone, Stem Cells 26 (6) (2008) 1636 1645. [116] G.W. Bennett, T.M. Ballard, C.D. Watson, K.C. Fone, Effect of neuropeptides on cognitive function, Exp. Gerontol. 32 (4 5) (1997) 451 469. [117] M.A. Upadhya, K.T. Nakhate, D.M. Kokare, U. Singh, P.S. Singru, N.K. Subhedar, CART peptide in the nucleus accumbens shell acts downstream to dopamine and mediates the reward and reinforcement actions of morphine, Neuropharmacology 62 (4) (2012) 1823 1833. [118] E. Borbe´ly, B. Scheich, Z. Helyes, Neuropeptides in learning and memory, Neuropeptides 47 (6) (2013) 439 450. [119] P.V. Gonzalez, H.B. Schio¨th, M. Lasaga, T.N. Scimonelli, Memory impairment induced by IL-1beta is reversed by alpha-MSH through central melanocortin-4 receptors, Brain Behav. Immun. 23 (6) (2009) 817 822. [120] P.R. Couceyro, E.O. Koylu, M.J. Kuhar, Further studies on the anatomical distribution of CART by in situ hybridization, J. Chem. Neuroanat. 12 (1997) 229 241. [121] J. Douglass, A.A. McKinzie, P. Couceyro, PCR differential display identifies a rat brain mRNA that is transcriptionally regulated by cocaine and amphetamine, J. Neurosci. 15 (1995) 2471 2481. [122] Y.L. Hurd, P. Fagergren, Human cocaine- and amphetamine regulated transcript (CART) mRNA is highly expressed in limbic- and sensory-related brain regions, J. Comp. Neurol. 425 (2000) 583 598. [123] O. Yermolaieva, J. Chen, P.R. Couceyro, T. Hoshi, Cocaine- and amphetamine regulated transcript peptide modulation of voltage-gated Ca21 signaling in hippocampal neurons, J. Neurosci. 21 (2001) 7474 7480. [124] G. Huguet, L. Aldavert-Vera, E. Ka´da´r, S. Pen˜a de Ortiz, I. Morgado-Bernal, P. Segura-Torres, Intracranial self-stimulation to the lateral hypothalamus, a memory improving treatment, results in hippocampal changes in gene expression, Neuroscience 162 (2) (2009) 359 374. [125] K. Schultz, S. Wiehager, K. Nilsson, J.E. Nielsen, S.G. Lindquist, L.E. Hjermind, et al., Reduced CSF CART in dementia with Lewy bodies, Neurosci. Lett. 453 (2009) 104 106. [126] M.A. Upadhya, K.T. Nakhate, D.M. Kokare, P.S. Singru, N.K. Subhedar, Cocaine and amphetamine-regulated transcript peptide increases spatial learning and memory in rats, Life Sci. 88 (2011) 322 334.

43

44

CHAPTER 2 Functional neuroanatomy and disorders of cognition

[127] A.P. Bharne, C.D. Borkar, S. Bodakuntla, M. Lahiri, N.K. Subhedar, D.M. Kokare, Pro-cognitive action of CART is mediated via ERK in the hippocampus, Hippocampus 26 (10) (2016) 1313 1327. [128] C.D. Borkar, A.P. Bharne, B. Nagalakshmi, A.J. Sakharkar, N.K. Subhedar, D.M. Kokare, Cocaine- and amphetamine-regulated transcript peptide (CART) alleviates MK-801-induced schizophrenic dementia-like symptoms, Neuroscience 375 (2018) 94 107. [129] C.D. Borkar, S. Sagarkar, A.J. Sakharkar, N.K. Subhedar, D.M. Kokare, Neuropeptide CART prevents memory loss attributed to withdrawal of nicotine following chronic treatment in mice, Addict. Biol. 24 (1) (2019) 51 64. [130] S. Sagarkar, N. Balasubramanian, S. Mishra, A.G. Choudhary, D.M. Kokare, A.J. Sakharkar, Repeated mild traumatic brain injury causes persistent changes in histone deacetylase function in hippocampus: implications in learning and memory deficits in rats, Brain Res. 1711 (2019) 183 192. [131] K. Yin, J. Jin, X. Zhu, L. Yu, S. Wang, L. Qian, et al., CART modulates betaamyloid metabolism-associated enzymes and attenuates memory deficits in APP/ PS1 mice, Neurol. Res. 39 (10) (2017) 885 894. [132] J.L. Jin, A.K. Liou, Y. Shi, K.L. Yin, L. Chen, L.L. Li, et al., CART treatment improves memory and synaptic structure in APP/PS1 mice, Sci. Rep. 5 (2015) 10224. [133] W. Jiao, Y. Wang, L. Kong, T. Ou-Yang, Q. Meng, Q. Fu, et al., CART peptide activates the Nrf2/HO-1 antioxidant pathway and protects hippocampal neurons in a rat model of Alzheimer’s disease, Biochem. Biophys. Res. Commun. 501 (4) (2018) 1016 1022. [134] M.M. Berglund, P.A. Hipskind, D.R. Gehlert, Recent developments in our understanding of the physiological role of PP-fold peptide receptor subtypes, Exp. Biol. Med. 228 (2003) 217 244. [135] P. Broqua, J.G. Wettstein, M.N. Rocher, B. Gauthier-Martin, J.L. Junien, Behavioral effects of neuropeptide Y receptor agonists in the elevated plus-maze and fear-potentiated startle procedures, Behav. Pharmacol. 6 (1995) 215 222. [136] R.M. Karlsson, A. Holmes, M. Heilig, J.N. Crawley, Anxiolytic-like actions of centrally administered neuropeptide Y, but not galanin, in C57BL/6J mice, Pharmacol. Biochem. Behav. 80 (2005) 427 436. [137] D. Verma, R.O. Tasan, H. Herzog, G. Sperk, NPY controls fear conditioning and fear extinction by combined action on Y1 and Y2 receptors, Br. J. Pharmacol. 166 (2012) 1461 1473. [138] M. Nakajima, A. Inui, A. Teranishi, M. Miura, Y. Hirosue, M. Okita, et al., Effects of pancreatic polypeptide family peptides on feeding and learning in mice, J. Pharmacol. Exp. Ther. 268 (1994) 1010 1014. [139] H. Ishida, Y. Shirayama, M. Iwata, S. Katayama, A. Yamamoto, R. Kawahara, et al., Infusion of neuropeptide Y into CA3 region of hippocampus produces antidepressant-like effect via Y1 receptor, Hippocampus 17 (2007) 271 280. [140] A. Thorsell, M. Michalkiewicz, Y. Dumont, R. Quirion, L. Caberlotto, R. Rimondini, et al., Behavioral insensitivity to restraint stress, absent fear suppression of behaviour and impaired spatial learning in transgenic rats with hippocampal neuropeptide Y overexpression, Proc. Natl. Acad. Sci. U.S.A. 7 (2000) 12852 12857.

References

[141] A.T. Sørensen, I. Kanter-Schlifke, M. Carli, C. Balducci, F. Noe, M.J. During, et al., NPY gene transfer in the hippocampus attenuates synaptic plasticity and learning, Hippocampus 18 (2008) 564 574. [142] R.J. Rangani, M.A. Upadhya, K.T. Nakhate, D.M. Kokare, N.K. Subhedar, Nicotine evoked improvement in learning and memory is mediated through NPY Y1 receptors in rat model of Alzheimer’s disease, Peptides 33 (2012) 317 328. [143] J.P. Redrobe, Y. Dumont, H. Herzog, R. Quirion, Characterization of neuropeptide Y, Y(2) receptor knockout mice in two animal models of learning and memory processing, J. Mol. Neurosci. 22 (2004) 159 166. [144] C.R. Gøtzsche, D.P. Woldbye, The role of NPY in learning and memory, Neuropeptides 55 (2016) 79 89. [145] R.N. McLay, W. Pan, A.J. Kastin, Effects of peptides on animal and human behavior: a review of studies published in the first twenty years of the journal Peptides, Peptides 22 (2001) 2181 2255. [146] I. Gantz, H. Miwa, Y. Konda, Y. Shimoto, T. Tashiro, S.J. Watson, et al., Molecular cloning, expression, and gene localization of a fourth melanocortin receptor, J. Biol. Chem. 268 (1993) 15174 15179. [147] R. Waltereit, M. Weller, Signaling from cAMP/PKA to MAPK and synaptic plasticity, Mol. Neurobiol. 27 (2003) 99 106. [148] Y. Shen, W.Y. Fu, E.Y. Cheng, A.K. Fu, N.Y. Ip, Melanocortin-4 receptor regulates hippocampal synaptic plasticity through a protein kinase A-dependent mechanism, J. Neurosci. 33 (2) (2013) 464 472. [149] I. Rainero, C. May, J.A. Kaye, R.P. Friedland, S.I. Rapoport, CSF alpha-MSH in dementia of the Alzheimer type, Neurology 38 (1988) 1281 1284. [150] A. Costa, P. Bini, M. Hamze-Sinno, A. Moglia, D. Franciotta, E. Sinforiani, et al., Galanin and α-MSH autoantibodies in cerebrospinal fluid of patients with Alzheimer’s disease, J. Neuroimmunol. 240 241 (2011) 114 120. [151] D. Giuliani, A. Bitto, M. Galantucci, D. Zaffe, A. Ottani, N. Irrera, et al., Melanocortins protect against progression of Alzheimer’s disease in tripletransgenic mice by targeting multiple pathophysiological path-ways, Neurobiol. Aging 35 (2014) 537 547. [152] K. Ma, J. McLaurin, α-Melanocyte stimulating hormone prevents GABAergic neuronal loss and improves cognitive function in Alzheimer’s disease, J. Neurosci. 34 (20) (2014) 6736 6745. [153] I. Machado, H.B. Schio¨th, M. Lasaga, T. Scimonelli, IL-1β reduces GluA1 phosphorylation and its surface expression during memory reconsolidation and α-melanocyte-stimulating hormone can modulate these effects, Neuropharmacology 128 (2018) 314 323. [154] I.M. Johansson, V. Birzniece, C. Lindblad, T. Olsson, T. Ba¨ckstro¨m, Allopregnanolone inhibits learning in the Morris water maze, Brain Res. 934 (2) (2002) 125 131. [155] H.F. Sakr, K.I. Khalil, A.M. Hussein, M.S. Zaki, R.A. Eid, M. Alkhateeb, Effect of dehydroepiandrosterone (DHEA) on memory and brain derived neurotrophic factor (BDNF) in a rat model of vascular dementia, J. Physiol. Pharmacol. 65 (1) (2014) 41 53. [156] G. Valenti, L. Ferrucci, F. Lauretani, G. Ceresini, S. Bandinelli, M. Luci, et al., Dehydroepiandrosterone sulfate and cognitive function in the elderly: the InCHIANTI Study, J. Endocrinol. Invest. 32 (9) (2009) 766 772.

45

46

CHAPTER 2 Functional neuroanatomy and disorders of cognition

[157] A. Rabinowitz, S.J. Cohen, D.A. Finn, R.W. Stackman Jr., The neurosteroid allopregnanolone impairs object memory and contextual fear memory in male C57BL/6J mice, Horm. Behav. 66 (2) (2014) 238 246. [158] S.K. Bengtsson, M. Johansson, T. Ba¨ckstro¨m, Long-term continuous allopregnanolone elevation causes memory decline and hippocampus shrinkage, in female wildtype B6 mice, Horm. Behav. 78 (2016) 160 167. [159] C. Singh, L. Liu, J.M. Wang, J.M. Wang, R.W. Irwin, J. Yao, et al., Allopregnanolone restores hippocampal-dependent learning and memory and neural progenitor survival in aging 3xTgAD and nonTg mice, Neurobiol. Aging 33 (8) (2011) 1493 1506. [160] R.W. Irwin, J.M. Wang, S. Chen, R.D. Brinton, Neuroregenerative mechanisms of allopregnanolone in Alzheimer’s disease, Front. Endocrinol. (Lausanne) 2 (2012) 117. [161] K.T. Nakhate, A.P. Bharne, V.S. Verma, D.N. Aru, D.M. Kokare, Plumbagin ameliorates memory dysfunction in streptozotocin induced Alzheimer’s disease via activation of Nrf2/ARE pathway and inhibition of β-secretase, Biomed. Pharmacother. 101 (2018) 379 390. [162] D.M. Holtzman, J.C. Morris, A.M. Goate, Alzheimer’s disease: the challenge of the second century, Sci. Transl. Med. 3 (77) (2011) 77sr1. [163] S.D. Skaper, Alzheimer’s disease and amyloid: culprit or coincidence? Int. Rev. Neurobiol. 102 (2012) 277 316. [164] J. Zhao, Y. Deng, Z. Jiang, H.G. Qing, Protein-coupled receptors (GPCRs) in Alzheimer’s disease: a focus on BACE1 related GPCRs, Front. Aging Neurosci. 8 (2016) 58. [165] J.S. Miners, N. Barua, P.G. Kehoe, S. Gill, S. Love, Aβ-degrading enzymes: potential for treatment of Alzheimer disease, J. Neuropathol. Exp. Neurol. 70 (11) (2011) 944 959. [166] R.J. O’Brien, P.C. Wong, Amyloid precursor protein processing and Alzheimer’s disease, Annu. Rev. Neurosci. 34 (2011) 185 204. [167] T. Fulop, J.M. Witkowski, K. Bourgade, A. Khalil, E. Zerif, A. Larbi, et al., Can an infection hypothesis explain the beta amyloid hypothesis of Alzheimer’s disease? Front. Aging Neurosci. 10 (2018) 224. [168] C.C. Curtain, F. Ali, I. Volitakis, R.A. Cherny, R.S. Norton, K. Beyreuther, et al., Alzheimer’s disease amyloid-beta binds copper and zinc to generate an allosterically ordered membrane-penetrating structure containing superoxide dismutase-like subunits, J. Biol. Chem. 276 (23) (2001) 20466 20473. [169] A. Takashima, K. Noguchi, G. Michel, M. Mercken, M. Hoshi, K. Ishiguro, et al., Exposure of rat hippocampal neurons to amyloid beta peptide (25 35) induces the inactivation of phosphatidylinositol-3 kinase and the activation of tau protein kinase I/glycogen synthase kinase-3 beta, Neurosci. Lett. 203 (1) (1996) 33 36. [170] M. Townsend, T. Mehta, D.J. Selkoe, Soluble Abeta inhibits specific signal transduction cascades common to the insulin receptor pathway, J. Biol. Chem. 282 (46) (2007) 33305 33312. [171] S.M.A. Naini, N. Soussi-Yanicostas, Tau hyperphosphorylation and oxidative stress, a critical vicious circle in neurodegenerative tauopathies? Oxid. Med. Cell. Longev. 2015 (2015) 1 17. [172] F. Liu, Z. Liang, C.X. Gong, Hyperphosphorylation of tau and protein phosphatases in Alzheimer disease, Panminerva Med. 48 (2) (2006) 97 108.

Further reading

[173] S.P. Braithwaite, J.B. Stock, P.J. Lombroso, A.C. Nairn, Protein phosphatases and Alzheimer’s disease, Prog. Mol. Biol. Transl. Sci. 106 (2012) 343 379. [174] D. Valdinocci, R.A. Radford, S.M. Siow, R.S. Chung, D.L. Pountney, Potential modes of intercellular α-synuclein transmission, Int. J. Mol. Sci. 18 (2) (2017) 469. [175] J.H. Power, O.L. Barnes, F. Chegini, Lewy bodies and the mechanisms of neuronal cell death in Parkinson’s disease and dementia with Lewy bodies, Brain Pathol. 27 (1) (2017) 3 12. [176] A.G. Kazantsev, A.M. Kolchinsky, Central role of α-synuclein oligomers in neurodegeneration in Parkinson disease, Arch. Neurol. 65 (12) (2008) 1577 1581. [177] H.A. Lashuel, C.R. Overk, A. Oueslati, E. Masliah, The many faces of α-synuclein: from structure and toxicity to therapeutic target, Nat. Rev. Neurosci. 14 (1) (2013) 38 48.

Further reading M.A. Aletrino, O.J. Vogels, P.H. Van Domburg, H.J. Ten Donkelaar, Cell loss in the nucleus raphes dorsalis in Alzheimer’s disease, Neurobiol. Aging 13 (1992) 461 468. E. Ciaramelli, C.L. Grady, M. Moscovitch, Top-down and bottom-up attention to memory: a hypothesis (AtoM) on the role of the posterior parietal cortex in memory retrieval, Neuropsychologia 46 (7) (2008) 1828 1851. E. Gould, P. Tanapat, N.B. Hastings, T.J. Shors, Neurogenesis in adulthood: a possible role in learning, Trends Cogn. Sci. 3 (5) (1999) 186 192. J. O’Keefe, J. Dostrovsky, The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat, Brain Res. 34 (1) (1971) 171 175.

47

CHAPTER

A cognitive system of elderly exercise evaluation with sensors and robots

3 Tatsuya Yamazaki

Niigata University, Niigata, Japan

3.1 Introduction While the life expectancy was 66.376 years in 2000 or 56.964 years in 1970, it became 71.429 years in 2015. Although there is still a gap for the life expectancy between in advanced countries and developing countries, it is forecast to be longer in future. In general, muscle mass lessens about 10% 20% in upper limbs and about 20% 40% in lower limbs at the age of 60 compared with those at the age of 20. Eventually, elderly people lose strength, balance, and flexibility because of muscle decrement; and these are risk factors causing elderly people to fall [1]. Therefore regular and moderate physical practice is important for physiological benefits, and the exercise is effective to keep the joint flexibility and to mitigate appearance of dementia [2]. Although elderly people are motivated to take exercise for keeping themselves healthy, appropriate watching and support is necessary to prevent unanticipated accidents. At this point a problem is encountered. Since the elderly population grows worldwide, necessary care is not always available. For example, at care facilities, the number of care facility staffs is limited, and sometimes it is difficult for them to watch the elderly exercise. Watching is important. Continuous practice of the exercise is important to get effectiveness and other’s advice motivates the continuous practice. One solution is to introduce ICT (information and communications technology) into the elderly care support. One of the earliest physical exercise systems for the elderly was developed by Macek and Kleindienst [3]. The system employed an ultrasound sensor to measure the posture height of the user, a heart rate monitor, and a microphone for speech recognition. The system motivated an elderly person to do regular physical activity based on an easy exercise in a monitored environment. The user load was tracked by monitoring heart rate and by scanning movement patterns using statistical estimators. The experiments were conducted in a laboratory situation. As the regular and moderate physical activity practice, Bleser et al. [4] focused in aerobic activity and strength exercises for the elderly persons. They used networked wearable sensors to capture the user’s motions as a tool of ICT. Cognitive Informatics, Computer Modelling, and Cognitive Science, Volume 2. DOI: https://doi.org/10.1016/B978-0-12-819445-4.00003-5 © 2020 Elsevier Inc. All rights reserved.

49

50

CHAPTER 3 A cognitive system of elderly exercise evaluation

Then they introduced a personalized home-based strength exercise trainer designed for the elderly to guide him/her at home through a personalized exercise program. Meanwhile, an advanced sensor called Kinect1 was developed and has become popular, because it has a human body model to capture human body motion data by depth and camera sensors. Using Kinects, Ganesan and Anthon [5] implemented a prototype game to help encourage the elderly to exercise. The Kinect-based application was tested just for basic usability. Before implementation, they gathered data through an interview with an expert in aging and physical therapy as well as a focus group with the elderly about the topics of exercise and technology. Webster and Ozkan [6] reviewed Kinect-based elderly care and stroke rehabilitation systems and suggested future works in this direction. In the review, they focused on fall detection and fall risk reduction for elderly care. One of the major issues in the case of single Kinect sensing is occlusion, where some part of a body is hidden by the other part. Because of the occlusion problem, human body capturing might be incomplete. To mitigate the occlusion problem, multiple Kinect sensors are used to compensate each other. Jo et al. [7] suggested a body tracking method for multiple users by utilizing multiple Kinects. They set up four Kinects in a laboratory-based experimental environment. In addition to this, they also used motion sensors to track the data of users. Still, there are two problems to make use of multiple Kinect sensors. One is synchronization of image frames captured by multiple Kinect sensors and the other is calibration for the Kinect sensors. In this chapter, we propose a cognitive system to watch elderly persons during exercising and to evaluate their exercise. The system consists of three processes: the first process is behavior measurement using a Kinect sensor, the second one is exercise analysis, and the last one is feedback. In the first process, we present body joint data acquisition by the Kinect sensor. In the second process, we extract features to evaluate the elderly person’s exercise from the measured data. Finally, the evaluation is provided to the elderly person as a feedback in the third process. A robot is used as an intimate interface. In addition, we introduce solutions for the two abovementioned problems regarding multiple Kinects.

3.2 System overview The proposed system, shown in Fig. 3.1, basically consists of the Kinect sensor, a data processing PC, and a robot that provides advice messages. They are connected by a local area network to exchange the data and the messages. When one 1

KINECT is a trademark of Microsoft Corporation.

3.2 System overview

FIGURE 3.1 Overview of the proposed system.

FIGURE 3.2 20 joint positions measured by the Kinect sensor.

Kinect is used, it is located just in front of the exercising person. Fig. 3.2 shows the 20 joint positions obtained by the Kinect infrared sensor. In this paper, 10 joint points (1 head, 1 neck, 2 shoulders, 2 elbows, 2 wrists, and 2 hands) as shown in Fig. 3.3 are used, because when the person exercises with sitting on a chair, only the upper body movement is evaluated. However, it can be extended to the full body exercise.

51

52

CHAPTER 3 A cognitive system of elderly exercise evaluation

FIGURE 3.3 10 joint points in an upper body.

FIGURE 3.4 The arm-push-up angle (front view).

3.3 Elderly exercise measurement With the cooperation of a care facility located in Niigata city, exercise measurement experiments were conducted there. We set up the proposed system without a robot at the care facility and measured and recorded body joint movement data from eight elderly persons and one care facility staff. Among a series of exercises which they performed, two typical exercises have been selected for evaluation analysis: arm-pushing-up and chest-opening exercises. By the arm-pushing-up exercise, people push up the left and right arms alternately. On the other hand, people open and close the left and right elbows simultaneously while bending the elbows at a right angle. For the arm-pushing-up exercise the height at which an arm pushes up is important. Therefore the angle configured by the neck, shoulder, and elbow joints with the center of the shoulder joint is selected for evaluation shown in Fig. 3.4. We call this angle as the arm-push-up angle hereafter. For the chest-opening exercise, how far the elbows open outward should be notified. Therefore the angle configured by the neck, right elbow, and left elbow joints with the center of the

3.4 Exercise evaluation

FIGURE 3.5 The chest-open angle (overhead view).

FIGURE 3.6 The arm-push-up angle extracted from the care facility staff exercise data.

neck joint is selected for evaluation shown in Fig. 3.5. Hereafter, this angle is called as the chest-open angle. Note that Fig. 3.5 is an overhead view, which is a figure viewed from upward, while Fig. 3.5 is a front view.

3.4 Exercise evaluation As defined in the previous section, the arm-push-up angle has been extracted from the care facility staff data, which is presented in Fig. 3.6, where the horizontal axis shows time and the vertical axis shows the arm-push-up angle. Since the arm-push-up angle changes periodically, it is found that the arm-pushing-up exercise was carried out. The arm-push-up angle data are, however, very noisy, so that it is difficult to divide into each period and to find out the maximum and

53

54

CHAPTER 3 A cognitive system of elderly exercise evaluation

minimum points in the period, which have extreme values. As one of the simplest noise reduction methods, the moving average method is known widely. In the moving average method, a measured datum is replaced by the averaged value in a defined window. In this paper, we use the moving average method whose window size is 2n 1 1, where the measured datum itself, its n previous data, and its n following data are used for averaging. After operated by the moving average method six times, the data were successfully divided into each period and four maximum values and three minimum values are extracted, which are shown as vertical lines inserted in Fig. 3.6. Since the maximum height to which an arm reaches is important for the arm-pushing-up exercise, we focus on the maximum values in this exercise. Other eight elderly persons’ measurement data are operated by the moving average method to extract the maximum values. Since the degree of noise differs depending on each person, the operation number of the moving average method becomes different depending on the data. Therefore the moving average methods are operated until the same number of the maximum values as the staff is detected; namely, the criterion is to find four maximum values from the measured data. We call the eight elderly persons as User A, User B, . . ., User H; and the operation numbers of the moving average method are shown in Table 3.1. Except for the left shoulder data of Users A and C, the moving average method successfully works to extract four maximum values and the extracted result of the right shoulder data of User F is shown in Fig. 3.7. Unavailability for Users A and C is caused by failure in data measurement. It is also found that the operation numbers differ from each elderly person because of the degree of noise included in the data. Then the averaged maximum value is calculated from the four maximum values for each person as shown in Tables 3.2 and 3.3. Table 3.2 presents the Table 3.1 The operation number of the moving average method to the right and left arm-push-up angle data for each person. Person

Operation number for the right shoulder data

Operation number for the left shoulder data

Staff User User User User User User User User

6 5 6 4 15 5 10 7 8

7 NA 9 NA 10 5 9 14 10

A B C D E F G H

NA, Not available.

3.4 Exercise evaluation

FIGURE 3.7 The arm-push-up angle of the right shoulder data of User F.

Table 3.2 The averaged maximum value of the arm-push-up angle for the right shoulder and the relative value based on the staff data. Person

The averaged maximum value (degree)

Difference from the staff data

Staff User A User B User C User D User E User F User G User H

246.8 246.4 242.2 230.9 222.8 239.7 207.1 229.1 234.8

0.4 4.6 15.9 24.0 7.1 39.7 17.7 12.0

result for the right shoulder, and Table 3.3 presents the result for the left shoulder. Difference between the staff and each elderly person is also presented based on the staff data. From the results in Tables 3.2 and 3.3, it is found that the proposed system recognizes the exercise of the elderly person and evaluates how much it performs by comparison with the model data of the staff. It is possible to exchange the model data to their past exercise data to observe change over time.

55

56

CHAPTER 3 A cognitive system of elderly exercise evaluation

Table 3.3 The averaged maximum value of the arm-push-up angle for the left shoulder and the relative value based on the staff data. Person

The averaged maximum value (degree)

Difference from the staff data

Staff User A User B User C User D User E User F User G User H

245.4 NA 241.6 NA 206.8 213.7 187.5 234.8 242.6

NA 3.8 NA 38.6 31.7 57.9 10.6 2.8

NA, Not available.

FIGURE 3.8 The open-chest angle extracted from the care facility staff exercise data.

In the same way as the arm-push-up angle, the chest-open angle of the chestopening exercise is analyzed. Fig. 3.8 shows the chest-open angle extracted from the staff data, where the horizontal axis shows time and the vertical axis shows the chest-open angle. Compared with the arm-pushing-up data, the chest-open angle data seem to be less affected by noise. Indeed, just one operation of the moving average method has extracted eight maximum or minimum values, which are shown as vertical lines inserted in Fig. 3.8. Since the open width between the

3.5 Feedback by robot interface

Table 3.4 The operation number of the moving average method to the chestopen angle data and the averaged maximum value of the chest-open angle for each person as well as the relative value based on the staff data. Person

Operation number

The averaged maximum value (degree)

Difference from the staff data

Staff User User User User User User User User

1 2 1 2 3 NA NA 2 2

228.3 177.9 189.0 195.2 181.5 191.2 165.7 185.4 199.9

50.4 39.3 33.1 46.8 37.1 62.6 42.9 28.4

A B C D E F G H

NA, Not available.

right and left elbows is important for the open-chest exercise, we also focus on the maximum values. Other eight elderly persons’ measurement data are operated by the moving average method to extract the maximum values. The operation numbers of the moving average method are shown in Table 3.4. Except for the data of Users E and F, the moving average method successfully works to extract eight maximum values. The maximum values have been extracted manually for Users E and F and it is a subject for future study. Then the averaged maximum values are calculated, and the relative values based on the staff data are derived shown in Table 3.4. From the results in Tables 3.2 and 3.3, it is verified that the proposed system recognizes the chest-open exercise and evaluates its performance.

3.5 Feedback by robot interface As described in the previous sections, we can measure the positions of body joints during exercise, calculate the remarkable angles from the joint position data, and evaluate the degree of movement in the exercise. However, such evaluation in numerical form is not intuitively understandable for the exercising person. Moreover, in the case of the care facilities, it is needed to consider the elderly people’s body condition or characteristics to provide feedback and advice messages to them. Therefore we conducted a hearing survey to the care facility staffs. In the hearing survey, we asked the staffs how to provide advises to the elderly person, and we found that there are three kinds of advises depending on the elderly person’s condition. The good examples of advises are as follows. The first one is “it

57

58

CHAPTER 3 A cognitive system of elderly exercise evaluation

Table 3.5 The averaged subjective evaluation score and the averaged relative value based on the staff data for the arm-pushing-up exercise. Person

The subjective evaluation score

Difference from the staff data

User A User B User C User D User E User F User G User H

5.0 5.0 4.0 3.0 1.5 2.5 5.0 4.0

0.4 4.2 15.9 31.3 19.4 48.8 14.2 7.2

is going nice,” and it is named the fine message in this paper. The second one is “let’s go ahead!” and it is named the encouragement message. The third one is “please do not overdo it,” and it is named the suppression message hereafter. It is noted that message selection needs comparison between the current exercise data and their past averaged exercise data as described in Section 3.4. It means that the proposed system becomes cognitive by memorizing the past averaged exercise data as the model data for each elderly person, and by providing intuitive and understandable messages for the elderly person. The next question is what the thresholds to alter the advises are. Normally, the fine message is moderate and it will be altered to the encouragement message in the case of underperforming or it will be altered to the suppression message in the case of overperforming. To determine whether these thresholds to be altered, we asked two care facility staffs to evaluate the arm-pushing-up exercise videos of the eight elder persons. The staffs scored the exercises in five levels subjectively. The score of 5 is the best and the score of 1 is the worst. The subjective evaluation scores averaged by two staffs are presented in Table 3.5 with the averaged relative values based on the staff data. The averaged relative values are calculated by averaging the right and left shoulder data in Tables 3.2 and 3.3. From Table 3.5, roughly speaking, the subjective evaluation score decreases below 4.0 when the relative value of the arm-push-up angle based on the reference staff data becomes more than 15.9. It means that the staff may regard the exercise as underperforming when difference of the arm-push-up angles between the elder person and the staff becomes more than 15.9. Thus a value Width_encourge such as 15.9 can be determined as the threshold between the fine and encouragement messages. On the other hand, when the arm-push-up angle exceeds the reference staff data, say Reference_staff, it can be regarded as the overperformance. Therefore the threshold between the fine and suppression messages can be the reference staff data itself.

3.6 Multiple Kinect application for occlusion problem

FIGURE 3.9 A prototype system and a person performing the arm-pushing-up exercise.

We have implemented these thresholds and an utterance mechanism in a robot and constructed the cognitive system of elderly exercise evaluation shown in Fig. 3.1. In the case of the arm-pushing-up exercise, the arm-push-up angle is measured for an elderly person. When the arm-push-up angle Reference_staff-Width_encourge between and Reference_staff, the robot utters “it is going nice.” When the arm-push-up angle becomes less than Reference_staff-Width_encourge, the robot utters “let’s go ahead!” Moreover, when the arm-push-up angle becomes more than Reference_staff, the robot utters “please do not overdo it” to restrain the overperformance of the elderly person. The final prototype system has been implemented as presented in Fig. 3.9. Some parts in Fig. 3.9 have been mosaicked to protect privacy.

3.6 Multiple Kinect application for occlusion problem To mitigate the occlusion problem presented in Section 3.1, multiple Kinect sensors are used to compensate each other. There are two problems to make use of multiple Kinect sensors. One is synchronization of image frames captured by multiple Kinect sensors and the other is calibration for the Kinect sensors. We describe solutions for these two problems. Hereafter, we assume two Kinect sensors as a case of multiple Kinect sensors without loss of generality.

59

60

CHAPTER 3 A cognitive system of elderly exercise evaluation

3.6.1 Frame synchronization Each Kinect sensor is connected to its controlling PC. The basic idea is to synchronize the system clock between the controlling PCs to detect the frames corresponding to the same scene. Although NTP (Network Time Protocol) or Simple NTP is used to synchronize PCs globally via Internet, local synchronization is enough for the system run in a care facility. Accordingly, two controlling PCs are directly networked and the “net time” command is used to synchronize these system clocks. Fig. 3.10 shows an experimental setting with local system clock synchronization. In this experiment setting, we test the accuracy of frame synchronization with the “net time” command. In the experiment a common time display is presented on the frames captured by each Kinect sensor and the corresponding frames can be found out by detecting the frames presenting the same time display. As a result of that, we confirmed time difference between corresponding frames from two Kinect sensors. In an experiment, we send out the “net time” command and collect 10 times differences. Then the average value is calculated from the 10 times differences as the result of one experiment. Table 3.6 shows the results of five experiments and verifies that the time difference between the corresponding frames from two Kinect sensors is smaller than the frame interval, which is usually 33.3 ms.

FIGURE 3.10 An experimental setting with local system clock synchronization environment.

Table 3.6 The results of five experiments. Experiment

Average of 10 times differences (ms)

1 2 3 4 5 Average

19.5 16.6 16.0 12.4 12.9 15.5

3.7 Conclusion

FIGURE 3.11 Extension of the proposed system with multiple Kinect sensors.

3.6.2 Sensing data integration without calibration When two Kinect sensors are used, their coordinates for rotation and translation differ from each other. Calibration is to derive a transformation from one coordinate into the other, or to determine the world coordinate. Conventionally, the same objects with accurate position parameters are used in the initial phase before the experiments [8]. Using these objects, the coordinate transformation matrices are calculated. Since such initial setting annoys the elderly persons waiting for exercise. Our proposed system compensates missing data by one Kinect sensor by the other sensing data. Fig. 3.11 shows an exercise environment with two Kinect sensors. We specify a main Kinect sensor and a sub Kinect sensor from the two Kinect sensors. The main Kinect sensor can be determined as the one that has less missing data. Suppose there is a missing data a4 5 {a4x, a4y, a4z} from the main Kinect sensor, the sensing data b4 5 {b4x, b4y, b4z} corresponding the missing data from the sub Kinect sensor, and at least three corresponding sensed data ai 5 {aix, aiy, aiz} and bi 5 {bix, biy, biz} from both the main and sub Kinect sensors, where i 5 1, 2, and 3. The transformation matrix is derived from the three corresponding sensed data ai and bi (i 5 1, 2, and 3). Specifically, a set of corresponding sensing data is matched with the same coordinates. Then, by constructing the same plane with three corresponding sensing data, we obtain the transformation matrix. Finally, the missing data a4 is calculated from the corresponding sensing data b4 by using the transformation data.

3.7 Conclusion As the elderly population grows, regular and moderate physical practice is important for elderly people to keep joint flexibility and to mitigate appearance of

61

62

CHAPTER 3 A cognitive system of elderly exercise evaluation

dementia. Although care facilities must play an important role to provide preventive care and rehabilitation services, shortage of care staffs will be an inevitable issue. Thus we are studying on the introduction of ICT to the care facilities. One of the ordinary services for the elderly persons at the care facilities is an exercise. To enhance the exercise effect, proper guidance or support is necessary, but the number of care facility staffs is not enough. Therefore we propose a cognitive system to watch elderly persons during exercising. The system also provides evaluation and proper feedback for their exercise. The proposed system consists of three processes: a sensing process, an evaluation step, and a feedback process. In the first sensing process a depth sensor detects and tracks user skeleton data, since noncontact are desirable for the elderly people. The depth sensor does not impose any physical burden on the elderly people. In the second evaluation process, body joint position data are extracted as features and the angles to be evaluated are calculated. Finally, the third feedback process returns an advice to the elderly person, but numerical evaluation is not cognitive for the exercising person. Then, a hearing survey to the care facility staffs was conducted to determine appropriate feedback. After the hearing survey, we could determine three kinds of advises depending on the elderly person’s condition. These advices were feedbacked to the exercising elderly person via a robot utterance. A robot seems to be intimate for the elderly people. Moreover, there is one severe problem in the sensing process, that is, occlusion where a body part hides some other part of the body. To mitigate the occlusion problem, multiple cameras are usually used to reduce the hidden body parts, but calibration among the cameras is needed to align their coordinates, which is a too complicated task for the care facility staffs. Therefore we propose a method to align the multiple camera coordinates without any calibration and to construct a three-dimensional skeleton model. The proposed system has been constructed as a prototype. With the collaboration of a care facility located in Niigata city, several experiments were conducted to collect exercise data and to test the feedback effects. More improvement will be implemented in the proposed system for practical realization.

Acknowledgment I gratefully acknowledge to Mr. Takuya Hirosawa, who was a master course student at Niigata University and contributed to this study so much.

References [1] V. Scott, S. Peck, P. Kendall, Prevention of Falls and Injuries Among the Elderly, A Special Report From the Office of the Provincial Health Officer, Ministry of Health Planning, Victoria, BC, 2004.

References

[2] D. Warburton, C. Nicol, S. Bredin, Health benefits of physical activity: the evidence, Can. Med. Assoc. J. 174 (6) (2006) 801 809. [3] J. Macek, J. Kleindienst, Exercise support system for elderly: multi-sensor physiological state detection and usability testing, in: IFIP TC13 International Conference on Human-Computer Interaction (INTERACT 2011) 2, Lisbon, Portugal, 2011, pp. 81 88. [4] G. Bleser, D. Steffen, M. Weber, G. Hendeby, D. Stricker, L. Fradet, et al., A personalized exercise trainer for the elderly, J. Ambient Intell. Smart Environ. Arch. 5 (6) (2013) 547 562. [5] S. Ganesan, L. Anthon, Using the Kinect to encourage older adults to exercise: a prototype, in: CHI 2012 Extended Abstracts on Human Factors in Computing Systems, Austin, Texas, 2012. [6] D. Webster, C. Ozkan, Systematic review of Kinect applications in elderly care and stroke rehabilitation, J. Neuroeng. Rehabil. 11 (2014) 108. [7] H. Jo, H. Yu, H. Kim, J. Sung, A study of multiple body tracking system for digital signage of NUI method, Adv. Sci. Technol. Lett. 86 (2015) 91 95 (Ubiquitous Science and Engineering 2015). [8] M. Caon, Y. Yue, J. Tscherrig, E. Mugellini, O. Abou, Context-aware 3D gesture interaction based on multiple Kinects, in: The First International Conference on Ambient Computing, Applications, Services and Technologies (AMBIENT 2011), Barcelona, Spain, 2011.

63

CHAPTER

Models of making choice and control over thought for action

4

Indrajeet Indrajeet1, Shruti Goyal2, Krishna P. Miyapuram2 and Supriya Ray1 1

Centre of Behavioural and Cognitive Sciences, University of Allahabad, Prayagraj India Centre for Cognitive and Brain Sciences, Indian Institute of Technology, Gandhinagar, India

2

4.1 Outline of review Our focus in this review is on recent computational models on both perceptual and value-based decision-making. We have discussed in great details on “hierarchical drift diffusion model” (HDDM) that is capable of estimating the influence of trial-by-trial variability in measurements (e.g., functional magnetic resonance imaging (fMRI) and electroenchephalography (EEG)) on perceptual decisionmaking parameters. Models of perceptual choices can be contrasted with models that involve pure monetary or value-based outcomes. Experience-based information results from our interaction with the environment. This has largely been overlooked by the existing models of choice under uncertainty such as “prospect theory.” In a study where experience-based information is incorporated along with the descriptive information, we have found that the probability weight associated with rare events is less than the objective probability. We suggest that the probability-based models of choice thus need to incorporate experience-based information. Two related models are the Bayesian models and reinforcement learning, both of which have been extensively applied to choices under uncertainty. Recent advances of successful application of models of perceptual choices such as HDDM in value-based scenarios allow us to discuss on whether we could use the valuation models of choice in perceptual scenarios. To probe cognitive processes underlying perceptual decision-making, subjects are compelled to select just one option from a set of limited alternatives and subsequently report their decision by generating a corresponding movement in every trial. On the contrary, studies that address subjects’ ability to countermand require them either to initiate or inhibit a stereotypical movement directed to an obvious target. Whether these two processes are exclusive, or how perceptual ability may influence the ability to control a motor action is poorly understood. “cancellable rise-to-threshold” (CRTT) model, which may be considered as an alternative to “Race” model of countermanding, suggests that the detection of the stop signal decelerates movement preparation and eventually withhold a planned movement. Cognitive Informatics, Computer Modelling, and Cognitive Science, Volume 2. DOI: https://doi.org/10.1016/B978-0-12-819445-4.00004-7 © 2020 Elsevier Inc. All rights reserved.

65

66

CHAPTER 4 Models of making choice and control over

We have modified CRTT model to account for our finding that the perceptual quality of the stop signal determines the magnitude of deceleration in movement planning. More recent models, like “linear approach to threshold explaining space and time” (LATEST) model that elegantly captures the choice of when and where to elicit a saccadic eye movement, and “rational decision-making (RDM)” model of movement inhibition, have also been discussed. We conclude with outstanding issues in this field and future scope of recent frameworks to address those issues.

4.2 Introduction Decision-making is a process of choosing a target object or action among multiple potential options. Adaptive organisms learn to choose a course of action that results in achieving a favorable goal. For optimal goal-directed behavior, a control strategy is also required that involves not only the selection but also inhibition of thoughts and corresponding action based on their possible outcomes. Have you ever faced an episode in life when some thoughts are stuck in your head for quite some time, for example, the eagerness of commencing a dream job in a foreign land or the contemplation to retaliate an abusive colleague in the next encounter? In the first case, accepting the offer would require you to leave your family and friends behind, and in the second, your action would possibly risk your job. Usually the excitement of taking a course of action subsides after realizing that the outcome may not be rewarding in the long run. These real-life episodes require us to develop the ability to timely inhibit the ongoing thought for an action if it becomes irrelevant or detrimental due to any change in the context. The ability to control our thoughts and action according to the need of the moment to achieve a goal is referred to as the executive control in cognitive science. The pioneering studies on cognitive control of perception and action are reviewed in great details elsewhere [1
6]. The control process is critical for gaining a balance between competing goals in the unpredictable dynamic environment in parallel via functionally dissociable neural networks [7]. How we select a goal, plan an action, evaluate risk, and inhibit the urge to execute the action remain as central issues for further exploration in cognitive neuroscience. The control over the thought for an action has been investigated thoroughly using different paradigms (e.g., stop signal, go/no-go, and alternating runs), with the help of cuttingedge techniques such as imaging, electrophysiology, and computational modeling. Scientists investigating neural mechanisms underlying decision-making often study saccadic eye-movements and the corresponding network in the brain as a model decision maker. Our vision has structural limit for sampling information from the environment. A clear vision is available at the foveae, and the acuity exponentially decreases away from the point of gaze. Saccades are fast movements of the eyes that bring objects of interest on the foveae. Many studies

4.3 Models of perceptual decision

reviewed here used saccadic task; however, their findings are not limited to the oculomotor systems, and often the phenomena are observed in effectors other than the eyes as well. Computational modeling enables progress in our understanding of mechanisms underlying cognitive processes. Models and empirical findings coevolve with each other [8]. Several types of computational model have been designed over decades to explain human behavior and understand underlying neural mechanisms (reviewed in [9]), which may be broadly classified as connectionist/parallel distributed processing models [10], rational and Bayesian models [11], dynamical systems models [12], symbolic logic-based models (e.g., adaptive control of thought
rational (ACT-R) [13]), and hybrid models (e.g., [14,15]). There are many aspects that determine the quality of a model; for example, explanatory adequacy, falsifiability, optimality of parameters, complexity, generalizability, and predictability are few to name [16]. Usually a model is designed and dedicated to account for a particular cognitive phenomenon; there is hardly any particular type of models that is adequate to describe the repertoire of human behavior. One of the reasons might be that human behavior itself is not optimal always, as McClelland [17] commented, “I see no clear distinction between the sufficiency, optimality, and empirical adequacy criteria, in part because I view human performance as inherently probabilistic and error prone.” A good model is distinguished by (1) testable predictability (i.e., making prediction about the changes in behavior and/or physiology due to changes in model parameters) and (2) prescriptive ability (i.e., describe limitations of the model and possible ways to overcome it). In addition, since neuroscience provides a rich source of constraints and information for modeling, a model is often considered robust if it is “neurally plausible.” In this review, we systematically look into relatively recent computational frameworks of perceptual decision-making, economic decision-making, and executive control.

4.3 Models of perceptual decision Decision-making forms a bridge between sensory and motor processes. Decisions can be distinguished from choice in which the latter refers to committing an action that reveals the selection of one option among several alternatives. The decision itself can be construed as the deliberation process that leads to choice [18
20]. Two major classes of decision-making models can be considered [21]. The “good-based” economic models and the action-based models differ by considering whether the decision processes are distinct from or integrated with the sensorimotor processes, respectively [22]. While the former approach considers a single decision variable computed by comparing subjective values of various alternatives [23
25], the latter emphasizes the computation of decision variable that leads to action selection (e.g., [26
28]). Regardless, these two approaches

67

68

CHAPTER 4 Models of making choice and control over

agree conceptually and mechanistically the pivotal role played by the decision variable in models of making choice. Decision-making has often been considered synonymous with the successful transformation of sensory inputs to a choice, that is, motor action. Accordingly, decision-making can be viewed in the sensory domain or in the motor domain. In an uncertain and dynamic world the noisy sensory evidence needs to be integrated to arrive at a categorical judgment about the stimulus. Perceptual decisionmaking is defined as interpreting sensory information and translating it into behavior. Typical tasks for perceptual decision-making include sensory discrimination in different modalities (e.g., tactile, visual, and auditory), for example, identification of the dominant direction of motion of a set of moving dots, referred to as a random dot kinematogram [25]. These paradigms help to establish the mechanisms underlying perceptual decisions. The computational modeling approach can be effectively used for understanding the latent parameters that underlie the process of perceptual decision-making [29]. The major class of computational models that explain perceptual decisions are from the class of sequential sampling models (Fig. 4.1; [30,31]). The advantage of this approach is that we can not only model what choice is committed, but also the response time that was taken to reach the decision [26,32]. Sequential sampling models can be considered as an extension of the signal detection theory that tries to determine which of the alternating hypothesis (e.g., h1: stimulus present vs h2: stimulus absent) has given rise to the observed sensory evidence “e” [33]. The decision variable is typically represented by the likelihood ratio (LR). l12 ðeÞ 5

pðejh1 Þ pðejh2 Þ

The central assumption of sequential sampling models is that stimulus representations in the brain are inherently variable or noisy. Thus these models are well-suited for understanding how we navigate in an uncertain environment. Because sensory information is noisy, it is necessary to accumulate evidence over successive samples until it reaches a threshold. In its general form the sequential probability ratio test allows constructing the decision variable from multiple independent pieces of evidence e1 ; e2 ; . . . ; en as log LR Log LR12 5

n X Pðei jh1 Þ i51

Pðei jh2 Þ

One subset of sequential sampling models is the drift-diffusion model (Fig. 4.2). A drift process (i.e., the integration of decision variables) characterizes the accumulation of evidence growing toward one of the two decision boundaries. The diffusion process accounts for the noise characteristics of input sensory evidence. The diffusion process is the extension of the random-walk model in continuous time. The diffusion process models the within-trial variability, which in turn

Sequential sampling models

Relative evidence criteria: random walk/diffusion models

Continuous time continuous evidence: diffusion models

Discrete time continuous evidence: random walk models

Absolute evidence criteria: accumulator models

Discrete time discrete evidence: recruitment model

discrete time continuous evidence: accumulator model

continuous time, discrete evidence: poisson counter model

continuous time continuous evidence: nonstochastic linear ballistic accumulator model

Hybrid accumulator/diffusion models absolute evidence criteria Decay in drift Constant drift Ornstein–Uhlenbeck (standard) Wiener (OU) model diffusion model

Independent evidence totals with or without decay in drift dual diffusion model

Inhibition between evidence totals, decay in drift: leaky competing accumulator (LCA) model

FIGURE 4.1 Different types of sequential sampling models of decision-making. From R. Ratcliff, P.L. Smith, S.D. Brown, G. McKoon, Diffusion decision model: current issues and history. Trends Cogn. Sci. 20 (2016) 260
281.

CHAPTER 4 Models of making choice and control over

Response density (upper boundary)

Non-decision time (t) (

t Drif

rate

Threshold (a)

Upper response boundary (v)

Bias (z)

70

Lower response boundary Reponse density (lower boundary)

Time

FIGURE 4.2 Schematic of drift diffusion model for 2AFC task. Noisy sensory evidence is accumulated over time, which is represented as a drift-process. The accumulation continues until the net accumulated evidence crosses one of the two decision boundaries. Each of these boundaries represents a choice. Upon reaching the boundary the drift-process initiates the corresponding response. The gap between the two boundaries (i.e., threshold a) determines the amount of accumulated evidence until a choice is made and a response is executed. The relative evidence for or against a particular choice determines drift-rate (v), which is the speed with which the accumulation process approaches one of the two boundaries. The time taken by the drift-process to reach the threshold is called RT, which varies from trial to trial due to noisy drift processes as shown by gray wobbling traces. However, RT is not merely comprised the decision-making process. The delay in perception, movement initiation, and execution together give rise to a nondecision time (t), and a bias (z) toward one of the two boundaries in the starting point of the drift process influences RT. All of these parameters together result in skewed RT distributions for any of the choices. An analytic solution to the resulting probability distribution of times (τ) taken by diffusion processes before termination N   P 2 2 2 2 ke ð2k π τ=2a Þ sinðkπz Þ. 2AFC, is expressed as f ðτjv ; a; z Þ 5 π=a2 e ð2vaz
ðv τ=2ÞÞ 3 k51

Two-alternative-forced-choice; RT, response time. Modified from T.V. Wiecki, I. Sofer, M.J. Frank, HDDM: hierarchical Bayesian estimation of the drift-diffusion model in python, Front. Neuroinform. 7 (2013) 14 [34].

suggests that the accumulation of sensory evidence with a given drift rate does not end up in the same response time or the same decision boundary. The accumulation process XðtÞ  is described by the stochastic differential equation dX ðtÞ 5 v dt 1 s dWðt , where WðtÞ represents the Weiner noise process (i.e., idealized Brownian motion) with amplitude s and v is the drift rate. The drift diffusion model also incorporates a bias parameter “z” at the starting point that accounts for evidence accumulation in favor of one of the two choices. This parameter usually represents differences in prior probabilities of the two

4.3 Models of perceptual decision

outcomes. It is also possible that that payoffs influence the starting point parameter [35]. The full diffusion model gives the percent correct responses as e22va=s 2 e22vz=s ; e22va=s2 2 1 2

Pðv; a; zÞ 5

2

or ð1 2 z=aÞ if drift is zero where a is the boundary separation. The cumulative distribution of reaction times (RTs) is given by Gðt; v; a; zÞ 5 Pðv; a; zÞ 2

  2 2 2 2 2 2 N 2k sin kπz=a e21=2ððv =s Þ1ðk π s =a ÞÞt πs2 2vz=s2 X       e 3 a2 v2 =s2 1 k2 π2 s2 =a2 k51

The EZ-diffusion model [36] gives a simple analytical alternative to estimating the parameters of the DDM with Pc being the empirical probability of correct responses and VRT is the variance of the response times, s is the scaling parameter conventionally set to the value 0.1. Drift rate and boundary separation is given by        1=4 v 5 sign Pc 2 1=2 s logitðPc Þ P2c logitðPc Þ2Pc logitðPc Þ1Pc 2 1=2  =VRT  and a 5 s2 logitðPc Þ=v, respectively, where logitðPc Þ 5 log Pc =1 2 Pc . Nondecision processes such as sensory encoding and motor actions during decision-making are represented by nondecision time parameter “Ter.” Thus response time is partitioned as RT 5 Decision time 1 Ter . Traditional estimation of model parameters fit different models for each subject, ignoring the similarities across individuals or fit the same model for all subjects ignoring the individual differences. The HDDM allows efficient estimation of model parameters simultaneously at group and subject level, allowing certain parameters to be fixed and others to be variable between subjects [34]. In cognitive modeling where the data from individuals are scant to reliably estimate the parameters of DDM, hierarchical property of the Bayesian approach is advantageous with the assumption that subjects are similar to each other within each group although each one is unique. Bayes’ theorem that changes the direction of conditionality calculates the probability of parameter θ given data y is PðθjyÞ 5 PðyjθPÞð3yÞ PðθÞ, where PðθjyÞ is posterior, PðyjθÞ is likelihood, PðθÞ is prior, and PðyÞ is overall probability of evidence irrespective of θ. While in DDM, the termination time estimation requires an infinite summation, HDDM uses integration of the likelihood function for variability in drift-rate, nondecision time and bias, followed by Markov chain Monte Carlo to estimate the joint posterior distribution of all model parameters [37,38]. The linear ballistic accumulator (LBA) model assumes a ballistic as opposed to the DDM models in which a stochastic process of evidence accumulation is considered. In the LBA model, RT and choice variability is accounted for by between-trial differences in average drift-rate. The LBA model, which is a simplistic and complete model of decision-making with a set of nominal assumptions, keeping the general principle is the same as that of DDM [39]. The LBA model assumes that separate accumulators accumulate evidence independently, which allows the model to naturally extend from two-alternative to multialternative

71

72

CHAPTER 4 Models of making choice and control over

decision-making. The LBA model can also be seen as an extension of the LATER (linear approach to threshold with ergodic rate) model, which relies on the random-walk approach for choice modeling [40]. Usher and McClelland [41] proposed the leaky, competing accumulator (LCA) model, which introduces the concept of inhibition of competing alternatives. Like the LBA, the LCA suggests the use of separate evidence accumulators for each alternative. However, the leakage of evidence accumulation and lateral inhibition among accumulators (i.e., competition) form the key characteristics of the LCA model. Several variations of the DDM exist which include variability of different parameters. Bogacz et al. [42] analyzed multiple variants such as the Ornstein
Uhlenbeck, race, and LCA models and demonstrated that these can be reduced to the DDM. Further comparisons of fitting multiple models suggest the general applicability of the DDM [43,44]. Recently, Mahesan et al. [45] conducted a study in which the participants were required to discriminate between a left or right oriented Gabor patch. The perceptual decision was preceded by payoff information providing high or low rewards to the two orientations. The payoffs were independently manipulated from the perceptual decisions. Diederich and Busemeyer [35] suggested that such experimental manipulations can effectively be explained through a two-stage hypothesis with independent evidence accumulation in the first stage acting as the prior bias in starting point for the second stage. Using extensive analysis with HDDM, Chawla and Miyapuram [20] demonstrated that starting point bias parameter tracks the payoff information supporting the two-stage hypothesis. The alternate hypothesis would be that the dynamic bias parameter, that is, the drift rate encodes the differential payoffs, which does not seem to be the case [46].

4.3.1 Fast decision-making Response time in typical perceptual discrimination choice tasks includes processes that transform sensory stimulation to motor commands. To measure the time duration exclusively consumed by the perceptual stage, Stanford et al. [47] took advantage of a novel task called “compelled-saccade” task that did not require masks or additional stimuli like other tasks to provide the duration of perceptual choices alone (e.g., color discrimination). They found that monkeys can make accurate color discrimination in less than 30 ms, pretty much close to humans [48]. Moreover, ongoing motor plan for the target was accelerated and that for distracter was decelerated, by the cue information at the point of time when the choice performance rose above chance that validated the behavioral findings.

4.3.2 Intuitive decision-making How does a player catch the ball in cricket or a baseball game with great accuracy and in very limited time? Theoretically, the flying balls follow parabolic

4.4 Models of economic decision

trajectories. To get the right trajectory the brain needs to estimate many complex parameters, for example, velocity, distance, and angle of the projectile path at the time zero (starting). But, other things (e.g., air resistance, wind, and spin) in real worlds do not let balls move in parabolic trajectories. This requires the brain to estimate additional parameters. But experimental studies have found that players perform relatively poor in estimating the landing point of the ball [49
51]. In practice, experienced players adopt several rules of thumb. For example, gaze heuristics works well for the ball in the air (for more on heuristics refer to Ref. [52]). The gaze-angle is measured between the moving object and eye taking ground as a reference. This heuristic does not require brain to calculate parameters such as wind speed, and velocities. Though gut feeling is used interchangeably with intuition or hunches, there are subtle differences between gut feeling and intuitive judgment [53
55]. The former is a judgment experienced quickly in consciousness, but the awareness about the cause behind it is not complete and potentially can trigger an action; while the latter is fast, inaccessible to conscious awareness, prone to erroneous outcome, based on heuristics, and requiring almost no deliberation. A deliberate judgment on the other hand is slow, accessible to conscious awareness, consumes cognitive resources, and based on rules and rationale [56
61]. However, this definition of deliberate judgment has been challenged [62,63] and advocated to be continuous rather than dichotomous [64,65]. For further details on the interplay between intuition and deliberation refer to De Neys [66] and Evans and Frankish [67].

4.4 Models of economic decision Of interest is also to consider whether the models of perceptual decision are applicable to value-based economic decision. Indeed, generic theories of decisionmaking [68,69] suggest different stages such as representation, valuation, action selection, and outcome evaluation. Decision field theory (DFT) [70] provides a sequential sampling model for value-based decisions. The nature of these models emphasizes the dynamic rather than the static nature of preferences, the latter of which forms the primary basis of neoclassical and modern economic models. The sequential sampling models can be effectively used to explain contextual effects [71], more specifically the influence of irrelevant alternatives. Usher and McClelland [72] have extended the LCA model to explain the loss aversion, a phenomenon that suggests that losses loom larger than gains (i.e., weighted approximately twice as the equivalent gain amounts). Recently, the DDM has been used for value-based decisions [73]. Further, Bitzer et al. [74] have shown that the DDM is equivalent to the Bayesian model, which has been used with choices based on reinforcement learning [75]. Comparing the common process of selection of choice, Lynn et al. [76] have suggested the integration of signal detection and economic perspectives for decision-making. The first step in the

73

74

CHAPTER 4 Models of making choice and control over

integrated signals and economic framework is combining the subjective value and expected value from the prospect theory and signal detection, respectively. The second step is to introduce weighting functions for probabilities associated with perceptual uncertainty. Such a framework extends and builds on similar earlier proposals to study perceptual and economic decisions together [20,21]. A major consideration in such an ambitious attempt is how prior experience is incorporated into making the choice. Models of perceptual choices can be contrasted with those that involve pure monetary or value-based outcomes. Experiencebased information results from our interaction with the environment. This has largely been overlooked by the existing models of choice under uncertainty such as the Prospect theory. In recent years contrasting literature has emerged for choices based on description versus experience [77]. Studies have found that when information about outcome and its probability is learned from description versus when learned from experience leads to different choices [78
80]. This is referred to as “Description
Experience Gap” [81]. In the descriptive paradigm, information about the outcome and outcome probabilities were clearly stated. In the experience-based paradigm, information about outcome and outcome probabilities had to be extracted based on sampling the two provided options with no stated information regarding the outcome or associated probabilities. Hertwig et al. [80] presented participants with a choice between a sure option and a gamble in two formats. They found overweighting of small probabilities as suggested by prospect theory [82]. However, in experience-based task, people behaved as if they were underweighting the rare events [78,80,83]. Underweighting of probabilities reflects small decision weight being associated with an event relative to its objective probability [84]. Results similar to experience-based paradigm are observed when both descriptive and experience-based information are available to the decision maker [85
88]. The existing models of value-based decision-making are inefficient at accounting these findings, which suggests that a separate theory of decision-making based on experience is much needed [80,83]. There are two major accounts for “description
experience gap,” According to the information asymmetry account, differences in the information acquisition stage of belief-based account of decisions under uncertainty results to this gap. Belief-based account [89] proposes two stages of decision-making under uncertainty. The first stage is an information acquisition stage, where the decision maker aims at constructing probability distribution for the alternatives of the uncertain scenario. Second stage concerns about the decision made given the probability distribution generated in the first stage governed by prospect theory. Information asymmetry account stresses that the probability distributions formulated for the alternatives in the two paradigms are incomparable. This is because of the sampling bias, which is inherent to the experience-based paradigm [90,91]. According to the psychological account, decision-making process differs between the two paradigms. Description and experience-based paradigms involve a different evaluation process to compare options. Studies report mixed findings when

4.5 Models of movement inhibition

accounting for the sampling bias. Few studies have found that when sampling bias (probability distribution in the first stage) is taken care of, gap diminishes [90,92], whereas others have found that the gap remains even after sampling bias is eliminated [93,94]. Wulff et al. [95] reanalyzed the results of the previous studies on description
experience gap and found differences in probability weighting between the two. They stated, “choices were still systematically different and the gaps were mostly consistent with either underweighting of rare events—or at least with less overweighting of rare events (including linear weighting) than in decisions from description” [95]. Goyal and Miyapuram [87] provided the same descriptive information to two groups with one group receiving no feedback on the choices made, while the other group receiving feedback on every choice made. They found underweighting of small probabilities in both loss and gain domains when feedback was available on the descriptive choices. Feedback was found to influence attractiveness and discriminability components of probability weighting mediating the choices made. Given the role played by experience as feedback on the choices, they have suggested a modification to the existing account of decision-making. Adding a third stage to the Belief-based account, that is, a feedback stage would account for experience-based choices. When feedback is not available, this stage would remain inactive and would lead to choices as predicted by prospect theory [87].

4.5 Models of movement inhibition Coutlee and Huettel [96] suggested the convergence of decision neuroscience literature with those of cognitive control with the central role of prefrontal cortex in flexible selection of a prepotent response for which an instantaneous reinforcement, be it positive or negative, is associated. Cognitive control is invoked when it is necessary to override a prepotent response that would otherwise inevitably occur. Experimental approaches for studying cognitive control require participants to successfully inhibit and act contrary to the prepotent response or overlearned tendency. Examples include the Stroop task, Simon task, and the stop-signal paradigm. In the color Stroop task, participants have to name the color of a word, for example, “red” that is printed in possibly a different color (e.g., blue) by inhibiting the prepotent response of reading the letters. Variants of the Stroop task can include the emotional Stroop task with emotional descriptor words superimposed on face stimuli. The Simon task refers to inhibiting the standard visuomotor mapping when participants have to respond to the side opposite to the stimulus (e.g., right side response when a stimulus appears on the left side). In the stop-signal task, a response to a stimulus under preparation needs to be inhibited upon encountering an unexpected stop signal. Therefore, theories of cognitive control refer to different psychological processes that form part of goal-directed behavior by adaptively adjusting our immediate behavior.

75

76

CHAPTER 4 Models of making choice and control over

Choosing an action is accompanied by successfully inhibiting unwanted action. Cortical neurons that encode decision rules for planning an action in a go/no-go task deselect nonpreferred rule before selecting the preferred rule evident from differential modulation of firing rate [97]. Understanding the computational mechanisms of action-selection is therefore imperative to understand decision-making. Many behavioral paradigms are used to study how subjects cancel urges to initiate an action, for example, Simon task, Countermanding, Stroop Task, anti-saccade, Wisconsin Card Sorting Test, Go/ No-Go task, and Eriksen flanker task. In this chapter, we will focus only on stop-signal task and its variants due to lack of space. Stop-signal task (or countermanding) is a popular tool used to investigate the inhibitory control in laboratory settings [98
102]. It involves making a go response or movement instructed by a go signal in major proportion of trials, but to cancel the preplanned response or movement if a stop signal appears at random variable delay from the onset of the go signal (stop-signal delay or SSD) in a small percentage of trials. Inhibition of a planned movement (i.e., countermanding) necessitated by the abrupt changes in the context or goal is a critical component of executive function. Race model [99] has been widely considered as the primary tool for quantitative study of inhibitory control in the oculomotor domain since its inception. In this model, two processes compete with each other to reach a fixed threshold; one of these processes, called GO, triggers saccadic eye movement when it reaches the threshold before the other one, called STOP (Fig. 4.3). In the last decade, many novel variants of race model are designed to accommodate the recent behavioral and physiological data (for recent review refer to Ref. [100,103,104]). Boucher et al. [105] designed interactive race model that describes the cancellation of a saccadic eye movement due to an interruption in the rising activity of motor neurons in the frontal eye fields (FEFs) of the primate brain. Subsequently, new variants, the “spiking” [106] and “augmented” [107], in the interactive race model in the form of incorporation of a pretarget module that accounts the dynamic activity of FEF’s gaze-holding (or fixation) neurons. Further, to delineate the distinct role of bottom-up (stimulus-driven) and topdown inputs to the neural integrators, “boosted-fixation model” and “blockedinput models” variants of race model were designed [108]. Blocked-input 1.0 assumes that instead of direct inhibition of go activation by the stop-integrator, stop-integrator blocks the input to the go-integrator preventing it to reach to threshold and trigger the motor action, whereas Blocked-input 2.0 assumes that besides the STOP process, a top-down process also inhibits the GO process. Fixation activation is boosted by a top-down process in “boosted-fixation” model. Imaging and electrophysiology study on human and monkeys have shown that while dlPFC (dorso-lateral prefrontal cortex) encodes context dependency of cognitive control, FEF signals success or failure regardless of the context in a saccade-countermanding task [109].

4.5 Models of movement inhibition

(A)

(B)

R

p es

on

se

tim

e

s

0m

50

Stop trials

Re

o sp

ns

et

im

Non-Cancel

e

Cancel

T

PP

Noncanceled stop trials (%)

No-stop trials

100 80

SSD (P = .5)

60 40 20 0 0

50

100

150

200

250

300

Stop signal delay (ms)

D

SS

(C)

(D) Theshold

O

G

SSD

ST OP

Noncancel

SSRT

Frequency

Activation

G

O

Median no-stop RT

SSD (P = .5)

SSRT

Cancel

Time from target onset

FIGURE 4.3 (A) Schematic of stop-signal task. Each trial starts with a fixation spot for a variable duration at the center. When the subject successfully fixates the spot for a stipulated period (say 500 ms), it disappears (go signal) and a target (here a white square) appears in the periphery at the same time. A subject is required to generate a saccade to the target (no-stop trials; upper panel). The delay between the target and saccade onset is the RT. However, in a small proportion of trials, the fixation spot (stop signal) reappears after a variable time interval, called SSD. In these trials the subject is instructed to stop his/her saccade (stop trials; lower panel). Subjects are rewarded for successfully stopping saccade (canceled trials); however, in a few trials, subjects generate saccade to the target despite the stop signal (noncanceled trials). The delay between the stop-signal and noncanceled saccade onset is referred to as PPT. The broken circle represents the point of gaze. (B) Inhibition function: fraction of noncanceled stop trials is plotted as a function of SSD. As SSD increases, the fraction of noncanceled stop trials increases too. The SSD for which the occurrence of canceled and noncanceled stop-trials are equally likely (P 5.5) is calculated from the fit. (C) Probable results of an independent race between a GO and a STOP process to reach a threshold. If the GO process reaches the threshold before the STOP, a response is produced (upper: noncanceled trials); whereas if the STOP process reaches the threshold before GO, the response is canceled (lower: canceled trials). (Continued)

77

CHAPTER 4 Models of making choice and control over

4.5.1 Proactive control Anticipation about the future state of the environment is critical to attain optimal behavioral outcomes. For example, a driver slows down in the accident prone or crowed area indicated by environmental cues (notice board, experience in the past) to successfully stop if required. The delay of response in expectation of requirement to stop increases the probability to inhibit an action or make another alternative action [99]. The slowing in generating a response in anticipation is referred to proactive control [110
115]. Traditionally, the likelihood of the stop signal in a trial is specified by a cue in the beginning in tasks designed to study proactive control. The positive correlation between RT and probability of appearance of the stop signal is considered as an effect of proactive control. This relationship is often present when the subject pool is healthy adults [116,117], whereas proactive control is underdeveloped (or absent) in children [113]. Deficient proactive control is reported in old age population [118] and patients afflicted with psychiatric disorders [119]. Stop-signal paradigm involves two competing goals—to generate the go response as quickly as possible or stop the go response if stop signal is present. There is a trade-off between responding as soon as possible that will result in more error in stopping or exert proactive control by slowing the responding to go signal that will result in more error in go trials. Several experiments have reported a strategic change in responding following stop trials that may be a signature of proactive control [110,120
123]. In trials following the stop trials, go reaction time was prolonged irrespective of the outcome of the stop trials (successful or unsuccessful stop) [124]. But other studies suggest that the change in the strategic adjustment occurs only if the outcome of the preceding trial was a failure in stopping [111,120,122,123]. The strategic adjustment after an unsuccessful stop might be a post error slowing—increase in response threshold after an error [125,126]. Verbruggen and Logan [111] presented precues specifying the likelihood of appearance of stop signal in next few trials. The study found evidence supporting both the hypothesis (dual-ask requirement and proactive adjustment) proposed as a possible explanation for the increase in go reaction time. In stop context, accuracy and go reaction time both were significantly greater than accuracy and RT in

L

78

(D) The relationship among SSD, SSRT, and no-stop RT distribution. The race model predicts that only faster GO processes win the race, dividing the no-stop RT distribution into two halves. The area on the left side of the distribution (painted in dark gray) represents the noncanceled reaction time distribution. The stop-signal RT or SSRT is calculated by subtracting SSD (P 5.5) from the median no-stop RT. The average time STOP process takes to reach the threshold is estimated from SSRT. PPT, Parallel processing time; RT, response time; SSD, stop-signal delay; SSRT, stop-signal response time.

4.5 Models of movement inhibition

no-stop context. The results hold even in experiment where the context was set to change in each trial (but also see studies, e.g., [127,128] suggesting proactive adjustment occurs only at the start of stop-signal block. Subsequently, only reactive control operates trial-wise). Modeling the data under the diffusion-model framework revealed a rise in the response threshold providing evidence in support of the proactive adjustment hypothesis. The nondecision effects were partly attributed to dual-task requirements. It was concluded that both proactive adjustment (93 ms) and dual-task effect (9 ms) might be present in the prolonged no-stop RT, but the increase in RT is mostly due to proactive adjustment control. However, Bissett and Logan [129] termed the prolongation in go RT as poststop-signal adjustments are determined by both local (outcome and/or type of previous trials) and global factors (e.g., probability of noncanceled trials, likelihood of stop trials), whereas proactive adjustment is not a consequence of cumulative slowing following stop-signal trials. Many studies investigate the proactive control and its neural underpinnings taking only objective information about the probability of stop signal contained in the cue stimulus (e.g., [130
132]). But researchers have demonstrated the markedly variable nature of participants’ expectation about the stop signal in a single block [112,133]. Pas et al. [134] used an unbiased design where they provided a gap interval (in the range of 1000
2000 ms) that separated cue from other stimuli and contingent responses in the trial. The cue indicated the objective likelihood of the stop signal; subsequently participants were asked to indicate their subjective prediction of the appearance of the stop signal in forthcoming few stimuli [112,118]. They found that the subjective expectation results in the slowness in the go response. The role of objective and subjective expectation in proactive control needs to be combined to build a more complete model [134]. It is concluded that reactive and proactive control coexist together and interconnect to produce an efficient response inhibition system [112,114,115,135
139]. For a better overview about the cognitive neuroscience of stopping response including both proactive and reactive control, refer to Aron [140].

4.5.2 Estimation of stopping efficacy Popularity of race model is majorly for methods it provides to estimate the mean covert latency of the cancelation process of the response called stop-signal response time (SSRT) by using empirically recorded parameters such as RT in no-stop trials and error rate in stopping. According to race model, the shorter the SSRT, the better the ability to cancel a preplanned movement. SSRT has become an indispensable tool to evaluate the ability of response inhibition in healthy and various patient populations [141], for example, ADHD (e.g., [142
144]), impulsivity (e.g., [145,146]), Parkinson’s disease (e.g., [147]), OCD (e.g., [148]), schizophrenia (e.g., [149
151]), anxiety disorder (e.g., [152]), depression (e.g., [153]), and substance dependence (e.g., [154,155]).

79

80

CHAPTER 4 Models of making choice and control over

Methods derived from race model provide only the summary measure of SSRT (e.g., mean SSRT). However, crucial information might be gained if the entire SSRT distribution can be estimated. This has motivated researchers to adopt distribution models, for example, ex-Gaussian distribution to estimate the entire distributions (e.g., [156
158]). Matzke et al. [159] derived a method to approximate the complete distribution of latency of the stop process (SSRT) by combining the notions of race model and HDDM. In addition to independence assumption of the race model the method supposes an ex-Gaussian parametric distribution for RTs in no-stop trials and SSRTs. The method is not limited to only ex-Gaussian distribution; other parametric forms will also suffice, but exGaussian distribution is often used as a distributional model producing excellent fits to the distribution of reaction times [158,160,161]. Bayesian Hierarchical model adds an advantage to estimate SSRT distribution for each participant and group requiring relatively few observations often encountered in experimental and clinical studies. Another method to estimate the SSRT distribution without parametric assumption is the Colonius method [162], but it requires unrealistically large number of trials per subject.

4.5.3 Trigger failures In a race model account of stopping, a response can be inhibited only if fast stop process reaches the threshold before the go process. However, if a participant is unable to encode or correctly interpret the signal, the stop process cannot even be initiated or triggered in those trials. Such incidences of trigger failures (TF) create a severe theoretical and methodical challenge to the interpretation of countermanding studies [98,99]. The observed poor performance in stopping is often attributed to the SSRT, but variable nature of the stop process might be due to high rate of TFs. Moreover, it has been shown that SSRT is dramatically overestimated by standard methods if the data contains TFs [163]. A rough indication about the TFs can be inferred by the lower asymptote in inhibition function typically higher than zero in TF data. Though the inhibition function reflects the rate of TFs, it also contains other parameters, that is, mean and variance of go and stop processes [99]. Matzke et al. [159] introduced a Bayesian model to approximate the probable rate of TFs along with full distribution of SSRTs. They demonstrated that even the modest amount of TFs (B8%
9%) in data can substantially distort the estimates of SSRTs in data from two published studies [149,164]. The TF modeling method provides novel insights that change the perspective to see some basic or clinical study. For example, SSRT calculated by TF model did not correlate with measures of impulsivity or risk-seeking behavior, rather the rate of TF of stop and go processes significantly correlated with the measure of impulsivity, possibly a signature of a lapse of attention [165], otherwise often observed correlation. Weigard et al. [166] found that difficulties in stopping in countermanding task among children with ADHD are primarily induced by failure to trigger the inhibitory process instead of deficient inhibitory process. Though TF

4.5 Models of movement inhibition

provides an elegant perspective, neither it accounts for the race model violation at the shorted SSD often observed [167] nor can calculate the rate of TF for a particular SSD. Since it is fundamentally dependent on the assumption of race model, it might be inappropriate to bring TF explanation of the data if the assumptions of the race model (independence) or the parametric form (e.g., Ex-Gaussian or any other) assumed does not hold true. Despite commendable success of the race model in explaining countermanding behavior, the neural correlates of STOP rising to a threshold remain unclear. Several studies have also reported violation of the independence assumption of the race model [167
170]. Using simulation techniques Salinas and Stanford [171] pointed out that the stop signal must be detected in time to successfully withhold saccade. However, lack of empirical evidence to evaluate their CRTT model kept the relationship between the detectability of the stop signal and its likely influence on performance in countermanding task obscure. We extended the CRTT model to link the detectability of the stop signal and performance of saccade inhibition (Fig. 4.4). In the revised version of CRTT model, ongoing saccade planning starts decelerating at the onset of the stop signal, and the magnitude of deceleration is proportional to stop-signal detectability [170]. In the classical countermanding task, we found that in an uncertain situation, human participants optimized performance having their saccadic eye-movements strategically postponed until an endogenous subjective deadline. We derived a metric, namely “saccade procrastination time” or SPT, on the foundation of CRTT framework, from the fit of noncanceled RT plotted against parallel processing time (PPT) (Fig. 4.4C). When the inhibitory control failed, the average saccadic reaction time increased at an exponential rate with the PPT. PPT is the maximum time duration when both go and stop signals can be processed simultaneously before the fixation is moved. We refer to the intercept of the best fit on the ordinate as SPT. We endorse that this metric is more robust in the estimation of the ability to inhibit saccade than the conventional metric called “stop-signal response time” or SSRT. Unlike SSRT, SPT does not depend on race model’s assumption that GO and STOP are independent, which is often violated. It is also immune to stopping performance and the distribution of noncanceled saccade latencies at different SSDs (unpublished data, manuscript in review).

4.5.4 Bayesian rational decision-making The race model account of action control offers a simple, elegant description of the data produced by countermanding experiments, but it is agnostic about how different cognitive processes might be involved in the controlling process of an action [172]. There is growing evidence suggesting the critical role of cognitive processes, for example, detection [170,171], attention [173], and memory [174]. It is also suggested that the stop latency (SSRT) is influenced by different task demands, modality/intensity of stop-signal [175,176], the salience of the stop signal [177,178], presence or absence of distracters [173], reward [179], etc.

81

CHAPTER 4 Models of making choice and control over

(A)

ΔRT

(B)

RT

Activity

RT

GO

SSD

PPT

PPT (C)

Stop signal

GO

Noncanceled RT (ms)

1100

Threshold

Activity

82

Countermand

900 700 500 300 0

Stop signal

200

400

600

800

1000

Parallel processing time (ms)

FIGURE 4.4 (A) Schematic of CRTT model of saccade countermanding. Time is plotted along the abscissa. The onset of the target triggers a preparatory process (GO), shown by an arrow, which rises at a variable rate to reach a threshold (gray line) of activity. In stop-trials, after a random delay (SSD), a stop signal appears (gray arrowhead). The stop-signal onset decreases the speed with which GO rises, causing a deviation from the expected trajectory of GO (dotted line). If the deceleration is not enough strong, a saccadic eye-movement is generated, when GO reaches the threshold as indicated by a broken black line on abscissa in the upper panel. The temporal gap between saccade onset and stop-signal onset is called PPT. During this period, both the target and stop signal are processed simultaneously. Decelerated GO gives rise to saccadic RT delayed by ΔRT. The lower panel shows that the GO is prevented from reaching the threshold resulting in successful saccade inhibition when the deceleration is strong. (B) CRTT model suggests that noncanceled RT increases as an exponential function of PPT. (C) Data collected from healthy humans who performed a countermanding task supported CRTT model [170]. The intercept of the fit on the ordinate is referred to as SPT. CRTT, Cancellable rise-tothreshold; PPT, parallel processing time; RT, reaction time; SSD, stop-signal delay; SPT, saccade procrastination time.

Understanding the role of cognitive processes in the control of action is important theoretically as well as for therapeutic purpose. The deficit in inhibitory control might be due to perturbed cognitive process, for example, perception, attention, and therapy or drugs need to target the deficient cognitive process and not the inhibitory control process. For example, ADHD was believed to be a problem of deficient inhibitory process [180
182], but it is suggested that the poor attention in ADHD might give rise to inhibitory problems [165,166]. RDM model for response inhibition [172] attempts to incorporate sensory processing, choices of action, and associated costs related to each behavioral

4.5 Models of movement inhibition

outcome (e.g., error in the go response, stop error, delay in responding, and correct stop). The RDM model contains two principal constituents: (1) monitoring process—it mimics sensory perceptual inference, identity-learning, probability of stimuli, and its associated time durations. It employs hierarchical Bayesian tools to achieve these goals; (2) decision process—by using stochastic control theory, it converts moment by moment the current expectations into a choice of action by combining the accumulated sensory evidence. In RDM, there are three competing alternative choices of actions at a given point of time, that is, two likely go choice options and hold for an additional time unit. Different possible actions’ relative value depends on the experiment specific parameters (e.g., stop-signal ratio, the difficulty level of go-signal discrimination) and subject specific parameters (cost or reward valuation, learning rate). The continual decision to wait longer results in successful stopping.

4.5.5 Optimal Bayesian statistical inference Incoming sensory information of stop and go signal is integrated with priors of identity of go signal and other parameters (e.g., proportion of stop trials and SSD) by the monitoring process. Generation of sensory inputs in the form of Bayesian inference is denoted in graphical model (see Figure 1B in Ref. [172]). The model consists of two hidden variables: (1) variable d represents go-signal identity, dAf0; 1g and (2) variable s represents the trial type (stop or go), sA0; 1. Experimental parameters of the stop-signal task set the priors for s and d. The stop trials ratio is typically set to 0.25, so Pðs 5 1Þ 5 0:25 and go stimulus corresponds to two equally probable choices, for example, left or right key, so Pðd 5 1Þ 5 0:5. A stream of identical and independent inputs (iid) conditioned on d is generated in each trials, that is, x1 ; x2 ; . . . ; xt ; . . . ; where t encodes a small increment in time with reference to the onset of the go stimuli. The likelihood is quantified as pðxt jd 5 0Þ 5 f0 ðxt Þ and pðxt jd 5 1Þ 5 f1 ðxt Þ. Two Bernoulli distributions containing rate parameters, that is, qd and 1 2 qd represent f0 and f1 , respectively. The occurrence of the stop signal is encoded in a variable zt . θ specifies the onset time of stop signal, thus z1 5 . . . 5 zθ21 5 0 and zθ 5 . . . 5 1. A constant hazard rate is assumed for the onset of the stop signal such that pðθjs 5 1Þ 5 ð12λÞθ21λ . Another stream of iid conditioned on zt is generated such that pðyt jzt 5 0Þ 5 g0 ðyt Þ and pðyt jzt 5 1Þ 5 g1 ðyt Þ. Two Bernoulli distributions containing rate parameters qs and 1 2 qs represent g0 and g1 , respectively. Posterior probability of go-signal identity can be computed by Bayes’ Rule:

ptd 5

t pt21 d  f 1 ðx Þ

 5 t21 f ðxt Þ t pt21 0 d f1 ðx Þ 1 1 2 pd

t   p0d L f1 xi i51

  t p0d L f1 ðxi Þ 1 1 2 p0d L f0 ðxi Þ t

i51

i51

83

84

CHAPTER 4 Models of making choice and control over

Posterior probability for occurred stop signal can be computed iteratively: ptz 5

    g1 ðyt Þ pt21 1 1
pt21 hðtÞ z z       hðtÞ 1 g0 ðyt Þ 12pt21 ð1 2 hðtÞÞ 1 1
pt21 g1 ðyt Þ pt21 z z z

where hðtÞ is the posterior probability of occurrence of the stop signal in the next iteration (only if it has not occurred by now): hðtÞ 5

r ∙ Pðθ 5 tjs 5 1Þ rλe2λt 5 2λðt21Þ r ∙ Pðθ . t 2 1js 5 1Þ 1 ð1 2 rÞ re 1 ð1 2 r Þ

4.5.6 Decision process as optimal stochastic control A response deadline D in go-trials and opportunity cost c per unit time is assumed. A penalty is attached for committing an error (e.g., error in stopping). In this model, action space has two elements (go and wait) with associated expected costs (Q-factors):       Qtg bt 5 ct 1 cs pts 1 1 2 pts min ptd ; 1 2 ptd    

  Qtw bt 5 1fD . t11g V t11 bt11 jbt bt11 1 1fD5t11g cðt 1 1Þ 1 1 2 pts     V t bt 5 min Qtg ; Qtw

In an optimal decision the action with the smallest Q-factor is selected. The cost associated with the go response decreases over time and when it goes slightly downward, the go response is made. But in some trials due to stochastic process, pd evolves faster enough for go response before the stop signal can be processed, leading to stop error trials. In trials where cost of going is greater than cost of waiting, a successful stop trial occurs. The RDM can capture the inhibition function, RT distribution, reward influence on SSRT, etc., in countermanding data.

4.5.7 Linear approach to threshold explaining space and time model for decisions in space and time We can move gaze from one location to another only twice or thrice in a second. It poses a crucial decision of the system to make about when and where to move the gaze. A thorough understating about the gaze shifts in both temporal and spatial domains will help to elucidate the online information processing strategies of the brain to achieve ongoing behavior. Saccadic reaction time (RT) and decisions are intricately linked. Two properties of the reaction times distribution need to be explained: (1) average saccade latency (B200 ms) are longer than it should be expected by its underlying neural processes (nerve conduction time, synaptic delay, etc.) and (2) even in identical stimulus condition, RT varies randomly and

4.5 Models of movement inhibition

is positively skewed. Carpenter [40] noted that the variance in the normal distribution of the reciprocal of RT determines the skewness in the RT distribution. In LATER (linear rise-to-threshold at ergodic rate) model [183], reciprocal of RT is the rate of underlying decision-making process for elicitation of saccade. LATER models focused mainly on explaining RT variability from tasks with discrete trials and with very simple display. But in real-world scene view, there are always many targets to look. Moreover, the LATER model does not take activity at the central vision into account, but studies have shown the modulation of fixation duration with the influence of information in the central vision in scene viewing [184,185] and reading [186]. LATEST model provides a novel saccadic decision process taking both central and peripheral vision into consideration [187]. Fig. 4.5 shows a schematic of the model. The decision process weighs the comparative expected benefit for prolonging a gaze at the current location and the benefit to make a saccade to the target location. In terms of evaluating the hypothesis that the goal will be better achieved by shifting the gaze to target at periphery (GO) relative to the hypothesis that maintaining the fixation at the current location (STAY) can be expressed as log QT ðHGO ; HSTAY Þ 5 log QT21 ðHGO ; HSTAY Þ 1 supportðHGO ; HSTAY Þ, where QT and QT21 are posterior and prior odd, respectively, and “support” denotes sensory evidence being accumulated to support competing hypotheses. In this stay-or-go theoretical approach, evaluation of the decision to generate a saccade from the present fixating location to the future potential location, the average rate of decision signal’s growth (μ) can be split in two principle independent components—(1) tendency to maintain the fixation at the present location (μSTAY ) and (2) propensity to generate a saccade to a different location (μGO ). μSTAY is a function of only the information content at the current fixation location (μ1 ), whereas μGO is function of two components—sensory (visual) information existing at the target location (μ2 ) and retinocentric position corresponding to the target location taking the present fixation location as reference (μ12 ). In scene viewing, at a given point of time, there are multiple possible target locations to move the eyes across the visual field. LATEST model assumes that the visual system possess the ability to accumulate information in parallel from the whole visual field. Thus the model suggests that there is a separate stay-or-go decision signal each rising to threshold for all possible target locations. When the decision signal for any of the location reaches a threshold, a saccade is generated to that location at that point of time. Multiple stay-or-go decisions at the periphery race to reach a threshold that creates a dynamic spatial map for where to move the eye in real time. The same temporal dynamics of the decisions of stayor-go determines the timing of the saccade too. Since the information at central vision is critical for the evolution of the spatial map, the map is retinotopic in nature. LATEST model provides an elegant model where race among multiple stay-or-go decision units explains the spatial and temporal dynamic of the eye movement.

85

86

CHAPTER 4 Models of making choice and control over

FIGURE 4.5 Schematic representation of evaluation process for possible target locations in LATEST framework. μ represents the average rate of growth for decision signals in different decision units. In the figure the gaze is fixed at location 1 and stay-or-go evaluation for two potential locations; 2 and 3 are undergoing. The information at the present point of gaze (location 1) (μ1 ) determines the stay signal for all other units. The signal at different rates in different decision unit rises to reach their corresponding thresholds. The outcome of race among multiple decision units decides the location for a shift of gaze. In this schematic, a person is looking at a car (location 1), while a person on a two-wheeler waiting nearby (location 2), and a faulty streetlight is flickering. The signal of decision unit corresponding to location 3 (flickering streetlight) finish to the threshold first due to greater salience, evoking a saccade to location 3. LATEST, linear approach to threshold explaining space and time. Modified from B.W. Tatler, J.R. Brockmole, R.H.S. Carpenter, LATEST: a model of saccadic decisions in space and time, Psychol. Rev. 124 (2017) 267.

4.6 Discussion Making choices is ubiquitous in everyday behavior. Models of choice, whether prescriptive or descriptive, have traditionally emphasized selecting the most suitable option. This chapter focuses on control over thought for action (cf. [96]).

4.6 Discussion

Inhibiting the unwanted option has been extensively investigated using the stop-signal paradigm. Correspondingly, in this chapter, a number of computational models have been described that aim to explain the choice (i.e., action), the reaction time distribution, and action inhibition. The primary goal of the computational modeling approach is to improve our understanding of a specific cognitive process leading to certain behavior. Two domains of decision-making research have investigated perceptual and value-based decisions mostly independently [76]. The major challenge in bringing prevalent theories in these domains under one umbrella stems from different sources of uncertainty inherent in different types of tasks used for research on perceptual and value-based decision-making. In most of the perceptual decision-making tasks the reward remains constant across trials, while the uncertainty in stimulus discrimination varies. On the contrary, in the plurality of value-based decision-making tasks, the stimulus remains unambiguous and unchanged across trials, while the uncertainty in reward varies. Furthermore, unlike in value-based decision-making tasks, rapid response is emphasized in perceptual decision-making tasks. Computational models of making simple perceptual decision inspired by neuronal dynamics in fine spatiotemporal scale have thrived due to the advancement in techniques for simultaneous recording of activity of multiple neurons in different layers of one or more than one brain area(s) of awake and behaving nonhuman primates. The same technique is quite likely to be futile if applied in economic decision-making tasks, because these tasks are naturally complex, and training animals to perform difficult tasks in an ineffable way is nearly impossible. Till date the primary source of physiological correlates of economic decision, therefore, originates from functional imaging on human participants with high spatial resolution, however, with relatively poor temporal precision. Neurally inspired models of perceptual choice have suggested a sequential sampling approach for accumulating sensory evidence in a noisy environment that leads to a choice on exceeding a certain threshold. At the one end, decision-making can be viewed as a sensory process, while, at the other end, the choices are tightly coupled with the corresponding motor actions [188]. Accordingly, this chapter has reviewed a rich set of models at the interface of decision-making and executive control (specifically, action/response inhibition). Further, inspired by the experience-based paradigms of economic choice, we have highlighted multiple stages of decision-making in an updated belief-based account. These two approaches are complementary to the two major approaches in the studies on decision-making: good-based models and action-based models [22]. The good-based approach emphasizes computation of decision variable as an economic value that is distinct from sensorimotor aspects of choice. This model implies that decision-making mechanisms involve serial processing, suggesting that action planning begins after a choice is made. These models fall short in dynamic and uncertain environments where the availability of the goods and their values can change over time. For example, in repeated choice scenarios, feedback about outcomes can be used to update the beliefs about the competing alternatives [87]. The action-based approach suggests that choices are made by parallel

87

88

CHAPTER 4 Models of making choice and control over

activation among competing alternatives until one of them is selected. A contrasting approach to the action-based theory is that decision-making can be considered to be abstract in nature. This view is particularly relevant when considering cognitive tasks that involve visual categorization (reviewed in Ref. [189]) or judgment and not necessarily require making a choice (i.e., action selection). How goods’ values and action’s cost are integrated is of paramount importance for models of choice. This chapter expanded on the action-based approach and emphasized control of thought over actions. More specifically, action inhibition accounts for how choices are made by not only competitively selecting the most suitable option but also inhibiting the unwanted options. Similar suggestions have earlier been proposed in good-based approaches from the perspective of counterfactual reasoning. For instance, the regret theory provides an alternative explanation of how the prospects of unobtained outcomes influence future and current choices. Models of reinforcement learning have variants that include fictive learning signals from unobtained outcomes (see Ref. [18]). Christopoulos et al. [190] proposed a biologically plausible approach that integrated dynamic neural fields for cue perception, motor planning, and valuation of goods and action. This approach relies on and extends the sequential sampling models and uses reinforcement learning for valuation. In an attempt to bridge the gap between theoretical frameworks for two types of decision-making, Busemeyer and Townsend [70] introduced DFT for an economic decision that in principle is built on sequential sampling of information like what the diffusion process does to mimic neural activity during perceptual decision [19,191]. In DFT, attention-weighted samples of attributes of goods are accumulated to make an economic choice. The accumulation of information has been at the core of many models to encompass different aspects of complex value-based decision, for example, multialternative DFT [192], leaky competitive accumulation for loss aversion and inhibition [72], and multiattribute preference accumulation for the rating task [193]. Given that the estimation of the weighted average of contributions from all attributes may be time-consuming, recent studies suggest that we may just focus on a subset of attributes and resort to “take-thebest” heuristic for quicker decision [194]. However, a recent parallel connectionist accumulator model suggests that there is an automatic means that considers all attributes yet makes a rapid decision [195,196]. From the models described in this chapter, we propose an extension that includes inhibitory control and repeated decision-making scenarios. Thus both our past experiences with the alternatives and a competitive selection among current alternatives would determine the choice. While the past experience would update the priors or beliefs, the inhibition, and selection processes would contribute to the computation of decision variables [197]. Although this proposal is not new, we are suggesting that there is a need for developing both paradigms and computational models that integrate multiple aspects of decision-making. This would contribute toward an understanding of mechanisms underlying decision-making, and cognition in general [198].

References

Conflict of interest The authors declare no competing financial interests.

Acknowledgments The first author was supported by Junior Research Fellowship by University Grants Commission, India. The corresponding author thanks Wellcome Trust India Alliance for the research grant and fellowship.

References [1] E.K. Miller, The prefrontal cortex and cognitive control, Nat. Rev. Neurosci. 1 (1) (2000) 59. [2] E.K. Miller, J.D. Cohen, An integrative theory of prefrontal cortex function, Annu. Rev. Neurosci. 24 (2001) 167
202. [3] A. Miyake, P. Shah, Models of Working Memory: Mechanisms of Active Maintenance and Executive Control., Cambridge University Press, 1999. [4] S. Monsell, J. Driver (Eds.), Control of Cognitive Processes: Attention and Performance XVIII, vol. 18, MIT Press, 2000. [5] A. Baddeley, Exploring the central executive, Q. J. Exp. Psychol. Sect. A 49 (1996) 5
28. [6] A.D. Baddeley, S. Della Sala, Working memory and executive control, Philos. Trans. R. Soc. London, Ser. B Biol. Sci. 351 (1996) 1397
1404. [7] W.W. Seeley, V. Menon, A.F. Schatzberg, J. Keller, G.H. Glover, H. Kenna, et al., Dissociable intrinsic connectivity networks for salience processing and executive control, J. Neurosci. 27 (2007) 2349
2356. [8] R.M. Shiffrin, Perspectives on modeling in cognitive science, Top. Cogn. Sci. 2 (4) (2010) 736
750. [9] R. Sun (Ed.), The Cambridge Handbook of Computational Psychology (Cambridge Handbooks in Psychology)., Cambridge University Press, Cambridge, 2008. Available from: ,https://doi.org/10.1017/CBO9780511816772.. [10] G. Houghton, Connectionist Models in Cognitive Psychology, Psychology Press, 2004. [11] K. Doya, S. Ishii, A. Pouget, R.P.N. Rao, Bayesian Brain: Probabilistic Approaches to Neural Coding., MIT press, 2007. [12] R.D. Beer, Dynamical systems and embedded cognition, in: K. Frankish & W. Ramsey (Eds.), The Cambridge Handbook of Artificial Intelligence. Cambridge University Press. (812) 2014, 856
873. [13] J.R. Anderson, C.J. Lebiere, The Atomic Components of Thought. Psychology Press, 2014. [14] C. Lebiere, R.C. O’Reilly, D.J. Jilk, N. Taatgen, J.R. Anderson, The SAL integrated cognitive architecture, in: AAAI Fall Symposium: Biologically Inspired Cognitive Architectures, 2008, pp. 98
104.

89

90

CHAPTER 4 Models of making choice and control over

[15] S. Wernier, R. Sun, An overview of hybrid neural systems, in: S. Wernier, R. Sun (Eds.), Hybrid Neural Syst., Berlin, Germany, 2000, pp. 1
13. [16] K. Lamberts, R. Goldstone, Handbook of Cognition, Sage, 2004. [17] J.L. McClelland, The place of modeling in cognitive science, Top. Cogn. Sci. 1 (2009) 11
38. [18] K.P. Miyapuram, V.S.C. Pammi, Understanding decision neuroscience: a multidisciplinary perspective and neural substrates, Progress in Brain Research., Elsevier, 2013, pp. 239
266. [19] J.D. Schall, Neural basis of deciding, choosing and acting, Nat. Rev. Neurosci. 2 (2001) 33. [20] M. Chawla, K.P. Miyapuram, Context-sensitive computational mechanisms of decision making, J. Exp. Neurosci. 12 (2018). Available from: https://doi.org/10.1177/ 1179069518809057. [21] C. Summerfield, K. Tsetsos, Building bridges between perceptual and economic decision-making: neural and computational mechanisms, Front. Neurosci. 6 (2012) 70. [22] N.J. Wispinski, J.P. Gallivan, C.S. Chapman, Models, movements, and minds: bridging the gap between decision making and action, Ann. N. Y. Acad. Sci. (2018). [23] P.R. Montague, B. King-Casas, J.D. Cohen, Imaging valuation models in human choice, Annu. Rev. Neurosci. 29 (2006) 417
448. [24] C. Padoa-Schioppa, Neurobiology of economic choice: a good-based model, Annu. Rev. Neurosci. 34 (2011) 333
359. [25] L.P. Sugrue, G.S. Corrado, W.T. Newsome, Choosing the greater of two goods: neural currencies for valuation and decision making, Nat. Rev. Neurosci. 6 (2005) 363. [26] J.I. Gold, M.N. Shadlen, The neural basis of decision making, Annu. Rev. Neurosci. 30 (2007) 535
574. [27] J.I. Gold, A.A. Stocker, Visual decision-making in an uncertain and dynamic world, Annu. Rev. Vis. Sci. 3 (2017) 227
250. [28] T.D. Hanks, C. Summerfield, Perceptual decision making in rodents, monkeys, and humans, Neuron 93 (2017) 15
31. [29] M.J. Mulder, L. Van Maanen, B.U. Forstmann, Perceptual decision neurosciences—a model-based review, Neuroscience 277 (2014) 872
884. [30] R. Ratcliff, P.L. Smith, S.D. Brown, G. McKoon, Diffusion decision model: current issues and history, Trends Cogn. Sci. 20 (2016) 260
281. [31] R. Ratcliff, P.L. Smith, A comparison of sequential sampling models for two-choice reaction time, Psychol. Rev. 111 (2) (2004) 333. [32] B.U. Forstmann, R. Ratcliff, E.J. Wagenmakers, Sequential sampling models in cognitive neuroscience: advantages, applications, and extensions, Annu. Rev. Psychol. 67 (2016) 641
666. [33] J.I. Gold, M.N. Shadlen, Banburismus and the brain: decoding the relationship between sensory stimuli, decisions, and reward, Neuron 36 (2002) 299
308. [34] T.V. Wiecki, I. Sofer, M.J. Frank, HDDM: hierarchical Bayesian estimation of the drift-diffusion model in python, Front. Neuroinform. 7 (2013) 14. [35] A. Diederich, J.R. Busemeyer, Modeling the effects of payoff on response bias in a perceptual discrimination task: bound-change, drift-rate-change, or two-stageprocessing hypothesis, Percept. Psychophys. 68 (2006) 194
207.

References

[36] E.-J. Wagenmakers, A practical solution to the pervasive problems of p values, Psychon. Bull. Rev. 14 (2007) 779
804. [37] R. Ratcliff, F. Tuerlinckx, Estimating parameters of the diffusion model: approaches to dealing with contaminant reaction times and parameter variability, Psychon. Bull. Rev. 9 (2002) 438
481. [38] D. Gamerman, H.F. Lopes, Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference, Chapman and Hall/CRC, 2006. [39] S.D. Brown, A. Heathcote, The simplest complete model of choice response time: Linear ballistic accumulation, Cogn. Psychol. 57 (2008) 153
178. [40] R.H.S. Carpenter, Oculomotor procrastination, in: D.F. Fisher, R.A. Monty, J.W. Senders (Eds.), Eye Movements: Cognition and Visual Perception, 1981. [41] M. Usher, J.L. McClelland, The time course of perceptual choice: the leaky, competing accumulator model, Psychol. Rev. 108 (2001) 550. [42] R. Bogacz, E. Brown, J. Moehlis, P. Holmes, J.D. Cohen, The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks, Psychol. Rev. 113 (2006) 700. [43] R. Ratcliff, R. Childers, Individual differences and fitting methods for the two-choice diffusion model of decision making, Decision 2 (2015) 237. [44] C. Donkin, S. Brown, A. Heathcote, E.-J. Wagenmakers, Diffusion versus linear ballistic accumulation: different models but the same conclusions about psychological processes? Psychon. Bull. Rev. 18 (2011) 61
69. [45] D. Mahesan, M. Chawla, K.P. Miyapuram, The effect of reward information on perceptual decision-making, International Conference on Neural Information Processing, Springer, 2016, pp. 156
163. [46] D. van Ravenzwaaij, M. Mulder, F. Tuerlinckx, E.-J. Wagenmakers, Do the dynamics of prior information depend on task context? An analysis of optimal performance and an empirical test, Front. Psychol. 3 (2012) 132. [47] T.R. Stanford, S. Shankar, D.P. Massoglia, M.G. Costello, E. Salinas, Perceptual decision making in less than 30 milliseconds, Nat. Neurosci. 13 (2010) 379. [48] C. Bodelo´n, M. Fallah, J.H. Reynolds, Temporal resolution for the perception of features and conjunctions, J. Neurosci. 27 (2007) 725
730. [49] B.V. Saxberg, Projected free fall trajectories. II. Human experiments, Biol. Cybern. 56 (1987) 177
184. [50] J.T. Todd, Visual information about moving objects, J. Exp. Psychol. Hum. Percept. Perform. 7 (1981) 795. [51] T.G. Babler, J.L. Dannemiller, Role of image acceleration in judging landing location of free-falling projectiles, J. Exp. Psychol. Hum. Percept. Perform. 19 (1993) 15. [52] G. Gigerenzer, W. Gaissmaier, Heuristic decision making, Annu. Rev. Psychol. 62 (2011) 451
482. [53] H.A. Simon, What is an “explanation” of behavior? Psychol. Sci. 3 (1992) 150
161. [54] G. Gigerenzer, Gut Feelings: The Intelligence of the Unconscious, Penguin, 2007. [55] J. Haidt, The emotional dog and its rational tail: a social intuitionist approach to moral judgment, Psychol. Rev. 108 (2001) 814. [56] S.A. Sloman, The empirical case for two systems of reasoning, Psychol. Bull. 119 (1996) 3. [57] S. Chaiken, Y. Trope, Dual-Process Theories in Social Psychology, Guilford Press, 1999.

91

92

CHAPTER 4 Models of making choice and control over

[58] S. Epstein, A. Lipson, C. Holstein, E. Huh, Irrational reactions to negative outcomes: evidence for two conceptual systems, J. Pers. Soc. Psychol. 62 (1992) 328. [59] J.S.B.T. Evans, Intuition and reasoning: a dual-process perspective, Psychol. Inq. 21 (2010) 313
326. Available from: https://doi.org/10.1080/1047840X.2010.52105. [60] M.B. Ferreira, L. Garcia-marques, S.J. Sherman, J.W. Sherman, Automatic and controlled components of judgment and decision making, J. Pers. Soc. Psychol. 91 (2006) 797
813. Available from: https://doi.org/10.1037/0022-3514.91.5.797. [61] D. Kahneman, A perspective on judgment and choice: mapping bounded rationality, Am. Psychol. 58 (2003) 697. [62] G. Gigerenzer, H. Brighton, Homo heuristicus: why biased minds make better inferences, Top. Cogn. Sci. 1 (2009) 107
143. [63] G. Gigerenzer, T. Regier, How do we tell an association from a rule? Comment on Sloman, Psychol. Bull. 119 (1996) 23
26. [64] G. Keren, Y. Schul, Two is not always better than one: a critical evaluation of twosystem theories, Perspect. Psychol. Sci. 4 (2009) 533
550. [65] A.W. Kruglanski, G. Gigerenzer, Intuitive and deliberate judgments are based on common principles, Psychol. Rev. 118 (2018) 97. [66] W. De Neys (Ed.), Dual Process Theory, Routledge, London, 2018. 2.0. [67] J.S.B.T. Evans, K. Frankish, In Two Minds: Dual Processes and Beyond, Oxford University Press, Oxford, 2009. [68] A. Rangel, C. Camerer, P.R. Montague, A framework for studying the neurobiology of value-based decision making, Nat. Rev. Neurosci. 9 (2008) 545. [69] K. Doya, Modulators of decision making, Nat. Neurosci. 11 (2008) 410. [70] J.R. Busemeyer, J.T. Townsend, Decision field theory: a dynamic-cognitive approach to decision making in an uncertain environment, Psychol. Rev. 100 (1993) 432. [71] J.R. Busemeyer, S. Gluth, J. Rieskamp, B.M. Turner, Cognitive and neural bases of multi-attribute, multi-alternative, value-based decisions, Trends Cogn. Sci. 23 (2019) 251
263. [72] M. Usher, J.L. McClelland, Loss aversion and inhibition in dynamical models of multialternative choice, Psychol. Rev. 111 (2004) 757. [73] I. Krajbich, A. Rangel, Multialternative drift-diffusion model predicts the relationship between visual fixations and choice in value-based decisions, Proc. Natl. Acad. Sci. U.S.A. 108 (2011) 13852
13857. [74] S. Bitzer, H. Park, F. Blankenburg, S.J. Kiebel, Perceptual decision making: driftdiffusion model is equivalent to a Bayesian model, Front. Hum. Neurosci. 8 (2014) 102. [75] M.L. Pedersen, M.J. Frank, G. Biele, The drift diffusion model as the choice rule in reinforcement learning, Psychon. Bull. Rev. 24 (2017) 1234
1251. [76] S.K. Lynn, J.B. Wormwood, L.F. Barrett, K.S. Quigley, Decision making from economic and signal detection perspectives: development of an integrated framework, Front. Psychol. 6 (2015) 952. [77] T. Rakow, B.R. Newell, Degrees of uncertainty: an overview and framework for future research on experience-based choice, J. Behav. Decis. Making 23 (2010) 1
14. [78] G. Barron, I. Erev, Small feedback-based decisions and their limited correspondence to description-based decisions, J. Behav. Decis. Making 16 (2003) 215
233.

References

[79] I. Erev, E. Ert, A.E. Roth, E. Haruvy, S.M. Herzog, R. Hau, et al., A choice prediction competition: choices from experience and from description, J. Behav. Decis. Making 23 (2010) 15
47. [80] R. Hertwig, G. Barron, E.U. Weber, I. Erev, Decisions from experience and the effect of rare events in risky choice, Psychol. Sci. 15 (2004) 534
539. [81] R. Hertwig, I. Erev, The description
experience gap in risky choice, Trends Cogn. Sci. 13 (2009) 517
523. [82] A. Tversky, D. Kahneman, Advances in prospect theory: cumulative representation of uncertainty, J. Risk Uncertainty 5 (1992) 297
323. [83] E.U. Weber, S. Shafir, A.-R. Blais, Predicting risk sensitivity in humans and lower animals: risk as variance or coefficient of variation, Psychol. Rev. 111 (2004) 430. [84] G. Barron, E. Yechiam, The coexistence of overestimation and underweighting of rare events and the contingent recency effect, Judgm. Decis. Making 4 (2009) 447. [85] T. Lejarraga, C. Gonzalez, Effects of feedback and complexity on repeated decisions from description, Organ. Behav. Hum. Decis. Process. 116 (2011) 286
295. [86] L. Weiss-Cohen, E. Konstantinidis, M. Speekenbrink, N. Harvey, Incorporating conflicting descriptions into decisions from experience, Organ. Behav. Hum. Decis. Process. 135 (2016) 55
69. [87] S. Goyal, K.P. Miyapuram, Feedback influences discriminability and attractiveness components of probability weighting in descriptive choice under risk, Front. Psychol. 10 (2019) 962. [88] R.K. Jessup, A.J. Bishara, J.R. Busemeyer, Feedback produces divergence from prospect theory in descriptive choice, Psychol. Sci. 19 (2008) 1015
1022. [89] C.R. Fox, A. Tversky, A belief-based account of decision under uncertainty, Manage. Sci. 44 (1998) 879
895. [90] T. Rakow, K.A. Demes, B.R. Newell, Biased samples not mode of presentation: Reexamining the apparent underweighting of rare events in experience-based choice, Organ. Behav. Hum. Decis. Process. 106 (2008) 168
179. [91] L. Hadar, C.R. Fox, Information asymmetry in decision from description versus decision from experience, Judgm. Decis. Making 4 (2009) 317. [92] A.R. Camilleri, B. Newell, Within-subject preference reversals in description-and experience-based choice, in: Proceedings of the Annual Meeting of the Cognitive Science Society, 2009. [93] C. Ungemach, N. Chater, N. Stewart, Are probabilities overweighted or underweighted when rare outcomes are experienced (rarely)? Psychol. Sci. 20 (2009) 473
479. [94] R. Hau, T.J. Pleskac, J. Kiefer, R. Hertwig, The description
experience gap in risky choice: the role of sample size and experienced probabilities, J. Behav. Decis. Making 21 (2008) 493
518. [95] D.U. Wulff, M. Mergenthaler-Canseco, R. Hertwig, A meta-analytic review of two modes of learning and the description-experience gap, Psychol. Bull. 144 (2018) 140. [96] C.G. Coutlee, S.A. Huettel, The functional neuroanatomy of decision making: prefrontal control of thought and action, Brain Res. 1428 (2012) 3
12. [97] S. Ray, S.J. Heinen, A mechanism for decision rule discrimination by supplementary eye field neurons, Exp. Brain Res. 233 (2015) 459
476.

93

94

CHAPTER 4 Models of making choice and control over

[98] G.D. Logan, On the ability to inhibit thought and action: a users’ guide to the stop signal paradigm, in: D. Dagenbach, T.H. Carr (Eds.), Inhibitory Processes in Attention, Memory, and Language, Academic Press, San Diego, CA, 1994, pp. 189
239. [99] G.D. Logan, W.B. Cowan, On the ability to inhibit thought and action: a theory of an act of control, Psychol. Rev. 91 (1984) 295
327. Available from: https://doi. org/10.1037/0033-295X.91.3.295. [100] D. Matzke, F. Verbruggen, G.D. Logan, The stop-signal paradigm, Stevens’ Handb. Exp. Psychol. Cogn. Neurosci, vol. 5, 2018, pp. 1
45. [101] M.A. Vince, The intermittency of control movements and the psychological refractory period, Br. J. Psychol. Gen. Sect. 38 (1948) 149
157. [102] J.S. Lappin, C.W. Eriksen, Use of a delayed signal to stop a visual reaction-time response, J. Exp. Psychol. 72 (1966) 805. [103] V. Cutsuridis, Behavioural and computational varieties of response inhibition in eye movements, Philos. Trans. R. Soc., B 372 (2017). Available from: https://doi.org/ 10.1098/rstb.2016.0196. [104] J.D. Schall, T.J. Palmeri, G.D. Logan, Models of inhibitory control, Philos. Trans. R. Soc. B 372 (2017). Available from: https://doi.org/10.1098/rstb.2016.0193. [105] L. Boucher, T.J. Palmeri, G.D. Logan, J.D. Schall, Inhibitory control in mind and brain: an interactive race model of countermanding saccades, Psychol. Rev. 114 (2007) 376. [106] C.-C. Lo, L. Boucher, M. Pare´, J.D. Schall, X.-J. Wang, Proactive inhibitory control and attractor dynamics in countermanding action: a spiking neural circuit model, J. Neurosci. 29 (2009) 9059
9071. [107] K. Wong-lin, P. Eckhoff, P. Holmes, J.D. Cohen, Optimal performance in a countermanding saccade task, Brain Res. 1318 (2009) 178
187. Available from: https:// doi.org/10.1016/j.brainres.2009.12.018. [108] G.D. Logan, M. Yamaguchi, J.D. Schall, T.J. Palmeri, Inhibitory control in mind and brain 2.0: blocked-input models of saccadic countermanding, Psychol. Rev. 122 (2015) 115. [109] K.Z. Xu, B.A. Anderson, E.E. Emeric, A.W. Sali, V. Stuphorn, S. Yantis, et al., Neural basis of cognitive control over movement inhibition: human fMRI and primate electrophysiology evidence, Neuron 96 (6) (2017) 1447
1458. [110] F. Verbruggen, Proactive inhibitory control: a general biasing account, 2016. Available from: ,https://doi.org/10.1016/j.cogpsych.2016.01.004.. [111] F. Verbruggen, G.D. Logan, Proactive adjustments of response strategies in the stop-signal paradigm, J. Exp. Psychol. Hum. Percept. Perform. 35 (2009) 835. [112] M. Vink, R. Kaldewaij, B.B. Zandbelt, P. Pas, S. du Plessis, The role of stop-signal probability and expectation in proactive inhibition, Eur. J. Neurosci. 41 (2015) 1086
1094. [113] M. Vink, B.B. Zandbelt, T. Gladwin, M. Hillegers, J.M. Hoogendam, W.P.M. van den Wildenberg, et al., Frontostriatal activity and connectivity increase during proactive inhibition across adolescence and early adulthood, Hum. Brain Mapp. 35 (2014) 4415
4427. [114] J. Chikazoe, K. Jimura, S. Hirose, K. Yamashita, Y. Miyashita, S. Konishi, Preparation to inhibit a response complements response inhibition during performance of a stop-signal task, J. Neurosci. 29 (2009) 15870
15877.

References

[115] B. Ballanger, Top-down control of saccades as part of a generalized model of proactive inhibitory control, J. Neurophysiol. 102 (2009) 2578
2580. [116] M. Vink, R.S. Kahn, M. Raemaekers, M. van den Heuvel, M. Boersma, N.F. Ramsey, Function of striatum beyond inhibition and execution of motor responses, Hum. Brain Mapp. 25 (2005) 336
344. [117] T.S. Braver, The variable nature of cognitive control: a dual mechanisms framework, Trends Cogn. Sci. 16 (2012) 106
113. [118] I. Kleerekooper, S.J.H. van Rooij, W.P.M. van den Wildenberg, M. de Leeuw, R.S. Kahn, M. Vink, The effect of aging on fronto-striatal reactive and proactive inhibitory control, NeuroImage 132 (2016) 51
58. [119] M. Vink, N.F. Ramsey, M. Raemaekers, R.S. Kahn, Striatal dysfunction in schizophrenia and unaffected relatives, Biol. Psychiatry 60 (2006) 32
39. [120] R.J. Schachar, S. Chen, G.D. Logan, T.J. Ornstein, J. Crosbie, A. Ickowicz, et al., Evidence for an error monitoring deficit in attention deficit hyperactivity disorder, J. Abnorm. Child Psychol. 32 (2004) 285
293. [121] K.M. Sharika, S.F.W. Neggers, T.P. Gutteling, S. Van der Stigchel, H.C. Dijkerman, A. Murthy, Proactive control of sequential saccades in the human supplementary eye field, Proc. Natl. Acad. Sci. U.S.A. 110 (2013) E1311
E1320. [122] F. Verbruggen, G.D. Logan, B. Liefooghe, A. Vandierendonck, Short-term aftereffects of response inhibition: repetition priming or between-trial control adjustments? J. Exp. Psychol. Hum. Percept. Perform. 34 (2008) 413. [123] E.E. Emeric, J.W. Brown, L. Boucher, R.H.S. Carpenter, D.P. Hanes, R. Harris, et al., Influence of history on saccade countermanding performance in humans and macaque monkeys, Vision Res. 47 (2007) 35
49. [124] M. Rieger, S. Gauggel, Inhibitory after-effects in the stop signal paradigm, Br. J. Psychol. 90 (1999) 509
518. [125] P.M.A. Rabbitt, Three kinds of error-signalling responses in a serial choice task, Q. J. Exp. Psychol. 20 (1968) 179
188. [126] G.J.M. Van Boxtel, M.W. Van Der Molen, J.R. Jennings, Differential involvement of the anterior cingulate cortex in performance monitoring during a stop-signal task, J. Psychophysiol. 19 (2005) 1
10. [127] G. Rinkenauer, A. Osman, R. Ulrich, H. Mu¨ller-Gethmann, S. Mattes, On the locus of speed-accuracy trade-off in reaction time: inferences from the lateralized readiness potential, J. Exp. Psychol. Gen. 133 (2004) 261. [128] V. Stuphorn, J.D. Schall, Executive control of countermanding saccades by the supplementary eye field, Nat. Neurosci. 9 (2006) 925. [129] P.G. Bissett, G.D. Logan, Balancing cognitive demands: control adjustments in the stop-signal paradigm, J. Exp. Psychol. Learn. Mem. Cogn. 37 (2011) 392. [130] J. Duque, L. Labruna, S. Verset, E. Olivier, R.B. Ivry, Dissociating the role of prefrontal and premotor cortices in controlling inhibitory mechanisms during motor preparation, J. Neurosci. 32 (2012) 806
816. [131] S. Jahfari, C.M. Stinear, M. Claffey, F. Verbruggen, A.R. Aron, Responding with restraint: what are the neurocognitive mechanisms? J. Cogn. Neurosci. 22 (2010) 1479
1492. [132] J. Van Belle, M. Vink, S. Durston, B.B. Zandbelt, Common and unique neural networks for proactive and reactive response inhibition revealed by independent component analysis of functional MRI data, NeuroImage 103 (2014) 65
74.

95

96

CHAPTER 4 Models of making choice and control over

[133] P. Pas, H.E. Van den Munkhof, S. Du Plessis, M. Vink, Striatal activity during reactive inhibition is related to the expectation of stop-signals, Neuroscience 361 (2017) 192
198. [134] P. Pas, S.D. Plessis, H.E. van den Munkhof, T.E. Gladwin, M. Vink, Using subjective expectations to model the neural underpinnings of proactive inhibition, Eur. J. Neurosci. 49 (2019) 1575
1586. [135] F. Verbruggen, G.D. Logan, Response inhibition in the stop-signal paradigm, Trends Cogn. Sci. 12 (2008) 418
424. [136] X. Chen, K.W. Scangos, V. Stuphorn, Supplementary motor area exerts proactive and reactive control of arm movements, J. Neurosci. 30 (2010) 14657
14675. [137] J. Duque, D. Lew, R. Mazzocchio, E. Olivier, R.B. Ivry, Evidence for two concurrent inhibitory mechanisms during response preparation, J. Neurosci. 30 (2010) 3793
3802. [138] F. Karayanidis, S. Jamadar, H. Ruge, N. Phillips, A. Heathcote, B.U. Forstmann, Advance preparation in task-switching: converging evidence from behavioral, brain activation, and model-based approaches, Front. Psychol. 1 (2010) 25. [139] J.A. King, F.M. Korb, D.Y. von Cramon, M. Ullsperger, Post-error behavioral adjustments are facilitated by activation and suppression of task-relevant and taskirrelevant information processing, J. Neurosci. 30 (2010) 12759
12769. [140] A.R. Aron, From reactive to proactive and selective control: developing a richer model for stopping inappropriate responses, Biol. Psychiatry 69 (2011) e55
e68. [141] J. Lipszyc, R. Schachar, Inhibitory control and psychopathology: a meta-analysis of studies using the stop signal task, J. Int. Neuropsychol. Soc. 16 (2010) 1064
1076. [142] R.M. Alderson, M.D. Rapport, M.J. Kofler, Attention-deficit/hyperactivity disorder and behavioral inhibition: a meta-analytic review of the stop-signal paradigm, J. Abnorm. Child Psychol. 35 (2007) 745
758. [143] E.M. Bekker, J.L. Kenemans, M.R. Hoeksma, D. Talsma, M.N. Verbaten, The pure electrophysiology of stopping, Int. J. Psychophysiol. 55 (2005) 191
198. [144] D.P. Munoz, I.T. Armstrong, K.A. Hampton, K.D. Moore, Altered control of visual fixation and saccadic eye movements in attention-deficit hyperactivity disorder, J. Neurophysiol. 90 (2003) 503
514. [145] L.J. Castro-Meneses, B.W. Johnson, P.F. Sowman, The effects of impulsivity and proactive inhibition on reactive inhibition and the go process: insights from vocal and manual stop signal tasks, Front. Hum. Neurosci. 9 (2015) 529. [146] G.D. Logan, R.J. Schachar, R. Tannock, Impulsivity and inhibitory control, Psychol. Sci. 8 (1997) 60
64. [147] P. Joti, S. Kulashekhar, M. Behari, A. Murthy, Impaired inhibitory oculomotor control in patients with Parkinson’s disease, Exp. Brain Res. 177 (2007) 447
457. [148] S.R. Chamberlain, N.A. Fineberg, A.D. Blackwell, T.W. Robbins, B.J. Sahakian, Motor inhibition and cognitive flexibility in obsessive-compulsive disorder and trichotillomania, Am. J. Psychiatry 163 (2006) 1282
1284. [149] J.C. Badcock, P.T. Michie, L. Johnson, J. Combrinck, Acts of control in schizophrenia: dissociating the components of inhibition, Psychol. Med. 32 (2002) 287
297. [150] D. Matzke, M. Hughes, J.C. Badcock, P. Michie, A. Heathcote, Failures of cognitive control or attention? The case of stop-signal deficits in schizophrenia, Atten. Percept. Psychophys. 79 (2017) 1078
1086. Available from: https://doi.org/ 10.3758/s13414-017-1287-8.

References

[151] K.N. Thakkar, J.D. Schall, L. Boucher, G.D. Logan, S. Park, Response inhibition and response monitoring in a saccadic countermanding task in schizophrenia, Biol. Psychiatry 69 (2011) 55
62. [152] A.N. Savostyanov, A.C. Tsai, M. Liou, E.A. Levin, J.-D. Lee, A.V. Yurganov, et al., EEG-correlates of trait anxiety in the stop-signal paradigm, Neurosci. Lett. 449 (2009) 112
116. [153] M.A. Lau, B.K. Christensen, L.L. Hawley, M.S. Gemar, Z.V. Segal, Inhibitory deficits for negative information in persons with major depressive disorder, Psychol. Med. 37 (2007) 1249
1259. [154] M.T. Fillmore, C.R. Rush, Impaired inhibitory control of behavior in chronic cocaine users, Drug Alcohol Depend. 66 (2002) 265
273. [155] C.R. Li, X. Luo, P. Yan, K. Bergquist, R. Sinha, Altered impulse control in alcohol dependence: neural measures of stop signal performance, Alcohol Clin. Exp. Res. 33 (2009) 740
750. [156] D.A. Balota, M.J. Yap, Moving beyond the mean in studies of mental chronometry: the power of response time distributional analyses, Curr. Dir. Psychol. Sci. 20 (2011) 160
166. [157] D. Matzke, E.-J. Wagenmakers, Psychological interpretation of the ex-Gaussian and shifted Wald parameters: a diffusion model analysis, Psychon. Bull. Rev. 16 (2009) 798
817. [158] R. Ratcliff, B.B. Murdock, Retrieval processes in recognition memory, Psychol. Rev. 83 (1976) 190. [159] D. Matzke, C.V. Dolan, G.D. Logan, S.D. Brown, E.J. Wagenmakers, Bayesian parametric estimation of stop-signal reaction time distributions, J. Exp. Psychol. Gen. 142 (2013) 1047
1073. Available from: https://doi.org/10.1037/a0030543. [160] A. Heathcote, S.J. Popiel, D.J. Mewhort, Analysis of response time distributions: an example using the Stroop task, Psychol. Bull. 109 (1991) 340. [161] W.E. Hockley, Retrieval processes in continuous recognition, J. Exp. Psychol. Learn. Mem. Cogn. 8 (1982) 497. [162] H. Colonius, A note on the stop-signal paradigm, or how to observe the unobservable, Psychol. Rev. 97 (1990) 309. [163] G.P.H. Band, M.W. Van Der Molen, G.D. Logan, Horse-race model simulations of the stop-signal procedure, Acta Psychol. (Amst) 112 (2003) 105
142. [164] M.E. Hughes, W.R. Fulham, P.J. Johnston, P.T. Michie, Stop-signal response inhibition in schizophrenia: behavioural, event-related potential and functional neuroimaging data, Biol. Psychol. 89 (2012) 220
231. [165] P. Skippen, D. Matzke, A. Heathcote, W.R. Fulham, P. Michie, F. Karayanidis, Reliability of triggering inhibitory process is a better predictor of impulsivity than SSRT, Acta Psychol. (Amst) 192 (2019) 104
117. [166] A. Weigard, A. Heathcote, D. Matzke, C. Huang-Pollock, Cognitive modeling suggests that attentional failures drive longer stop-signal reaction time estimates in attention deficit/hyperactivity disorder, Clin. Psychol. Sci. 7 (2019) 856
872. 2167702619838466. [167] P.G. Bissett, R.A. Poldrack, G.D. Logan, Severe violations of independence in response inhibition tasks are pervasive and consequential, 2019. Available from: ,https://doi.org/10.31234/osf.io/kpa65..

97

98

CHAPTER 4 Models of making choice and control over

¨ zyurt, H. Colonius, P.A. Arndt, Countermanding saccades: evidence against [168] J. O independent processing of go and stop signals, Percept. Psychophys. 65 (3) (2003) 420
428. ¨ zyurt, P.A. Arndt, Countermanding saccades with auditory stop [169] H. Colonius, J. O signals: testing the race model, Vision Res. 41 (2001) 1951
1968. [170] I. Indrajeet, S. Ray, Detectability of stop-signal determines magnitude of deceleration in saccade planning, Eur. J. Neurosci. 49 (2019) 232
249. [171] E. Salinas, T.R. Stanford, The countermanding task revisited: fast stimulus detection is a key determinant of psychophysical performance, J. Neurosci. 33 (2013) 5668
5685. [172] P. Shenoy, J.Y. Angela, R.P. Rao, A rational decision making framework for inhibitory control, Adv. Neural. Inf. Process. Syst., 2010, pp. 2146
2154. [173] F. Verbruggen, T. Stevens, C.D. Chambers, Proactive and reactive stopping when distracted: an attentional account, J. Exp. Psychol. Hum. Percept. Perform. 40 (2014) 1295
1300. [174] P. Chiappe, L.S. Siegel, L. Hasher, Working memory, inhibitory control, and reading disability, Mem. Cognit. 28 (2000) 8
17. [175] S. Morein-Zamir, A. Kingstone, Fixation offset and stop signal intensity effects on saccadic countermanding: a crossmodal investigation, Exp. Brain Res. 175 (2006) 453
462. [176] M. Van Der Schoot, R. Licht, T.M. Horsley, J.A. Sergeant, Effects of stop signal modality, stop signal intensity and tracking method on inhibitory performance as determined by use of the stop signal paradigm, Scand. J. Psychol. 46 (2005) 331
341. [177] R. Montanari, M. Giamundo, E. Brunamonti, S. Ferraina, P. Pani, Visual salience of the stop-signal affects movement suppression process, Exp. Brain Res. 235 (2017) 2203
2214. [178] P. Pani, F. Giarrocco, M. Giamundo, R. Montanari, E. Brunamonti, S. Ferraina, Visual salience of the stop signal affects the neuronal dynamics of controlled inhibition, Sci. Rep. 8 (2018) 14265. [179] L.A. Leotti, T.D. Wager, Motivational influences on response inhibition measures, J. Exp. Psychol. Hum. Percept. Perform. 36 (2010) 430. [180] R.A. Barkley, Behavioral inhibition, sustained attention, and executive functions: constructing a unifying theory of ADHD, Psychol. Bull. 121 (1997) 65. [181] E.M. Bekker, C.C. Overtoom, J.L. Kenemans, J.J. Kooij, I. De Noord, J.K. Buitelaar, et al., Stopping and changing in adults with ADHD, Psychol. Med. 35 (2005) 807
816. [182] C. Hanisch, R. Radach, K. Holtkamp, B. Herpertz-Dahlmann, K. Konrad, Oculomotor inhibition in children with and without attention-deficit hyperactivity disorder (ADHD), J. Neural Transm. 113 (2006) 671
684. [183] I. Noorani, R.H.S. Carpenter, The LATER model of reaction time and decision, Neurosci. Biobehav. Rev. 64 (2016) 229
251. [184] G. Underwood, L. Humphreys, E. Cross, Congruency, saliency and gist in the inspection of objects in natural scenes, Eye Movements., Elsevier, 2007, pp. 563
VII.

Further reading

[185] J.M. Henderson, P.A. Weeks Jr, A. Hollingworth, The effects of semantic consistency on eye movements during complex scene viewing, J. Exp. Psychol. Hum. Percept. Perform. 25 (1999) 210. [186] K. Rayner, Eye movements in reading and information processing: 20 years of research, Psychol. Bull. 124 (1998) 372. [187] B.W. Tatler, J.R. Brockmole, R.H.S. Carpenter, LATEST: a model of saccadic decisions in space and time, Psychol. Rev. 124 (2017) 267. [188] S.B.M. Yoo, B.Y. Hayden, Economic choice as an untangling of options into actions, Neuron 99 (3) (2018) 434
447. [189] D.J. Freedman, J.A. Assad, Neuronal mechanisms of visual categorization: an abstract view on decision making, Annu. Rev. Neurosci. 39 (2016) 129
147. [190] V. Christopoulos, J. Bonaiuto, R.A. Andersen, A biologically plausible computational theory for value integration and action selection in decisions with competing alternatives, PLoS Comput. Biol. 11 (2015) e1004104. [191] R. Ratcliff, Theoretical interpretations of the speed and accuracy of positive and negative responses, Psychol. Rev. 92 (1985) 212. [192] R.M. Roe, J.R. Busemeyer, J.T. Townsend, Multialternative decision field theory: a dynamic connectionist model of decision making, Psychol. Rev. 108 (2001) 370. [193] S. Bhatia, T.J. Pleskac, Preference accumulation as a process model of desirability ratings, Cogn. Psychol. 109 (2019) 47
67. [194] M.D. Lee, T.D.R. Cummins, Evidence accumulation in decision making: unifying the “take the best” and the “rational” models, Psychon. Bull. Rev. 11 (2004) 343
352. [195] M. Brusovansky, M. Glickman, M. Usher, Fast and effective: Intuitive processes in complex decisions, Psychon. Bull. Rev. 25 (2018) 1542
1548. [196] A. Glo¨ckner, B.E. Hilbig, M. Jekel, What is adaptive about adaptive decision making? A parallel constraint satisfaction account, Cognition 133 (2014) 641
666. [197] M.D. Lee, B.R. Newell, Using hierarchical Bayesian methods to examine the tools of decision-making, Judgm. Decis. Making 6 (2011) 832
842. [198] M.N. Shadlen, R. Kiani, Decision making as a window on cognition, Neuron 80 (2013) 791
806.

Further reading P. Shenoy, A.J. Yu, Rational decision-making in inhibitory control, Front. Hum. Neurosci. 5 (2011) 1
10. Available from: ,https://doi.org/10.3389/fnhum.2011.00048.. M.A. Bellgrove, C.D. Chambers, A. Vance, N. Hall, M. Karamitsios, J.L. Bradshaw, Lateralized deficit of response inhibition in early-onset schizophrenia, Psychol. Med. 36 (2006) 495
505. D.P. Hanes, W.F. Patterson, J.D. Schall, Role of frontal eye fields in countermanding saccades: visual, movement, and fixation activity, J. Neurophysiol. 79 (1998) 817
834.

99

CHAPTER

Speech recognition technique for identification of raga

5

Snehlata Barde and Veena Kaimal Department of MATS School of Information Technology, MATS University, Raipur, India

5.1 Introduction Voice or speech detection is a technology that allows one to input speech to a system as inputs to trigger an action. Voice identification and speech detection are two diverse biometric modalities. Though these two modalities are separated technically, both are dependent on human voice. The gesture technology on which the Xbox game works is based on a gesture-recognition technique that is contactless biometry. So there is no direct contact between the device and the input. Similarly, speech and voice identification are also no-direct-contact softwarebased technologies. These biometric devices are used very commonly for their convenience in regular use. Fig. 5.1 shows a block diagram of speech-recognition system.

5.2 Speech recognition Generally, we use both word voice and speech for recognition that can be defined as follows: Voice recognition: Voice recognition, also commonly referred to as voiceprint. It identifies and authenticates vocal modalities. It is the measure of incomparable human vocal characteristics extracted from the sound produced while speaking. Speech recognition: Speech recognition is a technology that is widely used today. Nowadays, this technology is widely used to replace the input methods such as typing, clicking, touch screens, and other ways. It is a means by which we can make devices and software that are more user friendly and increase productivity.

Cognitive Informatics, Computer Modelling, and Cognitive Science, Volume 2. DOI: https://doi.org/10.1016/B978-0-12-819445-4.00005-9 © 2020 Elsevier Inc. All rights reserved.

101

102

CHAPTER 5 Speech recognition technique for identification of raga

FIGURE 5.1 Block diagram of speech-recognition system.

5.3 Applications of speech recognition There are plenty of applications that use speech recognition, such as militarybased applications, an aid for impaired persons (if a person is with crippled or no hands or fingers); the devices used, based on speech technology in the field of medical sciences, have also emerged as useful mobile apps for people who do not have a clear vision and in robotics. Speech recognition, today, is no more a new story because of its propagation among common devices such as computers and mobile phones. Today’s smartphones are making interesting use of speech recognition. The iPhone, Android are devices that actively apply this technology. It allows us to initiate a call or to contact by giving just speech instructions. Speech and voice recognition together become a powerful method for authentication and offer hands-free interface for smartphones and other gadgets. We have speech recognition on our personal mobile devices such as Alexa that responds to voice commands to perform basic task. Voice recognition is used as authentication methods in preventing fraud. In certain situations, it may be used in combination with face recognition for authentication purposes such as mobile app authentication. This technology can be used in identifying those customers who and thus the company may emphasize the level of security required. The speech-recognition technology has made a revolution in the field of mobile technology. The devices became easier to use than before. This technology gains a great scope in the area of IOT-focused chip designers. Voice recognition is now used as any other biometric authentication techniques for login solutions. The integration of voice-supported car infotainment systems too is gaining worldwide popularity as several countries have launched “hands-free” regulations that control the use of mobile phones while driving. Developers in the voice and speech-recognition market focus on innovation that is expected to accelerate market growth over the forecast period. By using speech-recognition technology on smartphones, doctors and clinicians can

5.5 A brief history of Indian music

translate their voices into rich, detailed clinical descriptions recorded in the electronic health record system. Another important area of application of speech-processing system is music that plays a vital role in our lives.

5.4 Speech analyses in music information retrieval Music information retrieval (MIR) is a science of retrieving information about music, both statically and dynamically. Music is an art, which is loved and used by every human being; researches in this field have also started since the 1990s. MIR is a high learning curve and is beyond the knowledge of text information retrieval. Music is a subject in itself; therefore it requires a strong understanding of theoretical, analytical, and practical aspects. Music differs from text in many ways. The most important fact is that music is multifaceted, so research in this field is challenging. The two main approaches, in which MIR is dealt with, are based on metadata and content. Initially, the researches were focused on music indexing, classification of music based on compositions, artists, and other static information that involved text-based retrieval. The content-based MIR actually focuses on musical content retrieval such as pitch, melody, and rhythm that can be used for extracting audio features for identification and classification of songs.

5.5 A brief history of Indian music The origin of music is not known to any of us, but it has been there since the existence of life. When talking about music, according to Dr. Ashok Ranade, an eminent musician, the Indian music has a musical pentad, that is, it can be categorized into five forms. They are as follows: 1. 2. 3. 4. 5.

primitive music or tribal music folk music or Lok sangeet religious music or Bhajans or Bhakti songs popular music or film songs art music or classical music

Indian classical music has its own historic importance. Indian classical music or shastriya sangeet can be further classified as Carnatic and Hindustani. Both these types have originated from a common base called the “Samveda” which is one of the four Vedas of the Aryans. In northern India, the influence of Persian and Arabic invaders gave rise to Hindustani sangeet, whereas South India, which was uninfluenced by intruders, continued the same tradition and was called Carnatic sangeet. Indian classical music is based on raga or melody and hence

103

104

CHAPTER 5 Speech recognition technique for identification of raga

referred to as the backbone of music. A raga is an ear-pleasing sound that is produced when swaras are arranged in a specific pattern. Swaras can be understood as the most fundamental element of the building block called “raga” of music. Some types of music such as classical, Western, and folk are fundamentally built on swaras. In classical music, typically, we have “sapta swaras” or seven swaras—“Sa, Ri, Ga, Ma, Pa, Dha, Ni, Sa`” (Sa` denotes the upper Sa in the octave). Each of these swaras has a separate range of frequency. These notes (swaras) are arranged in their increasing order of frequencies. This arrangement in the ascending order of frequencies is called “Arohan” and the same when denoted as “Sa`, Ni, Dha, Pa, Ma, Ga, Ri, Sa,” that is in the descending order of frequencies, is called “Avrohan.” When a raga in Carnatic music has all these complete notes arranged in the same fashion as previously discussed, it is called “Janak raga” (Sampurna raga). In Carnatic music, ragas have been broadly classified as “Janak raga” (parent raga) and “Janya raga” (child raga). Sometimes, a raga may contain only five or six swaras in its Arohan and Avrohan. Such ragas are called “Audava raga” and “Shaudav raga.” Here we will consider “Sampurna raga” also called complete raga or parent raga or Janak raga. This is a superficial way of looking at the notes and using them as a classification tool by the number of notes that each raga has. There is a requirement of a minimum of five notes in any raga except for some exceptions. Therefore we have ragas with five, six, and finally seven swaras. Technically, the ragas with five swaras are called “Audava raga,” ragas with six swaras are called “Shaudav raga,” and ragas with all the seven swaras are called “Sampurna ragas.”

5.6 Mathematical structure of Carnatic music Above this classification, the Carnatic music has a well-structured mathematical classification theory called the “Melakarta” theory. To understand this, first we need to know about swarasthanas. There are 12 swarasthanas altogether. A swarasthana can be understood as the frequency denoted by that particular note. Recalling the sapta swaras, Sa, Ri, Ga, Ma, Pa, Dha, and Ni, Sa and Pa are called achala swaras or sthai swaras, that is, Sa and Pa have a fixed frequency, whereas all other swaras, that is, Ri, Ga, Dha, and Ni, have three variations each, and Ma has two variations (Fig. 5.2). Fig. 5.3 depicts the detailed structure of swarasthanas of the notes (swaras). The swarasthanas are based on the frequency variations of the notes. In the previous figure, including Sa` (upper Sa), there are 17 swarasthanas. If you watch carefully, it could be found that few of the variants are equivalent to the next note in the octave. Example: R2
G1; R3
G2;

D2
N1; D3
N2

5.6 Mathematical structure of Carnatic music

SWARASTHANAS SNO 1

Swaras Sa

2 3 4

Ri

5 6 7

Ga

8 9 10

Ma Pa

11 12 13

Dha

Name Shadjamam Shuddha Rishabham Chaturshruti Rishabham Shatshruti Rishabham Shuddha Gandharam Sadharana Gandharam Antar Gandharam Shuddha Madhyamam Prati Madhyamam Panchamam Shuddha Deivatam Chaturshruti Deivatam Shatshruti Deivatam Shuddha Nishadam

14

Symbol S

Equivalent Fixed

Possible combinations

G1,G2,G3

R1 R2

G1

G2,G3

R3

G2

G3

G1 G2 G3 M1 M2 P

Fixed

D1

N1,N2,N3

D2

N1

N2,N3

D3

N2

N3

N1

Kaisila 15

Nishadam

N2

Kaikali 16

Ni

Nishadam

17

Sa

Shadjamam

N3 S

FIGURE 5.2 Diagrammatic representation of the raga identification in Carnatic music as a constraint satisfaction problem.

FIGURE 5.3 Detailed structure of swarasthanas of the notes (swaras).

105

106

CHAPTER 5 Speech recognition technique for identification of raga

Therefore in a Sampurna raga’s Arohan and Avrohan, we can find combinations with these equivalents. The “Melakarta” theory was first put forward by Sri Raamamaatya, a famous musician of the 16th century. In continuation, Sri Venkatamakhi, a gifted musicologist in the 17th century, came up with a new mela system, commonly known as the Melakarta system or theory. Earlier, only 12 swarasthanas were considered. Sri Venkatamakhi worked out the number of combinations of the 12 swarasthanas. One essential consideration was that each scale had to have all the seven swaras in order. Let us look at our schema of the following 12 swarasthanas as given: 1. Shadjamam 2. Shuddha Rishabham 3. Chaturshruti Rishabham Shuddha Gandharam 4. Shatshruti Rishabham Sadharana Gandharam 5. Antar Gandharam 6. Shuddha Madhyamam 7. Prati Madhyamam 8. Panchamam 9. Shuddha Deivatam 10. Chaturshruti Deivatam Shuddha Nishadam 11. Shatshruti Deivatam Kaisika Nishadam 12. Kaikali Nishadam We have “Sa, R1, R2, G1, G2, M1, M2, Pa, D1, D2, N1, N2.” We need to figure out how many combinations of the 7 swaras can be made when we have these 12 swarasthanas. Sri Venkatamakhi essentially made that possible. He reworked on the nomenclature of these swaras. We speak of “R1 R2” and need a scale to have all the seven swaras, Sa, Ri, Ga, Ma, Pa, Dha, Ni. Now, if we have swarasthanas 2 and 3, we will not have “Ga.” So, in that case, it is not a Sampurna scale. Therefore we take a scale that can accommodate swarasthana2 as well as swarasthana 3. For this, what he did was quite ingenious, that is, the third swarasthana was introduced such that there are two variants of “Ri,” two variants of “Ga,” and two variants of “Dha” and “Ni” each. He spoke of the three variants each of “Ri, Ga, Dha, and Ni,” and he distributed it as shown in the following: R1 with G1 or G2 or G3 R2 with G2 or G3 R3 with G3 Madhyamam M1 or M2 Similarly, D1 with N1 or N2 or N3 D2 with N2 or N3 D3 with N3

5.6 Mathematical structure of Carnatic music

FIGURE 5.4 Twelve swarasthanas along with equivalent notes.

Now, the fourth swarasthana can be “R1, R2, and R3” or “G2,” and the fifth swarasthana will be “G3.” So, essentially, the third swarasthana could be either “R2 or G1,” the fourth swarasthana could be either “R3 or G2,” and the fifth swarasthana has to be “G3”, also with “Dha and Ni.” The 10th swarasthana is “D2 or N1,” the 11th swarasthana is “D3 or N2,” and 12th swarasthana is N3. The previous explanation can diagrammatically be summarized as given in Fig. 5.4. The Melakarta system was proposed from this. For example, let us consider a raga with Arohana as “Sa, R1, G1, M1, Pa, D1, N1.” Now keeping the first three variants constant, that is, R1, G1, and M1 constant, we have six different combinations. Those are as follows: D1-N1 D1-N2 D1-N3

107

108

CHAPTER 5 Speech recognition technique for identification of raga

FIGURE 5.5 The 72 Melakarta ragas.

D2-N2 D2-N3 D3-N3 Therefore with just one set of “Ri and Ga,” that is, R1 and G1, we have six possible combinations of “Dha and Ni”. Similarly, we have six possible combinations of Ri and Ga also. Therefore we have 6 3 6 5 36 possible combinations with just “M1” and another 36, possible with the note “M2.” This is the 72 Melakarta scheme propounded by Venkatamakhi in his Chathurdandi Prakashika. An immensely influential work and this idea have firmly entrenched in the minds of all Carnatic musicians. After Venkatamakhi and his Chathurdandi Prakashika, the Melakarta system was handled and was taken forward by the later musicologists. And the 72 Melakarta system or the chart of the 72 Melakarta, which is used in today’s Carnatic music, is shown in Fig. 5.5. In the abovementioned Melakarta structure, R1 is denoted with Ra, R2 with Ri, and R3 with Ru. Similarly, Ga, Gi, and Gu are used for denoting G1, G2, and G3. The Carnatic music has a well-structured layout that helps in easy classification of raga. A raga that has all the seven swaras of the octave in its ascend (Arohana) and descend (Avrohana) is called as “Janak” raga in Carnatic music. It is also called a complete raga. There are 72 such “Janak” ragas in Carnatic music called the Melakarta ragas.

5.7 Digital speech processing

These 72 Sampurna (Janak) ragas are further classified into 12 chakras based on the common swaras shared by them. Before moving into the classification of ragas into chakras, we need to first understand the swaras. A raga is said to be Melakarta raga if it has all the seven swaras of the octave. The upper Shadjamam S of the succeeding octave must be included making it an eight-swara sequence, Sa, Ri, Ga, Ma, Pa, Dha, Ni, Sa. The same swaras must be in the Arohana and Avarohana. All the swaras strictly increase in frequency for Arohana and strictly decrease for Avarohana, and then the raga is called a Krama Sampurna raga or the Janak raga. The seven swaras present in Arohana and Avrohana of a “Janak” raga consist of two constant (achala) swaras and five variant (chala) swaras. Each swara has a specific range of frequency; therefore these frequencies can be used for identifying the swara sequence present in the audio signal taken as input. The 12 “Chakras” in which these ragas are classified are as follows: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12.

Indu Chakra Netra Chakra Agni Chakra Veda Chakra Bana Chakra Rutu Chakra Rishi Chakra Vasu Chakra Bhrama Chakra Dishi Chakra Rudra Chakra Aditya Chakra

The chakras contain six ragas each. The first 36 ragas, belonging to Indu, Netra, Agni, Veda, Bana, and Rutu chakras, have Shuddha Madhyama (M1) and ragas from 37 to 72, belonging to chakras Rishi, Vasu, Bhrama, Dishi, Rudra, and Aditya chakra, have Prati Madhyama (M2). In each chakra the earlier (Purvanga) swaras Sa, Ri, Ga, Ma are the same, while the later swaras (Uttaranga) swaras Dha and Ni are different.

5.7 Digital speech processing Speech is a natural mode of communication. It can be used as biometry, speech coding, extraction of speech signal parameters, speech prosody, etc. which are the other aspects of digital speech processing. Many mission critical operations in aerospace and defense, industrial applications, and other high-end applications also use digital signal processing for a proficient result. Real-time communications for system monitoring and diagnostics also make the use of digital signal processing. Digital speech processing is implemented on digital signal processing and is the

109

110

CHAPTER 5 Speech recognition technique for identification of raga

FIGURE 5.6 Steps of speech processing.

major technology used in MIR today. There are many speech-analysis tools available both as free and open source software (FOSS) and even licensed software. Few commonly used speech-analysis tools are Praat, Wavesurfer, Goldwave, Transcriber, Cool Edit, and many more to add to the list. The main steps involved in an audio signal processing are given in Fig. 5.6. The flowchart in the previous figure shows the steps involved in speech processing. The audio signals are first converted into discrete signals. The feature extraction is the most important step involved in speech processing. First, we need to remove all the irrelevant features and extract the relevant ones. Then by using any classifier, we need to compare and match the extracted features.

5.8 Proposed methodology for classification of raga Formally, a constraint satisfaction problem (CSP) can be defined as a set of variables V1, V2, V3, . . ., Vn and a set of constraints C1, C2, C3, . . ., Cm such that each

5.8 Proposed methodology for classification of raga

variable Vi is contained in any one of the nonempty domains Di of possible values. The state of values that satisfies all the constraints of the constraint set is called a model. Therefore, mathematically a CSP can be defined as a tuple R 5 hV; D; Ci

where R is a relation, V is an ordered set of variables {v1, . . ., vn}, D is a set of domains {D1, . . ., Dn}, Di is a finite set of possible values for variable xi, and C is a set of constraints {C1, . . ., Cm}. A constraint specifies the allowed combinations of values for a variable. As discussed in the previous section, each chakra contains six ragas. Each of these 72 ragas is referred to as complete raga as they have all the seven swaras in the octave in their ascending order of frequencies. Out of these, the Shadjamam and Panchamam have fixed scale, whereas all the other swaras have their variations [1]. The first 36 ragas belonging to Indu, Netra, Agni, Veda, Bana, and Rutu chakras have Shuddha Madhyama (M1) and ragas from 37 to 72 belong to chakras Rishi, Vasu, Bhrama, Dishi, Rudra, and Aditya chakra have Prati Madhyama (M2). In each chakra the earlier (Purvanga) swaras Sa, Ri, Ga, Ma are the same, while the later swaras (Uttaranga) Dha and Ni are different. Classification of raga as seen in the previous section can be done on the basis of the no. of swaras present in Arohana and Avrohana, Melakarta ragas and Janya (child) ragas. As we have already seen the mathematical layout of the Carnatic music, we propose that the previous problem of identification of raga in Carnatic music can be posed as a CSP. Fig. 5.5 indicates the representation of the raga identification in Carnatic music as a CSP. For example, we have Arohan or ascent of a raga as the given input signal. Let us input the Arohan of a raga which is given as the input signal. That is, X: {Sa, R2, G3, M2, P, D2, N3} represents the set of variables. D: {Sa, Ri, Ga, Ma, Pa, Dha, Ni} represents the domain set. C: {if R1 then (G1|G2|G3), if R2 then (G2|G3) . . .} represents the set of domain. The diagram in Fig. 5.2 shows when an input signal is given; first, it identifies the domain to which it belongs and then based on the set of constraints it identifies the Melakarta of the input raga: X: {Sa, R2, G3, M2, P, D2, N3} Therefore the set X contains the following values {Sa, R2, G3, M2, P, D2, N3}. The set D consists of all the swaras with respect to their swarasthanas. The constraint set C consists of rules such as {if R1 then (G1, G2, G3); if R2 then (G2, G3) . . .}

111

112

CHAPTER 5 Speech recognition technique for identification of raga

Based on the previous example, we can draw a test case for this problem. The structure is as given later. From the previous parse tree, we can clearly analyze the following inferences: When we provide the input signal, which is an Arohana (ascend) of “Mechakalyani raga,” that is, X: {Sa, R2, G3, M2, P, D2, N3} Then we get the following results: 1. In the previous input, Sa and Pa are fixed. Therefore the first step is to check for the variants. 2. The comparison starts from “Madhyamam.” If the Arohana contains M1 (Shudhha madhyamam) then the Melakarta ragas ranging 136 need to be scanned, else if it is M2 (Prati madhyamam), then Melakarta ragas from 37 to 72 are to be scanned. 3. Next step is to scan for Rishabham, that is, if (R1, R2 or R3); if (R1 then (G1 or G2 or G3)) else if (R2 then (G2 or G3)) else R3-G3 4. Now once Madhyamam, Rishabham, and Gandharam are identified then we just need to check for the Daivatam and Nishadam. 5. Step 3 repeats for Daivatam, that is, 6. if (D1, D2 or D3); If (D1 then (N1 or N2 or N3)) else if (D2 then (N2 or N3)) else D3-N3 7. Finally, we identify the Melakarta number for the given input. In our example, we have taken the input set as X:{ Sa, R2, G3, M2, P, D2, N3}. The flowchart in Fig. 5.7 shows the diagrammatic representation of identification of raga for the given input.

5.9 A practical example using Praat Based on the previous theory of “chala” and “achala” swara, we have taken a small testing sequence of achala swara, that is, Sa and Pa as an input audio signal to Praat acoustic signal-processing tool. Praat provides an easy interface for feature extraction such as pitch, formant, intensity, and spectrogram. Following is the reading of the small piece of testing data which is a .wav audio file containing the frequency of “Arohana swaras” (ascent) of the raga Mechakalyani. These swaras “Sa, Ri, Ga, Ma, Pa, Dha, Ni, Sa`” were audio recorded and then were given as inputs to Praat for feature extraction. An input audio signal was given as

5.9 A practical example using Praat

FIGURE 5.7 Flowchart for identification of “Mechakalyani raga.”

input. Fig. 5.8 displays the graph plotted by the Praat tool for the previously given input. The graph in Fig. 5.9 shows the details of formant, pitch, spectrogram, intensity, and pulse for the visible part 8.99 seconds. The graph in Fig. 5.10 plots the details of the visible pitch contour between the selected time interval. The graph in Fig. 5.11 shows the visible intensity contour for a selected time period.

113

FIGURE 5.8 The graphical representation of the input audio signal.

FIGURE 5.9 Formant, pitch, spectrogram, and intensity for the input audio signal.

FIGURE 5.10 Visible pitch contour between 4.00 and 7.00 seconds.

5.9 A practical example using Praat

FIGURE 5.11 Visible intensity contour.

FIGURE 5.12 Visible formant contour.

The graph shown in Fig. 5.12 indicates the most important feature used in extraction. This graph shows the formant contour for the specific time period selected.

115

116

CHAPTER 5 Speech recognition technique for identification of raga

FIGURE 5.13 Visible spectral slice.

Fig. 5.13 shows the spectral property of the selected slice of input. These feature extraction methods will help us in further identifying the basic swaras and ragas of a given piece of music.

5.10 Conclusion At the end, we concluded that Praat is an interactive and user-friendly tool that allows easy feature extraction from sound signals. There are many recent researches in MIR techniques using digital speech/signal processing for dynamically retrieving information and is still an upcoming area. Therefore we would like to take this study further using signal processing through Praat as the fundamental technique. We also concluded that the classification of raga in carnatic music (CM) can be posed as a CSP. We find better chances of identification and classification of ragas. There are many recent researches in MIR techniques using digital speech/signal processing for dynamically retrieving information, which is still an upcoming area. We conclude that the proposed methodology will provide an easier approach for the classification of ragas.

Reference [1] S. Chakraborty, et al., Pattern classification of Indian classical ragas based on object oriented concepts, Int. J. Adv. Comput. Eng. Archit. 2 (2) (2012).

Further reading

Further reading S. Chakraborty, et al., An interesting application of simple exponential smoothing in music analysis, Int. J. Soft Comput. Artif. Intell. Appl. 2 (4) (2013) 3744. P. Dighe, et al., Swara histogram based structural analysis and identification of Indian classical ragas, Int. Soc. Music Inf. Retr. (2013). S. Dutta, et al. Raga verification in Carnatic music using longest common segment set, in: 16th International Society for Music Information Retrieval Conference, 2015, pp. 605611. http://en.wikipedia.org/wiki/Glossary of Carnatic_music_terms. http://onlinecourses.nptel.ac.in. http://www.rfwireless-world.com/Terminology/automatic-speech-recognition-system.html. Y. Jadoul, et al., Introducing Parselmouth: a Python interface to Praat, J. Phonetics 71 (2018) 605611. R. Joseph, et al., Carnatic raga recognition, Indian J. Sci. Technol. 10 (13) (2017) 17485110326. M. Kamble, Raga identification techniques of Indian classical music: an overview, in: Innovation in Engineering Science and Technology (NCIEST-2015), 2015, pp. 100105. V. Kaimal, S. Barde, Introduction to identification of raga in Carnatic music and corresponding Hindustani music, Int. J. Comput. Sci. Eng. 6 (6) (2018) 955958. V. Kaimal, S. Barde, Raga identification using Praat: an acoustic signal processing tool, Int. J. Manage. Technol. Eng. 9 (3) (2019) 970974. P. Kirthika, et al., Frequency based audio feature extraction for raga based musical information retrieval using LPC and LSI, J. Theor. Appl. Inf. Technol. 69 (3) (2014) 582588. V. Krishnan, Mathematics of Melakartha Ragas in Carnatic Music, UMass Lowell, 2012. G.K. Kouduri, et al., A survey of Raaga recognition techniques and improvements to the state-of-the-art, Proc. Sound Music Comput. (2011). J. Larrosa, G. Valiente, Constraint satisfaction algorithm for graph pattern matching, Math. Struct. Comput. Sci. 12 (4) (2002) 403422. J. Serra, et al., Assessing the tuning of sung Indian classical music, Int. Soc. Music Inf. Retr. (2011). K. Srimanip, et al., A comparative study of Carnatic and Hindustani raga systems by neural network approach, Int. J. Neural Networks 2 (1) (2012) 3538. P. Rajshri, et al., Harmonium raga recognition, Int. J. Mach. Learn. Comput. 3 (4) (2013) 352356. P. Rao, et al., Audio metadata extraction: the case for Hindustani classical music, in: Proc. of SPCOM, Department of Electrical Engineering, Indian Institute of Technology Bombay, 2012. S. Shetty, K.K. Achary, S. Hegde, Clustering of ragas based on jump sequence for automatic raga identification, in: K.R. Venugopal, L.M. Patnaik (Eds.) Wireless Networks and Computational Intelligence, ICIP, CCIS vol. 292, 2012, pp. 318328. R. Shridhar, et al., Raga identification of Carnatic music for music information retrieval, Int. J. Recent Trends Eng. 1 (1) (2009) 571574. Y. Yorozu, M. Hirano, K. Oka, Y. Tagawa, Electron spectroscopy studies on magneto optical media and plastic sub state interface, IEEE Transl. J. Magn. Jpn. 2 (1987) 740741. Digests 9th Annual Conf. Magnetics Japan, 301, 1982.

117

CHAPTER

Future of cognitive science

6

Shankru Guggari1, H. Nagendra2, Santosh R. Desai3 and V. Umadevi1 1

Department of Computer Science and Engineering, B.M.S. College of Engineering, Bengaluru, India 2 Department of Electronics & Communication, Poojya Doddappa Appa College of Engineering, Kalaburagi (Gulbarga), India 3 Department of Electronics and instrumentation, B.M.S. College of Engineering, Bengaluru, India

6.1 Introduction Cognitive science is an interdisciplinary domain catering to multiple fields such as philosophy, biology, computer science, linguistics, and psychology. It is anticipated to be used in solving various problems that exist in human computer interaction, human learning, and user modeling. It has varied success from understanding human mind to its ability to adapt the frequent changes in the environment. More recently, it has gained popularity in the study of unconscious mind and its capability in processing the information. Inspite of its success in understanding the mind, there are certain things that influence its performance such as body functions, internally and externally, and movement of the objects. There are some of the key challenges that can be explored [1]:

• understanding the interaction between human being and environment, • measuring the differences in the performance within himself/herself, across human beings along with life span, and

• carrying out the study to build comparative metamodeling. Cognitive neuroscience is an emerging interdisciplinary field of cognitive psychology and neuroscience. It unravels the mysterious connection between brain, mind, and body. It also explains the relationships among neural systems and psychological structures and behavior. Neuroscience relates to the science of neurons and the nervous system. Cognitive neuroscience pertains to mechanisms of the most complex neuronal systems, associated with higher mental functions such as language, memory, attention, or mental representations. In this chapter, main challenges and their future research directions existing in different domains are explained briefly. Further, the roles of cognitive science in

Cognitive Informatics, Computer Modelling, and Cognitive Science, Volume 2. DOI: https://doi.org/10.1016/B978-0-12-819445-4.00006-0 © 2020 Elsevier Inc. All rights reserved.

119

120

CHAPTER 6 Future of cognitive science

terms of big data, philosophy, psychology, linguistics, social science, cognition control, and cognitive image processing are being discussed concisely. This chapter is divided into following sections: role of cognition science in varied domains are explained in Section 6.2. Future of cognitive neuroscience and its enhancement is discussed in Section 6.3. Conclusions are given in Section 6.4.

6.2 Role of cognitive science in varied domains In general, cognitive science is the study of humans, machines, and animals. It began with intellectual movements in 1950. The main aim is to understand the intelligence principles for developing better intelligent machines or devices. Some of the key domains that can be explored for future study are discussed in the following sections.

6.2.1 Cognitive science for big data Cognitive computing has ability to examine the various types of data to get high insights. It comprises various tools such as machine learning, Internet of Things, data visualization, and predictive analytics. Cognitive systems have some key features such as learning ability, ability to use existing program to gather more knowledge. Some of the key features are listed as follows [2]:

• The work carried out in the combination of both cognitive science and big data is at initial stage.

• Cognitive computing and big data (CCBD) analytics provides high chance to • • • • •

do application-oriented research such as health care, banking services, and retail industry. Characteristics such as evaluation, interpretation, decision-making, and observations are needed to be explored for cognitive computing. Focus on the interface for CCBD on decision-making and its implications. In network analysis, decision of a node to touch when there is a miss in the link. Quality of the data plays a crucial role in cognitive computing and leads to develop new innovation products by taking care of security to minimize the related risks. CCBD can be used to understand the concerns in terms of information security, trust, and privacy. As the big data analytics and cognitive science are expensive, we need to bring these to small and medium enterprises. Cloud computing is one such technique. Previously enterprise resource planning was used only for large enterprises. Now, due to the advances in the computing technique, it is available in all industries.

6.2 Role of cognitive science in varied domains

The future of cognitive science and computing science [3] is widely used in ambient intelligence and includes augmented reality to improve the user experience. However, there is a need to explore the combination of these two techniques to predict the hardware capacity and development of three-dimensional web portal.

6.2.2 Cognitive science for philosophy Philosophy is an interdisciplinary study of mind. It is closely related to mind in terms of experimental and theoretical work like cognitive science. Following are the issues being listed that can be explored in future:

• need to address emotions and consciousness challenges in human thinking • significance of physical world in human thoughts • understand the effects of dynamic and social changes in human actions and thoughts

• address the issues related to mind in traditional philosophy • concerns about practice of metatheoretic issues and its fundamental assumptions

• explanation about the core concepts of cognitive science

6.2.3 Brain machine interface Brain machine interface (BMI) is a dominant research area to improve the quality of life for disabled individuals. It is a new approach to establish artificial communication between animals. There are two aspects: (1) controlling of neural activity and (2) controlling of external equipment using neural activity. Following are some of the research issues that need to be addressed in future:

• Elucidate the characteristics of multibrain system with its computational abilities as compared to nontuning techniques [4].

• Develop unique techniques to control thoughts. • Design of a novel hybrid technique for learning by using both brain and • • • • •

artificial intelligence [5]. Extensive study needs to be carried out for brain-to-brain learning [4]. Significance in monitoring the health [6]. Availability of resources and its proper usage. Develop modalities for acquiring neurological signals and establish bench mark methods [6]. Generation of bimanual functionality for BMI systems [7].

In emerging natural language system, understanding evolutionary cultural forces are considered as the top priority in research [8].

121

122

CHAPTER 6 Future of cognitive science

6.2.4 Cognition science for psychology Cognition psychology deals with cognitive mental process and describes how people remember, learn, perceive, and think about information. In general, it explains acquiring of the information, storing, processing, and language communication. Some of the applications are driving behavior, face/object recognition, etc. Few challenges faced by cognitive psychologists are as follows [9]:

• Need to declare whether cognition is internal or not, that is, it is only related to brain or the combination of both body and nature (environment).

• In comparative psychology, researchers need to clarify that cognition is mutually exclusive with associative models.

• Required to describe on low-level behaviors to be considered as cognitive. • Significance of shared mental models in trading zones [10]. • Interactional expertise for acquisition and deployment in trading zones [10]. Embodied cognition is related to thoughts and emotions of living human beings with respect to human sensor and motor system. It has the following challenges that need to be addressed [11]:

• develop an interoceptive mechanism for psychological processing to understand the changes in the body;

• social psychology: understanding the body-related metaphors such as bodily movements and capacities, which can influence thoughts and emotions;

• cognitive psychology: need to explore what kind of information is processed in place of body-environment constraints;

• role of actions in learning; • test the hypothesis which influences the computation; and • design of sophisticated representations to establish relationship between sensor and effector systems. Cognitive hearing science is an interdisciplinary science with combination of both hearing and cognition [12]. It is demonstrated with the following topics: (1) language processing with respect to the condition of listening, (2) advantage in auditory communication techniques, and (3) techniques to measure the increase in the performance after training and ageing. Some of the challenges that need to be looked into for future research in this area are mentioned:

• develop framework to optimize signal processing hearing based on capacity of user cognition;

• demonstrate cognition techniques to explore other domains such as speed of information processing, attention, and phonological skills;

• explore deeper understanding about perceptual learning to describe the inference based on context; and

• an integrated research (such as auditory, visual, and cognitive) that needs to be carried out in clinical practices.

6.2 Role of cognitive science in varied domains

6.2.5 Cognition social science Social cognition shaped around the “theory of mind.” It draws inferences to represent the mental state of people and translate it into a “false belief.” It helps in the development of theory theory, understanding the interactionist and perceptual of other minds and simulation theory. Social recognition also performs empirical hypothesis tests. Theory theory claims the mental state of other people and achieves causal relationship between observable and unobservable mental states. The future direction of theory theory model must deal with the following paradigms [13]:

• Folk psychology: need to look into how it is implemented in the individual. • Inferential knowledge: explore mind reading inferential knowledge on unobservable mental states.

• Propositional attributes: elaborate with respect to mind reading. It carries information about rationality and conceptual thoughts.

• Defend third person’s mind reading in terms of observed and unobserved mental states.

• Development of more precise taxonomies related to mental processes as compared to old theories [14].

• Design generalized high-level frameworks for mind [14]. Perceptual cognitive skills [15] involve locating an individual, adding new information about the environment to the existing knowledge, and requiring capabilities to select and execute the actions. There are three important areas in perceptual cognitive skill training need to explore in future: (1) combination of perception and action: need to identify whether this kind of training help to improve the physical performance. (2) Generalization of practice structure: develop a systematics arrangement to practice. (3) Contextual information: investigate this kind of information that will help in the improvement of the performance.

6.2.6 Role of cognitive science in linguistics Cognitive linguistics is an explanation about grammatical structure, also known as functionalist [16]. It mainly describes mental power based on introspection and analyzes language from linguistic point and mainly focuses on language. Language is used as a tool in organizing, processing, and transferring the information. Some of the tasks ahead in this area are listed:

• building a bottom up grammatical typology • development of multimodel communication [17] • language awareness in terms of conscious mechanisms that needs to be studied

• explore more quantitative methods with respect to cognitive linguist

123

124

CHAPTER 6 Future of cognitive science

• need to emphasis social aspects of linguistics Some of the future research works in linguistic laws and human voice [18] are (1) exploring structure of acoustic, (2) establishing mathematical models for selforganized criticality (SOC), and (3) requiring an explanation about the relationship between SOC and linguistic.

6.2.7 Cognitive control It is the process where goals influence the behavior. It supports goal-oriented complex thoughts and adaptive responses, unstructured environments without human interface. Architecture of cognition control includes sensors, actuators, and environment agents. The industrial world has been witnessing appreciable development in the past few decades. This in turn has resulted in long-lasting influence in almost all the areas, including control discipline at the cost of increased complexity. Motivated by the need, cognitive systems are being roped to encourage interdisciplinary approach to the scope of the study. Robotics, sea exploration, unmanned aerial vehicles, automation, space systems, and process plants are some of the fields where cognitive control will help in providing greater degree of autonomy [19,20]. Thus cognition has helped in opening up a new subdomains and stimulating interdisciplinary approach in the field of study. In order to achieve novel cognitive task with satisfactory performance, the system under control needs to assemble and interconnect effectively the mental processes and parametric adjustments [21]. Basically, the cognitive control comprises learning and planning and symbolizes environmental model and perception cycle [22]. According to the proposed cognitive control architecture, perception, action, learning, and knowledge, cognition forms the major components of the system under discussion. Though the architecture has a drawback of missing out the interactions between the inputs and outputs of the environmental model separately, the issue needs to be addressed explicitly with the aid of dominating and neighboring disciplines, namely, artificial intelligence, operations research, cognitive science, real-time embedded system, rapid advancement in the field can be expected in the future. However, there are always challenges that remain to be encountered; adaptive management, modeling and estimation, and response to rare and sudden events are mentionable among them. Control community can and will contribute to the cognitive control by using effective and efficient solutions to tackle stability, controllability, robustness, etc. thereby opening new avenues for researchers in area of cognitive control [20].

6.2.8 Cognitive image processing Recent developments in computer vision with the support of Internet of Things, the applications fields are expanding exponentially. Images enable us to develop complex systems to tackle the problem at different phases. In this context,

6.3 Future of cognitive neuroscience and cognitive enhancement

Deep-learning techniques are gaining popularity to build high-level models by taking help of cognition to understand images. Some of the issues to focus in future are as follows:

• effectively deal with large volume of big data (image) [23] • explore research work by combining both computer vision and video coding [23]

• develop effective frameworks for quality assessment and edge detection [23] • build novel methods for target tracking using cognition association network [24] • developing models for analysis of high resolution satellite images

6.3 Future of cognitive neuroscience and cognitive enhancement The brain conveys information in the form of a very small electrical rhythm or pulse. Every single emotion, memory, or actions are decoded into tiny electrical signals in the central nervous system (CNS). The synchronized sum of these electrical signals is represented as electrical signal. This electrical signal consists of various frequencies called electroencephalogram (EEG) band of frequencies. Each of these brain waves are associated with a particular band of frequencies (0.5 30 Hz) and impact every aspects of functional state such as physical, behavioral, emotional, and particularly cognitive functions [25,26]. The cognitive functions of human can be determined by computing energy or power present in EEG frequencies. Each EEG frequencies are related to a certain type of cognitive functions and the power or energy in these frequencies varies depending on the activity of the neurons in the brain. The increase in the power or energy in particular part of the brain indicates that a particular part of the brain is highly activated [27,28]. The five important EEG frequency bands are theta (0 4 Hz), delta (4 8 Hz), alpha (8 12 Hz), beta (13 30 Hz), and gamma (30 70 Hz). The alpha wave is associated with the restive state of mind and involves the mind body coordination. The increase of alpha activity indicates the improvement of particular cognitive abilities such as hand eye coordination and memory. Beta waves are active in the wakeful period and involves in cognitive functions. This activity represents mental activation and problem-solving skills. The increase in this frequency activity indicates enhancement of certain cognitive functions [26,29]. Theta waves are prominent in sleep and are the bed of intuition. This frequency band represents a range of cognitive functions such as problem-solving abilities and learning. An increased theta activity indicates improvement in these abilities [27]. Some studies used delta wave activity during various cognitive abilities such as focused and strategic tasks. Gamma frequencies are highly localized. These frequencies represent higher order of thinking and knowledge processing abilities [30]. The research in neuroscience has revealed that each of these bands of brain

125

126

CHAPTER 6 Future of cognitive science

waves plays a vital role in the conscious, subconscious, and superconscious states of the mind and body. Overarousal of some the waves in specific brain centers results in a corresponding anomaly. The technology has given the boost to neuroscience in learning the wave patterns in detail with the help of electroencephalography. This instrument uses sensors placed across the scalp according to international standards, which can pick up and record the electrical activity of the brain. These wave patterns give a wealth of information to neurologists on the neurophysiological condition of an individual [26].

6.3.1 Scope for neuroscience research and challenges Neuroscience researches have attracted huge amount of funding and public attention. Basically, the research on the human brain requires to develop algorithms and computer simulations of the human brain. This will help to study the neural activity mapping and proving a functional connection of the whole neural system. These in turn helps to improve the better understanding of the functions of the human brain. Researchers have also gained insight into the neural basis of learning, memory, emotion, and decision-making. Scientists have also developed new range of technological and computational tools for efficient and accurate recording of neural activity from numerous neurons at the same time for mapping the connections between the neurons in the brain for correlating structures and neural activity to test the role of neurons in human behavior. Even today there is a big gap between cognitive neuroscience and the clinical laboratories. Researchers are finding it difficult to completely understand the human brain and to build new computing algorithms and tools. Scientists are attempting to recreate the human brain as a computer prototype to investigate new results for neuroscience and clinical researches. Ultimately, it will help neuroscientists to discover the underlying principles governing the brain structure and functions. This could in turn help to construct new algorithms for the diagnosis and treatment of the brain abnormalities [31 33].

6.3.2 Cognitive enhancement The ability of a person to regulate and perform a task greatly depends upon a wide range of cognitive functions. The cognitive abilities are correlated with augmentations of planning, memory, psychomotor control, and visual-spatial abilities. Cognition is emphasized as the process that is used to comprehend the information. These processes include perception, attention, understanding, memory information, and using them to guide the behavior. Cognitive enhancement is defined as acceleration of mind’s core capabilities through internal/external processing units. Brain study is one of the immense limits in the cognitive learning. The unraveling of its mystery is very complex and intellectually challenging in the research. The functioning of the CNS can be investigated at various levels of system. Biologists study the characteristics of molecules that perform tasks

6.3 Future of cognitive neuroscience and cognitive enhancement

significant for brain function. Physiologists research the properties of individual nerve cell or ensembles. Psychologists investigate patterns of behavior and its modification and learning. Computational neuroscientists venture to bring these domains together to model higher level of brain functions. Thus the brain research necessitates a multidisciplinary means, to bridge the gap between the traditional disciplines and facilitate association among scientists with very different experimental methodologies [34,35]. The present neuroscientific study on understanding which particular neuron in the brain reacts to various kinds of video/audio and sensation stimuli by learning the working of the complex neural system. The current research is based on three paradigms: first, the necessity to better cognize neural coding, sensory, and memory orientation; second, to help paralyzed sufferers and those persons suffering from peripheral sensory dysfunctions; and third, to determine cognitive benefits of having neural prostheses and decision-making [33,36]. The recent past has shown an exponential growth in the area of neuroscience research, opening the alternative path for different cognitive enhancements. The current neuroscientific disciplines are in their improving phases of cognitive enhancements for human beings. The recent development in the field of neuroscience and imaging techniques has helped the scientists to understand the mechanism of how brain functions. These hundred billions of neurons with possibility of hundred trillion connections between them, packed into the 1.5 k of the brain, are becoming less enigmatic to the scientists and researchers. The human enhancement is a direct result of increased understanding of human physiology, cognition, and enhancing skills of maneuvering these and other attributes of human [32,37]. There are four types of enhancement [28,34]: (1) physical, (2) cognitive, (3) emotional, and (4) moral. In this study, we will explore cognitive enhancements, methods of which are as follows:

• Cognitive enhancement via conventional: education, training, and the

•

•

application of peripheral information processing systems can be termed traditional methods of cognitive enhancement. They are highly validated and culturally acknowledged. Other types of psychological interventions, such as yoga, meditation, martial arts, physical exercises, and computer or video games. The human cognitive enhancement has taken the form of education, which has been communicated from generation to generation. Cognitive enhancement via unconventional: the methods of improving cognition by means of unconventional methods, such as ones requiring nootropic drugs, gene rehabilitation, or neural transplant, are almost all considered as at their experimental stage. Cognitive enhancement via pharmaceutical: these kinds of interventions may have various types and many of them are age-old traditional types. This type of enhancement makes use of drugs and chemicals to enhance certain cognitive functions.

127

128

CHAPTER 6 Future of cognitive science

FIGURE 6.1 Block diagram of cognitive study tree.

It is a fact that certain memory-related tasks are enhanced by families of pharmacological drugs, which help to improve cognitive abilities. The combination of various pharmacological agents administered at different times could improve the learning process, increasing memory retention, and memory selection. Stimulants enhance neuron activity by releasing neuromodulators and provide synaptic change. Biomedical enhancement methods results in cognitive performance. For easy understanding, we represent it in tree structure as shown in Fig. 6.1.

6.3.3 Ethical issues and concerns of cognitive enhancement It has few social and ethical practical challenges. Risk is a major fear for all kinds of enhancement. Enhancement does have prices, but often it is a trade of between different abilities [31,35]. Conventional types of cognitive enhancements, such as

References

education, mental techniques, neurological health, and external systems, are largely accepted. On the other hand, unconventional types of cognitive enhancement such as drugs, implants, and direct brain computer interfaces tend to evoke moral and social concerns. Society will have to accept or reject the emerging technologies to enhance cognitive functions of human beings. Though it has the potential to improve the cognitive performances and support some aspects of human lives, it also produces various serious ethical problems. There has been a consistent conflict between proponent and opponents of cognitive enhancement [38]. In a nutshell, cognitive enhancement will improve the quality of human lives: better homes, safer cars, and more effective medicines. Enhancing the human mind improves certain cognitive abilities of the individual. Any techniques that improve the human intellect will pay dividends to the society. Thus an enhancement technology will provide enormous benefits to the human society.

6.4 Conclusion This chapter discusses the future roles of the cognitive science in different domains. It briefly gives information about the domain then explains the future research themes with respect to the domains. Similarly, we describe cognitive neuroscience and its enhancement with future research directions. In summary, cognition science is going to improve in the following fields:

• It contributes in the growth of high-order cognition, possibly it will add more domains and methods to improve the quality of the life.

• Embodied cognition is gaining the popularity and allows degree of abstract representation.

• Knowledge representations with the combination of both statistics and neural techniques enable the development of powerful web search techniques.

• Comparative cognition needs to help deeper understanding of cognition components.

References [1] E. Davelaar, Cognitive science—future challenges of an interdisciplinary field, Front Psychol 1 (2010) 7. [2] S. Gupta, A.K. Kar, A. Baabdullah, W.A. Al-Khowaiter, Big data with cognitive computing: a review for the future, Int. J. Inf. Manage. 42 (2018) 78 89. [3] R. Adams, Cognitive science meets computing science: The future of cognitive systems and cognitive engineering, in: Proceedings of the ITI 2009 31st International Conference on Information Technology Interfaces, 2009, pp. 1 12. [4] M. Pais-Vieira, M. Lebedev, C. Kunicki, J. Wang, M.A. Nicolelis, A brain-to-brain interface for real-time sharing of sensorimotor information, Sci. Rep. 3 (2013) 1319.

129

130

CHAPTER 6 Future of cognitive science

[5] Z. Wu, N. Zheng, S. Zhang, X. Zheng, L. Gao, L. Su, Maze learning by a hybrid brain-computer system, Sci. Rep. 6 (2016) 31746. [6] A. Hassan, M.N. Huda, F. Sarker, K.A. Mamun, An overview of brain machine interface research in developing countries: opportunities and challenges, in: 2016 5th International Conference on Informatics, Electronics and Vision (ICIEV), 2016, pp. 396 401. [7] M.A. Lebedev, A.J. Tate, Future developments in brain-machine interface research, Clinics 66 (2011) 25 32. [8] M. Schouwstra, H. de Swart, B. Thompson, Interpreting silent gesture: cognitive biases and rational inference in emerging language systems, Cogn. Sci. 43 (7) (2019) e12732. [9] C. Buckner, E. Fridland, What is cognition? angsty monism, permissive pluralism(s), and the future of cognitive science, Synthese 194 (11) (2017) 4191 4195. [10] M.E. Gorman, Trading zones, interactional expertise, and future research in cognitive psychology of science, Top. Cogn. Sci. 2 (1) (2010) 96 100. [11] J.I. Davis, A.B. Markman, Embodied cognition as a practical paradigm: Introduction to the topic, the future of embodied cognition, Top. Cogn. Sci. 4 (4) (2012) 685 691. [12] A. Stig, L. Thoms, L. Bjorn, M.K. Pichora-fuller, The emergence of cognitive hearing science, Scand. J. Psychol. 50 (5) (2009) 371 384. [13] N. Gangopadhyay, The future of social cognition: paradigms, concepts and experiments, Synthese 194 (3) (2017) 655 672. [14] M.A. Sushchin, The future of cognitive science and the problem of experience, Mediterranean Journal of Social Sciences 6 (6 S2) (2015) 74. [15] D.P. Broadbent, J. Causer, A.M. Williams, P.R. Ford, Perceptual-cognitive skill training and its transfer to expert performance in the field: future research directions, Eur. J. Sport Sci. 15 (4) (2015) 322 331. [16] D. Geeraerts, H. Cuyckens, J. Nuyts, Cognitive linguistics and functional linguistics, 2012. [17] D. Divjak, J. Klavan, N. Levshina, Cognitive linguistics: looking back, looking forward, Cogn. Ling. 27 (2006) 447 463. [18] I. Torre, B. Luque, L. Lacasa, J. Luque, A. Hernndez-Fernndez, Emergence of linguistic laws in human voice, Sci. Rep. 7 (2017) 1 9. [19] S. Nefti-Meziani, J.O. Gray, Advances in cognitive systems, 2010. [20] T. Samad, A. Annaswamy, The impact of control technology, 2011. [21] T. Egner, The Wiley handbook of cognitive control, 2017. [22] M. Fatemi, S. Haykin, Cognitive control: theory and application, IEEE Access 2 (2014) 698 710. [23] E. Diamant, Cognitive image processing: the time is right to recognize that the world does not rest more on turtles and elephants, arXiv 1411.0054. [24] C. Xiu, Z. Chai, Target tracking based on the cognitive associative network, IET Image Process 13 (3) (2019) 498 505. [25] A. Holm, K. Lukander, J. Korpela, M. Sallinen, K. Mller, Estimating brain load from the EEG, ScientificWorldJournal 9 (2009) 639 651. [26] E. Niedermeyer, F.H. Lopes da Silva (Eds.), Neurocognitive Functions and the EEG Electroencephalography: Basic Principles, Clinical Applications, and Related Fields, 2, Lippincott Williams and Wilkins, 2005, pp. 661 681.

References

[27] A. Gevins, The future of electroencephalography in assessing neurocognitive functioning, Electroencephalogr. Clin. Neurophysiol. 106 (1998) 165 172. [28] M.A. Bell, C. Kimberly, Using EEG to study cognitive development: issues and practices, Trends Cogn. Sci. 13 (2012) 281 294. [29] L.M. Ward, Synchronous neural oscillations and cognitive processes, Trends Cogn. Sci. 7 (2003) 553 559. [30] E. Basar, C. Basar-Eroglu, S. Karakas, M. Schurmann, Are cognitive processes manifested in event-related gamma, alpha, theta and delta oscillations in the EEG? Neurosci. Lett. 259 (3) (1999) 165 168. [31] G.A. Wiggins, J. Bhattacharya, Mind the gap: an attempt to bridge computational and neuroscientific approaches to study creativity, Front. Hum. Neurosci. 8 (2014) 540 555. [32] U. Hasson, H.C. Nusbam, Emerging opportunities for advancing cognitive neuroscience, Trends Cogn. Neurosci. 1898 (2019) 1 3. [33] S. Grillner, N. Ip, C. Koch, W. Koroshetz, H. Okano, M. Polachek, et al., Worldwide initiatives to advance brain research, Nat. Neurosci. 19 (2019) 1118 1122. [34] N. Bostrom, R. Roache, Smart policy: cognitive enhancement and the public interest, Contemp. Read. Law Soc. Justice 2 (2010) 68 84. [35] N. Bostrom, A. Sandberg, Cognitive enhancement: methods, ethics, regulatory challenges, Sci. Eng. Ethics 15 (3) (2009) 311 341. [36] E.S. Boyden, Optogenetics and the future of neuroscience, Nat. Neurosci. 18 (2015) 1200 1201. [37] R.E. Kandel, H. Markram, P.M. Matthews, R. Yuste, C. Koch, Neuroscience thinks big (collaboratively), Nat. Rev. Neurosci. 14 (2013) 659 664. [38] D.A. Raichlen, G.E. Alexander, Adaptive capacity: an evolutionary neuroscience model linking exercise, cognition, and brain health, Trends Neurosci. 40 (7) (2017) 408 421.

131

CHAPTER

Application of virtual reality systems to psychology and cognitive neuroscience research

7

C.S.N. Koushik, Shruti Bhargava Choubey and Abhishek Choubey Sreenidhi Institute of Science and Technology, Ghatkesar, Hyderabad, India

7.1 Introduction With the advancement of virtual reality, it is very much useful in the fields such as cognitive neuroscience. They are used by many numbers of developers and researchers. Virtual reality uses a three-dimensional (3-D) development for the visualization along with a good amount of programing in it. Virtual reality is used for the social understanding of individuals and their behavior for the inputs that are used to visualize things such as heartbeats on the ECG, eye tracking devices, and gaming based on users’ emotional capability through the spatial navigation techniques and anxiety disorder techniques and others. The actions specified in this are fixed such as moving forward and jumping that can be visualized with the help of programing tools, GUI, scripting languages, etc. Virtual reality can be used to simulate the thinking process that is going on the basis of the emotional state in which a user exists; as a result user’s activities can be reflected on a screen and the process state can be changed in accordance with that. These actions are used to create a library of the models that are 3-D. The end devices can be configured, but their behavior cannot be customized. The position, location, and the time of the user are noted. The GUI and the custom systems differ on the level of their interaction and utility. All the external constraints are related to cognitive neuroscience, which can be used for simulation purposes such as gaming, etc. They can also be implemented using Python tools, which have a large number of predefined libraries, reducing the level of complexity for a user. Python can be used as the programing language for virtual reality and as a result the level of advancement can be increased by using the principles such as artificial intelligence (AI) and deep learning for better results (Fig. 7.1).

Cognitive Informatics, Computer Modelling, and Cognitive Science, Volume 2. DOI: https://doi.org/10.1016/B978-0-12-819445-4.00007-2 © 2020 Elsevier Inc. All rights reserved.

133

134

CHAPTER 7 Application of virtual reality systems

Psychology

Physiology

Cognitive hexagon

AI

Neuroscience

Linguistics

Anthropology

FIGURE 7.1 Cognitive hexagon [1].

7.1.1 Cognitive science Cognitive science is the scientific study of analyzing how the mind works and the analysis is done based on how the nervous system perceives or receives, under a particular condition, the data that is being given as inputs to it. Through cognitive science one can enhance their language, perception, memory, attention, reasoning, and emotion and predict the requirement for the best next alternative. It plays a major role in the decision-making process of an individual which can be represented as computational procedures for an effective understanding of the analysis to be performed. It could be understood from the topics such as linguistics, psychology, AI, philosophy, neuroscience, and anthropology. Psychology is that branch of science where one deals with emotions that affect one’s behavior as per the situation that one pertains to exist; in a simple way, it can be said that it basically depicts how a person is affected when one is subjected to the surroundings and the emotional state or any trauma, by studying how the person’s brain reacts to that sensory inputs. It can help various researchers to understand the difference between humans and animals as to how they tend to differ under various subjected conditions. Human brain can be mainly affected by depression or stress and when one individual enters into that state. it can affect his/her behavior tremendously. A person’s mentality can be easily affected by the hormonal levels in the body as well; for instance, the person can become angry or irritated or suddenly depressed for the change in the hormonal level, and it can even develop various sorts of complexes that can affect the personality traits of the individual to a great extent. Physiology deals with how the organs or any part of the body work/works in an efficient manner with the changes in the inputs or emotions or chemical

7.1 Introduction

substances secreted in the body and the duration for which these are subjected to the body. The physiology may be considered only to the operation/functionality of the part, but when studied further, it can have an effect on the brain’s thinking process as to how it affects its functionality. If the body falls ill, it has a very great impact on the body as the person can have a trauma based on the intensity of the sickness and that can develop stress or depression and can even develop suicidal tendencies, which can affect the concentration of the brain. Linguistics and philosophy deal with the language/literature in its own respective manner, as it can leave an impact on a person who comes across it. Literary works have a great impact on an individual’s mind, and as a result the behavior of the individual can also vary. The person can be motivated or diverted as a result the behavioral patterns need to be analyzed to prevent any sort of disruptions that can be done to work. The individual can take it in an optimistic way or can take the entire work of the author in a pessimistic way, which can greatly do harm to the individual at some point of time in the mere future. Hence, in the field of cognitive neuroscience, it plays an important role in determining the behavioral pattern of an individual when subjected to a wide range of emotional stimuli having a great impact on the individual’s mindset. It plays a major role in the AI systems or neural network systems wherein the implementation of the next steps in the process occurs in an accurate sequence; as a result there is an ease in the representation of outcomes and understanding the behavioral patterns of the network in the system. It can also be implemented with the help of the data analytic algorithms to analyze the future pattern in the behavior of the system model that is being depicted. It becomes an important work of the psychiatrists to do the understanding of the individual’s mind and diagnose if required.

7.1.2 Virtual reality It is the science of understanding various virtual environments by applying various fields such as computer applications/programing, mathematics, science, and analyzing actions. The environments are basically simulated or shown pictorially to clients or users based on emotional aspects of the users. Input devices send to systems the information related to users’ movements. There are various devices such as Google Lens that are useful for virtual reality simulation. It gets inputs based on the sensory nature of the user-like vision, hearing, tactile, gestures, etc. The inputs can provide outputs based on the percentages of affecting levels of outputs, that is, it depends upon the level of dependency of the output on the input applied. The outputs that can be generated are generally virtual environments or any actions that the user needs, especially without which he/she may lose many events such as games and research. Various devices such as haptic and skinput sensitive are used to generate the response for the inputs that are applied [2] (Fig. 7.2).

135

136

CHAPTER 7 Application of virtual reality systems

Virtual reality applications: •Mathematics •Statistics •Probabilities •Robotics •AI •Designing •Computer applications •Simulations •Electric engineering

FIGURE 7.2 Virtual reality applications [2].

7.2 Literary survey review Virtual reality finds various applications in the fields of the healthcare/physiology and psychology and in various real-time applications in product development and gaming, which helps various scientists to help them in their research activities. There are basically many mining or data science algorithms on the data that they have collected, which can simulate the results in a 3-D virtual environment. The most famous algorithms used include k-means clustering that plays a major role in the unsupervised or semisupervised learning aspects for various users. It can even have various algorithms used along with principles such as AI and machine learning to enhance the output of the systems. There can be various visualization tools used for it to have the visualization to be done easily with the latest advancements in technology [1,2].

7.2.1 Cognitive neuroscience/physiology Physiology is the branch of biology that deals with the normal functioning of the body and its organs. The people with both mental or physical or any sickness can be diagnosed very easily as with the help of the virtual reality, the symptoms of these diseased persons can be visualized and can be understood very easily. The people suffering from various mental syndromes, stress disorders, and anomalies of the organ functioning can be diagnosed easily without any difficulty due to simulation with the help of the computerized models that are very accurate [3]. It has the potential to revolutionize medical industry by increasing the automation and reducing the chances of loss of lives and using of robots that are controlled by the humans who can see the simulation and complete the surgeries very effectively.

7.2 Literary survey review

In the exposure therapy, people suffering from disorders are subjected to stimuli with which they are affected very badly, that is, stimuli such as allergycausing agents and fear-causing agents such that they can be diagnosed very easily. Parkinson’s syndromes etc. can be detected. The data from the people are collected, upon which analysis is done by finding out the standard errors, or the outliers are detected, in the computerized simulated virtual environment. The dataset is taken in which the subject to be tested is present, which is tested or compared with the standard dataset. The dataset is scanned from which all the outliers are separated and the useful data for that sickness is recorded, which founds the basis for the unsupervised or semisupervised learning. For example, for people suffering from various mental disorders or syndromes, all the aspects are recorded, such as their behavior at various instants and the symptoms in those situations, which are analyzed. It forms an unsupervised learning which is analyzed from the data that is quite huge. The mining algorithms such as the k-means clustering or the association rules (Apriori) algorithm can be applied in order to understand the causes and what sets of symptoms can be present at each instant of time and that particular location [3,4]. The final processed data can be simulated in a virtual environment. It can be even used to predict the amount of medicines to be taken by the affected people based on the level of sickness that can be detected based on the AI and machine learning algorithms. The datasets can be affected based on the duration for which they are affected. It can even be used to detect cancer in the body. The activities of the brain are recorded, and with the help of EEG signals the functioning of the entire central nervous system can be controlled. In the case of various diseases that an individual possesses, it can also be detected in the person due to the simulation of all the inputs that the system receives. An AI system is needed in order to detect the disorders, wherein with the help of various neural networks, all those disorders can be found. Various algorithms of deep learning can be applied in order to detect the disorder and suggest the best suitable treatment for the disorder and even predict the further stages that can be found in the individual when the treatment is neglected. Systems with these software programs tend to operate efficiently rather than the hardware part only. Its accuracy can be increased to a greater extent.

7.2.2 Cognitive psychology Psychology is the branch of science of understanding the human behavior. For the people who are psychotic or act like maniacs or are suffering from various problems where they cannot interact with people socially properly due to their past experiences that have played a major role in it, their behaviors are studied by recording every aspect carefully as that can provoke them at times. Hence, it is essential to record their activities without offending them and comparing them with the dataset for the worst sort of the psychotic behavior from which the data can be actually retrieved based on the traits that they possess. All the unwanted

137

138

CHAPTER 7 Application of virtual reality systems

Direct referrals

Psychotic patients

Screening eligibility

Diagnosis

Excluded from treatment

Allot treatment procedure and follow up

FIGURE 7.3 Flow chart for psychotic people [5].

Sensory input

Animal’s central nervous system

Reflex got to the motor system

Virtual reality system

FIGURE 7.4 Physiological behavior gesture in animals [4].

data is filtered, that is, the outliers are detected upon the preprocessing stages and then the filtered data can be analyzed and the best treatment can be given to them. Their behavior can be simulated and predicted for their next activities [5]. There are various sorts of psychotic people like paranoids for whom techniques vary based on the inclusion method, for which techniques such as GPTS (Green Paranoid Thoughts Scale), wherein the thought process of the paranoids is understood, and based on the outcomes, they can be empathized and they can be cured; similarly, there are SBQs (safety-based questionnaires), used at the public places, which show how they behave at those places and others. Based on the outcomes of the analysis, a various set of treatments (T) can be given [5,6] (Figs. 7.3 and 7.4). Similarly, even the case of any animal’s or any person’s behavior can be studied by understanding the behavior with the help of the unsupervised learning.

7.2 Literary survey review

The input of every movement is taken in order to understand the all-round behavioral aspects of the organism at every time and location instants. Based on its present emotional aspect and behavioral patterns, it is analyzed, and its future behavior and the movements can be predicted, and future locations can be predicted. The dataset of the various aspects is taken, in which the condition is recorded in a table and the outcomes are generated upon which the data can be processed to generate the accurate simulation results. Simultaneously, the data can be recorded for various instances at that time and location, and the data can be processed for various aspects and its future outcomes as well, that is, every minute aspect is time stamped [4]. The data can be collected from various sources, based on the level of the hormones and their secretions and their body condition at that time. These aspects can even be used to understand what will be their future behavioral traits that can be used solely for research purposes [6]. Animals like mice, rabbit, and other animals can be used for research purposes with utmost care of protecting the animals. As these animals tend to behave similar to the humans, they have been selected and they even tend to have similar human tendencies at the time of difficult situations even though they lack in brilliance compared to humans. All these aspects like when they are happy, sad, angry, concerned nature, tensed, and what times they are alert can be predicted with the help of the virtual reality. All the aspects can be predicted when they subjected to those particular conditions that they can generate the various outputs in an emotional manner that are useful to predict in any research work. If the person has any sort of phobias from spiders, snakes, dogs, dust, close rooms, then the person is made to be subjected to these respective fear-causing agents due to which, under the exposure therapy, the person’s behavior can be understood, which can play a major role in overcoming the fear. The person’s data for all adrenaline rush and emotional fear levels can be recorded, from which the intensity of the fear can be analyzed from the processed data. The processed data is taken based on the mean, variance, and whisker ranges. The data will be simulated on the display screen and the treatment can be given [3,7]. The data visualization can be done in box plots. The behavioral traits in every condition of time and location are recorded for the sake of the treatment or research, which can generate the various possibilities of the individual to do in the next upcoming future, based on the type of sensory inputs that are given to the individual based on the phobias. These alone can be given to the individual as a simulated sensory input rather than a way to predict the future for the sake of treatment of individuals. It can be even used for individuals that suffer from various syndromes as well as a part of their treatment. A particular fundus camera can be used in order to record the conditions and that can be given to an AI system, which, with the help of the neural networks, can predict the suitable next steps of action in either research or in treatment to provide medication and further course of preventive steps that are to be taken (Fig. 7.5). Even in gaming, virtual reality plays a major role in the user’s performance in the game and increases the chances of winning the games. The player’s

139

140

CHAPTER 7 Application of virtual reality systems

Virtual reality process

Sensory input

Central nervous system of an individual

Virtual reality system

AI system that has recorded all the aspects

Display on a screen

FIGURE 7.5 Virtual reality process [6].

movements are recorded, which helps the computerized model to generate the outputs by predicting the next best move for the player. All the movements are recorded either physically from the motion sensor or from the dataset, and then based on the association rules application (Apriori algorithm), the best moves are supported to the user’s present moves. The output efficiency can be enhanced by the use of the AI. The analysis of the data can be achieved based on the behavioral and questionnaire basis to process the data [8]. The moves can be affected by the emotional level of the user and type of the games that the user wants to play. It helps one to have an empathy with the user and generate better results. Based on the level of interest of the gamer, the level of performance can be enhanced such that the developed idea or procedure can help both the gamer and the developer. It can even help the developer to enhance the interface of the individual and even the use of game in the market. It can be used in various malls in order to attract more customers, which can be used as a perfect marketing strategy. All the emotional aspects are recorded from various users, which can be used to predict the market sales of the product in the future. Virtual reality can show all the scenarios of the game in a detailed manner to the gamer to increase the chances of winning through virtual simulation and even predicting what must be done in the next step by an AI system (Fig. 7.6). When a person reads a book or novel or any scientific content, based on his/ her understanding capabilities, the person’s thought process can be simulated and the level of imagination of the reader can be depicted. It can be useful in the field of education, which can help students very much in understanding contents of the subject and future possible topics in advance and even in the field of business management for future prediction of sales. It can help very much in the research aspects for the scientists to simulate the alternative paths or ideas to their areas of research. So, whenever the reader reads a material, based on the intensity of the

References

Sensory input

Human’s central nervous system

Reflex for the motor system

Action required in gaming like mobility etc.

Virtual reality system

FIGURE 7.6 Learning with virtual reality For gaming [9,10].

emotional connectivity with the literature, the future sales of the book, future emotional state of the reader, and continuity of the story can be predicted or the scientific processes can be altered to have better optimal results. It helps kids in understanding lessons very easily through the simulation and even analyzing the things with the help of AI and machine leaning for the predictions of their character or behavior [9,10].

7.3 Conclusion Virtual reality plays an important role in understanding the behavior of human being through cognitive science. It may enhance the outcomes of the task that is carried out by the individual and decrease the load on the person. Virtual reality plays an important role in the field of neuroscience/brain’s physiology, wherein the functionality of the brain of the individual can be studied and analyzed for better treatment and surgeries if required. It even plays an important role in understanding the behavior of the individuals for their interests in literature, gaming, education, and even understanding their mental behavior in between others. Virtual reality may reduce the cost of production or investment, which is reduced by simulating the requirement in software and meeting requirements in an effective manner by estimating the drawbacks. Principles like AI and machine learning can be implemented to enhance the outcomes of the situation and requirements.

References [1] O. Krasnyak, M. Fanguy, E. Tikhonova, Linguistic Approaches in Teaching History of Science and Technology Courses Through a Content Block on Cognitive Sciences Article, September 2016. Available from: ,https://doi.org/10.17323/2411-7390-20162-3-32-44..

141

142

CHAPTER 7 Application of virtual reality systems

[2] L.S. Machado, R.M. De Moraes, Intelligent Decision Making in Training Based on Virtual Reality, Atlantis Computational Systems, Article, January 2010. Available from: ,https://doi.org/10.2991/978-94-91216-29-9_4.. [3] T.D. Parsons, A.A. Rizzo, Affective outcomes of virtual reality exposure therapy for anxiety and specific phobias: a meta-analysis, J. Behav. Ther. Exp. Psychiatry 39 (3) (2008) 250 261. [4] D.A. Dombeck, M.B. Reiser, Real neuroscience in virtual worlds, Curr. Opin. Neurobiol. 22 (2012) 3 10. [5] R. Pot-Kolder, W. Veiling, C. Geraets, M. van der, Effect of virtual reality exposure therapy on social participation in people with a psychotic disorder (VRETp): study protocol for a randomized controlled trialGaag, Pot-Kolder et alTrials 17 (2016) 25. Available from: https://doi.org/10.1186/s13063-015-1140-0. [6] D.A Dombeck, L. Tian, Functional imaging of hippocampal place cells at cellular resolution during virtual navigation, Nat. Neurosci. 2010. Available from: ,https:// doi.org/10.1038/nn.2648.. [7] A.S. Carlin, H.G. Hoffman, S. Weghorst, Virtual reality and tactile augmentation in the treatment of spider phobia: a case report, Behav. Res. Ther. 35 (2) (1997). Available from: https://doi.org/10.1016/S0005-7967(96)00085-X. [8] M. Gonza´lez-Franco, D. Pe´rez-Marcos, B. Spanlang, M. Slater, The contribution of real-time mirror reflections of motor actions on virtual body ownership in an immersive virtual environment, in: IEEE Virtual Reality 2010, 20 24 March, Waltham, MA. [9] M.K. Barbour, T.C. Reeves, The reality of virtual schools: a review of the literature, Comput. Educ. 52 (2009) 402 416. [10] I. Marfisi-Schottman, S. George, Supporting teachers to design and use mobile collaborative learning games, in: Proceedings of the International Conference on Mobile Learning, 28 February 2 March 2014, Madrid, Spain, pp. 3 10.

Further reading A.W. Logue, Watson’s Behaviorist Manifesto: Past Positive and Current Negative Consequences, Greenwood Press/Greenwood Publishing Group, Westport, CT, 1994. S.V. Adamovich, G.G. Fluet, E. Tunik, A.S. Merians, Sensorimotor training in virtual reality: a review, NeuroRehabilitation 25 (2009) 29 44. E. Aksay, G. Gamkrelidze, H.S. Seung, R. Baker, D.W. Tank, In vivo intracellular recording and perturbation of persistent activity in a neural integrator, Nat. Neurosci. 4 (2001) 184 193. M. Alcan˜iz, et al., GeRTiSS: generic real time surgery simulation, in: Studies in Health Technology and Informatics, IOSPress, 2003. American Association for the Advancement of Science, Project 2061: Science for All Americans, 1989. N.H. Anderson, Cognitive algebra: integration theory applied to social attribution, In: L. Berkowitz (Ed.), Advances in Experimental Social Psychology, 1974. N.H. Anderson, Integration theory applied to cognitive responses and attitudes, In: R.E. Petty, T.M. Ostrom, T.C. Brock (Eds.), Cognitive Responses in Persuasion, 1981.

Further reading

L.E. Baum, An inequality and associated maximization technique in statistical estimation for probabilistic functions of Markov processes, in: Inequalities, 1972. L.E. Baum, T. Petrie, Statistical inference for probabilistic functions of finite state Markov chains, Ann Math. Stat. 37 (1996) 1554 1563. T. Becker, G. Thornicroft, M. Leese, P. Mccrone, S. Johnson, M. Albert, et al., Social networks and service use among representative cases of psychosis in south London, Br. J. Psychiatry 171 (1) (1997) 15 19. J. Benda, T. Gollisch, C.K. Machens, A.V. Herz, From response to stimulus: adaptive sampling in sensory physiology, Curr. Opin. Neurobiol. 17 (4) (2007) 430 436. A. Benko, C. Sik La´nyi, History of artificial intelligence, In: M. Khosrow-Pour (Ed.), Encyclopedia of Information Science and Technology, second ed., 2009. doi: 10.4018/ 978-1-60566-026-4.ch276. A. Berardi-Wiltshire, P. Petrucci, Bringing linguistics to life: an anchored approach to teaching linguistics to non-linguists, Te Reo, 2015. J.C. Bezdek, A review of probabilistic, fuzzy and neural models for pattern recognition, J. Intell. Fuzzy Syst. 1 (1993) 1 25. N. Bonnet, J. Cutrona, Improvement of unsupervised multi-component image segmentation through fuzzy relaxation, in: Proc. of IASTED Int. Conf. on Visualization, Imaging and Image Processing, 2001. C. Borgelt, R. Kruse, Graphical Models: Methods for Data Analysis and Mining, Wiley, 2002. H. Borotschnig, et al., Fuzzy relaxation labeling reconsidered, in: Proc. IEEE World Congress on Computational Intelligence, FUZZ-IEEE, 1998. P. Brown, S.C. Levinson, Politeness: Some Universals in Language Usage, Cambridge University Press, Cambridge, 1987. B.G. Buchanan, E.H. Shortlife, Rule-Based Expert Systems: The MYCIN Experiments of the Stanford Heuristic Programming Project, Addison-Wesley, 1985. J. Cheng, R. Greiner, Learning Bayesian belief network classifiers: algorithms and system, in: Proc. Fourteenth Canadian Conference on Artificial Intelligence, 2001. N. Chomsky, Knowledge of Language: Its Nature, Origin, and Use, Greenwood Publishing Group, Westport, CT, 1986. Chomsky, N. (2016). N. Chronis, M. Zimmer, C.I. Bargmann, Microfluidics for in vivo imaging of neuronal and behavioral activity in Caenorhabditis elegans, Nat. Methods (2007). Cognition 65, 71 86 I. Feinberg, Efference copy and corollary discharge: implications for thinking and its disorders, Schizophr. Bull. 4 (1978) 636 640. R.A. Johnson, D.W. Wichern, Computational intelligence in complex decision systems, in: Applied Multivariate Statistical Analysis, fifth ed. Prentice Hall, 2001. J.A. Coyne, Science, religion, and society: The problem of evolution in America, Evolution 66 (2012) 8. N. Creanza, et al., A comparison of worldwide phonemic and genetic variation in human populations, Proc. Natl. Acad. Sci. U.S.A. 112 (5) (2015) 1265 1272. E. Daprati, et al., Looking for the Agent: An Investigation Into Consciousness of Action and Self-Consciousness in Schizophrenic Patients, 1997. M.R. Dawson, Understanding Cognitive Science. Blackwell, Oxford, 1998. A.P. Dempster, N.M. Laird, D.B. Rubin, Maximum likelihood from incomplete data via EM algorithm, J. Royal Stat. Soc. (1977). C. Doring, C. Borgelt, R. Kruse, Fuzzy Clustering of Quantitative and Qualitative Data, in: Proc. of the 2004 NAFIPS, 2004.

143

144

CHAPTER 7 Application of virtual reality systems

M. Drake, Radical Preacher Brainwashed Young Men to Unleash Paris and Brussels Terror Attacks, 2016. D. Dubois, H. Prade, What are fuzzy rules and how to use them, Fuzzy Sets Syst. 84 (1996) 169 185. U.W. Eisenecker, AI: The tumultuous history of the search for artificial intelligence, AI Commun. 8 (1) (1995) 45 47. N. Evans, S.C. Levinson, The myth of language universals: language diversity and its importance for cognitive science, Behav. Brain Sci. 32 (2009) 429 492. C. Everett, et al., Climate, vocal folds, and tonal languages: connecting the physiological and geographic dots, Proc. Natl. Acad. Sci. U.S.A. 112 (5) (2015) 1322 1327. L.S.N. Fatima, M. Rosa, The virtual reality challenges in the health care area: a panoramic view, ACS, 2008. Available from: ,https://doi.org/10.1145/1363686.1363993.. T. Franze´n, Go¨del’s Theorem: An Incomplete Guide to Its Use and Abuse, AK Peters, Wellesley, MA, 2005. S. Freud, The unconscious, J. Nerv. Ment. Dis. 56 (3) (1922) 291 294. Y.N. Harari, The theater of horror, Guardian (2015). K.S. Fu, T.S. Yu, Statistical Pattern Classification Using Contextual Information, Research Studies Press, 1980. A. Gande, V. Devarajan, Instructor station for virtual laparoscopic surgery: requirements and design, in: Proc. of Computer Graphics and Imaging, 2003. Z. Gao, A. Le´cuyer, A VR simulator for training and prototyping of telemanipulation of nanotubes, in: Proc. ACM Simp. on Virtual Reality Software and Technology, 2008. O. Garaschuk, O. Griesbeck, A. Konnerth, Troponin C-based biosensors: a new family of genetically encoded indicators for in vivo calcium imaging in the nervous system, Cell Calcium (2007). A. Gorini, C.S. Capideville, G. De Leo, F. Mantovani, G. Riva, The role of immersion and narrative in mediated presence: the virtual hospital experience, Cyberpsychol. Behav. Soc. Netw. (2011). Available from: https://doi.org/10.1089/cyber.2010.0100. D.S. Greenberg, A.R. Houweling, J.N. Kerr, Population imaging of ongoing neuronal activity in the visual cortex of awake rats, Nat. Neurosci. (2008). K.P. Harden, Genetic influences on adolescent sexual behavior: why genes matter for environmentally oriented researchers, Psychol. Bull. 140 (2) (2014) 434 465. P. Harstela, The future of timber harvesting in Finland, Int. J. For. Eng. 10 (1999) 2. R.M. Hazen, J. Trefil, Science Matters, 1991. R.A. Hummel, S.W. Zucker, On the foundations of relaxation labeling processes, in: IEEE Transactions on Pattern Analysis and Machine Intelligence, 1983. A. Iriki, M. Taoka, Triadic (ecological, neural, cognitive) niche construction: a scenario of human brain evolution extrapolating tool use and language rom the control of reaching actions, in: Philosophical Transactions in the Royal Society of London, 2012. W. James, The Principles of Psychology, Dover Publ. (reprinted 1950), 1890. M. Jeannerod, The representing brain: neural correlates of motor intention and imagery, Behav. Brain Sci. 17 (1994) 187 245. N.W. John, et al., Web-based surgical educational tools, in: Studies in Health Technology and Informatics, IOSPress, 2001. B. Johnson, Bombing Syria is not the whole solution but it’s a good start, The Telegraph, 2015.

Further reading

B-H. Juang, L.R. Rabiner, The segmental K-means algorithm for estimating parameters of hidden Markov models, in: IEEE Trans. Acoustics, Speech and Signal Processing, 1990. P.J. Krause, Learning probabilistic networks, Knowl. Eng. Rev. 1998. B. Kuhn, W. Denk, R.M. Bruno, In vivo two-photon voltage-sensitive dye imaging reveals top-down control of cortical layers 1 and 2 during wakefulness, Proc. Natl. Acad. Sci. U.S.A. (2008). T.S. Kuhn, The Structure of the Scientific Revolutions. University of Chicago Press, Chicago, IL, 1962. D.R. Kumar, F. Aslinia, S.H. Yale, J.J. Mazza, Jean-Martin Charcot: the father of neurology, Clin. Med. Res. 9 (2011) 46 49. R. Kurzweil, The Singularity Is Near: When Humans Transcend Biology, 2006. G. Lakoff, M. Johnson, Conceptual metaphor in everyday language, J. Philos. (1980). G.W. Leibniz, Theodicy: Essays on the Goodness of God, the Freedom of Man and the Origin of Evil, Wipf and Stock Pub, Eugene, OR, 2000. Linking past and present. A view of historical comments about language, AILA Review 24 (1) (2011) 55 67. J. Locke, Drafts for the Essay Concerning Human Understanding, and Other Philosophical Writings. Clarendon Press, Oxford, 1990. G. Lokhorst, Descartes and the Pineal Gland, The Stanford Encyclopedia of Philosophy, 2015. G. Luksys, et al., Computational dissection of human episodic memory reveals mental process-specific genetic profiles, Proc. Natl. Acad. Sci. U.S.A. 112 (35) (2015) E4939 E4948. L. Luo, E.M. Callaway, K. Svoboda, Genetic dissection of neural circuits, Neuron (2008). E.M. Macdonald, R.L. Hayes, A.J. Baglioni, The quantity and quality of social networks of young people with early psychosis compared with closely matched controls, Schizophr. Res. (2000). L.S. Machado, R.M. Moraes, An online evaluation of training in virtual reality simulators using fuzzy Gaussian mixture models and fuzzy relaxation labeling, in: Proc. IASTED International Conference on Computers and Advanced Technology in Education (CATE’2003), 2003. L.S. Machado, R.M. Moraes, VR-based simulation for the learning of gynaecological examination, Lect. Notes Comput. Sci. (2006). L.S. Machado, et al., Assessment of gynecological procedures in a simulator based on virtual reality, in: Proc. Seventh International FLINS Conference on Applied Artificial Intelligence, 2006. L.S. Machado, A.N. Mello, R.D. Lopes, V. Odone Fo, M.K. Zuffo, A virtual reality simulator for bone marrow harvest for transplant, Stud. Health Technol. Inf. 81 (2001) 293 297. E.A. Maguire, N. Burgess, J. O’Keefe, Human spatial navigation: cognitive maps, sexual dimorphism, and neural substrates, Curr. Opin.: Neurobiol. (1999). G. Maimon, A.D. Straw, M.H. Dickinson, Active flight increases the gain of visual motion processing in Drosophila, Nat. Neurosci. (2010). W. Mason, J.W. Vaughan, H. Wallach, Computational social science and social computing, Mach. Learn. (2014).

145

146

CHAPTER 7 Application of virtual reality systems

N. McDermott, Chief Medic Warns Britain: Obesity Is as Big a Risk as Terrorism to Health and NHS, The Sun, December 11, 2015. G. Miesenbock, The optogenetic catechism, Science (2009). J. Millar, How Evil ISIS Are Like a DISEASE Spreading to Become a Global Terror Group, The Daily Express, August 4, 2016. M. Minsky, The Emotion Machine, Pantheon, New York, 2006. E. Nieh, et al., Decoding neural circuits that control compulsive sucrose seeking, Cell (2015). A. Miri, K. Daie, R.D. Burdine, E. Aksay, D.W. Tank, Regression-based identification of behavior-encoding neurons during large-scale optical imaging of neural activity at cellular resolution, J. Neurophysiol. (2011). M. Moutoussis, J. Williams, P. Dayan, R.P. Bentall, Persecutory delusions and the conditioned avoidance paradigm: towards an integration of the psychology and biology of paranoia, Cogn. Neuropsychiatry (2007). Available from: https://doi.org/10.1080/ 13546800701566686. U. Neisser, Five kinds of self-knowledge, Philos. Psychol. 1 (1988) 35 59. M. Nyer, J. Kasckow, I. Fellows, E.C. Lawrence, S. Golshan, E. Solorzano, et al., The relationship of marital status and clinical characteristics in middle-aged and older patients with schizophrenia and depressive symptoms, Ann. Clin. Psychiatry (2010). G. O’Regan, Marvin Minsky, Giants of Computing, Springer, London, 2013. Olga Krasnyak, Mik Fanguy, Elena Tikhonova, Linguistic approaches in teaching history of science and technology courses through a content block on cognitive sciences, J. Language & Education 2 (3) (2016). D. Opri¸s, S. Pintea, A. Garc´ıa-Palacios, C. Botella, S. ¸ Szamosko¨zi, D. David, Virtual reality exposure therapy in anxiety disorders: a quantitative meta-analysis, Depress. Anxiety (2012). Available from: https://doi.org/10.1002/da.20910. M.B. Orger, A.R. Kampff, K.E. Severi, J.H. Bollmann, F. Engert, Control of visually guided behavior by distinct populations of spinal projection neurons, Nat. Neurosci. (2008). F. Parker, K. Riley, Linguistics for Non-Linguists: A Primer With Exercises, third ed., Allyn & Bacon, Boston, MA, 2000. M.A. Paveau, Do non-linguists practice linguistics? An anti-eliminative approach to folk theories, AILA Rev. 24 (2011) 40 54. R. Perkins, Unemployment rates among patients with long-term mental health problems: a decade of rising unemployment, Psychiatr. Bull. (2002). Available from: https://doi. org/10.1192/pb.26.8.295. G. Perry, R. Mace, The lack of acceptance of evolutionary approaches to human behaviour, J. Evol. Psychol. 8 (2) (2010) 105 125. Pinker, S. (1994). S. Pinker, The cognitive niche: coevolution of intelligence, sociality, and language, Proc. Natl. Acad. Sci. U.S.A. 107 (2010) 8993 8999. C.B. Rohde, F. Zeng, R. Gonzalez-Rubio, M. Angel, M.F. Yanik, Microfluidic system for on-chip high-throughput whole-animal sorting and screening at subcellular resolution, Proc. Natl. Acad. Sci. U.S.A. 104 (2007) 13891 13895. C. Ryan, C. Jetha´, Sex at Dawn: The Prehistoric Origins of Modern Sexuality, 2010. K. Schwab, The Fourth Industrial Revolution, World Economic Forum, Geneva, Switzerland, 2016. Schwab, N. (2015, December 15). J. Shear, Explaining Consciousness: The Hard Problem. MIT Press, Cambridge, MA, 1999.

Further reading

V. Stoykova, Teaching corpus linguistics. Proc. Soc. Behav. Sci. 2014. Available from: ,https://doi.org/10.1016/j.sbspro.2014.07.513.. M.J. Tarr, W.H. Warren, Virtual reality in behavioral neuroscience and beyond, Nat. Neurosci. (2002). L. Tian, S.A. Hires, T. Mao, D. Huber, M.E. Chiappe, S.H. Chalasani, et al., Imaging neural activity in worms, flies and mice with improved GCaMP calcium indicators, Nat. Methods (2009). N. Tinbergen, The Study of Instinct, Oxford University Press, 1951. N. Burgess, E.A. Maguire, J.: O’Keefe, The human hippocampus and spatial and episodic memory, Neuron (2002). V.A. Uspensky, Go¨del’s incompleteness theorem, in: Theoretical Computer Science, 1994. T.A. van Dijk, in: A. Wilton, H. Wochele (Eds.), Discourse and Context. A Sociocognitive Approach, Cambridge University Press, Cambridge, 2008 (2011). P. Wintour, Boris Johnson Says Assad Must Go If Syrians’ Suffering Is to End, The Guardian, July 19, 2016. P. Yourgrau, A World Without Time: The Forgotten Legacy of Go¨del and Einstein. Basic Books, New York, 2009.

147

CHAPTER

Electrodermal activity and its effectiveness in cognitive research field

8

Abdul Momin1, Ambika Shahu2, Sudip Sanyal3 and Pavan Chakraborty1 1

Department of Information Technology, Indian Institute of Information Technology, Allahabad, India 2 Cognitive Science Lab, International Institute of Information Technology, Hyderabad, India 3 Department of Computer Science, BML Munjal University, Gurugram, India

8.1 Introduction It is almost 140 years (in the early 1880s), researchers first observed a close relationship between electrodermal activity (EDA) and psychological factors. The observation makes it an immediate interest among the researchers. The psychophysiological significance of the signal and the ease of acquisition of an eventoriented electrodermal response (ERP) was the major reason behind this rapid and widespread use of this newly observed phenomenon. Speaking very broadly, EDA could be collected from skin and the signal majorly captures the autonomic nervous system (ANS) activities. But this is not all about the EDA signal and its application. Before starting experiment with EDA, one should cover all the basic principles of EDA. What are those psychological and psychophysiological significances? Where can we employ EDA signals? What kind of experimental setup we need to make the best use of EDA signal? How to process and analyze the EDA signals? These thoughts are eminent if someone wants to work with EDA signals. We will try to touch every base in this chapter but before going into all those details, it is always good to know a bit of history of the subject.

8.2 History of electrodermal activity signal, psychophysiological, and physiological mechanism behind electrodermal activity It was DuBois-Reymond (in 1849) [1] who first discovered the presence of electrical phenomena presented in human skin. They observed that electrical current is flowing from one limb at rest to other. But unfortunately, they were completely unaware of the newly observed phenomena and failed to justify the observed Cognitive Informatics, Computer Modelling, and Cognitive Science, Volume 2. DOI: https://doi.org/10.1016/B978-0-12-819445-4.00008-4 © 2020 Elsevier Inc. All rights reserved.

149

150

CHAPTER 8 Electrodermal activity and its effectiveness

phenomena. So they reached out to other fellow scientists and after an elaborated discussion, DuBois-Reymond opined that the observed phenomena could be due to the muscle action potentials. The actual relationship between sweat gland and current flow in skin came to lights when Hermann and Luchsinger [2] conducted their experiment. They conducted their experiment with curarized cats and established that the electric stimulation of the sciatic nerve resulted in sweat secretion. They also tried to replicate the experiment conducted by DuBois-Reymond [1] and proved the significance of sweat glands in EDA of skin [1]. These were the best known innovation history of EDA summarized by Neumann and Blanton [1]. Before going into the evolution of EDA, may be this is the good time to introduce our readers about different terminology and measurement process of EDA, and this will help them to understand the overall concept of EDA. The term EDA, which we use now, was first used by Johnson and Lubin in 1966 [3]. Before them many researchers used different terminologies to refer EDA of skin. They have used different terms mainly based on the acquisition process. The actual measurement and the electrical process underlying will make things a bit lengthy and will also make the readers a bit confused. However, to give our reader a clear picture on electrodermal process, we have tried to summarize the most important things related to electrodermal recordings in a few lines. For the sake of simplicity, we will not discuss the electrical process associated with it and will leave it for the curious readers. Mainly, we can collect the electrodermal recordings either by applying external current on skins surface or just measuring the potential difference originating in the skin. EDA measurement process, where we apply some external current on skin surface, is generally known as exosomatic process. In exosomatic, either we apply alternate current or direct current (DC) on skin to record the EDA. For using the method where we apply DC, we do two things: either we can keep the voltage or the current constant. When we keep the voltage constant, we record the EDA directly in skin conductance (SC) units. On the other hand, if we keep the current constant, then we record EDA in skin resistance (SR) units. Likewise, we can keep the effective voltage or effective current constant. Researchers have used the term skin admittance (SY) when they have used the effective voltage as constant and in the case of constant effective current they have recorded the EDA as skin impedance (SZ). In the other process, also known as the endosomatic process, we do not apply any external current or voltage; we simply measure the potential difference originating in the skin. This is also called skin potential (SP). Except these, we associate two more terminologies with EDA, namely, “response (R)” and “level (L),” and these two terminologies further divide electrodermal response into two more categories. These are phasic (EDR; 1/4 electrodermal response or reaction) and tonic [1/4 electrodermal level (EDL)] phenomena. These two phenomena play a significant role in the analysis of EDA, and we will talk about it in detail later. Galvanic skin response was another terminology, which was in use during early days of EDA research. But the modern literature does not recommend the use of this terminology. In a nutshell, the same

8.2 History of electrodermal activity signal, psychophysiological

phenomena have been named differently on the basis of measuring processes. But too many aliases were creating confusion among researchers. To make things more clear, Johnson and Lubin [3] first used the term “EDA” to refer to all the electrodermal phenomena present in skin. Just 1 year later, the Society of Psychophysiological Research [4] proposed for standardization of EDA. We hope this section will help the readers to understand the different terminologies they will encounter during the literature survey. But during early phase of EDA, there were several hypotheses regarding EDA studies. Without discussing those hypotheses, the section will remain incomplete. In 1879 Vigouroux [5] first observed the relationship between psychological factors and EDA, but he presumed something different. He was working with hysterical patients and tried to measure the SR changes. He saw that the amount of SR changes is proportional to the amount of anesthesia applied on hysterical patients. He tried to explain the observed phenomena as part of the central process and ignored the fact that it might be due to any local changes. It will be worthwhile to mention here that his assumption was in-line with then trending research on “autonomic nervous control of blood flow.” Anyway, Fe´re´ [6] and Tarchanoff [7] could be names as the main pioneer considering all the factors associated with EDA. These two scientists led the way and discovered the essentials of electrodermal phenomena and also tried to establish the relationship with psychological factors. Readers might get amazed knowing that these two scientists were from different countries and they were unaware of each other’s works when they started working on the electrodermal phenomena. Fe´re´ and Tarchanoff came to know about each other’s works only when they published their work in the same journal [1,8]. Although they both worked on electrodermal phenomena, their works have significant difference. Fe´re´ [8] used the exosomatic (applying external DC) to investigate the SR changes following sensory or emotional stimulation [1] and presumed that the changes in SR were due to the vasomotor phenomena [9]. On the other hand, Tarchanoff [7] tried the endosomatic approach and tried to observe the SP changes during different cognitive processes (such as imagination and mental arithmetic). Furthermore, Tarchanoff also did mention the relationship between sweat gland activity and secretory nerves [7]. This was quite new at that time and contrary to the vasomotor theory. Later researchers noticed that the plethysmographic changes and EDR changes are two independent phenomena. Hence, the interpretation of SRR as vasomotor phenomena lacks some essential details and is no longer in practice. Another hypothesis, which suggested that EDR phenomena are due to involuntary muscle activity, also got dropped due to the lack of evidence [9,10]. Similarly, theory of polarization of skin behind the EDR phenomena [11 13] also came into existence during that time but failed to grab the attention [13]. The most decisive study related to endosomatic EDA was carried out by Gildemeister and Rein [12] in 1928 29. The duo made some notable contribution toward endosomatic EDA concept. But the most valid theory was proposed by Richter [14] in 1929. He proposed the casual mechanism for EDA, including both epidermal and sweat gland mechanism [11]. This theory is

151

152

CHAPTER 8 Electrodermal activity and its effectiveness

still in use and regarded valid [14,15]. Apart from these many theories, if we talk about the use of EDA in the psychological studies then it is Veraguth [8] who first talked about the perfect relationship. After his article on EDA, psychiatrists and psychologists got interested in EDA. The research studies involving EDA got a boost after Veraguth article [8]. After this early phase, Darrow [16 18] emerged as one of the pioneers in EDA research. A thorough early research on electrodermal methodologies and psychological studies could be found in the book of Woodworth and Schlosberg [19]. During the early days of EDA research the main barrier was the physical limitation of EDA instruments, and researchers found it difficult to integrate EDA in different studies. But in the last half of the previous century, we saw some massive improvements in equipment (such as oscilloscope, polygraph, and especially the modern computer technology and integrated amplifiers). These technologically improved and more sophisticated instruments gave the EDA researchers a new dimension to explore. Though initially researchers tried to investigate nonhuman (mainly animals) subjects [20,21] and it is a matter of fact that we cannot generalize these things for humans, the study of Wang [22] and Sequeria and Roy [23] on the central origin of EDA added the much-needed background knowledge of EDA recordings. The basic research on human subjects started shortly after this period. The story of development of various EDA devices and their developments are very fascinating, but unfortunately due to limited scope we will not discuss it here but enthusiastic researchers may go through the book on EDA written by Boucesin [24]. Rather, a short discussion on the physiological facts of EDA will be a great exposure for our readers. In the next few paragraphs, we will talk about background physiological mechanism of EDA. To understand the physiological mechanism behind EDA, we need to know the architecture of the skin, as skin is the primary source of EDA. But skin architecture is very complex in nature, and our readers, coming from different backgrounds, may find themselves at a loss. So for simplicity, we will discuss basic mechanism of skin and sweat glands, which influence the EDA recordings the most. We would like to show a sample structure of human skin (taken from Boucesin’s book [24]). This will help our readers to visualize two basic components (i.e., sweat gland and ANS) of skin and EDA recordings as well (Fig. 8.1). It is evident from the above picture that sweat glands are surrounded by the nerve nets (typically ANS), but except this there are many other effectors and sensory organs present in the skin. These free ending nerves, present in the skin, carry different signals and influence the behavior of sweat secretion. This is the main concept behind EDA recordings. To be precise, thermoregulation process, which constitutes a phylogenetically highly developed autonomic system with effects on kidney and cardiovascular functions [25], plays a significant role in sweat gland secretion [26]. The EDA recordings record the changes in the secretion of sweat, and measure of the secretion changes could be used to study the different aspects of different neural activity. However, skin is reached by numerous vegetative fibers (such as sympathetic nerves, pilo-erecting muscles) and the

8.2 History of electrodermal activity signal, psychophysiological

FIGURE 8.1 The structural diagram of human skin. The efferent sympathetic innervation is indicated by dashed lines. Taken from W. Boucsein, Electrodermal Activity, Springer Science & Business Media, 2012.

effect of vasoconstrictor on the blood vessel is also prominent here. These nerves are so overlapped with each other that sometimes a microscope fails to distinguish them from each other. Due to this overlapping nature, the claim of association of EDA with neural activity was debatable for a long time and made the EDA concept so difficult to study. But over a period, researchers are able to prove the claims through their studies and concept of EDA is well established today [14,26 28]. The diagram in Fig. 8.2 is an illustration of the different pathways [24]. The diagram in Fig. 8.2 shows the sympathetic pathway is in close proximity of lateral pyramidal tract and anterolateral tract. Furthermore, the distance is very less with sudomotor fiber and other sympathetic fibers [27]. These pathways are collectively regarded the classical pathways for the perception of pain, temperature, degree of consciousness, and defensive response. Nevertheless, the impact of somatosensory afferent at spinal cord level is well accepted. Fig. 8.3 shows the important pathways from hypothalamus to spinal cord [27]. In the above picture, sections marked with 1, 2, and 3 are paraventricular, posterior, and supramamillary nuclei of hypothalamus. The pathways marked with “M”

153

FIGURE 8.2 Pictorial representation of connection between sweat gland and different organs. Taken from W. Boucsein, Electrodermal Activity, Springer Science & Business Media, 2012.

8.2 History of electrodermal activity signal, psychophysiological

FIGURE 8.3 The limbic system in medial section. Taken from H. Schliack, R. Schiffter, Neurophysiologie und-pathophysiologie der Schweißsekretion, in: Normale und Pathologische Physiologie der Haut II, Springer, Berlin, Heidelberg, 1979, pp. 349 458; W. Boucsein, Electrodermal Activity, Springer Science & Business Media, 2012.

represent the Papez circuit, which connects hippocampus to mamillary, anterior thalamus, and cingulate gyrus. The pathway “T” and “A” represent the tegmental midbrain and amygdala correspondingly [27]. Whether the course of hypothalamic-reticular-spinal pathway is ipsilateral, or partly or wholly contralateral, was a matter of debate earlier. But in 1979, Schliack and Schiffter [27] claimed that his pathway is ipsilateral. They further claimed that the thalamus and the Broadman area 6 of the frontal lobe (area in close proximity with precentral motor) could provoke the sweat gland activity [27]. Furthermore, they proposed that hypothalamus should be regarded as the main center for the sweat secretion [27,29], though there are other cerebro-efferent pathways present sudomotor neuron (spinal cord). They identified that limbic system also influences the sweating and probably contributes to emotional sweating [27]. Though this knowledge

155

156

CHAPTER 8 Electrodermal activity and its effectiveness

helps us to understand the connection between Central Nervous System (CNS) and EDA, most of the studies were on animals and could not be generalized for human. Venables and Christie [30] jotted down evidences of association between EDA and different brain levels. Well later, several researchers [31 35] carried out several studies on this topic and now the fact is well established that brain activity (cortical as well as subcortical) does correlate with EDA activity.

8.2.1 Application of electrodermal activity Now after discussing the psychophysiological and physiological factors associated with EDA, the readers must be thinking about the possible application of EDA in psychophysiology. In this subsection, we will discuss about the possible applications of EDA. But before beginning the discussion, we must discuss two very important terms associated with EDA, that is, phasic and tonic components (Fig. 8.4). Typically, the tonic component is referred to as the slow changing component of EDA signal and captures the background characteristics of EDA signal. The tonic component captures the level-shift characteristic of EDA signal [36,37]. On the other hand, the phasic component is the fast changing component of EDA signal. The phasic component is termed the stimulus-driven EDA [36,37]. Based on these two components, the researchers create their experimental setup and signal analysis procedure. Anyway, scientists [38 40] found that these two components are equally important and useful in psychophysiology. In 1985, Spinks et al. carried out another important study, which readers may like to follow up. They have compared the different views on the significance of orienting

FIGURE 8.4 A sample EDA signal collected using Biopac EDA system. The blue line is phasic signal and the red line is the tonic signal [36]. EDA, Electrodermal activity.

8.2 History of electrodermal activity signal, psychophysiological

response (OR). There are a huge number of works available on these two topics, but we can summarize creation of these two electrodemal as follows [41,42]: 1. An interaction between hippocampus and amygdala creates phasic electrodermal. 2. The interaction between hippocampal and basal ganglia structures may create a phasic or tonic electrodermal. 3. A tonic electrodermal hippocampal information processing creates tonic elctrodermal, which we can associate with increased attention or arousal. There are plenty of researches available on the significance of phasic, tonic, and OR, and we will recommend to study them carefully, as the entire spectrum of EDA (experiment setup and analysis) depends on these two component mostly. Anyway, without making this more complicated for our readers, we will discuss the possible applications of EDA now. The discussion will revolve around the application of EDA in generalized psychophysiological state analysis.

8.2.2 Electrodermal activity as an indicator of general arousal Generally, arousal is a special state (psychological or physiological) controlled by the reticular activating system located in human brain. This phenomenon also affects the other part of the human brain and sometimes the effects could be seen as changes in heartbeat, pulse rate, blood pressure, behavior, etc. There are several theories, which had tried explaining arousal state differently. But in 1951 Duffy et al. [43] proposed the energetic view of arousal. He described the general arousal as an organic overall excitations [43]. Following his work, it was clear that general arousal process could be measured using CNS indicators, parameters of the ANS (mostly tonic EDA [44]), and the endocrine system. There are several works using phasic and tonic EDA parameters for general arousal [44 47]. Few researchers have shown that both the phasic and tonic may have different parameters [45,46]. While the trend was to study with the general arousal theories, few researchers opted for micro-theories of activating (local) processes [47] due to the lack of clear concept. The unitary arousal [48] theories, which were prevailing till 1970, lose its popularity, and the multidimensional arousal system [49,50] takes the place. But in either case, the effectiveness of EDA systems in arousalrelated studies got acceptance widely.

8.2.3 Electrodermal activity in different sleep stages The need of sleep does not need any special mention. It is already known to us that sleeplessness may create many mental and physical issues. Researchers have proved that the analysis of different sleep stages is very crucial in determining several issues related to health. As we have already discussed in our earlier section that EDA comes from ANS, furthermore, the relationship between ANS and circadian influence is widely accepted [51]. Thus EDA became a useful measure

157

158

CHAPTER 8 Electrodermal activity and its effectiveness

to evaluate the sleep stages. Hot et al. [52,53] showed the effectiveness of EDA measure in diurnal variation measurement. Johnson and Lubin [3] observed some significant electrodermal response during deep sleep. They reported that the EDR response is highest during the deep sleep phase. The claim was contradiction to the previous belief of slow wave EDR during sleep. However, later and more recently, researchers found that EDA activity is as useful as other brain signal in sleep pattern related studies [54,55].

8.2.4 Electrodermal indices of emotion and stress The use of ANS variables, in measuring emotional activity, was controversial and one of the most important topics in psychophysiology in the early days of EDA research. But later researchers proved their claims through their studies and different experiments. In our earlier discussion, we have mentioned about the Papez circuit (Fig. 8.3). The Papez circuit was hypothesized by Papez [56]. He claims that neuronal activity in limbic system is the reason behind the emotional excitement. He also discussed about the involvement of ANS programs stored the hypothalamus and different emotional states. Gray [57] extended Papez’s idea and studied the anxiety model. A more detailed study of Panksepp [58] and Wundt [59] holds the claims that ANS variables could be used to quantify the emotional state. In more recent time, researchers have carried out a wide range of researches using EDA [60 63], where they coupled EDA with heart rates and other signals. A review on emotions by Kreibig [62] is a good resource for readers. The recent technological advancement in EDA device helped researchers to study the impact of EDA in other fields. Some emerging fields are personality traits analysis [64], different psychological disorders [65,66], attention [67], decision-making [68,69], spatial knowledge and navigation [70], and other psychological and cognitive research topics [24]. Boucesin [24] has summarized them in his book.

8.3 Experiment design—a good experiment design 8.3.1 Experimental design 8.3.1.1 Experiment design Lindquist [71] defined experimentation saying, “Research design is the plan, structure and strategy of investigation conceived so as to obtain answer to research question and to control variance.” Winer [72] compared an experiment’s design with an architect’s plan for a building’s structure. An experiment’s designer plays a similar role to an architect. As we have seen earlier, experimentation is about answering a question. The answer depends on different factors, including what was the question and what we did during the experiment. In understanding experimental design, measurement, and statistical analysis, the concept of

8.3 Experiment design—a good experiment design

“variance” is fundamental. Researchers should avoid prematurely designing a complex experiment in the light of variance control. The more difficult the design will be, the more noise the design will have. One of experimental design’s most important factors is to control variance [73]. The variance control is a measure of a scores dispersion or spread. It describes how different the scores may be from each other. Experimental design’s main functions are to maximize the effect of systematic variance and control the source of variance from outside, that is, extraneous variance. In experimental variable manipulation, systematic variance is the variability in the dependent measure. Although, in addition, the dependent variable may be influenced by extraneous variables, unless controlled, the experiment results may be affected by an extraneous variable that tends to mask the experimental variable effect. Basic procedures such as randomization and elimination are in place to control extraneous variables. For all such uncontrolled scenarios the term experimental error or error variance is used. Maximizing systematic variance, controlling extraneous variance, and minimizing error variance are important in an experimental design. In analyzing the appropriateness, meaningfulness, and usefulness of a research study, validity is very crucial. Experimental validity refers to the manner in which variables affect both the research outcomes and the overall population generalizability. Validity is related to the control of secondary variables in experimental situations. Secondary or extraneous variation may influence the dependent variable to an extent, where the conclusion drawn becomes invalid. It is divided into two groups: (1) internal validity and (2) external validity. Internal validity refers to the ability of a study to determine whether there is a causal relationship between one or more independent variables and one or more dependent variables. While designing the experiment, researchers need to be aware of aspects that can reduce a study’s internal validity and do whatever they can to manage these threats. If left unnoticed, these threats may decrease validity to the point where results may become meaningless. External validity refers to a study’s generality. External validity threats can lead to significant results within a sample group, but an inability to generalize this to the general population.

8.3.1.2 Types of experiments The three critical aspects of experimental design are as follows: the first is that the research question is at the heart of an experiment: if we do not know what the question is, we cannot find an answer. While this may seem obvious, the central question is not specifically specified by many experiments. The second point is derived from the first: the more accurately the question is formulated, the clearer the answer to the question will be what we have to do. The third observation is that creating an accurate, explicit question of research not only involves the use of specific terms, but also the listing of known assumptions. Once the critical aspects are set, the experiment goals can range from connecting ideas to understand cause and effect to confirming a prespecified relationship. We can engage in confirmatory experimentation if we have a specific hypothesis. On the other

159

160

CHAPTER 8 Electrodermal activity and its effectiveness

hand, we do not have a specific hypothesis and must therefore engage in exploratory research. How data can be analyzed from such studies depends on how the study can be designed. The selection of the experimental design will affect the type of statistical analysis to be used on the data. If the study generates statistically significant results, then a change in the independent variable can be said to have caused a change in the dependent variable. In behavioral sciences, specifically social sciences, full control over experimental situation cannot always be exercised; such experimental situations are part of quasiexperimental design. The experimental situations in which the experimenter is able to manipulate the independent variable and has the freedom to randomly assign subjects to the treatment groups and control the extraneous variables are described as true experiments. This experimental design is addressed in our book chapter. In such cases, a large number of subjects are studied, the subjects are randomly assigned to the treatment groups, the experimenter has complete control over independent variables scheduling. There are three types of experimental design: between subjects, within subjects and mixed. In between subjects design, each subject is observed under one of the treatment condition. In within subjects design, each subject is observed under all the treatment conditions of the experiment. Some factors are among subjects in mixed design and some factors are within subjects. Each of these types of experimental design has its own advantages and disadvantages; within-subject design requires fewer participants and increases the chance to discover a real difference between your conditions; between-subject designs minimize learning effects across conditions, lead to shorter sessions, and can be easier to setup and analyze.

8.3.1.3 Hypothesis Experimental hypothesis is where definitions are operationalized in order to be able to test specific scenarios. For example, you can operationalize the number of smiles and laughs and other related actions as happy emotion. In order to measure and test one of two hypotheses, you then statistically hypothesize: the null, or H0, which is noneffect (i.e., no difference between samples or populations, or anything tested), and an alternative hypothesis, H1. The alternative hypothesis is that a difference or an effect exists. One mean may be greater than another, or they may not be equal. Statistical testing therefore aims at testing the truth of a theory or part of a theory. In other words, if they are accurate, it is a way to look at predictions. We do not test the alternate hypothesis. We do this because we base our testing on falsification logic. Two types of errors can result from a hypothesis test: Type I error: A Type I error occurs when the researcher rejects a null hypothesis when it is true. The likelihood of committing an error of Type I is called the level of significance. Also called alpha, this probability is often denoted by α. Type II error: A Type II error occurs when a false null hypothesis is not rejected by the researcher. The likelihood of committing a Type II error is called beta, often

8.3 Experiment design—a good experiment design

referred to as β. The probability that a Type II error will not be committed is called the test power. The probability of the occurrence of Type I error is assigned to a statistical test. This is the likelihood that you will reject the null hypothesis when the null is true and should therefore not have been rejected. The statistical testing process can lead to statements of probability, but only under certain conditions, about the theories under consideration. Statistical testing and hypothesizing are representative of theory when it is connected to theory conceptually (verbally and operationally). This means that in order for these statements to represent the overarching theory, there must be a logical and direct association between the statistical probability statements and the theory. The experimental and conceptual hypotheses forge this connection.

8.3.1.4 Stimulus Stimulus selection depends on experimental context and the research question being addressed; whether the experiment would be conducted on an existing dataset or a new one would be created. If the stimulus is produced from the scratch, the question that arises is how to sample data for a new dataset to create a suitable set. Creating an optimal stimulus requires balancing two components— experimental specificity and generalizability. The elements in a stimulus set be precise enough to answer the research question, yet not too specific to obstruct generalization. In cognition and neuroscience field, where it is important to reproduce and revalidate findings on mind and brain, specifically psychophysiological experimental data like functional magnetic resonance imaging, publishing datasets has become highly novel.

8.3.1.5 Measure of performance The objectivity of methods and tasks varies depending on the research question. The tasks devised to answer a research question involving high-order cognitive processes such as creativity and problem-solving are very different from tasks involving low-order cognitive functions such as shape and color perception. There are several ways in which the research question can be tackled ranging from self-reporting descriptive data to interacting with the stimulus to measuring how the brain, heart, or sweat glands respond and function. These tasks can be roughly categorized into the following three categories metatasks, direct-tasks, and physiological tasks. Metatasks such as free description, rating scales, or multiple choice are tasks referring to broader questions. Free-description tasks focus on spoken or written descriptive outcomes that can be vague and difficult to uniquely interpret. For example, a study conducted by Kaulard asked participants to describe the expression using a maximum of four words allowing a baseline outcome without biasing the participants to great degree to determine common and frequent words used to refer to a particular expression [74,75]. This form of task requires introspection or prediction. Metatasks are handy in understanding participant’s belief system and mental model and the influence of it. This method

161

162

CHAPTER 8 Electrodermal activity and its effectiveness

is specifically helpful in the field of user-experience research and design thinking to obtain insights within the natural boundaries of a phenomenon. Userexperience researchers conduct interviews with individuals or small groups, post surveys, or questionnaire to validate their hypothesis and design more meaningful solutions. On the other hand, a direct task requires participants to firsthand perform the actions. For example, to understand how people change lanes while driving, Wallis et al. designed a direct task-controlled experiment where the participant rotates a steering wheel control to move one lane to another [76]. This method provides direct evidence of how a participant would respond in a given scenario closer to the real-world behavior of interest. Simulating a real-world scenario in an experimental setup is inexpedient. Controlling all possible elements that can play unwanted sources of variance or noise is impossible in real world; therefore it is crucial to manipulate and control the stimuli, environment, and the interaction in a laboratory setup. To ensure objectivity and precision, there are experimental protocols on environmental setup, stimulus presentation, and response recording using means such as keyboard, joystick, or mouse to be followed. However, this measure also does not accurately and perfectly fit to higher order cognitive functions. Bodily reactions such as gaze, posture, breath, or sweat can provide deeper insights, in addition to the experimental measures described earlier. Physiological tasks are the most precise measure of performance capturing body’s reaction in the form of eye movements, neural firing, heart rate, blood pressure, body temperature, etc. High-accuracy bodily response can be measured using minimal invasion to fetch details of psychophysiological information. This can also help fathom out what exact elements of the stimulus were perceived along with respective bodily reaction and achieve unbiased responses. The focus of this chapter is SC and electrical properties across the skin. EDA signals have a widespread user ranging from basic to clinical research. As sweating alters the resistance of the skin to conducting currents, it is a good measure for emotional responses and arousal [77,78]. Using EDA signals, Ayzenberg et al. proposed a system for automatic annotation and monitoring of cell phone activities such as email, phone calls, calendar, and associated stress responses of the users [79]. Another study presented a novel emotion recognition system by processing EDA signals [80]. While designing an experiment involving EDA signals, the following limitations must be taken into consideration.

8.3.2 External and internal influences There are several external (environmental and demographic) and internal factors, which may create undesirable troubles in EDA recording. These factors create artifacts and biases in EDA recording, and analyses of these data are troublesome and often lead to wrong conclusion. To avoid this kind of unwanted situation, researchers have raised their voice for a controlled experimental setup over the

8.3 Experiment design—a good experiment design

years. A controlled experimental setup helps us to analyze the ground truth data and test our hypothesis. Though modern equipment are well advanced and equipped with technology that reduces several external or internal influences automatically, it is always better to test the hypothesis in a controlled setup before implementing it in real time. In this section, we will talk about few known external and internal factors that influence the EDA recording and therefore should be taken care of during experiment design phase.

8.3.3 Climatic conditions Air temperature and humidity of the environment could massively impact the EDA. Studies have shown that temperature has impacted EDA signal recordings [10,81 83]. However, temperature has different impact on different EDA components. For example, SC level decreases when temperature decreases [83], whereas SR level increases when the temperature decreases [84]. However, researchers have agreed on that the seasonal changes (winter and summer) also influence the EDA recordings [10,81,85]. There are several others studies where researchers have observed the impact of temperature on EDA recordings. The other factor, humidity, received a mixed reaction from the researchers. Few researchers [86] reported nonsignificant impact of humidity on EDA, whereas others have found significant impact of humidity on EDA [87,88]. Nonetheless, in recent times researchers have recommended a controlled humidity as the humidity could impact the skin temperature and a low skin temperature is not good for EDA recording [87]. Though there are no hard and fast rules for choosing a temperature or humidity, we strongly recommend a comfortable ambient (also known as thermoneutral zone) [25] for experimental setup.

8.3.4 Internal or physiological influences In our earlier section, we have discussed the thermoregulation process and its contribution in EDA. We have also discussed the contribution of ANS on different physiological process. Thus anything that could impact the skin temperature or core temperature [10,81,89,90] also impacts the EDA recording. Even the menstrual cycle [89], hot flushes, blood flow [91], etc. which change the skin temperature may affect the EDA recordings. Similarly, skin moisture also may affect the EDA recordings [91]. So researchers recommend few things such as the participant should be healthy and not having her menstrual cycle and there should not be any wrapping that may change the blood flow, which should be considered during experimental setup.

8.3.5 Demographic characteristics Few demographics characteristic also impact the EDA recordings. With age the human skin formation does change, and this also affects the sweat gland activity

163

164

CHAPTER 8 Electrodermal activity and its effectiveness

and amount of salt in the sweat [92]. As we already know that the sweat gland activity is one of vital cogs in EDA recording, the age also plays a crucial role in EDA recordings. Similarly, we have already discussed that few physiological factors such as the menstrual cycle [89] and blood flow [91] influence the EDA recordings. These physiological factors are very much gender dependent, and thus the gender also influences the EDA recordings. So it is advisable that the researchers should choose their participants carefully to remove any kind of biasness from their studies.

8.4 Electrodermal activity signal collection sites and pretreatment of sites 8.4.1 Electrodermal activity signal collection sites Where to place the electrode or which sites to choose for EDA signal collection? Researchers often get confused on this particular question. In this section we will try to address this topic in short. Researchers generally prefer the palmar or volar surface as the preferred site. Till now several attempts and proposals had been made to standardize the EDA collection site and placing of electrode but unfortunately there are no standard procedures regarding EDA collection site. However, Venables and Christie [81] and van Dooren and Janssen [93] studied the different EDA collection procedure and summarize their findings as follows: 1. Choose a location where we can place the electrode easily and the site is less prone to external disturbance. 2. The area is relatively free from scarring or damaging. 3. The surface area size is enough to place the electrode. 4. The Palmar site shows some significantly distinguished EDA. Furthermore, they recommended the middle finger (medial phalanx) and index finger (also recommended by Edelberg and Brown [14]) as the preferred EDA collection area (Fig. 8.5: location marked as sites A and B). They pointed out the fact that these two areas are less prone to movement and scarring effects. In their previous study [30], they have also observed the fact that placing the electrode at locations A and B (Fig. 8.5) helps one to avoid EDA asynchrony. However, few studies showed that the distal phalanx (Fig. 8.5) could be another important location as these sites are more sensitive to habituation [94]. Venables and Christie [81] also made some useful recommendation for choosing the hand (right or left). They suggested that the nondominant hand is more suitable than the dominant hand as the nondominant hand is less callous. However, what if the thickness of the finger is not adequate for placing the electrode or the finger is damaged? In such a case, Edelberg and Brown [14] suggested an alternative. The electrode could be placed on thenar or hypothenar

8.4 Electrodermal activity signal collection sites

FIGURE 8.5 A pictorial representation of different electrodermal activity collection sites. Adapted from P.H. Venables, M.J. Christie, Electrodermal activity, Tech. Psychophysiol. 54 (3) (1980).

eminence (Fig. 8.5: locations C and D). These two locations also exhibit similar EDA activity as the fingers. In some experimental design, we need to engage both hands of the participants. In such scenarios placing of electrode on hands becomes difficult and we need to place the electrode on some other place. Edelberg and Brown [14] suggested one such alternative. According to him, the medial part (Fig. 8.6) of the foot sole could be used as an alternative collection site. The emotional sweating pattern of this location is the same as the hand. In more recent time, Westerink et al. [95] used a wristband like EDA instrument for ambulatory monitoring system. Later, in 2010 Poh et al. [96] proposed different and robust wristband for EDA signal collection. They tried to collect the signal from vocal side of the distal forearm.

165

166

CHAPTER 8 Electrodermal activity and its effectiveness

FIGURE 8.6 Plantar location for EDA collection marked as A and B. EDA, Electrodermal activity.

Anyway, till now there are no standard sites for EDA collection. But based on the several recommendation and study, it is a good practice to collect the signal from hand (finger or wrist band).

8.4.2 Pretreatment of sites There are many recommendations made by researchers in this regard as the degree of hydration and concentration of electrolyte influenced the overall EDA. We can summarize the recommendation as follows: 1. Wash the site with soap [81] before placing the electrodes. 2. Alternatively, alcohol swab could be used [97].

8.6 Analysis of electrodermal activity signal

However, it is worthwhile to mention that both the procedures come with some advantages and disadvantages. So pretreatment of the site is highly subjective and depends on the experiment, participants, and other factors.

8.5 Artifacts removal from the electrodermal activity signal There are several sources of artifacts, which may create noise in EDA signal. Some of them are external, whereas some of them are internal or physiological. Most of the currently available devices take care of the external artifacts, but the artifacts coming from the physiological things are still a matter of concern. Most of the physiological artifacts occur due to movement. The movement could be the muscle movement, movement due to blood flow, etc. This is why researchers recommend their participants to sit or lie quietly during the experiments. However, though sitting quietly avoids few artifacts, but these do not ensure an artifact-free recording. Mainly, when we try to analyze the long recordings (hours to day long), these kinds of artifacts create a real problem. So addressing of this kind of artifacts is a major concern in EDA research. Recently, after the rise of the wearable device, researchers have tried to remove the mental artifacts from EDA recordings. Researchers have used different kinds of smoothing techniques [98] to remove the artifacts. Few researchers have also tried the low-pass filter [99 101] for noise removal or reduction. All these approaches are good to reduce the small variation in the signal but in long recordings these techniques fail to remove the movement artifacts. Boucsein [24] suggested some artifact-removal techniques in his book. One of them is heuristic method. Kocielnik et al. [100] adopted a similar heuristic approach where they have discarded the data points, which are more than or less than a specific threshold. Storm et al. [102] tried a modified version of this approach where they have considered the maximum and minimum amplitude, maximum slope, and minimum width along with the other factors. Hedman [103] used two wristbands to counter the artifact problem. After recording, he compared the output of the two signals and has discarded the data points whenever there is an abnormality in any one of the output signals. Sano et al. [104] have used features like (mean, first-order derivative, second-order derivative of amplitude) to automatically identify the artifacts of EDA signal. Waveletbased [105] and machine-learning approaches [106] have been also employed to identify and remove artifacts. Nevertheless, this area is still challenging as before and researchers are trying to find the best approach to counter this problem.

8.6 Analysis of electrodermal activity signal In our earlier section, we have mentioned two components of EDA, that is, phasic and tonic components. We have also hinted the reader about the importance of these two components in EDA analysis. Zangro´niz et al. [107] show different

167

CHAPTER 8 Electrodermal activity and its effectiveness

800

(A)

µS

700 600 500

Raw 0

µS

800

200

300

400

500

600

700

(B)

EDA Events 0

800

µS

100

600

400

100

200

300

400

500

600

700

(C)

600

400

EDA SCL 0

100

µS

168

100

200

300

400

500

600

(D)

700 SCR

50

0

0

100

200

300 400 Samples

500

600

700

FIGURE 8.7 Different stages in EDA signal processing: (A) raw EDA signal before filtering; (B) raw EDA signal after low-pass filtering and stimuli onset; (C) estimation of EDA baseline using a cubic spline approximation; and (D) SCR obtained after the convolution process. EDA, Electrodermal activity; SCR, skin conductivity response. Adopted from R. Zangro´niz, A. Mart´ınez-Rodrigo, J. Pastor, M. Lo´pez, A. Ferna´ndez-Caballero, Electrodermal activity sensor for classification of calm/distress condition, Sensors 17 (10) (2017) 2324 after permission.

stages of EDA processing in their study “classification of calm and distress state” [107]. In this section, we will discuss the analysis in detail (Fig. 8.7).

8.6.1 Phasic electrodermal activity Phasic component also termed “response.” The gestalt characteristic of this component helps the researchers to study the signal (because the noise removal is

8.6 Analysis of electrodermal activity signal

much easier) more conveniently. In phasic component analysis, we try to use the parameters mentioned next:

8.6.1.1 Latency Among all the available biosignals, electrodermal response (EDR) latency is the highest. As EDR is stimulus driven, we need to take care of the EDA latency to analyze the signals more accurately. Many researchers [12,14,81] have observed this in their study and have proposed different time windows. Edelberg [12] proposed a time window between 1.2 and 4 seconds. He also suggested that the time window suits his experiments when the room temperature is comfortable. Venables and Christie [81] opined that the 4 second time window is too long as EDR latency and proposed for a time widow between 1 and 3 seconds. While there were several proposals on EDR latency, Levinson and Edelberg [108] propose a new method for latency calculation. According to their suggestion, the time window should be calculated based on experiment setup and researchers should consider the EDR latency of all the participants before choosing a time window. Stern and Walrath [109] consider the individual modal value in their EDR latency standardization proposal and found that for the most of the cases the latency is 0.5 second. However, there are few cases where the EDR latency calculation is not easy and often impossible. To handle such kind of situation, Levinson and Edelberg [108] proposed for the calculation of point of EDR maximum to calculate the EDR latency. Anyway, there are several opinion and proposal for EDR latency calculation but most of them have agreed that the EDR latency is dependent on experiment, participants, and room environment. So though there is no standard time window available, but the window between 0.5 and 3 seconds works for the most of the cases [110].

8.6.1.2 Amplitude The amplitude is another very important parameter in phasic component. Whenever we talk about the high arousal or low arousal, amplitude plays an important role for such identification. Though the use of the term “amplitude” is arguable and many researchers [81] proposed the term “magnitude,” the main reason behind such suggestion was the zero response during no stimuli. However, later researchers come to conclusion that “magnitude” is best suited for the missing data handling, as we take average of the data points to replace the missing data and this average value is different from the mean of EDR amplitude. The EDR helps us to find the onset and recovery time, which are important factors when we are discussing the arousal. In recent time researcher have used this parameters in emotion research as well as in other research [109 111].

8.6.1.3 Shape of electrodermal responses The shape of the electrodermal response (EDR) is very helpful to derive meaningful conclusion. There two very important parameters associated with EDR shape. One is parameter of ascent and the other is descent or recovery parameter. To calculate the parameter of ascent or EDR rise time, it is necessary to find the onset

169

170

CHAPTER 8 Electrodermal activity and its effectiveness

and peak of the EDR. Ideally, the span between response onset and response peak represent the rise time or EDR rise time [81]. But problem arises when the curve is flat and we can define the peak unambiguously. Edelberg and Brown [14] recommended the first derivative but the calculated minima from this method are the EDR minima not the EDA minima. Foerster [112] did come up with an alternative threshold-based method in his computational approach, but this was not enough to solve the problem. Thom [113] modified the threshold-based method slightly to improve its effectiveness. They have also suggested the maximum inclination of the curve to be taken account of during the rise time calculation. This recommendation is very significant and important. Nevertheless, both pointed out some significant points (such as latency and EDR speed), which plays a significant role in determining the parameter of ascent. The other parameter, descent (recovery parameters), calculation gets a bit complicated as the decline ends more or less asymptotically. In 1937 Darrow proposed the half-life concept where he considered the amplitude as the “total amount.” His proposal was inline with the electrophysiology theory. Edelberg [12] proposed for a graphical matching method. There are several proposals available on how to calculate the descent [81]. But nonetheless the parameters of ascent and descent are really helpful when we need to study any kind of arousal [111].

8.6.2 Area measurements The area under the curve is a very useful analysis when we need to measure the strength of the effect. In 1937 Traxel [114] first proposed this and later Sano and Picard [101], Healey and Picard [111], and Bach et al. [115] have successfully used this measure in their studies.

8.6.3 Tonic electrodermal activity Tonic component (EDL) comes after the phasic parameters. So knowing the phasic component is a prerequisite if someone wants to dig into the tonic component of EDA. The calculation of EDL is not as easy as it seems. It is a known fact that ascent and descent time of EDR is 0.5 and B4 seconds, but this is not the case with EDL; the ascent and descent time is 10 30 seconds. If the point under consideration falls out in the range of the electrodermal response then a temporal shift of that point is necessary. However, this kind of shift is not possible for online recording of EDA, and we avoid such situations using averaging method. Nonetheless, there are many cons associated with the calculation of EDL and can create a number of artifacts (such as movement artifacts). So it is always recommended to choose a good EDL calculation method during tonic component analysis. Venables and Christie [81] have proposed some good technique for it. Anyway, except the abovementioned points, researchers have used several other features (such as spectral density [116]) and methods [95,96,99 101,106,110,111]. This field is still in growing phase and there is a lot to explore.

References

8.7 End remarks Though it was believed that EDA is a very naı¨ve signal, recent development shows that EDA could be used in different psychological studies. Inclusion of EDA in different psychology-related and HCI studies is increasing gradually. Simplicity of the device, portability, easy to wear for a long time span, easy analysis technique of signal, etc. are few of the factors that could be named the most important factors for this upraise. Furthermore, few recent studies show that EDA could be used for complex cognitive phenomena study. This is also a crucial point that lured modern researchers toward EDA. Anyway, as we have discussed earlier, though EDA device is very simple and the signal is simple in nature compared to other biosignals. But there are a few tricks that we need to consider while we are designing the experiment. In this chapter, we have tried to give a glimpse of the most important points related to psychological and physiological significance of EDA signal and we have tried to cover all the influencing factors, which may play a pivotal role during research studies (comprise EDA). It will be worth mentioning that EDA is still in growing phase (in terms of its applications and technical development), and it is also highly possible that in few years the research will find some new information regarding EDA. But this chapter will give a good insight to the readers on basics of EDA signal and will help them start their research experiment.

References [1] E. Neumann, R. Blanton, The early history of electrodermal research, Psychophysiology 6 (4) (1970) 453 475. ¨ ber die Secretionsstro¨me der Haut bei der Katze, [2] L. Hermann, B. Luchsinger, U Pflu¨gers Arch. Eur. J. Physiol. 17 (1) (1878) 310 319. [3] L.C. Johnson, A. Lubin, Spontaneous electrodermal activity during waking and sleeping, Psychophysiology 3 (1) (1966) 8 17. [4] M.J. Christie, Electrodermal activity in the 1980s: a review, J. R. Soc. Med. 74 (8) (1981) 616 622. [5] R. Vigouroux, Sur le role de la resistance electrique des tissues dans l’electro-diagnostic, C. R. Soc. Biol. 31 (1879) 336 339. [6] C. Fe´re´, Note on changes in electrical resistance under the effect of sensory stimulation and emotion, C. R. Seances Soc. Biol. 5 (1888) 217 219. [7] J. Tarchanoff, De´charges e´lectriques dans la peau de l’homme sous l’influence de l’excitation des organes des sens et de diffe´rentes formes d’activite´ psychique, C. R. Seances Soc. Biol. (Paris) 41 (1889) 447 451. [8] O. Veraguth, Das psychogalvanische reflexpha¨nomen, Karger, 1909. [9] B. Sidis, L. Nelson, The nature and causation of the galvanic phenomenon, Psychol. Rev. 17 (2) (1910) 98. [10] M.J. Christie, P.H. Venables, Mood changes in relation to age, EPI scores, time and day, Br. J. Soc. Clin. Psychol. 12 (1) (1973) 61 72.

171

172

CHAPTER 8 Electrodermal activity and its effectiveness

[11] C.P. Richter, Physiological factors involved in the electrical resistance of the skin, Am. J. Physiol.—Legacy Content 88 (4) (1929) 596 615. [12] R. Edelberg, Electrodermal recovery rate, goal-orientation, and aversion, Psychophysiology 9 (5) (1972) 512 520. [13] W. Boucsein, Principles of electrodermal phenomena, Electrodermal Activity, Springer, Boston, MA, 2012, pp. 1 86. [14] R. Edelberg, C.C. Brown, Methods in Psychophysiology, 1967, pp. 1 53. [15] D.C. Fowles, The eccrine system and electrodermal activity, Psychophysiology: Systems, Processes, and Applications, vol. 1, 1986, pp. 51 96. [16] C.W. Darrow, The functional significance of the galvanic skin reflex and perspiration on the backs and palms of the hands, Psychol. Bull. 30 (4) (1933) 712. [17] C.W. Darrow, The equation of the galvanic skin reflex curve: I. The dynamics of reaction in relation to excitation-background, J. Gen. Psychol. 16 (2) (1937) 285 309. [18] C.W. Darrow, Neural mechanisms controlling the palmar galvanic skin reflex and palmar sweating: a consideration of available literature, Arch. Neurol. Psychiatry 37 (3) (1937) 641 663. [19] R.S. Woodworth, H. Schlosberg, Experimental Psychology, Holt, New York, 1954. [20] R. Ladpli, G.H. Wang, Spontaneous variations of skin potentials in footpads of normal, striatal and spinal cats, J. Neurophysiol. 23 (4) (1960) 448 452. [21] M.H. Bagshaw, D.P. Kimble, K.H. Pribram, The GSR of monkeys during orienting and habituation and after ablation of the amygdala, hippocampus and inferotemporal cortex, Neuropsychologia 3 (2) (1965) 111 119. [22] G.H. Wang, The Neural Control of Sweating., University of Wisconsin Press, 1964. [23] H. Sequeira, J.C. Roy, Neural control of electrodermal activity, Cent. Nerv. Control Auton. Funct. 11 (1997) 259 293. [24] W. Boucsein, Electrodermal Activity, Springer Science & Business Media, 2012. [25] G. Thews, E. Mutschler, P. Vaupel, Human anatomy, physiology and pathophysiology, in: Human Anatomy, Physiology and Pathophysiology, 1985. [26] F. Sargent, Depression of sweating in man: so-called sweat gland fatigue, Adv. Biol. Skin 3 (1962) 163 212. [27] H. Schliack, R. Schiffter, Neurophysiologie und-pathophysiologie der Schweißsekretion, Normale und Pathologische Physiologie der Haut II, Springer, Berlin, Heidelberg, 1979, pp. 349 458. [28] M. Kunimoto, K. Kirno¨, M. Elam, T. Karlsson, B.G. Wallin, Neuro-effector characteristics of sweat glands in the human hand activated by irregular stimuli, Acta Physiol. Scand. 146 (2) (1992) 261 269. [29] R. Schiffter, P. Pohl, Course of the descending central sympathetic pathway. Results of sweat secretion tests following stereotaxic subthalamotomy, Arch. Psychiatr. Nervenkr. 216 (4) (1972) 379 392. [30] M.J. Christie, P.H. Venables, Site, state, and subject characteristics of palmar skin potential levels, Psychophysiology 9 (6) (1972) 645 649. [31] H. Sequeira, J.C. Roy, Cortical and hypothalamo-limbic control of electrodermal responses, Progress in Electrodermal Research, Springer, Boston, MA, 1993, pp. 93 114. [32] J.C. Roy, H. Sequeira, B. Delerm, Neural control of electrodermal activity: spinal and reticular mechanisms, Progress in Electrodermal Research, Springer, Boston, MA, 1993, pp. 73 92.

References

[33] A.A. Dube, M. Duquette, M. Roy, F. Lepore, G. Duncan, P. Rainville, Brain activity associated with the electrodermal reactivity to acute heat pain, NeuroImage 45 (1) (2009) 169 180. [34] A. Mobascher, J. Brinkmeyer, T. Warbrick, F. Musso, H.J. Wittsack, R. Stoermer, et al., Fluctuations in electrodermal activity reveal variations in single trial brain responses to painful laser stimuli—a fMRI/EEG study, NeuroImage 44 (3) (2009) 1081 1092. [35] J. Fan, P. Xu, N.T. Van Dam, T. Eilam-Stock, X. Gu, Y.J. Luo, et al., Spontaneous brain activity relates to autonomic arousal, J. Neurosci. 32 (33) (2012) 11176 11186. [36] ,https://www.biopac.com/.. [37] E.N. Sokolov, Perception and the Conditioned Reflex., Macmillan, New York, 1963. [38] Y. Nagai, H.D. Critchley, E. Featherstone, P.B.C. Fenwick, M.R. Trimble, R.J. Dolan, Brain activity relating to the contingent negative variation: an fMRI investigation, NeuroImage 21 (4) (2004) 1232 1241. [39] M.E. Dawson, A.E. Schell, D.L. Filion, The electrodermal system, Handbook of psychophysiology, 2000, pp. 200 223. [40] R.J. Barry, E.N. Sokolov, Habituation of phasic and tonic components of the orienting reflex, Int. J. Psychophysiol. 15 (1993) 30 42. [41] D.C. Fowles, The three arousal model: implications of Gray’s two-factor learning theory for heart rate, electrodermal activity, and psychopathy, Psychophysiology 17 (2) (1980) 87 104. [42] K.H. Pribram, D. McGuinness, The neuropsychology of attention: emotional and motivational controls, The Brain and Psychology, Academic Press, New York, 1980, p. 95. [43] T.E. Duffy, S.R. Nelson, O.H. Lowry, Cerebral carbohydrate metabolism during acute hypoxia and recovery 1, J. Neurochem. 19 (4) (1972) 959 977. [44] A.J. Silverman, S.I. Cohen, B.M. Shmavonian, Investigation of psychophysiologic relationships with skin resistance measures, J. Psychosom. Res. 4 (1959) 65 87. [45] N.R. Burch, T.H. Greiner, A bioelectric scale of human alertness: concurrent recordings of the EEG and GSR, Psychiatr. Res. Rep. 12 (1960) 183. [46] J. Fahrenberg, P. Walschburger, F. Foerster, M. Myrtek, W. Mu¨ller, An evaluation of trait, state, and reaction aspects of activation processes, Psychophysiology 20 (2) (1983) 188 195. [47] L.J. Levine, S.L. Burgess, Beyond general arousal: effects of specific emotions on memory, Soc. Cognit. 15 (3) (1997) 157 181. [48] D.B. Lindsley, L.H. Schreiner, W.B. Knowles, H.W. Magoun, Behavioral and EEG changes following chronic brain stem lesions in the cat, Electroencephalogr. Clin. Neurophysiol. 2 (1 4) (1950) 483 498. [49] A. Routtenberg, The two-arousal hypothesis: reticular formation and limbic system, Psychol. Rev. 75 (1) (1968) 51. [50] A. Routtenberg, M. Sloan, Self-stimulation in the frontal cortex of Rattus norvegicus, Behav. Biol. 7 (4) (1972) 567 572. [51] J. Rutenfranz, Zur Frage einer Tagesrhythmik des elektrischen Hautwiderstandes beim Menschen, Eur. J. Appl. Physiol. Occup. Physiol. 16 (2) (1955) 152 172. [52] P. Hot, J. Naveteur, P. Leconte, H. Sequeira, Diurnal variations of tonic electrodermal activity, Int. J. Psychophysiol. 33 (3) (1999) 223 230.

173

174

CHAPTER 8 Electrodermal activity and its effectiveness

[53] P. Hot, P. Leconte, H. Sequeira, Diurnal autonomic variations and emotional reactivity, Biol. Psychol. 69 (3) (2005) 261 270. [54] A. Sano, R.W. Picard, Toward a taxonomy of autonomic sleep patterns with electrodermal activity, in: 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE, August 2011, pp. 777 780. [55] A. Sano, R.W. Picard, R. Stickgold, Quantitative analysis of wrist electrodermal activity during sleep, Int. J. Psychophysiol. 94 (3) (2014) 382 389. [56] J.W. Papez, A proposed mechanism of emotion, Arch. Neurol. Psychiatry 38 (4) (1937) 725 743. [57] J.A. Gray, Pre´cis of The neuropsychology of anxiety: an enquiry into the functions of the septo-hippocampal system, Behav. Brain Sci. 5 (3) (1982) 469 484. [58] J. Panksepp, Toward a general psychobiological theory of emotions, Behav. Brain Sci. 5 (3) (1982) 407 422. [59] R. Reisenzein, A structuralist reconstruction of Wundt’s three-dimensional theory of emotion, The Structuralist Program in Psychology: Foundations and Applications, 1992, pp. 141 189. [60] S. Khalfa, P. Isabelle, B. Jean-Pierre, R. Manon, Event-related skin conductance responses to musical emotions in humans, Neurosci. Lett. 328 (2) (2002) 145 149. [61] A. Drachen, L.E. Nacke, G. Yannakakis, A.L. Pedersen, Correlation between heart rate, electrodermal activity and player experience in first-person shooter games, in: Proceedings of the Fifth ACM SIGGRAPH Symposium on Video Games, ACM, July 2010, pp. 49 54. [62] S.D. Kreibig, Autonomic nervous system activity in emotion: a review, Biol. Psychol. 84 (3) (2010) 394 421. [63] H. Sequeira, P. Hot, L. Silvert, S. Delplanque, Electrical autonomic correlates of emotion, Int. J. Psychophysiol. 71 (1) (2009) 50 56. [64] D.C. Fowles, Motivational effects on heart rate and electrodermal activity: implications for research on personality and psychopathology, J. Res. Pers. 17 (1983) 48 71. [65] S.E. Crowell, T.P. Beauchaine, L. Gatzke-Kopp, P. Sylvers, H. Mead, J. ChipmanChacon, Autonomic correlates of attention-deficit/hyperactivity disorder and oppositional defiant disorder in preschool children, J. Abnorm. Psychol. 115 (1) (2006) 174. ¨ hman, Electrodermal activity and vulnerability to schizophrenia: a review, Biol. [66] A. O Psychol. 12 (2 3) (1981) 87 145. ¨ hman, F. Esteves, A. Flykt, J.J. Soares, Gateways to consciousness: emotion, [67] A. O attention, and electrodermal activity, Progress in Electrodermal Research, Springer, Boston, MA, 1993, pp. 137 157. [68] A. Bechara, H. Damasio, A.R. Damasio, G.P. Lee, Different contributions of the human amygdala and ventromedial prefrontal cortex to decision-making, J. Neurosci. 19 (13) (1999) 5473 5481. [69] B. Figner, R.O. Murphy, Using skin conductance in judgment and decision making research, A Handbook of Process Tracing Methods for Decision Research, 2011, pp. 163 184. [70] L. Holper, N. Ja¨ger, F. Scholkmann, M. Wolf, Error detection and error memory in spatial navigation as reflected by electrodermal activity, Cognit. Process. 14 (4) (2013) 377 389.

References

[71] E.F. Lindquist, Design and Analysis of Experiments in Psychology and Education, Houghton Mifflin, 1956. [72] B.J. Winer, Multifactor experiments having repeated measures on the same elements, Statistical Principles in Experimental Design, 1971, pp. 514 603. [73] F.N. Kerlinger, Foundations of Behavioral Research, Holt, Rinehart and Winston, New York, 1986. [74] K. Kaulard, D.W. Cunningham, H.H. Bu¨lthoff, C. Wallraven, The MPI facial expression database: a validated database of emotional and conversational facial expressions. PloS one, 7(3), e32321. [75] K. Kaulard, C. Wallraven, D.W. Cunningham, H.H. Bu¨lthoff, Laying the foundations for an in-depth investigation of the whole space of facial expressions, J. Vis. 10 (2010) 606. [76] G. Wallis, A. Chatziastros, H.H. Bu¨lthoff, An unexpected role for visual feedback in vehicle steering control, Curr. Biol. 12 (2002) 295 299. [77] R. Vetrugno, R. Liguori, P. Cortelli, et al., Sympathetic skin response: basic mechanisms and clinical applications, Clin. Auton. Res. 13 (4) (2003) 256 270. [78] W. Boucsein, Electrodermal Activity., Plenum Press, New York, 1992. [79] Y. Ayzenberg, J. Hernandez Rivera, R. Picard, FEEL: frequent EDA and event logging— a mobile social interaction stress monitoring system, in: CHI’12 Extended Abstracts on Human Factors in Computing Systems, ACM, May 2012, pp. 2357 2362. [80] K.H. Kim, S.W. Bang, S.R. Kim, Emotion recognition system using short-term monitoring of physiological signals, Med. Biol. Eng. Comput. 42 (3) (2004) 419 427. [81] P.H. Venables, M.J. Christie, Electrodermal activity, Tech. Psychophysiol. 54 (3) (1980). [82] J.E. Conklin, Three factors affecting the general level of electrical skin-resistance, Am. J. Psychol. 64 (1) (1951) 78 86. [83] W.F. Waters, R.L. Koresko, G.V. Rossie, S.A. Hackley, Short-, medium-, and longterm relationships among meteorological and electrodermal variables, Psychophysiology 16 (5) (1979) 445 451. [84] R.C. Wilcott, Effects of high environmental temperature on sweating and skin resistance, J. Comp. Physiol. Psychol. 56 (4) (1963) 778. [85] E. Neumann, Thermal changes in palmar skin resistance patterns, Psychophysiology 5 (2) (1968) 103 111. [86] L.E. Fisher, M.H. Winkel, Time of quarter effect: an uncontrolled variable in electrodermal research, Psychophysiology 16 (2) (1979) 158 163. [87] P.H. Venables, I. Martin, Skin resistance and skin potential, A Manual of Psychophysiological Methods, 1967, pp. 53 102. [88] W.W. Grings, Recording of electrodermal phenomena, Methods Physiol. Psychol. 1 (1974) 273 296. [89] J. Rutenfranz, Skin resistance to weak electric currents, Hautarzt 9 (7) (1958) 289 299. [90] R.L. Maulsby, R. Edelberg, The interrelationship between the galvanic skin response, basal resistance, and temperature, J. Comp. Physiol. Psychol. 53 (5) (1960) 475. [91] W. Boucsein, F. Schaefer, T. Sommer, Electrodermal long-term monitoring in everyday life, Progress in Ambulatory Assessment, 2001, pp. 549 560. [92] N.A. Fenske, C.W. Lober, Structural and functional changes of normal aging skin, J. Am. Acad. Dermatol. 15 (4) (1986) 571 585.

175

176

CHAPTER 8 Electrodermal activity and its effectiveness

[93] M. van Dooren, J.H. Janssen, Emotional sweating across the body: comparing 16 different skin conductance measurement locations, Physiol. Behav. 106 (2) (2012) 298 304. [94] A.S. Scerbo, L.W. Freedman, A. Raine, M.E. Dawson, P.H. Venables, A major effect of recording site on measurement of electrodermal activity, Psychophysiology 29 (2) (1992) 241 246. [95] J. Westerink, M. Ouwerkerk, G.J. de Vries, S. de Waele, J. van den Eerenbeemd, M. van Boven, Emotion measurement platform for daily life situations, in: 2009 Third International Conference on Affective Computing and Intelligent Interaction and Workshops, IEEE, September 2009, pp. 1 6. [96] M.Z. Poh, N.C. Swenson, R.W. Picard, A wearable sensor for unobtrusive, longterm assessment of electrodermal activity, IEEE Trans. Biomed. Eng. 57 (5) (2010) 1243 1252. [97] P. Walschburger, Zur Beschreibung von Aktivierungsprozessen: Eine Methodenstudie zur psychophysiologischen Diagnostik, Albert-Ludwigs-Universita¨t Freibg. im Breisgau (1976). [98] J. Hernandez, R.R. Morris, R.W. Picard, Call center stress recognition with personspecific models, in: International Conference on Affective Computing and Intelligent Interaction, Springer, Berlin, Heidelberg, October 2011, pp. 125 134. [99] M.Z. Poh, Continuous Assessment of Epileptic Seizures With Wrist-Worn Biosensors (Doctoral dissertation), Massachusetts Institute of Technology, 2011. [100] R. Kocielnik, N. Sidorova, F.M. Maggi, M. Ouwerkerk, J.H. Westerink, Smart technologies for long-term stress monitoring at work, in: Proceedings of the 26th IEEE International Symposium on Computer-Based Medical Systems, IEEE, June 2013, pp. 53 58. [101] A. Sano, R.W. Picard, Stress recognition using wearable sensors and mobile phones, in: 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, IEEE, September 2013, pp. 671 676. [102] H. Storm, A. Fremming, S. Ødegaard, Ø.G. Martinsen, L. Mørkrid, The development of a software program for analyzing spontaneous and externally elicited skin conductance changes in infants and adults, Clin. Neurophysiol. 111 (10) (2000) 1889 1898. [103] E.B. Hedman, In-Situ Measurement of Electrodermal Activity During Occupational Therapy (Doctoral dissertation), Massachusetts Institute of Technology, 2010. [104] S. Taylor, N. Jaques, W. Chen, S. Fedor, A. Sano, R. Picard, Automatic identification of artifacts in electrodermal activity data, in: 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), IEEE, August 2015, pp. 1934 1937. [105] W. Chen, N. Jaques, S. Taylor, A. Sano, S. Fedor, R.W. Picard, Wavelet-based motion artifact removal for electrodermal activity, in: 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), IEEE, August 2015, pp. 6223 6226. [106] Y. Zhang, M. Haghdan, K.S. Xu, Unsupervised motion artifact detection in wristmeasured electrodermal activity data, in: Proceedings of the 2017 ACM International Symposium on Wearable Computers, ACM, September 2017, pp. 54 57. [107] R. Zangro´niz, A. Mart´ınez-Rodrigo, J. Pastor, M. Lo´pez, A. Ferna´ndez-Caballero, Electrodermal activity sensor for classification of calm/distress condition, Sensors 17 (10) (2017) 2324.

Further reading

[108] D.F. Levinson, R. Edelberg, Scoring criteria for response latency and habituation in electrodermal research: a critique, Psychophysiology 22 (4) (1985) 417 426. [109] J.A. Stern, L.C. Walrath, Orienting responses and conditioning of electrodermal responses, Psychophysiology 14 (4) (1977) 334 342. [110] R. Yoshida, T. Nakayama, T. Ogitsu, H. Takemura, H. Mizoguchi, E. Yamaguchi, et al., Feasibility study on estimating visual attention using electrodermal activity, in: Eighth International Conference on Sensing Technology, September 2014, pp. 589 595. [111] J. Healey, R.W. Picard, Detecting stress during real-world driving tasks using physiological sensors, IEEE Trans. Intell. Transp. Syst. 6 (2) (2005) 156 166. [112] F. Foerster, Elektrodermale Aktivita¨t (EDA), Computerprogramme zur Biosignalanalyse, Springer, Berlin, Heidelberg, 1984, pp. 52 64. [113] E. Thom, Die Hamburger EDA-Auswertung, Elektrodermale Aktivitat. Grundlagen, Methoden und Anwendungen, 1988, pp. 501 514. ¨ ber das Zeitmass der psychogalvanischen Reaktion. Zeitschrift fu¨r [114] W. Traxel, U Psychologie: Organ der Deutschen Gesellschaft fu¨r Psychologie, 1957. [115] D.R. Bach, K.J. Friston, R.J. Dolan, Analytic measures for quantification of arousal from spontaneous skin conductance fluctuations, Int. J. Psychophysiol. 76 (1) (2010) 52 55. [116] H.F. Posada-Quintero, J.P. Florian, A.D. Orjuela-Can˜o´n, T. Aljama-Corrales, S. Charleston-Villalobos, K.H. Chon, Power spectral density analysis of electrodermal activity for sympathetic function assessment, Ann. Biomed. Eng. 44 (10) (2016) 3124 3135.

Further reading B.G. Wallin, Sympathetic nerve activity underlying electrodermal and cardiovascular reactions in man, Psychophysiology 18 (4) (1981) 470 476. E.N. Sokolov, Orienting reflex as information regulator, in: A. Leontyev, A. Luria, A. Smirnov (Eds.), Psychological Eesearch in USSR, Progress Publishers, Moscow, 1966, pp. 334 360. J.A. Spinks, D.A. Siddle, The effects of anticipated information on skin conductance and cardiac activity, Biol. Psychol. 20 (1) (1985) 39 50. E. Duffy, The concept of energy mobilization, Psychol. Rev. 58 (1) (1951) 30.

177

CHAPTER

Study of modern brainimaging and -signaling techniques for brain computer interface

9

Vikas Dilliwar and Mridu Sahu National Institute of Technology, Raipur, India

9.1 Introduction A brain computer interface (BCI) is a communication path between a brain and a computer-based device. In the 1970s a BCI-based research started in University of California. The primary focus of BCI research and development has been on neuroprosthetic applications such as restoring damaged hearing and sight. Brainsignal processing (brain signaling) is widely used for various clinical applications and brain machine interface. In the brain-signal processing, first, a brain-signal acquisition is performed with the help of various technologies and methods, then some preprocessing operations such as segmentation, filtering, and denoising are performed before the analysis and diagnosis of signal. The other important applications of brain-signal process are possible in the field of brain machine interface, for example, mind-controlled gaming and entertainment, security and authentication, healthcare, smart environments, and education [1]. Neuroimaging can be broadly categorized in two types. First is structural neuroimaging, which is helpful to diagnose the internal cranial disease such as tumors and injuries that concern with the structure of the brain. Second is functional imaging, applicable for analyzing the metabolic diseases, Alzheimer’s disease, and also useful for neurological and cognitive psychology analysis as well as building a BCI or brain-interacting support system [2 4]. Some brain-imaging and signaling techniques are used for brain-diagnosis process by researchers and neuroradiologists, such as computed tomography (CT) or computed axial tomography (CAT), diffuse optical imaging (DOI) or diffuse optical tomography (DOT), high-density DOT (HD-DOT), magnetic resonance imaging (MRI), functional MRI (fMRI), single-photon emission computed tomography (SPECT), cranial ultrasound (CU), near-infrared spectroscopy (NIRS), positron emission tomography (PET), magnetoencephalography (MEG), electroencephalography (EEG), electromyography (EMG), and electrooculography (EOG). However, every neuroimaging technique has some challenges. Cognitive Informatics, Computer Modelling, and Cognitive Science, Volume 2. DOI: https://doi.org/10.1016/B978-0-12-819445-4.00009-6 © 2020 Elsevier Inc. All rights reserved.

179

180

CHAPTER 9 Study of modern brain-imaging and -signaling techniques

This chapter covers the different brain-imagining and -signaling techniques, sleep wave recording and analysis standards and its advantages and limitations along with the diagnosis and analysis of sleep disorder standard [5,6].

9.2 Brain-imagining techniques Brain-imagining tools and techniques are very valuable and advantageous paraphernalia for neurologists and neuroresearchers to identify the abnormalities and various brain-related diseases and disorders of brain. These techniques are based on image (actual pictorial representation) of neurons system using different brainscanning techniques such as CT or CAT, DOI or DOT, HD-DOT, MRI, fMRI, SPECT, CU, NIRF, and fNIRF [5].

9.2.1 Computer tomography The word “tomography” came from Greek terms “tomos” (its meaning is cut, slice, partition, or section) and “graphein” (means write). The computer tomography is the imaging procedure that uses a special type of X-ray machine to draw the detailed internal pictures of focused body. CT scans and captures image of various human body parts such as head, neck, lungs, angiography, cardiac, abdominal and pelvic, axial skeleton, and extremities [7]. In the 1970s, CT became an essential equipment for the imaging of internal human body with the addition of X-ray and medical ultrasonography. Now, it has been used for the identification of various diseases and abnormality of body [8]. CT scans two-dimensional cross-sectional images of the human body; these images are captured by X-ray tube that is surrounded in 360 degrees of a patient. In this process a machine with X-ray and CT scanner in it moves around a table on which the patient is laid. Fig. 9.1 shows a diagram of a CT machine. Sometimes contrast agents have also been used in the CT for highlighting specific area or cleaner picture. These agents are called “dye”; iodine and barium are commonly used dyes in CT process, these may be injected from mouth, veins, or enema [8].

9.2.1.1 Computer tomography head CT head is a unique X-ray equipment to capture the brain images from bottom to top of cranium. This is useful for the identification of head injury, severe headache, dizziness, aneurysm, bleeding, stroke, and brain tumors, etc. CT head is also useful for finding out some brain-related indications such as acute head trauma, vascular-related disease, calcification, evaluating psychiatric disorders, brain herniation, and suspected mass or tumor. CT is used in different clinical processes such as an assistance system in the neurosurgical and internal therapy process, identification of skull lesions, cranial

9.2 Brain-imagining techniques

FIGURE 9.1 CT scanner machine.

nerve seizures, apnea, neurodegenerative disease, brain developmental delay, medicinal toxicity, congenital morphologic brain abnormalities, abusive head trauma and postmortem forensic investigations, brain death, and suspected shunt malfunctions or shunt revisions [9].

9.2.1.1.1 Benefits • Painless, noninvasive, and precise brain-imaging technique, with the capability • • •

of imaging the bone, soft tissue, and blood vessels of body at same time are the advantages of CT. This has been a cost-effective imaging tool for various clinical problems and is advantageous for emergency cases to quickly identify the internal injuries and bleeding. CT is less sensitive and useful for medical device implanted patient, rather than MRI. CT scanning reduces the exploratory surgery and surgical biopsy; moreover, the effect of radiation ends after CT process.

9.2.1.1.2 Risk and limitation • Slight chances of cancer due to excessive radiation during the process, so emission of proper proportion of radiation dose is required.

• Records noted by physician are essential for X-ray or CT scan for pregnant

•

women because of radiation can affect the baby; moreover, as children are more sensitive to radiation, the scan should be done by low-dose radiation technique. It is observed that if a person undergoes multiple CT scans before the age of 15, there is a possibility of increase in the chances of leukemia, brain tumors, and other cancers.

181

182

CHAPTER 9 Study of modern brain-imaging and -signaling techniques

9.2.2 Near-infrared spectroscopy based imaging equipment 9.2.2.1 Functional near-infrared spectroscopy Functional NIRS (fNIRS) is a noninvasive neuroimaging technique based on light attenuation or NIR spectrum with 700 900 nm wavelengths. Jobsis first reported the ability of brain mapping using NIR in 1977. In 1985 first cerebral study were accomplished by Ferrari et al. [10]. fNIRS is harmless, less time-consuming, painless, and easiest method to take brain imaging to place optical fibers system in the form of plastic disk over a head without compulsion of any static body position, meaning that fNRIS can be recorded in sitting, lying, walking, and other active body positions. NIRS is a progressive admirable technology that studies the human brain function. The measurement concept of NIRS (fNIRS) is based on “Beer Lambert law”; according to that, the intensity differs between transmitted and received light signal on brain’s tissue oxygen saturation and tissue hemoglobin contents. Now, the various noninvasive NIRS-based equipment with oximetry sensor are available for recognizing the cerebral ischemia. The application of NIRS includes brain mapping, splanchnic, renal, and spinal cord analysis [11,12].

9.2.2.2 Diffuse optical imaging or diffuse optical tomography DOI is a recent medical imaging technique based on NIRS, which have a wide clinical and research application based on psychology and pathology. In DOI technique, result image is built from multiple NIRS analyses in the same target with multimodality imaging. Two important applications of DOI are functional brain imaging and breast cancer detection; apart from this, there are other wide applications in biology and medicine. The architecture of DOI is based on light source, photo-detection, spectral separation unit, signal modulation, and imaging contrast unit [13]. The principle of DOI is based on counting the potential changes on deoxyhemoglobin (HbR) and total hemoglobin (HbT) concentration, so it captures the dissimilarity of oxygen utilization and blood pressure at the time of brain activation. The measurement of HbR and HbT concentrations are very important factors in neuroscience; many researchers are improving in the cognition of these factors with sensation, resolution, and accuracy. Optical image quality can be improved from assortment of wavelength calculation of the hemoglobin concentrations, filtration of physiological signal to improve the sensitivity of hemodynamic reaction of brain, enhance the image resolution and quantity accuracy [14].

9.2.2.3 High-density diffuse optical tomography The diffuse optical brain mapping is a noninvasive, safe, radiation-free, less timeconsuming, and portable technique. Recent expansion of HD-DOT has improved human cortical brain mapping with better image quality. The principle of HD-DOT is based on NIRS and biological tissues. HD-DOT system captures the

9.2 Brain-imagining techniques

FIGURE 9.2 Optical imaging process.

brain function from the depth of cortex with improved signal-to-noise ratio and triumphs over the previous challenges of the system such as high-fidelity image of brain, better recognition of spatial, and temporal distribution of active brain imaging. DOT and HD-DOT are normally used for neuroimaging analyses as well as for other clinical applications such as breast cancer detection and noniterative measurement approaches such as muscle, peripheral circulation, joint Imaging, and thyroid imaging. Fig. 9.2 shows the basic concept of optical imaging [15].

9.2.2.3.1 Advantages and disadvantages of optical imaging Advantages:

• • • • •

Cost effective, smaller and harmless, and less artifacts [16]. Temporal resolution is high around 10 Hz. Less sensitive of body parts movement, clinically applicable. Comfort in long-term recording and less harmful for environment. Advantageous for infants, children, or patients.

Disadvantages:

• Spatial resolution is less. • Absence of standard analyzing package. • Imprecise activation localization.

9.2.3 Magnetic resonance imaging MRI is a popular, noninvasive imaging technique capable of producing a 3D image of various internal body parts of human using strong magnetic field and radio waves. MRI is applicable for the imaging of head, bone, muscle, and heart and useful for the diagnosis of related diseases and disorders [17]. In the process of MRI, patient is correctly placed on the movable bed that rolls on the machine throughout the procedure. (At that time, radiologists and technicians will leave the room and observe from attached room.) The imaging machine connected with computer system receives the signal and produces a 3D image of

183

184

CHAPTER 9 Study of modern brain-imaging and -signaling techniques

the internal body of the patient. The MRI is a big tube-shaped machine with a round magnet with 20,000 G (harmless for human body) [18].

9.2.3.1 Magnetic resonance imaging head MRI head is a useful tool for capturing the clear and detailed picture of brain and cortical structure with the help of high magnetic field and radio waves. This is useful for the detection of head internal injury, brain tumors, blood clotting of head, brain cancers etc. Fig. 9.3 shows the image capturing process [19].

9.2.3.2 Functional magnetic resonance imaging fMRI is used for the observation and identification of the responsible region of brain at particular events with the help of variation of blood flow, for example, mortar effect, eyeblink, sleep stage, epilepsy, and tumors. The working principle of fMRI is same like MRI; this also measures the quantity of variation in oxygenated blood flow. If oxygenated blood is more than the particular region, it means that the brain area is more active compared to other region. This process is called BOLD (blood-oxygenation level-dependent response). Low temporal resolution and restricted computational dependency can be observed as a snag of fMRI [20].

9.2.3.2.1 Advantages of magnetic resonance imaging • This is a noninvasive and radiation-free imaging technique. • Contrasting against of MRI is fewer allergies prone which is helpful to observe the size, location, growth, and blood flow.

• The imaging of MRI is more cleared and detailed (create 100 of images) • • •

compared to other imaging techniques, and 3D view provides the more understandable image. MRI is able to imaging almost all parts of body in depth. MRI is a boon of cancer detection and suggests of proper treatment. Explosive radiation less property of MRI is useful for imaging of pregnant women and kids.

FIGURE 9.3 MRI process. MRI, Magnetic resonance imaging.

9.2 Brain-imagining techniques

• MRI is mainly used for screening of soft tissues (ligaments, cartilages, eyes, etc.). This is also helpful for identifying the blood circulation of particular organ and finding the blockages.

9.2.3.2.2 Disadvantages of magnetic resonance imaging • This is costly as compared to other imaging techniques. • MRI is not capable of identifying some cancer tissues (such as breast cancers). • MRI is painless for body, but the machine produces quite disturbing sound, •

which is problematic for claustrophobic patients and less useful for metalimplanted patients due to high magnetic resonance unit. Major accidents also take place due to high magnetic field of MRI machine. The magnetic field can attract metal objects such as hospital equipment (such as stretchers, chairs, and oxygen cylinders). Many cases due to this have been recorded in the past.

9.2.4 Single-photon emission computed tomography SPECT is a nuclear medicine-based imaging technique for cerebral function analysis, which was introduced in the 1980s. It produces a good quality image in 2D planner with increased signal-to-noise ratio. In the SPECT, images are produced by the ring type gamma camera with the help of photon emitted by the flow tracer or receptor binding substance (intrusive injected with radionuclide) attentive in the brain. The camera of SPECT system is normally gamma cameras with FWHM (full width at half maximum) facility; multiple heads (crystal system) and different focal length gamma cameras have been used [21]. In the SPECT imaging system, parallel gamma rays stroke the crystal detector {normally sodium iodide thallium [NaI (Tl)] crystal drop} that changes the gamma rays to visible light. This visible light transforms into electron and photon by using the photomultiplier tubes (PMT) and pulse-height analyzer (PHA). PMT emits an electron and PHA transforms the energy in the form of photon, which are used for the creation of image. Then all the information are gathered in a computing device to reproduce the image; this image represents the actual structure of the body organ. In an image, high or low pixel density may represent some abnormalities such as arthritis, diseases, fracture, and tumors. Some artifacts are also identified in the SPECT, for example, star artifacts (occurred due to back projection), motion artifacts (generated because of movements of the subject), edge packing artifacts (dye to over brightness in the edge of crystal), and some equipment related artifacts may also produce because of the damage of collimators, PMT, PHA, or crystal [22].

9.2.4.1 Advantages of single-photon emission computed tomography • The essential things observed in the SPECT are its correctness, good sensitivity, cost-effectiveness, and clear imaging.

185

186

CHAPTER 9 Study of modern brain-imaging and -signaling techniques

• It is useful for diagnosing the imaging of dementia, congested blood vessels, seizure detection, epilepsy, head injuries, etc.

• SPECT is also applicable for the cardiac implantable electronic devices.

9.2.4.2 Disadvantage of single-photon emission computed tomography • This is much costly compared to other brain-imaging and cardiograph techniques, and its availability is fewer.

• Some limitations have been observed in spatial resolution, and computation •

and interpretation of results. The radiation of SPECT has been troublesome but not very harmful.

PET is also nuclear medicine based imaging technique that gives the highresolution functional brain images. It gives more appropriate information compared to SPECT.

9.2.5 Cranial ultrasound CU is a brain-scanning technique using high-frequency sound signal. CU is mainly used for imaging the brain for premature babies, infants having “acoustic window” (skull is not completely developed), blood clotting or damaged brain tissues identified, observed cognitive abnormalities, damaged white matter of brain, and traced infected site or tumor. Moreover, CU is also applicable for intraoperative imaging of brain at the time of neurosurgery of adult (used for finding a margin of tumors at open skull condition) [23]. Another ultrasound-based imaging technique is transcranial Doppler ultrasound, which is useful for recognizing the blood flow of brain arteries, assessment of sickle cell disease, stenosis (arteries goes to hard), vasospasm, etc. At the time of CU imaging, radiologists instruct the patients for preimaging preparation such as avoiding nicotine-based products, wearing loose-fitting and comfortable cloths, and removing jewelry and ornaments. The best of scanning of infants will be under sleeping or calm stage. Then, liquid-based gel is applied on the external surface of scanning area and ultrasound is used to view the images on the display unit and record its significance. In the standard CU, approximately 10 views are captured and recorded from different angles [24]. The working principle of ultrasound imaging is based on the sound signal reflection of sonar system. In CU the high-frequency sound waves hit the object and reflect echo waves, and these waves are used to determine the nature, shape, size, and other things of the object. CU is able to imaging the organ’s size, shape, tissues, and vessel and is helpful to determine abnormal object and tumors [25].

9.2.5.1 Advantages of cranial ultrasound • CU is a harmless, noninvasive, radiation-free imaging technique. • It is easily available, easy to use, and cost effective. • CU imaging shows clear picture compared to X-ray.

9.3 Brain-signaling techniques

9.2.5.2 Limitations of cranial ultrasound • Specialized and experienced person required for CU imaging. • Sometime visualization of posterior structure may be poor. • In case of premature infant, visualization of frontal lobe is a complicated task due to small size of the head.

• Sometimes, parenchyma, abnormal myelination, and ischemia problems are identified after ultrasound.

• This is a motion-sensitive task, challenging, and time taking in case of active and crying baby position.

9.3 Brain-signaling techniques Naturally generated brain electrical and magnetic signals are also used for the study, analysis, and diagnosis of brain function, neuro-disease and disorders; EEG, EOG, EMG, and MEG are commonly used brain-signaling techniques. This can be used for the development of brain computing interface and man machine interactive devices.

9.3.1 Electroencephalography EEG is widely used because it is a noninvasive, cost-effective, accurate, easily available brain electrical signal-based brain-mapping technique. The discovery of EEG started in the 19th century from electrical signal recording from cortex of animal. In the 1920s, Hans Berger, a German neuropsychiatrist, first recorded a brain electrical signal. In EEG, brain electrical signals (produce by neurons) are picked by the electrodes and recorded or plotted the electrical signal using computer. EEG is helpful to measure the brain activity from the waves generated by the electroencephalogram. This is a painless brain technique used to diagnose the various brain diseases and disorders such as distortion of brain tissues, sleep disorder, epilepsy seizures, head injury, narcolepsies, brain infections, and different metabolic conditions [26]. In the procedure of EEG recording, first the hairs and scalp (shampoo can be used) of the patient is cleaned, then is comfortably laid or seated at the recording place; at the same time electrodes are placed on the patient’s scalp (a number of electrodes are depending on electrode placement standard such as 10 20 and 10 10). Gel can be applied in the concerned area where electrodes are applied for good conductivity between skin and electrodes. Then after, preliminary testing is conducted, such as relaxed position of subject, breathing measurement, and flashing light on eyes (observe change in brain pattern on display unit). In a normal condition, EEG recording is 30 60 minutes. In some cases, this may be more. The abnormality of brain can be recognized by the EEG wave pattern. This wave patterns are classified in different types, according to its frequency

187

188

CHAPTER 9 Study of modern brain-imaging and -signaling techniques

FIGURE 9.4 EEG signal capturing process. EEG, Electroencephalography.

bands: alpha, beta, gamma, theta, delta, and mu. Generation of each wave represents different normal and abnormal activities. Some artifacts are also recorded at the time of EEG recording, the removal of these artifacts are another important task for accurate EEG analysis; Fig.9.4 represents the basic EEG setup for signal recording [27,28].

9.3.1.1 Application of electroencephalography [28] • The major application of EEG is the study and analysis of nervous system by the researcher, neurologist, psychiatrist, and neuroradiologist.

• This is used to observe the premature infant’s neurodevelopment at the NICU. • This is also applicable for mapping the brain area and location before neurosurgery of epilepsy, tumor, etc.

• It is helpful to identify the sleep disorder, chronic neural problem, depression, etc. using EEG patterns.

9.3.1.2 Advantages of electroencephalography • This is a cost effective, fast, harmless, brain-mapping technique. • The silent nature of EEG recording can increase the accuracy of response. • Placements of electrodes are easy in the noninvasive manner. • Installation of EEG equipment is easy and allows to record in high temporal • • •

resolution in milliseconds. EEG has the ability to record all the potential of brain without responding the subject’s stimuli. EEG is capable to capture motor response. The cost of EEG equipment is less, compared to other brain-mapping devices.

9.3.1.3 Disadvantages of electroencephalography • The limitation or disadvantage of EEG is its less spatial resolution. • Identification of exact location of brain injury and brain stroke is a challenging task.

• Sometime, it requires a different procedure and experienced person for the identification of some brain seizures and brain disorders.

• Various artifacts are also recorded with brain signals. • Recording and analysis of EEG signal are time-taking processes.

9.3 Brain-signaling techniques

9.3.2 Magnetoencephalography MEG is a noninvasive brain-mapping technique that measures the electrical activity of brain using magnetic fields with high temporal resolution. MEG information has been used for pinpointing the epilepsy origin, neurosurgery, identification of tumors, and brain injury and is helpful to understand and diagnose the function of nervous system for neuroresearchers and neuroradiologists [29]. In 1968 David Cohen first used the copper coil based magnetic field for capturing brain electrical signal, which is called magnetoencephalograph. Then further superconducting quantum interference device detector was used at the head of subject for MEG signal measurement. In the 1980s MEG device manufacturer was started the MEG detector in the form of helmet-shaped vacuum container with around 300 sensors that covered the head. This MEG scanning system has been popular, advanced, safe and noise free. The advantages and limitations of MEG are shown in the following subsections [30,31].

9.3.2.1 Advantages of magnetoencephalography • This is noninvasive and capable of identifying the seizure location. • MEG sensors are more sensitive and accurate to capture the small magnetic field generated by the brain with exact position.

• This is radiation free and harmless for children. • Useful for epilepsy pinpoint diagnosis and surgery and applicable for motor or sensory mapping.

9.3.2.2 Limitations of magnetoencephalography • Expert neuroradiologist and technicians are required for MEG process. • Magnetic field used by MEG has a negative impact on implant medical device in the body and other metal-based substance.

• Very sensitive in nature; sometimes guardians are also not allowed to enter the MEG room while diagnosing children.

• Pregnant or expected pregnancy women may encounter some problem due to the magnetic field.

9.3.3 Electromyography EMG is a bioelectrical signal-based diagnosis technique that recognizes and records the muscles-based electrical signal. The equipment performs the EMG signal acquisition is called Electromyograph. An EMG has an application on various clinical uses like identification of neurological and neuromuscular troubles, biomedical instrument development and recent human computer-interface system (e.g., motor controller, biomechanical system, and prosthetic devices) etc. EMG signal acquisition involves some major steps such as decomposition, processing, and classification [32].

189

190

CHAPTER 9 Study of modern brain-imaging and -signaling techniques

FIGURE 9.5 EMG signal recording system. EMG, Electromyography.

Francesco Redi started the concept of EMG from the muscle signal analysis of electric fish in 1966. The first muscle recording was performed and introduced the word “electromyography” in 1890 by Marey. The clinical application of EMG started in the late 1960s. Cram and Steger launched the sensor-based EMG device with scanning facility on different types of muscles in the 1980s. Now the advance EMG recording equipment are available with low artifacts and accuracy [33]. In the process of EMG recording, electromyograph equipment senses the stimulated muscular cell from bioelectrical signal of human body. Then the recorded signal can be used to diagnose the abnormalities, muscle activation level, or biomechanical applications. Under the EMG diagnosis, nerves-based and needle electrode based distinct information is composed and finally depict the analysis and draw the final conclusion related to subject. Fig. 9.5 represents the equipment arrangement of EMG signal recording system [33].

9.3.3.1 Applications of electromyography EMG has wide applications in the field of medical research, clinical medicine, sports, biomechanics, ergonomics, various rehabilitations such as neurological, physical, and postsurgery. EMG is medically used for the diagnosis of muscle disorder, nerves disorder, root disorder, and neuropathy [3,34,35].

9.3.3.2 Advantages of electromyography • EMG can be used for the analysis of paralytic subject. • Useful for the recognition of motionless gesture and unvoiced speech. • EMG is applicable to control the prosthesis devices. • Man machine interface can be possible with the help of EMG, for example, wheelchair, robots, and computer-based games.

9.3.3.3 Limitations of electromyography • Needle (invasive type) used in EMG recording will be constraint, if patients have unwillingness or noncooperative mood.

• Additional fat (tissues) has an effect on EMG recording. • EMG signal recording is more accurate for young people rather than elder people.

• EMG recording on deep muscles can be painful due to intramuscular wires. • Only surface EMG is not competent to provide the complete information.

9.4 Sleep-based disorder analysis using neurodiagnosis techniques

9.4 Sleep-based disorder analysis using neurodiagnosis techniques Sleeping is an important factor in our daily routine, and a good sleep makes the life healthy. Sleeping is directly concerned with psychology, human behavior and disorders, brain signals, etc. The neurologists, psychiatrists and researchers are required the need of sleep test (sleep pattern) for the study and diagnosis of activities during the sleeping stage. Some sleep test methods are developed for the study of sleeping characteristics and disorder, that is, simple sleep studies, polysomnography, multiple sleep latency tests, and home sleep tests. The polysomnography is called golden standard from sleep test [36]. Sleeping study can be useful for the diagnosis of some sleep-related neurological and psychological disorders such as breathing, sleep apnea, seizures, motor movements related disorders, insomnia, stress, depression, discomfort, narcolepsy nighttime disorder (sleepwalk, night terror, bed wetting, etc.), and working shift problem. Sleep disorder can also be determined from sleep stage analysis of random eye movement sleep and nonrandom eye movement sleep [37].

9.4.1 Polysomnography Polysomnograhy (PSG) is a golden standard for sleep stage analysis. The equipment used in PSG is called polysomnograph. The polysomnograph records various parameters used in sleep analysis, for example, brain waves, eye movements, oxygen level in blood, heart rate, heart rhythm, air dispense on nose and mouth, and body movements. This multiparametric test is performed during night at the dedicated sleep room in the diagnostic centers by well-trained technicians and radiologists [38]. PSG is trustworthy and used universally for sleep test that includes EEG, EMG (of face, leg, and arm), EOG, ECG, positive airway pressure (PAP), and respiratory equipment. In PSG other evaluations can also be conducted such as temperature of body and esophageal. In some cases polysomnography is performed with video recording of EEG which is more efficient than polysomnography. This is called video-EEG polysomnography [39]. In the process of PSG, patient comes in sleep test center in the evening with the preparation of night stay. The PSG recording room is just like a bedroom with all sleeping facility of singles person. The sleeping area facilitates with deem light, video recording facility, and two-way audio system. At sleep time of subject, technicians place all the PSG-related equipment in the body and PAP machine if required. After recording the full night activity, all sensors and equipment are removed from the body and the patients are allowed to leave from sleep center. The measured recordings have adequate information related to sleep pattern, that is, brain signal and eye movement information (helpful for diagnosis of

191

192

CHAPTER 9 Study of modern brain-imaging and -signaling techniques

narcolepsy and random eye movement related sleep disorder), heart breathing rate and oxygen level in blood (helpful for suggestion of sleep apnea), movements of leg, and other abnormal movements that affect the sleep and other essential information. Then all information is combined together, and report will be sent to the recommended physician. Normally, patients are strictly advised to evade the alcoholic- and caffeine-related things before PSG [40].

9.4.1.1 Advantages of polysomnograhy • PSG is a noninvasive, painless sleep test. • Technicians are available for monitoring all equipment. • All sleep patterns are recorded for detailed analysis. • PSG is helpful tool for neurologists, neuroradiologists, and psychiatrists for the diagnosis and analysis of symptoms of sleep and neurological disorders.

9.4.1.2 Limitation of polysomnograhy • The adhesive is used for the attachment of sensors in the skin that makes skin • •

irritation and skin reaction also. Sometimes, the feeling of sleep room can be uncomfortable. PSG is expensive and availability in less.

9.5 Summary The advancement of computer-based medical instruments and various diagnostic tools and techniques increases the accuracy and makes the analysis and diagnosis of various internal problems on human body easier for the physicians and researchers. The diagnosis and understanding the nervous system of human brain is a very challenging task. Nowadays, this is possible with the help of different advanced computer-based neurodiagnosis techniques (brain imaging and signaling). Some limitations have been observed in specific neurodiagnosis techniques (mentioned in the chapter), but this is not a restriction for the diagnosis of any particular disease and disorder, which means it can be possible with other alternate methods. Other important advantages of computer-based brain-interactive tool have been observed in the attraction toward the brain-controlled inventions. This can be seen in the man machine interface, prosthetic medical devices, neurogaming, sports training, etc. So the BCI is the medical boon for the diagnosis of various brain diseases, disorders and is helpful to make life easier with braininteractive device specially for physically disabled, ALS, and other neurodisabled patients.

References

References [1] R.A. Ramadan, A.V. Vasilakos, Brain computer interface: control signals review, Neurocomputing (2017). Available from: https://doi.org/10.1016/j.neucom.2016. 10.024. [2] S. Bularka, A. Gontean, Brain-computer interface review, in: 2016 12th International Symposium on Electronics and Telecommunications, ISETC 2016 Conference Proceedings, 2016. ,https://doi.org/10.1109/ISETC.2016.7781096.. [3] S.N. Abdulkader, A. Atia, M.S.M. Mostafa, Brain computer interfacing: Applications and challenges, Egypt. Inf. J. (2015). Available from: https://doi.org/10.1016/j. eij.2015.06.002. [4] J. D’haese, J. Ackhurst, D. Wismeijer, H. De Bruyn, A. Tahmaseb, Current state of the art of computer-guided implant surgery, Periodontol. 2000 (2017). Available from: https://doi.org/10.1111/prd.12175. [5] J.H.W. Jacobsen, L.M. Parker, A.V. Everest-Dass, E.P. Schartner, G. Tsiminis, V. Staikopoulos, et al., Novel imaging tools for investigating the role of immune signalling in the brain, Brain Behav. Immun. (2016). Available from: https://doi.org/ 10.1016/j.bbi.2016.04.014. [6] W.E.A. Saad, N. Saad, Computer tomography for venous thromboembolic disease, Radiol. Clin. North Am. (2007). Available from: https://doi.org/10.1016/j. rcl.2007.04.011. [7] R.D. Neumann, D.A. Bluemke, Tracking radiation exposure from diagnostic imaging devices at the NIH, J. Am. Coll. Radiol. (2010). Available from: https://doi.org/ 10.1016/j.jacr.2009.07.012. [8] K. Abramovitch, D.D. Rice, Basic principles of cone beam computed tomography, Dent. Clin. North Am. (2014). Available from: https://doi.org/10.1016/j. cden.2014.03.002. [9] H. Goerne, P. Rajiah, Computed tomography, in: Right Heart Pathology: From Mechanism to Management, 2018. ,https://doi.org/10.1007/978-3-319-737645_35.. [10] M. Ferrari, L. Mottola, V. Quaresima, Principles, techniques, and limitations of near infrared spectroscopy, Can. J. Appl. Physiol. (2004). Available from: https://doi.org/ 10.1139/h04-031. [11] J. Tremblay, E. Mart´ınez-Montes, P. Vannasing, D.K. Nguyen, M. Sawan, F. Lepore, et al., Comparison of source localization techniques in diffuse optical tomography for fNIRS application using a realistic head model, Biomed. Opt. Express (2018). Available from: https://doi.org/10.1364/boe.9.002994. [12] P. Pinti, C. Aichelburg, S. Gilbert, A. Hamilton, J. Hirsch, P. Burgess, et al., A review on the use of wearable functional near-infrared spectroscopy in naturalistic environments, Jpn. Psychol. Res. (2018). Available from: https://doi.org/10.1111/ jpr.12206. [13] L.C. Enfield, A.P. Gibson, Review: a review of mechanisms of contrast for diffuse optical imaging of cancer, J. Infrared Spectrosc. (2012). Available from: https://doi. org/10.1255/jnirs.949. [14] S.A. Carp, Q. Fang, Diffuse optical imaging, in: Pathobiology of Human Disease: A Dynamic Encyclopedia of Disease Mechanisms, 2014. ,https://doi.org/10.1016/ B978-0-12-386456-7.07605-X..

193

194

CHAPTER 9 Study of modern brain-imaging and -signaling techniques

[15] S.M. Liao, High-density diffuse optical tomography of term infant visual cortex in the nursery, J. Biomed. Opt. (2012). Available from: https://doi.org/10.1117/1.jbo.17. 8.081414. [16] Y. Hoshi, Y. Yamada, Overview of diffuse optical tomography and its clinical applications, J. Biomed. Opt. (2016). Available from: https://doi.org/10.1117/1.jbo.21.9.091312. [17] J. Barentsz, S. Takahashi, W. Oyen, R. Mus, P. De Mulder, R. Reznek, et al., Commonly used imaging techniques for diagnosis and staging, J. Clin. Oncol. (2006). Available from: https://doi.org/10.1200/JCO.2006.06.5946. [18] M.F. Bellin, MR contrast agents, the old and the new, Eur. J. Radiol. (2006). Available from: https://doi.org/10.1016/j.ejrad.2006.06.021. [19] E.D. Gareth, K. Nisha, L. Yit, G. Soujanye, H. Emma, N.J. Massat, et al., MRI breast screening in high-risk women: cancer detection and survival analysis, Breast Cancer Res. Treat. (2014). Available from: https://doi.org/10.1007/s10549-014-2931-9. [20] A.L. Lin, H.Y. Monica Way. Functional magnetic resonance imaging, in: Pathobiology of Human Disease: A Dynamic Encyclopedia of Disease Mechanisms, 2014. ,https://doi.org/10.1016/B978-0-12-386456-7.07610-3.. [21] P. Herscovitch, Single-photon emission computed tomography (SPECT), in: Encyclopedia of the Neurological Sciences, 2014. ,https://doi.org/10.1016/B978-012-385157-4.00204-9.. [22] J.C. Masdeu, Single-photon emission computed tomography, in: Neurobiology of Disease, 2007. ,https://doi.org/10.1016/B978-012088592-3/50078-5.. [23] G. van Wezel-Meijler, S.J. Steggerda, L.M. Leijser, Cranial ultrasonography in neonates: role and limitations, Semin. Perinatol. (2010). Available from: https://doi.org/ 10.1053/j.semperi.2009.10.002. [24] S.J. Steggerda, G. van Wezel-Meijler, Cranial ultrasonography of the immature cerebellum: Role and limitations, Semin. Fetal Neonatal Med. (2016). Available from: https://doi.org/10.1016/j.siny.2016.04.011. [25] J.L. White, K.N. Sheth, Neurocritical care for the advanced practice clinician, in: Neurocritical Care for the Advanced Practice Clinician, 2017. ,https://doi.org/ 10.1007/978-3-319-48669-7.. [26] S. Sanei, J.A. Chambers, EEG signal processing, in: EEG Signal Processing, 2013. ,https://doi.org/10.1002/9780470511923.. [27] W. Klimesch, P. Sauseng, S. Hanslmayr, EEG alpha oscillations: the inhibitiontiming hypothesis, Brain Res. Rev. (2007). Available from: https://doi.org/10.1016/j. brainresrev.2006.06.003. [28] A.F. Jackson, D.J. Bolger, The neurophysiological bases of EEG and EEG measurement: a review for the rest of us, Psychophysiology (2014). Available from: https:// doi.org/10.1111/psyp.12283. [29] F. Lopes da Silva, EEG and MEG: relevance to neuroscience, Neuron (2013). Available from: https://doi.org/10.1016/j.neuron.2013.10.017. [30] S. Baillet, Magnetoencephalography for brain electrophysiology and imaging, Nat. Neurosci. (2017). Available from: https://doi.org/10.1038/nn.4504. [31] R.C. Burgess, Magnetoencephalography, in: Encyclopedia of the Neurological Sciences, 2014. ,https://doi.org/10.1016/B978-0-12-385157-4.00533-9.. [32] M. Gonza´lez-Izal, A. Malanda, E. Gorostiaga, M. Izquierdo, Electromyographic models to assess muscle fatigue, J. Electromyogr. Kinesiol. (2012). Available from: https://doi.org/10.1016/j.jelekin.2012.02.019.

Further reading

[33] B. Jonsson, Electromyographic kinesiology, in: New Concepts of the Motor Unit, Neuromuscular Disorders, Electromyographic Kinesiology, 2015. ,https://doi.org/ 10.1159/000394053.. [34] J. Garza-Ulloa, Introduction to human neuromusculoskeletal systems, in: Applied Biomechatronics Using Mathematical Models, 2018. ,https://doi.org/10.1016/b9780-12-812594-6.00002-0.. [35] D. Winter, EMG interpretation, in: Electromyography in Ergonomics, 2017. ,https://doi.org/10.1201/9780203758670.. [36] C. Karmakar, A. Khandoker, T. Penzel, C. Schobel, M. Palaniswami, Detection of respiratory arousals using photoplethysmography (PPG) signal in sleep apnea patients, IEEE J. Biomed. Health Inf. (2014). Available from: https://doi.org/ 10.1109/JBHI.2013.2282338. [37] B. van Alphen, M.H.W. Yap, L. Kirszenblat, B. Kottler, B. van Swinderen, A dynamic deep sleep stage in Drosophila, J. Neurosci. (2013). Available from: https://doi.org/10.1523/jneurosci.0061-13.2013. [38] B. Jafari, V. Mohsenin, Polysomnography, Clin. Chest Med. (2010). Available from: https://doi.org/10.1016/j.ccm.2010.02.005. [39] L.S. Teixeira, R.C. Granjeiro, C.A.P. De Oliveira, F.B. Ju´nior, Polysomnography applied to patients with tinnitus: a review, Int. Arch. Otorhinolaryngol. (2018). Available from: https://doi.org/10.1055/s-0037-1603809. [40] Q. Li, X. Zhao, D.H. Gong, Y.M. Geng, H.L. Zhang, P.X. Bi, Sleep disorders of acute thalamic stroke and its influence on plasma IL-17, J. Biol. Regul. Homeost. Agents 31 (2017) 745 751.

Further reading D.J.A. Margolis, J.M. Hoffman, R.J. Herfkens, R.B. Jeffrey, A. Quon, S.S. Gambhir, Molecular imaging techniques in body imaging, Radiology (2007). Available from: https://doi.org/10.1148/radiol.2452061117. L.F. Nicolas-Alonso, J. Gomez-Gil, Brain computer interfaces, a review, Sensors (2012). Available from: https://doi.org/10.3390/s120201211. A. Suhrbier, M. Riedl, H. Malberg, T. Penzel, G. Bretthauer, J. Kurths, et al., Cardiovascular regulation during sleep quantified by symbolic coupling traces, Chaos (2010). Available from: https://doi.org/10.1063/1.3518688.

195

CHAPTER

Reading an extremist mind through literary language: approaching cognitive literary hermeneutics to R.N. Tagore’s play The Post Office for neurocomputational predictions

10

Valiur Rahaman and Sanjiv Sharma Madhav Institute of Technology & Science, Gwalior, India

10.1 Introduction 10.1.1 Why transdisciplinary? The problem is that cognitive science, linguistics, literature, neuroscience/brain studies, and computation studies are being practiced separately to work in their specific domains. Interlink is tangentially touched upon when required in a research work. This separatist epistemic politics does not let the transdisciplinary research work be done easily where the interferences of researchers of various disciplines lead to an innovative conceptual domain of knowledge required for current and coming times. But an erratic derogatory remark “Jack of all trades, master of none” is often used attacking words for transdisciplinary/interdisciplinary researchers, while a researcher knows his domain well first before touching upon another subdomain or other domains of knowledge [1]. The first and foremost requirement of becoming a transdisciplinary research is that one must know one’s domain of knowledge deeply. One moves from one’s domain towards another domain under a compulsion—epistemological condition and force, without which one thinks one’s work remains incomplete. There is a compulsion, a force, a condition, and a rational cause behind the move of a sincere researcher.

10.1.2 Tagore’s The Post Office: a cognitive neurology The question is what that “compulsion” is, why that force works as a factor of developing or an emergence of innovative thought working behind all Cognitive Informatics, Computer Modelling, and Cognitive Science, Volume 2. DOI: https://doi.org/10.1016/B978-0-12-819445-4.00010-2 © 2020 Elsevier Inc. All rights reserved.

197

198

CHAPTER 10 Reading an extremist mind through literary language

transdisciplinary works. Keeping this hypothesis in mind, R.N. Tagore’s play The Post Office is studied here as a neurology. It is observed that the play was translated into many foreign languages and dramatized at affected areas of the European countries during the World War II (WW II), though the play was written originally in Bengali. It is a short play of three acts in which an orphan child dies due to intense captivity. He is characterized as a metaphor to understand a difference between extremism and nationalism, humanity and patriotism. It is used as a cognitive play to counsel affected minds of the people under threats of the WW II. The play was used as a tool of Cognitive-behavioral therapy (CBT) during disastrous war- and trauma-affected people. It was used as a medical tool to cure psychoneurologically disturbed people who were to die but when and how was not known to them. The primary function of a neuroliterature or neuropath witnessed through staging the play or reading or exhibiting is the emergence of the very same neuronal effect of which the author has procured once for representing through creative work.

10.2 Affecting factors to activate mirror neuron in R.N. Tagore The chapter argues for neuronal etiology of creative works, such as Tagore’s The Post Office written under traumatic circumstances owing to death of the dearest members of his family—his mother Sharda Devi (when the author was of 14), his sister-in-law Kadambari Devi (1858 84 aged 26), his wife Mrinalini Devi (1872?-1902 aged 30) , his younger daughter, Renuka (1891 1903 aged 12), and his youngest son Shamindranath (1896 1907 aged 11 years), followed by the death of his elder daughter Madhurilata (1886 1918 aged 32). Besides, he witnessed the Bengal partition (1909), under the British colonial divide and rule tricks, which was not less than a death of a nation having one culture, one language, and one soil. The partition saddened the author and mirrored in his song Bangalar mati, bangalar jol (The soil of Bengal, the water of Bengal). For him, it was nothing but a result of national/colonial “egoism,” a consequence that also resulted in Jallianwala Bagh Massacre (April 13, 1919) and the inhuman, heinous crimes from pre-Christian era to the day. The author witnessed saddening egoist extremism and revealed the very same empathetically in The Post Office, the play. Thus the author is observed as a sufferer from traumatic “mirror neuron” (Pellegrino et al., 1992; Gallese et al., 1996; Rizzolatti et al., 2004, 2001; Ramachandran V.S., 2000) [2,3, 4, 5, 6]. And the same seems transferred to the audiences be they translators or stage performers or depth readers. The death or a silence of a sportive child Amal saddens audiences as if death of a liberty causes death of humans.

10.3 Hypothesis

10.3 Hypothesis Nation, ideology, religion, soul, and the like entity of the universe under extreme care urge all the possible ways to waste them at large. They look like Amal the child under a captivity of intense patronizing care, devoid of all natural divinity, lively pleasure, and socialization leading his stark isolation. Hence Amal’s death mirrors the death of the nation, ideology, religion, soul, and the like. There is a chain of mirror neurons that work behind the creation of the play and its worldwide impact. Rabindranath Tagore’s The Post Office (1912, Dak Ghar) is all about the queries of Amal, an adopted child confined to his patronizing uncle, Madhav, suffering from an incurable sickness. All the characters—Madhav, who adopted Amal; Gaffer (in disguise of a Fakir, Act 2); Sudha, a little flower-gatherer; and all—direct their conversation toward the child only. It is only he who walks in everyone’s life. Why so? He cannot be a common character, as he seems to be. A world poet does not concentrate on characterizing a trivial thing. There is something that seems missed by us while reading The Post Office and we often narrate a simple story connected with a few incidents. The Post Office, written in 4 days, attracted many scintillating minds of that time and later ([7]:26). What is the reason behind its significant place in Tagore’s career? Why Amal worked as a soothing character during the WW II? What the play does is of a great remarkable factor of worldwide reputation for Tagore [8]. We often narrate The Post Office, the play as if there is nothing to understand in it. We simply narrate that Amal, an imaginative and interactive child, stands in adaptive uncle’s courtyard and talks to passer-by about the places they visit. Only one hope remains and it was to receive a letter from the King from a new post office nearby. It inspires Amal to imagine for receiving a letter from the King or being his postman, which seems a subject of mockery to the village headman. And his hope raised out of nothing, is fulfilled when the royal physician really comes with a herald with public announcement of arrival of the king. But Amal dies before their arrival; before Sudha comes to gift him flowers. That’s all about the play, yet it attracted leading symbolic modern writers, translators, and activists such as W.B. Yeats (1865 1939), a noble laureate (1923); Andre´ Paul Guillaume Gide aka Andre Gide (1869 1951), the French poet and Noble Laureate (1947), J.R. Jime´nez (1881 1951), Spanish poet and Noble Laureate (1956), and Korczak. They translated Dak Ghar (The Post Office) into their mother tongues, read, and staged for specific reasons. W.B. Yeats, a modern English literary writer, translated the play into English for the first time and wrote a tiny preface to it saying that the little play “conveys to right audience and emotion of gentleness and peace” [9]. The Irish Theatre performed the play in English for the first time in 1913 in London when Tagore was himself present there. The original play was staged in Calcutta (1917), successfully performed in Germany (Bhattacharya: 47 originally quoted in Wikipedia).

199

200

CHAPTER 10 Reading an extremist mind through literary language

It is an often discussed theme of liberation from captivity in terms of spirituality; and hope and zest for life echoed in major cities where atrocities were at their height during WW II. For instance, Andre´ Gide read The Post Office on the radio well before the Nazis captivated Paris. A Polish version of the play was performed under the supervision of Janusz Korczak (1879 1942), a Polish educator, pediatrician, children advocate, and author, in the Warsaw Ghetto (a German captivated Jewish residential district during Nazism, Jews would be exterminated guising resettlement after making them move out/deport from the district to Nazi concentration camps or mass killing canters). At that time an experience of uncertainty of life and death for the Jews had been defeating the courage of the common people. At such a time, Korczak decided to stage the play believing that the play would “help the children face death without fear” [10]. For him, Amal’s captivity was much akin to Jews captivity [10]. The captive condition of Amal, the orphan, is compared with “like everyone in the ghetto, awaiting an uncertain future” and the dying Amal was looked as “dying ghetto.” On an enquiry about why he had selected that particular play, he replied that it was necessary to learn to accept “serenely the angel of death” [10]. Korczak and his children were put to death after the performance of the Post Office was over. The question with which this chapter reverberates is how a tiny play written in Bengali language, a faraway dialect from the West people, could attract so much global attention in its time and later? Is there any connection of Amal’s existence, his life, his suffering, and his death with author’s contemporary history of the Wars, the question of humanity, the problems of defining nationalism as the West had defined once and resulted inhuman massacre at large? In order to answer this, affinities between key characters’ communication and situation in which a nation or humanity, or a soul flourishes or dies are observed. This kind of visualization of contextual or embedded meaning may also be represented through quantitative digital research tools [11]. But the categories of characters representing human race under threats are represented through Table 10.1 and Fig. 10.1. As Fig. 10.2 represents Amal’s isolation and his inclination towards nature while doctor and Madhav who represents monitoring and overcaring patronization respectively control Amal’s inclination.

10.4 Colonialism/nationalism or national extremism: symptoms psychoneurological disorders The Bengal Partition, Amritsar Massacre, and other colonial injustices deeply influenced his motor and related neuro-cells and made him disagree with Mahatma Gandhi on Swadeshi Movement based on Charkha concept [see Modern Review (1921, p. 25)] and his idea of India nationalism. He wrote Nationalism (1917) “The Cult of the Charkha” (Modern Review, 1925) for disseminating the

10.4 Colonialism/nationalism or national extremism

Table 10.1 Representation of The Post Office play in real world situation using representatives of Ideas. Details about characters of Post Office play Characters

Role/relationship/identity

Representatives of ideas

Amal Madhav

An orphan child Father of Amal

Doctor

For regular check up and monitoring and factor of control

Sudha

Passer-by, flower-gatherer

Royal physician

Stranger, visitor

Nature

A mode of communication for Amal Objects who are very close to Amal

Humanity, religion, nation, soul Authority, patronizing agent of a nation, humanity, religion, soul Priesthood, religious preacher, ministerial staff to patronizing agents, distorter of meanings contained in scriptures, a less experienced person Empathy, social/national connector/ fraternity Well-wisher harbinger of liberty and freedom All the creatures, living and nonliving

Autumn wind and Sun

External environment (abstract and concrete)

idea of global freedom from the terrorism of inhumanity. To note, his disagreement was not denial of the movement but on its limitation to its regions [12]. Nationalism (1917) is a compilation of his delivered lectures, is as an epistle on the death of nationalism itself. “Nationalism in the West” is one of a series of lectures delivered throughout the United States during the winter of 1916 17, “Nationalism in Japan” delivered in Japan in 1916. In the United States, he wrote “Nationalism in India” (1916), attacks on nationalist egoism. The poem “The Sunset of the Century,” which concludes the book, was written on the last day of the last century [12]. He defines nationalism with its inseparable value attained only by humanity and interconnectedness with one and all living in a society. He criticizes the root cause of the most difficult situation in the world, that is, two World Wars mediated with Great Depression and remarked, “In the West the national machinery of commerce and politics turns out neatly compressed bales of humanity which have their use and high market value; but they are bound in iron hoops, labeled and separated off with scientific care and precision” (1917, p. 17). And commerce and politics, I add, turned into technological advancement resulting bombs and experimental machines advanced and brutally used over humans in hurry. The sickness of nationalism/humanity is symbolically represented by Amal captivated in the name of care and put in jail from where Amal tries to maintain connectivity with pass goers hoping for an establishment of peach and new humanity dies at last being unknown to other characters.

201

CHAPTER 10 Reading an extremist mind through literary language

1858–1884

1873–1962

1891–1903

1896–1907

1886–1919

• Death of mother, Sharda Devi • 14 age

• Death of sister-in-law, Kadambari Devi • 4 months later in marriage • 26 age

• Death of his wife, Mrinaline Devi • 29 age

• Death of younger daughter, Renuka • 12 age

The Post Office (Dak Ghar) (1912) the three act play

R.N. Tagore

• Death of son, Shamindra Nath • 11 age

• Banglar Mati Banglar Jol • Nationalis m (1970) • The Cult of the Charkha (1925)

• Death of eldest daughter, Madhurilata • 32 age

• Death of Bangol partition 1909

Mirror neuron

1875

Mirror neuron

202

Soothers fr the sufferer Translator performer readers audience

• Develops patience audience • Againt social disparity

• Amritsar massacre April 13, 1919

• World War I 1914–1919

FIGURE 10.1 Psychoneurological etiology of the author and derived knowledge from mirror neuron.

Sudha Nature Flower gather

Mode of communication

Over caring patron

Madhav

Doctor Amal

Monitoring

FIGURE 10.2 Knowledge representation of key characters in The Post Office.

The play was translated and performed during the wars, of course, not for entertainment but for its serious issue symbolically presented. R.N. Tagore witnessed its failing impact, a few people understand the symbolical, indirect,

10.5 The mind of extremist: a neurological observation

literary, or artistic meaning engrained within a particular context. Later he started disseminated his views about nationalism worldwide as observed earlier. As said earlier, it was the period when he encountered the views of M.K. Gandhi about Swadeshi Nationalism and Nonalignment, when politics of nationhood was at its zenith in the world, when nations were preparing themselves to protest against colonization; when imperialism was threatened in the world. Empathy arousal incidents owing to these upheavals caused psychoneurological play to transfer the very same sentiments as experienced by its author.

10.5 The mind of extremist: a neurological observation The play comments on failure impact of books and scriptures on the disease condition. This connotes to how scriptures do not work for betterment of humanity if people became sick of their own profits, nation and racial profits. Humanity dies in utter captivity, in utter care. There must be a space of liberty of movement outside an encircling force. Amal could have lived a healthy life if he had a chance to go out and interact with natural agencies (human beings, animals, and rivers). Utter care kills Amal, a suffering child from isolation and a sense of captivity. The fundamentalists are utter caretakers of scriptures and general practices. Under such protective shields and controls people fail to interact with scriptures; people are not allowed to enjoy the right to question, to deny, and to accept; to adopt and to relieve. Amal’s compulsive captivity under physician’s instruction and Madhav’s so-called loving care symbolically represent a human soul, a religion, and a nationalism encircled or captivated with an undaunted shackle of iron-like words under laws. How can a human soul, a human sensibility, a true sense of nationality that carries an essence of global fraternity or humanity be long lived? How can a scripture be influential in a good life unless you experiment with it? As Amal dies not with any disease but with an extreme care so a soul dies, a religion dies, a nation and nationalism dies. Amal’s death symbolizes the death of a human soul. His death recalls the death of a religion under utter care. His death symbolizes the death of a true nationalism of undiscerned humanity based on an extreme care of a nation’s ideology. Madhav enjoys his own will. Amal loves to be in touch with natural agencies, but he is restricted to enjoy his will. Your extreme care interrupts one’s own will to enjoy life. In the beginning of the play, the Madhav physician dialogues connote to an existential helplessness of both the caretakers. Madhav asks to seek some ways to cure the sickness of Amal but the doctor helplessly advises Madhav with reference to scriptures, “What else can you do? The autumn sun and the damp are both very bad for the little fellow for the scriptures have it: ‘In wheezing, swoon or in nervous fret, In jaundice or leaden eye’”[13]. Madhav reacts and says, “Never mind the scriptures, please. Eh, then we must shut the poor thing up. Is there no other method?” (PO: p. 11 My Italics).

203

204

CHAPTER 10 Reading an extremist mind through literary language

Doctor goes on advising to care the child intensively. Madhav interrupts and remarks disdainfully, “Your system is very, very hard for the poor boy; and he is so quiet too with all his pain and sickness. It tears my heart to see him wince, as he takes your medicine” (PO: p. 11 My italics). But the doctor continues reading the book and goes on instructing him over and again. Here the doctor allegorically represents a person with extreme fundamentalism, a person who trusts in books more than requirements of a human being as if a person with an extreme nationalist approach imagines himself a sole instructor of national issues badly driven by individualized perceptions. Madhav adopts Amal the orphan child, and this adoption is much akin to a person’s actions to adopt a religion, takes care of it and controls it in the name of safeguarding it and follows distorted forms of its scriptures. Both the caregivers the doctor and the patron insist not to let Amal go out for any human or natural interaction. This insistence, in a governing or imperative mode, represents a patronizing restriction. And this patronizing restriction is dangerous for one’s liberty and one’s freedom. Amal longs for that freedom. Allegorically religion, nationalism, or patriotism under over-insistence dies as Amal dies under intense care. Hence the play seems simple written in children’s language but its message embedded in a language is encoded only when we understand the neurological condition of the author. Interaction with the environment, society, people, animals, and other natural forces makes an individual humble to accept the divine in Nature and receive power to accept. The play remained influential work of art for leading philosophic writers and activists of the world, because of its embedded message i.e. longing for freedom from extremist minds to receive power to accept. R.N. Tagore often quotes Upanishadic and Vedic resources of knowledge and believes in the divine power of Nature as Vedic verses exhibit human sublimation and its connection with Natural agencies. Nature is throughout divine. Everything that is impressive by its sublimity attributes to Nature. Nature is supposedly capable of affecting us. Mountains, rivers, springs, trees, plants are invoked as so many high powers (A. Barth). Amal is stopped going out for natural interaction represents a soul restricted to interact with divine nature, which is a root cause of its living existence, and is left isolated to wait for death. As a matter of fact, communication and interaction make you alive, you are dead otherwise. The caretakers of Amal were, in fact, suffering from “cosmicism syndrome” in Lovecraft’s terms: one’s state of mind when one denies all factors of humanity, peace, religious existence and stuck with an owned frenzied mind set, sense of insecurity, to control a situation leading to disastrous result. Lovecraft’s short story “The Outsiders” represents this aspect of human mind—a result of extreme care about oneself and related to one. I believe the factors behind the great wars in the world are the results of “cosmicized” mind. Sir R.N. Tagore’s definition of nation extrapolates the existence of cosmicized mind: “Have you not seen, since the commencement of the existence of the Nation, that the dread of it has been the one goblin-dread with which the whole world has been trembling?” (Tagore, 1917, p. 41). He adds, “Wherever there is a dark corner, there is the suspicion of its secret malevolence; and people

10.6 “Nation is the greatest evil for the Nation”?

live in a perpetual distrust of its back where it has no eyes” (Tagore, 1917, p. 41). He continues: “Every sound of footstep, every rustle of movement in the neighborhood, sends a thrill of terror all around. And this terror is the parent of all that is base in man’s nature. It makes one almost openly unashamed of inhumanity. Clever lies become matters of self-congratulation” (1917, p. 42). The Tagorian sense of “It makes one almost openly unashamed of inhumanity” is mirrored or extended further in Korczak’s Ghetto Diary for the same reason when he writes, “Wicked, shameful years—destructive, base. Pre-war years, lying, hypocritical. Cursed years” (Ghetto Diary: Part Two); and very much similar to his say, “Everything else has its limits, only brazen shamelessness is limitless” (Ghetto: July 22, 1942).

10.6 “Nation is the greatest evil for the Nation”? Tagore defines nation, its nature and causes of contamination: “The nation, with all its paraphernalia of power and prosperity, its flags and pious hymns, its blasphemous prayers in the churches, and the literary mock thunders of its patriotic bragging, cannot hide the fact that the Nation is the greatest evil for the Nation, that all its precautions are against it, and any new birth of its fellow in the world is always followed in its mind by the dread of a new peril” (Tagore, 1917, pp. 41 42, my italics). He goes on, “Its one wish is to trade on the feebleness of the rest of the world, like some insects that are bred in the paralyzed flesh of victims kept just enough alive to make them toothsome and nutritious” (Tagore, 1917, pp. 41 42). And therefore he concludes, “It is ready to send its poisonous fluid into the vitals of the other living peoples, who, not being nations, are harmless” (Tagore, 1917, pp. 41 42). Hence, if Amal represents a nation, and Madhav an extreme patron, and physician a factor of monitoring and controlling both Madhav and Amal, then the consequence will always be disastrous for a nation and other human institutions as Fig. 10.3 depicts. The extreme sense of care in someone is itself an evil for which one’s care is. In a similar way, let us understand R.N. Tagore’s statement in the context. He says, “Nation is the greatest evil for the nation,” which means that a nation that is founded on the principles of what I called “cosmicism syndrome,” a behavioral syndrome, a behavioral pattern, a reflective concurrence of traumatic whims to adopt and control the feeble, meek, and people with lower identity living outside or inside of a nation. A nation that fights for establishing the nation where people of a religion, a caste, a creed, or an ideology can only reside, for example, is the nation which itself is an evil for the nation and other nations. The founder obsessed with intense authoritativeness forgets to value the other’s existence, complementary or supplementary existence and owes a dread of “a new peril” to public. The perilous, Madhav represents this feature of authoritativeness. He is obsessed with authoritativeness of an extra care to an orphan, an adopted child

205

206

CHAPTER 10 Reading an extremist mind through literary language

Authority

Priesthood

Control the nation

Extremist mind of patron

Yes The death of a nation

No

The growth of nation

FIGURE 10.3 Representation of derived knowledge from The Post Office for finding impact of extremist mind for nationalism.

(represents a man performing extra care to religion or nation) that he forgets to ask Amal what his likes and dislikes are. Madhav could have spent times with Amal freely, could live with his freedom and choice. But, R.N. Tagore did not allow Madhav to be freed from the obsession in the play to represent the mind of an extremist. He did not let Madhav be a liberal to his child only for depicting the mind through literary language connoting extremists’ authoritativeness over nations’ problems of his time. In the first episode of talk with Amal, Madhav denies all Amal’s wishes to get fulfilled as he (Amal) expresses his will to go nearby the mountains he could visualize from his window. But, Amal seems a “crazy” child. To quote a few self-explicit narratives from The Post Office, Madhav forbids Amal from going out saying “Oh, you silly! As if there’s nothing more to be done but just get up to the top of that hill and away! . . . You don’t talk sense, my boy” (Post Office Act I). He further advises, “Now listen, since that hill stands there upright as a barrier, it means you can’t get beyond it. Else, what was the use in heaping up so many large stones to make such a big affair of it, eh!” (PO: Act 1, my italics). Amal comments on it wisely: “Uncle, do you think it is meant to prevent your crossing over? Its seems to me because the earth can’t speak it raises its hands into the sky and beckons. And those who live far and sit alone by their windows can see the signal” (my italics). He wants to contact environmental entities and says that he will “walk on, crossing so many streams, wading through water. Everybody will be asleep with their doors shut in

10.7 Amal as a religion under control

the heat of the day and I will tramp on and on seeking work far, very far” (Post Office: Act 1).

10.7 Amal as a religion under control In a letter written on October 5, 1895, he writes “The religion that only comes to us from external scriptures never becomes our own; our only tie with it is that of habit. To gain religion within is man’s great lifelong adventure” (Glimpses) [14]. The play depicts the birth and parentage of a religion or a soul or a nation and its adoption symbolically. In the guise of Amal’s parental identity, the birth of religion is disclosed herein meticulously in the following dialog between Gaffer and Madhav: Gaffer: Ah, well, and where did you pick him up? Madhav: He is the son of a man who was a brother to my wife by village ties. He has had no mother since infancy; and now the other day he lost his father as well (PO: Act I). How is a religion adopted? Who is her creator? How is she treated in human society? And how an adopting person is worried to save religion from her sickness and in doing so he captivates and kills her. Analogically, how is a nation created? Who is its creator? How is it treated in human society? And how authorities are worried to save a nation from its death and in doing so they captivate and kill her. The religion cries to liberate herself and contact humanity, but she is shackled with chained and leg cuffs. Sir R.N. Tagore’s The Post Office metaphorically, allegorically, and archetypically represents the most serious issue noted in the history of war, peace, and liberty: a nation in a “dark corner,” a nation with a “secret malevolence,” a “perpetual distrust,” a nation under a threat to take shelter in the dead is death of the nation itself (Tagore, 1917, p. 70). The above illustrations and arguments justify the hypothesis true. The play establishes two major ideas: first, Madhav’s extreme care of Amal has much affinity with a nation who “sedulously cultivates moral blindness as the cult of patriotism” and “ends her existence in a sudden and violent death.” Similarly, Madhav’s extreme care of Amal has much affinity with a culture being saved from its inflammation and contamination, forgetting values of external forces. Extremism starts with personal egoism and egotistical whims and frenzies and takes rest with mass loses [15]. Its basic factor lies with intense care to someone or something. Tagore depicts the very emotion as his mirror neuron imitates the incidents. Second, his words are draperies of neurological experiences activated by mirror neurons. It justifies that words are datasets of witnessed chains of events/incidents by their users. Literature or a language is a set of observed system of quantifiable experiences. Furthermore, knowledge can be extracted from literary language and predicts the influential factor for identification of extremist through experimental practices

207

208

CHAPTER 10 Reading an extremist mind through literary language

in semantic and sentiment analysis, semantic network, neuroartificial intelligence. If the etiology of cell mirror neurons is deeply studied, extremist sentiments instigated by social evils like orthodox, fundamentalism, racial egoism may be controlled. The mirror neurons and their causing cells activating other neurons causing creative impulses and cells of the author are responsible factors after such a work of art. An intensive study of the “mirror neurons” is yet to be done in terms of literary and language studies.

References [1] R.K. Pandit, V. Rahaman, Critical Pedagogy in Digital Era: Understanding the Importance of Arts & Humanities for Sustainable IT Development. Proceedings of International Conference on Digital Pedagogies (ICDP). May 12, 2019, 2019. Available at , https://ssrn.com/abstract 5 3387020 . or , https://doi.org/10.2139/ ssrn.3387020 . . [2] G. Hickok, The Myth of Mirror Neurons: The Real Neuroscience of Communication and Cognition, WW Norton, London, 2014. [3] L.M. Oberman, J.A. Pineda, V.S. Ramachandran, The Human Mirror Neuron System: A Link Between Action Observation and Social Skills, Social Cognitive and Affective Neuroscience 2 (1) (2006) 62 66. Available from: https://doi.org/10.1093/ scan/nsl022. [4] G. Hickok, Eight Problems for the Mirror Neuron Theory of Action Understanding in Monkeys and Humans, 2009. Available from https://www.ncbi.nlm.nih.gov/pmc/ articles/PMC2773693/ [5] G. Pellegrino, L. Fadiga, L. Fogassi, V. Gallese, G. Rizzolatti, Understanding Motor Events: a Neurophysiological Study, Experimental Brain Research 91 (1) (1992) 176 180. Available from: https://doi.org/10.1007/bf00230027. [6] V. Gallese, L. Fadiga, L. Fogassi, G. Rizzolatti, Action Recognition in the Premotor Cortex, Brain 119 (1996) 593 609. [7] N.S. IyerMusings on Indian Writing in English: Drama2007Sarup & SonsDelhi [8] K. Dutta, A. Robinson (Eds.), Rabindranath Tagore: An Anthology, Macmillan, 1998. [9] W.B. Yeats, Prefaces and Introductions: Uncollected Prefaces and Introductions by Yeats to Works by Other Authors and to Anthologies, Simon & Schuster, 1989. [10] J. Korczak, Ghetto Diary (Trans. Bachrach, J.). Betty Jean Lifton (Intro.), Yale University Press, New Haven, 2003. Available at , www.arvindguptatoys.com/arvindgupta/korczak/ghettodiary.pdf . Also available at , https://ia800401.us.archive.org/2/ items/GhettoDiary-English-JanuszKorczak/ghettodiary.pdf . Downloaded: 01.08.2019. [11] Valiur Rahaman, Introducing Digital Humanities, Yking Books. Print, India, Jaipur, 2016. [12] R.N. Tagore, Nationalism, Norwood Press, Norwood MA, 1917. Available at , http://www.archive.org/details/nationalism00tagorich . . Downloaded: 01.08.2019. [13] R.N. Tagore, The Post Office, Macmillan Company, New York, 1912. Available at , http://www.archive.org/details/postoffice00tago . . Downloaded: 01.08.2019.

Recommended Reading

[14] Tagore, R.N. Glimpses of Bengal Selected From the Letters of Sir Rabindranath Tagore 1885 1895. Available at Project Gutenberg. Downloaded: 01.08.2019. [15] L. Stankov, D. Higgins, G. Saucier, G. Kneˇzevi´c, Contemporary Militant Extremism: a Linguistic Approach to Scale Development, Psychol Assess. 22 (2) (2010) 246 258. Available from: https://doi.org/10.1037/a0017372.

Further Reading G. Hickok, Eight Problems for the Mirror Neuron Theory of Action Understanding in Monkeys and Humans, Journal of Cognitive Neuroscience 21 (7) (2009) 1229 1243. Available from: https://doi.org/10.1162/jocn.2009.21189. J. Korczak, Ghetto Diary (Trans. Bachrach, J.). Betty Jean Lifton (Intro.), Yale University Press, New Haven, 2003. Available at , www.arvindguptatoys.com/arvind-gupta/korczak/ghettodiary.pdf . Also available at , https://ia800401.us.archive.org/2/items/ GhettoDiary-English-JanuszKorczak/ghettodiary.pdf . Downloaded: 01.08.2019. L.M. Oberman, J.A. Pineda, V.S. Ramachandran, The Human Mirror Neuron System: A Link Between Action Observation and Social Skills, Social Cognitive and Affective Neuroscience 2 (1) (2006) 62 66. Available from: https://doi.org/10.1093/scan/nsl022. G. Pellegrino, L. Fadiga, L. Fogassi, V. Gallese, G. Rizzolatti, Understanding Motor Events: a Neurophysiological Study, Experimental Brain Research 91 (1) (1992) 176 180. Available from: https://doi.org/10.1007/bf00230027. Ramachandran, V.S. (2000). Mirror Neurons and Imitation Learning as the Driving Force Behind ‘‘the great leap forward’’ in Human Evolution. Available at http://williamlspencer.com/mirrorneurons.pdf G. Rizzolatti, L. Craighero, The Mirror-Neuron System, Annual Review of Neuroscience 27 (1) (2004) 169 192. Available from: https://doi.org/10.1146/annurev.neuro.27. 070203.144230. G. Rizzolatti, M.A. Arbib, Language Within Our Grasp, Trends in Neurosciences 21 (5) (1998) 188 194. Available from: https://doi.org/10.1016/s0166-2236(98)01260-0. R.N. Tagore, Nationalism, Norwood Press, Norwood MA, 1917. Available at , http:// www.archive.org/details/nationalism00tagorich . . Downloaded: 01.08.2019. Tagore, R.N. Glimpses of Bengal Selected From the Letters of Sir Rabindranath Tagore 1885 1895. Available at Project Gutenberg. Downloaded: 01.08.2019. R.N. Tagore, The Post Office, Macmillan Company, New York, 1912. Available at , http://www.archive.org/details/postoffice00tago . . Downloaded: 01.08.2019. V. Gallese, M.A. Gernsbacher, C. Heyes, G. Hickok, M. Iacoboni, Mirror Neuron Forum, Perspectives on Psychological Science 6 (4) (2011) 369 407. Available from: https:// doi.org/10.1177/1745691611413392.

Recommended Reading A. Bhattacharya, M. Renganathan, The Politics and Reception of Rabindranath Tagore’s Drama: The Bard on the Stage, Routledge, NY, 2015. Goldberg Benjamin, The Mirror and Man, University Press of Virginia, Charlottesville, 1985.

209

210

CHAPTER 10 Reading an extremist mind through literary language

Amy Cook, Shakespearean Neuroplay: Reinvigorating the Study of Dramatic Texts and Performance through Cognitive Science., Palgrave Macmillan, NY, 2010. Jacques Derrida, Writing and Difference, OUP, USA, 2004. Merlin Donald, Origins of the Modern Mind: Three Stages in the Evolution of Culture and Cognition, Harvard University Press, 1991. Alvin Goldman, Simulating Minds: The Philosophy, Psychology, and Neuroscience of Mindreading, Oxford University Press, Oxford, 2006. Stephen Halliwell, The Aesthetics of Mimesis: Ancient Texts and Modern Problems, Princeton University Press, Princeton, NJ, 2002. Deborah Jenson, Marco Iacoboni, Literary Biomimesis: Mirror Neurons and the Ontological Priority of Representation, California Italian Studies. 2 (2011). 1 Available at , https://escholarship.org/uc/item/3sc3j6dj . . S.T. Joshi., in: H.P. Lovecraft (Ed.), The Annotated, Dell Publishing Group, NY, 1997. T. Peter, Special Issue on Hermeneutics and Discourse Analysis, Discourse Studies (2011) 601 608. Valiur Rahaman, Introducing Digital Humanities, Yking Books. Print, India, Jaipur, 2016. V.S. Ramachandran, Sandra Blakeslee, Phantoms in the Brain: Probing the Mysteries of the Human Mind, William Morrow and Company, Inc, New York, 1998. B.O. States, Performance as Metaphor, Theatre Journal 48 (1) (1996). JHUP. Available at: , http://www.jstor.org/stable/3208711?origin 5 JSTOR-pdf . Accessed 24.8.13. L. Stankov, D. Higgins, G. Saucier, G. Kneˇzevi´c, Contemporary Militant Extremism: a Linguistic Approach to Scale Development, Psychol Assess. 22 (2) (2010) 246 258. Available from: https://doi.org/10.1037/a0017372. Jun. R.N. Tagore, Nationalism, Norwood Press, Norwood MA, 1917. Available at , http:// www.archive.org/details/nationalism00tagorich . . Downloaded: 01.08.2019. Tagore, R.N. (2005) Glimpses of Bengal Selected From the Letters of Sir Rabindranath Tagore 1885 1895. Available at Project Gutenberg. Downloaded: 01.08.2019. R.N. Tagore, The Post Office, Macmillan Company, New York, 1912. Available at , http://www.archive.org/details/postoffice00tago . . Downloaded: 01.08.2019. W.B. Yeats, Prefaces and Introductions: Uncollected Prefaces and Introductions by Yeats to Works by Other Authors and to Anthologies, Simon & Schuster, 1989.

CHAPTER

REAH: Resolution Engine for Anaphora in Hindi dialogue

11

Seema Mahato1 and Ani Thomas2 1

Dr. C.V. Raman University, Bilaspur, Chattisgarh, India Department of IT, Bhilai Institute of Technology, Durg, Chattisgarh, India

2

11.1 Introduction Proficient speakers avoid repetition of words or phrase in dialogue and therefore grammatically substitute it with another expression such as pronoun to denote the proceeding word or phrase that can convey and carry the same meaning after substitution. The substituted expression is known as anaphora and the word (or phrase) that it refers is known as antecedent or referent. Thus an anaphora is a pro-form which is used to mention an entity that appear prior to it, and the process that deals with the identification of such entities is known as anaphora resolution (AR). The referent may reside in local domain or may have long-distance connectivity with anaphor. The significance of AR can be understood by the fact that improper resolution of anaphora restricts and hinders the performance of many natural languages processing (NLP) application such as machine translation, question answering system, and essay grading. So many techniques of cognitive science have been applied to perform text analysis at varied linguistic stages in NLP application. Resolving anaphora needs significant amount of linguistic knowledge and excessive time to develop rules that can be automate utilizing machine learning techniques. For example, in the sentence “eksgu us jke ls mls iqLrd nsus dks dgk,” “mls” is referring to preceding noun phrase (NP) “eksgu.” So “mls” is anaphor and the NP “eksgu” is antecedent (referent) of “mls,” which avoid unnecessary restatement of NP. The investigation for AR by the authors is based on the acquisition of lexical knowledge provided by Hindi shallow parser at sentence level and chosen a rule-based heuristic approach to develop Resolution Engine for Anaphora in Hindi dialogue (REAH). The algorithms designed by undertaking the role of subject, object, and verb and their lexical relation that have been employed in the engine.

Cognitive Informatics, Computer Modelling, and Cognitive Science, Volume 2. DOI: https://doi.org/10.1016/B978-0-12-819445-4.00011-4 © 2020 Elsevier Inc. All rights reserved.

211

212

CHAPTER 11 REAH: Resolution Engine for Anaphora in Hindi dialogue

11.1.1 Categorization of Hindi anaphora Anaphora in Hindi can be categorized in two broad levels: sentence level and word level. The locality of anaphor and the referential link may be same or distal. When the location of anaphor and the antecedent collide, it indicates intrasentential anaphor. The antecedent of an anaphor may reside beyond its locality, and then it is referred as intersentential anaphor. Take the following sentence (s1): (s1) jke us [kqn ds fy, pk; cukbZ. As the referent of the anaphor in (s1) is located in the same sentence, the anaphor is an intrasentential. Consider sentence (s2), in which the antecedent and the anaphor are located in different sentences; therefore it is an intersentential anaphor.

(s2) caVh ds ikl dyj isfUlYl gS- ccyh dks og pkfg,Anaphor may refer to a word that may be a noun, adjective, adverb, verb, or other pronoun also. A pronominal anaphor refers to a noun antecedent, which occur earlier in text as in sentences (s1) and (s2), whereas a nonpronominal anaphor has antecedent other than noun or may have multiple or indefinite referent. Consider sentence (s3) in which the pronoun “;g” refers to the incident “ck< yk ldrh gS” mentioned in the earlier sentence. (s3) lqukeh leqæh fdukjksa ij rhozrk ls ck< yk ldrh g S; g tugkfu dk dkj.k cu ldrk gS.

11.1.2 Boundaries in anaphora resolution AR task is cumbersome as well as laborious task. Though research on Hindi AR have been in growth from last decade, but few boundaries still have to cross to overcome the restrictions faced.

11.1.2.1 Nonavailability of freeware Hindi discourse The availability of free Hindi corpora for evaluation in compare to foreign languages is minimal and available on trial or payment basis. Due to which researchers have to bind their experimental evaluation to the corpora that are subjected to accessibility terms. Other issue is with the nonavailability of corpora related to financial, agriculture, biomedical, etc. Therefore researchers have to develop synthetic datasets or corpus.

11.1.2.2 Efficiency of linguistic preprocessor AR requires qualitative processing of discourse and corpora prior to implementing the resolution algorithms that need linguistic preprocessors such as clausal splitter, sentential splitter, and linguistic analyzer at word, utterance or sentence level. The performance of tools benefits the resolution process by providing the relevant and significant data from discourse, which helps to improve overall result.

11.2 The state-of-the-art

11.1.2.3 No benchmark for POS tagging Hindi annotated corpus from different resources did not have similar Part-ofSpeech (POS) tag sets as no standard benchmark has been defined by NLP association. Hence, PP/PSP/CM is used to tag postpositions, PRP/PN to tag pronoun, PU/PUNC/SYM to tag punctuation, etc. Due to the absence of such standards, investigators have to face computational complexity in implementing and evaluating algorithms for different linguistic analyzers and therefore have to modify the algorithms to get fit into it which require unnecessary time for modifications.

11.1.2.4 Lack of efficient named entity recognizer Identifying an entity in text as person, location, organization, etc. needs a named entity recognizer (NER) tagger that helps to improve the AR. The available Hindi NER does not able to recognize correctly the foreign words, designation, rivers, products, etc. in text and so erroneous classifications by NER leads to degradation in performance of AR system. Moreover, the semantic feature to differentiate an entity as animate or inanimate also improves the resolution, but an efficient one for Hindi language is still lacking.

11.2 The state-of-the-art The state-of-the-art highlights the investigation done by different researchers on Hindi AR. It has been observed that in last decade there was a huge research gap in this field and researchers have been shifted from knowledge-rich or knowledge-poor to hybrid approach, and instead relying on single technology, different approaches have been merged with syntax-based or machine learning to develop a hybrid system. Ref. [1] presented a knowledge-rich approach by implementing centering theory and resolved zero (null) pronouns and third-person pronouns (TPP) in Hindi. Ref. [2] put forward a knowledge poor system for resolving pronouns and gaps in Hindi as well as in Tamil and Bengali. Ref. [3] proposed heuristic approach to resolve Hindi pronominal anaphors acquiring syntactic and semantic data obtained from head-driven phrase structure grammar. Ref. [4] resolved Hindi anaphora incorporating machine learning and semantic knowledge. Ref. [5] developed rule-based approach by modifying Hobb’s theory to resolve pronouns in Hindi. Ref. [6] put forward discourse-based approach to resolve Hindi pronominal anaphora. Ref. [7] resolved direct and indirect anaphora available in Hindi corpus “Emille.” Ref. [8] investigated mention for anaphors in Bengali, Hindi, and Tamil text by integrating data driven and statistical approach with the help of semantic and syntactic data. Ref. [9] handled pronominal anaphora employing animistic knowledge. Ref. [10] put forward a syntax-based machine learning system to find equivalence class of abstract and concrete anaphors employing soft and hard constraints. Ref. [11] proposed a system integrating Lappin and Leass and centering

213

214

CHAPTER 11 REAH: Resolution Engine for Anaphora in Hindi dialogue

algorithm and animistic knowledge. Ref. [12] conducted experiment using Gazetteer method and claimed that recency and animistic factor improved the accuracy to 83% to overall system. Ref. [13] resolved event anaphors using Hindi dependency Treebank by considering the features such as participant or arguments of verbs, dependencies among event anaphor and verb, distance and direction of pronoun to adjacent verb, and the presence of final verb. Ref. [14] focus and discuss the effect of AR on identification of opinion target.

11.2.1 Background of the authors Ref. [15] presented a syntax-based system for resolving personal pronouns and manually evaluated 86 simple sentences. The behavior of postpositions was observed in this experiment and the attribute features of each lexicon in annotated corpus have been studied. The authors developed and employed heuristic rules crafted from shallow features for resolving pronominal anaphors and the preliminary evaluation correctly resolved all the anaphors. Ref. [16] presented computational approach to resolve pronominal anaphora and evaluated 192 anaphors available in a synthetic corpus generated from different domains. The algorithms successfully resolved 145 pronouns by extending the search scope to previous 3 sentences. Ref. [17] highlights factors and constraints helpful in finding the equivalence class of anaphors in different genre of text that could definitely help to enhance the outcome of AR system. Ref. [18] discusses and compares the approach, constraints, strategies, and evaluation metrics utilized in the AR toolkits available for different foreign languages. Mahato and Thomas [19] presented approach in CEUR Workshop for automated essay grading system in Hindi-based on lexical and semantic data to ease assessment in e-learning. The proposed approach had suggested integrating Hindi WordNet to prevail over the issues associated with it.

11.3 The resolution engine The authors proposed a REAH to resolve anaphors in Hindi, which comprise of several unique modules that are integrated efficiently into two units: preprocessing and AR unit. This section discusses the overall functionality of REAH in terms of outcomes gained in pre- and postprocessing tasks carried out at different level of implementation in both units for identifying the equivalence class of Hindi anaphora. It also explains the rules or algorithms defined in individual unit with illustrations to achieve the substantial result.

11.3.1 The preprocessing phase The flow of information in preprocessing phase has been depicted in Fig. 11.1, which includes data annotation, elimination of irrelevant information,

11.3 The resolution engine

Sentences Words with POS tag and shallow features

Shallow parser

Relevant chunks

Anaphora list

List of NPs Term patterns set List of NPs matched with term patterns Filtration rules List of potential candidates

Anaphora resolution phase

FIGURE 11.1 Preprocessing phase.

identification of anaphors and NP, removal of irrelevant NPs, and selection of potential candidate. Once the list of potential candidates and anaphors prepared by the phase, it is forwarded to anaphora resolver to take final conclusion on antecedent.

11.3.1.1 Data annotation The authors have utilized Hindi shallow parser developed by Language Technology Research Centre to annotate the discourse, which tag each lexicon with part of speech as per defined in Hindi dependency tag sets ([20], pp. 1
38) and generate a Treebank that holds lexical knowledge represented in Shakti Standard Format (SSF). This Hindi lexical analyzer breaks a sentence into chunks to tokenize each idiom as noun, pronoun, verb, etc. ([10], p. 978). The lexical knowledge provided by the parser constitutes syntactic features such as gender, number, person, and case. Table 11.1 represents the output generated by Hindi shallow parser [21] for sentence “eksgu u s,d lsc [kk;k.” The alphabet notations in attribute features (af) in Table. 11.1 depicted in Fig. 11.2.

215

216

CHAPTER 11 REAH: Resolution Engine for Anaphora in Hindi dialogue

Table 11.1 Output generated by shallow parser.

1 mohana 2 ne 3 eka 4 seba 5 KAyA 6 .

unk unk unk unk unk unk

| ||

< fs af ‘mohana,

n, m,

sg, 3, d,

ne′ >

psp=Case marker The word Noun Masculine

Singular

Direct case

Third person

FIGURE 11.2 Notations in Hindi shallow parser.

11.3.1.2 Defining the term patterns Hindi has different grammatical case marker (CM) attached as suffix that represents specific case forms and follows the arguments of verb. The significance of each CM associated with grammatical case has been explained in Table 11.2, which has been considered as the key to resolve Hindi anaphors. The authors have defined few term patterns based upon the CMs mentioned in Table 11.2 by combining noun (N) and the CM in the form of [NP ((N, CM))]. The syntactic relation between N and CM makes easy to comprehend the role of each N in a sentence, which motivates the authors to define the term patterns.

11.3.1.3 Removal of irrelevant chunks and nonanaphoric Few tokens such as CCP, RBP, JJP, and BLK in the information obtained by shallow analyzer have been found irrelevant in resolution and hence have been eliminated. Anaphoric pronouns in Treebank are tagged as “PRP” chunks and specified by “pn” within attribute features. In few instances, it holds the attribute of adverb or adjective together such as “adv” or “adj” with “PRP” too. Therefore anaphors tagged as “PRP” and containing “adv” or “adj” attribute should be considered as nonanaphoric and eliminated from resolution. The elimination should be ignored for reflexive pronouns.

11.3 The resolution engine

Table 11.2 Role of case markers. Case marker(s)

Case

Role

Examples

Ø (null) or “uss” “dks” “ls’/ds}kjk” “ds fy;s/dks” “lss” “es/ij”

Karta Karma Karana Sampradaana Apaadaana Adhikarana

Subject Object Agent Recipient Source Spatial

lhrk [khj [kkrh gS Jke us cktkj es jfo dks ns[kkjke us pkdw ls lsc dkVk jke us eksgu dks [khj nh jke us pEep ls dVksjh ls [khj [kkbZ gjh us LorU=k laxzke es fgLlk fy;k Fkk-

11.3.1.4 Identification of intermediate clause Verb main (VM), VGF, or VGNF tag has been used by shallow parser to POS tag verb phrases. The VM chunk resides within VGF or VGNF chunk that have considered as baseline for defining different intermediate clause boundaries in a sentence. Clause boundaries improve identification of role played by noun or pronoun as subject and object. An independent VM chunk is ignored in determining intermediate boundary and has been removed.

11.3.1.5 Extraction of relevant noun phrases The NPs in the Treebank that are relevant have been dig out implementing multiple filtration rules defined on the basis of term patterns observing the frequency of cooccurrence of specific CM with subject or object. The filtration rules reduce the dimension of the information that have to be sent to next phase. Few of filtration rules have been discussed: Rule 1: NP chunks in the Treebank that include “pn,” “avy,” “adv,” “adj,” or “v” in its attribute features are considered irrelevant and have to be eliminated. Rule 2: NP chunks that include tag such as WQ, INTF, QC, and QF were considered as irrelevant and have to be eliminated. Rule 3: NP chunks that include JJ token having “adv” attribute feature have to be eliminated as it is irrelevant. Rule 4: NP chunk with null CM whose child NN or NNP chunk have exactly same attribute features and which are not an argument of final verb, have to be eliminated. Rule 5: When an NP has unassigned grammatical number as “unk” by the parser, then the number attribute of the nearest and succeeding VM token has been assigned to it. Rule 6: If an NP that has been eliminated by any rule abovementioned, then further reoccurrence of equivalent NP should also have to be eliminated. The Treebank generated by the shallow parser for the sentence (s4) has been captured and converted into proper format to make readable easily that has been shown in Table 11.3. (s4) “feM&Ms es aNih [kcj ds eqrkfcd yhuk us dgk fd ^^e'kgwj odhy jke tsBeykuh us eq>ls iwN dj eq>s xys yxk;k Fkk.”

217

Table 11.3 Corpus generated by shallow parser. Offset

Chunk

POS

Attribute features

1 1.1 1.2 1.3 2

((

NP XC SYM NN NP

2.1 2.2 3

Nih

3.1 3.2 4 4.1 5 5.1 6 6.1 6.2 6.3 6.4 7

eqrkfcd

7.1 8 8.1 9

tsBeykuh

9.1 10 10.1 11 11.1 12

iwN

((

VM NP PRP NP NN VGF

12.1

yxk;k

VM

,fsaf 5 ‘Ms,n,m,pl,3,d,0_es,a 0’ vpos 5 “vib3_4” head 5 “Ms” . ,fsaf 5 ‘feM,unk,,,,,,’ poslcat 5 “NM” . ,fsaf 5 ‘-,punc,,,,,,’ . ,fsaf 5 ‘Ms,n,m,pl,3,d,0,0’ name 5 “Ms” . ,fsaf 5 ‘[kcj,n,f,sg,3,d,0_dk,0’ vpos 5 “vib2_3” head 5 “[kcj” . ,fsaf 5 ‘Nih,adj,f,any,,any,,’ . ,fsaf 5 ‘[kcj,n,f,sg,3,d,0,0’ name 5 “[kcj” . ,fsaf 5 ‘yhuk,unk,,,,,0_us,’ vpos 5 “vib2_3” head 5 “yhuk” poslcat 5 “NM” . ,fsaf 5 ‘eqrkfcd,adj,any,any,,any,,’ . ,fsaf 5 ‘yhuk,unk,,,,,,’ name 5 “yhuk” poslcat 5 “NM” . ,fsaf 5 ‘dg,v,m,sg,any,,या,yA’ head 5 “dgk” . ,fsaf 5 ‘dg,v,m,sg,any,,या,yA’ name 5 “dgk” . ,fsaf 5 ‘fd,avy,,,,,,’ head 5 “fd” . ,fsaf 5 ‘fd,avy,,,,,,’ name 5 “fd” . ,fsaf 5 ‘jke,n,m,sg,3,d,0,0’ head 5 “jke “ . ,fsaf 5 “,punc,,,,,,’ . ,fsaf 5 ‘e'kgwj,adj,any,any,,any,,’ . ,fsaf 5 ‘odhy,n,m,sg,3,d,0,0’ poslcat 5 “NM” . ,fsaf 5 ‘jke,n,m,sg,3,d,0,0’ head 5 “jke “ . ,fsaf 5 ‘tsBeykuh,unk,,,,,0_us,’ vpos 5 “vib1_2” head 5 “tsBeykuh “poslcat 5 “NM” . ,fsaf 5 ‘tsBeykuh,unk,,,,,,’ name 5 “tsBeykuh “poslcat 5 “NM” . ,fsaf 5 ‘eS,pn,any,sg,1,o,ls,se’ head 5 “eqq>ls” . ,fsaf 5 ‘eS,pn,any,sg,1,o,ls,se’ name 5 “eqq>ls” . ,fsaf 5 ‘iwN,v,any,sg,3,,0_dj,0’ vpos 5 “tam1_2” head 5 “iwN “. ,fsaf 5 ‘iwN,v,any,any,any,,0,0’ name 5 “iwN “ . ,fsaf 5 ‘eS,pn,any,sg,1,o,dks,ko’ head 5 “eq>s “ . ,fsaf 5 ‘eS,pn,any,sg,1,o,dks,ko’ name 5 “eq>s “ . ,fsaf 5 ‘xyk,n,m,pl,3,d,0,0’ head 5 “xys” . ,fsaf 5 ‘xyk,n,m,pl,3,d,0,0’ name 5 “xys” . ,fsaf 5 ‘yxk,v,m,sg,any,,या_था,yA’ vpos 5 “tam1_2” head 5 “yxk;k” . ,fsaf 5 ‘yxk,v,m,sg,any,,या,yA’ name 5 “yxk;k” .

feM


Ms

((

[kcj

((

yhuk

(( dgk

(( fd

(( “ e'kgwj odhy jke

((

(( eq>ls

((

(( eq>s

(( xys

JJ NN NP JJ NN VGF VM CCP CC NP SYM JJ XC NNP NP NN NP PRP VGNF

11.3 The resolution engine

Table 11.4 Relevant features extracted from corpus. Offset

Chunk

POS

Attribute features

2

((

NP

3

((

NP

4.1 6 7

dgk

(( ((

VM NP NP

,fsaf 5 ‘[kcj,n,f,sg,3,d,0_dk,0’ vpos 5 “vib2_3” head 5 “[kcj” . ,fsaf 5 ‘yhuk,unk,,,,,0_us,’ vpos 5 “vib2_3” head 5 “yhuk” poslcat 5 “NM” . ,fsaf 5 ‘dg,v,m,sg,any,,या,yA’ name 5 “dgk” . ,fsaf 5 ‘jke,n,m,sg,3,d,0,0’ head 5 “jke “ . ,fsaf 5 ‘tsBeykuh,unk,,,,,0_us,’ vpos 5 “vib1_2” head 5 “tsBeykuh “poslcat 5 “NM” . ,fsaf 5 ‘eS,pn,any,sg,1,o,ls,se’ name 5 “eq . ls” . ,fsaf 5 ‘eS,pn,any,sg,1,o,dks,ko’ name 5 “eq . s “ . ,fsaf 5 ‘yxk,v,m,sg,any,,या,yA’ name 5 “yxk;k” .

8.1 10.1 12.1

PRP PRP VM

eq>ls eq>s yxk;k

Table 11.5 Filtration table generated using relevant features. Sentence ID

Clause ID

NPs within clause

Associated case marker

Number of NP

Behavior of neighboring VM

S1

C1

[kcj,n,f,

dk

sg

, fs , fsaf 5 ‘dg,v,m, sg,any,, ;k,yA’ name 5 “dgk” .

us

sg

NULL

sg

us

sg

S1

C1

S1

C2

S1

C2

sg,3, d,0_dk, ‘yhuk, unk,,,,,0_us, ‘jke,n,m, sg,3,d,0,0’ ‘tsBeykuh, unk,,,,,0_us

, fsaf 5 ‘yxk,v,m,sg, any,, ;k,yA’ name 5 “yxk;k” .

On implementing the filtration rules on the corpus shown in Table 11.3, all the irrelevant information and NPs that were not advantageous further get eliminated and the corpus left with relevant NPs that are considered as the prospective candidates for antecedent. The filtration rules are very effective in pruning the corpus and bring out the most relevant NPs. The relevant features obtained after employing filtration rules have been shown in Table 11.4. In Table 11.4 the NP chunk with offset 3 contains “unk” that indicates unknown features, which have to be determined. So as per rule 5 mentioned earlier, this chunk should be assigned the grammatical number of the nearest VM token and hence updated with singular “sg” number. The leftover corpus has to be converted into a filtration table to generate possible pairs of anaphor and antecedent and to check the probability of coreferential chain further. Table 11.5 shows the filtration table generated from the features in Table 11.4.

219

220

CHAPTER 11 REAH: Resolution Engine for Anaphora in Hindi dialogue

Table 11.6 List of anaphors to resolve. Sentence ID

Clause ID

Anaphor within clause

Category of anaphor

Number of anaphor

S1 S1

C2 C2

eq>ls

FPP FPP

sg sg

eq>s

Table 11.7 Anaphor and antecedent possible pairs. Anaphor 5.Potential antecedent pair for eq>ls

Anaphor 5.Potential antecedent pair for eq>s

eq>ls 5. [kcj_dk

eq>s 5. [kcj_dk

eq>ls 5. yhuk_us eq>ls 5. jke

eq>s 5. yhuk_us eq>s 5. jke

eq>ls 5. tsBeykuh_us

eq>s 5. tsBeykuh_us

From Table 11.3 a list of anaphors to be resolved also has been prepared with their respective pronoun category and clause identification that have been shown in Table 11.6. The filtration table helps in determining the possible antecedent of an anaphor and pairing them. Utilizing Tables 11.5 and 11.6, the possible pairs have been generated. As all the four NPs in the filtration table precede the anaphors to be resolved, each anaphor has four instances of possible antecedent, which have been shown in Table 11.7. To improve the accuracy a couple of distal factors have been computed, which will prune and finalize the abovementioned pairs before forwarding list to AR phase.

11.3.1.6 Distance factors Two distance factors, DF1 and DF2, have been used to prune the anaphor and antecedent possible pairs and to find the coreference chain, respectively. The factors compute the distance between the entities in terms of number of phrases. The token_id (offset number) generated by the parser for each token has been taken into account for the calculations. Therefore DF1 and DF2 are calculated as, DF1(anaphor, candidate antecedent) 5 (difference between token id of anaphor and possible antecedent) 2 1 and DF2(preceding anaphor, anaphor) 5 (difference between token id of anaphor and preceding anaphor) 2 1. DF1 count the number of phrases between anaphor and the preceding nearest NP that has been considered as possible antecedent. DF1 by pruning the (anaphor, antecedent) list by eliminating the insignificant pairs contribute in increasing the accuracy of resolution. Two rules have been defined to made decision on the computed value of DF1.

11.3 The resolution engine

Rule 1: when the participant anaphor belongs to personal pronouns and the computed value of DF1 is zero, then the candidate antecedent must be eliminated from its list of possible candidates. Rule 2: when the participant anaphor belongs to reflexive pronouns and the computed value of DF1 is zero and then the candidate antecedent should be considered as an actual antecedent. Consider the pairs list in Table 11.7, the DF1 between the entities for (eq>ls, tsBeykuh us) pair will be calculated as, DF1(eq>ls,tsBeykuhus) 5 [int(token_id(eq>ls)) 2 int(token_id(tsBeykuhus))] 2 1 5 [(8) 2 (7)] 2 1 5 0, The zero value indicates that even though the entity “tsBeykuh us” is in subject position could not be antecedent of the anaphor “eq>ls” and therefore removed from the potential candidate list. DF2 count the number of phrases between the couple of anaphors that falls in same grammatical root and same clause in a sentence. DF2 have been used to check the coreferential link between neighboring anaphors, which direct the engine to skip the task of resolving one of the coreferential anaphor. Thus the factor not only identifies corefer anaphors but also helps to reduce the load on engine and the overall processing time. Three rules have been drawn on DF2 when the computed value is less than or equal to one. Rule 1: When both the anaphors fall in same clause and belong to first- or second-person pronoun (FPP and SPP), then it is assumed that they are coreferential. “eq>ls” and “eq>s” in sentence (s4) belongs to FPP and the DF2 between them is calculated as, DF2(eq>,s eq>ls,) 5 [int(token_id(eq>s)) 2 int(token_id(eq>ls))] 2 1 5 [(10) 2 (8)] 2 1 5 1, which indicates “eq>ls” and “eq>s” are referring to same antecedent. Rule 2: When both the anaphors fall in same clause and belong to TPPs, then they are not coreferential. Rule 3: When one of the anaphor in a clause belongs to personal pronoun and other one is reflexive pronoun, then they are coreferential.

11.3.1.7 Identifying inanimate entity The authors have considered demonstrative (DEM) pronouns for marking inanimate entities in the Hindi dialogue. DEM pronouns, which do not have a NP antecedent, indicate the entities that have been referred by some other referring expression. DEM pronouns are POS tag with DEM. Consider the sentence (s5) below, (s5) bl isM ij dcwrjks adk >q.M foJke djrk Fkk“bl” in the phrase (bl isM ij) is DEM pronoun the NPs (isM ij) indicates an inanimate entity “isM.”

221

222

CHAPTER 11 REAH: Resolution Engine for Anaphora in Hindi dialogue

11.3.2 Anaphora resolution phase The resolution phase identified the actual antecedent for which individual modules, such as FPP for resolving anaphors belong to first-person pronouns, SPP for second-person pronouns, TPP for third-person pronouns, RFX for reflexive, LOC for locative, and DEM for demonstrative pronouns, have been developed that get invoke automatically. Separate rules have been defined for each module by observing how the NPs are framed within a sentence and the order of CMs (Fig. 11.3).

11.3.2.1 Constraints Few constraints such as priority of term pattern, syntactic position of noun and number agreement have been defined on the available features in the filtration table. These constraints are considered as a decision factors to identify the equivalence class for an anaphor. The constraints are discussed in the following paragraphs: Priority of term pattern: Utterance has multiple CMs, and on the basis of their order of appearance, the roles of NPs associated with them get changed accordingly. As previously discussed, the term patterns had been defined on the basis of CMs, it is necessary to set the priorities of term patterns. The priority order of term patterns defined for individual pronoun type differs. For example, to resolve

FPP resolver module Potential candidates list

SPP resolver module

Substitution module

Anaphora list TPP resolver module List from preprocessing phase

RFX resolver module

LOC resolver module

Resolution module

FIGURE 11.3 Resolution phase.

Mapping table

11.3 The resolution engine

FPPs, the priority has been defined as “uss” . “dk” . “ls” . null, and for SPPs, it is set as “ls” . “dks” . “dk” . null and so on. Syntactic position of noun: The syntactic position of noun for being head or subject of anaphor in a sentence has been efficiently added in the algorithms. The decision on a noun for being subject of the sentence is taken on the clause_id it belongs, CM attached to it, CM of succeeding nearest NP, and distance of noun from anaphor. Number agreement: Number agreement checks for the plurality commonness between candidate and the anaphor ([22], p. 4). On picking a candidate antecedent, the resolver checks the priority of term pattern attached to it, syntactic position, and finally decision to choose it as actual antecedent relying on number agreement.

11.3.2.2 Identifying the equivalence class On the basis of pronoun type, respective resolution algorithms have been defined to find the equivalence class. The algorithms explained later were employed on the updated features available in filtration table, provided by preprocessing phase to the resolution phase. The algorithms lookup for the actual antecedent within the scope of previous five sentences but to resolve reflexive or DEM, the scope has been limited to previous two sentences. A step-by-step explanation of the algorithms for resolving anaphors are as follows.

11.3.2.2.1 Algorithm for resolving first-person pronouns FPP includes “eSs” and its inflected form such as “eSus,” “eq>s,” and “eq>ls,” which refers to a subject that can be the speaker or agent of the action.

Algorithm: For i 5 (token_id(anaphor)
1) to 1 For j 5 sid(anaphor) to 1 Identification of NP with ‘us’ Check for head noun Check for number agreement Consider NP as antecedent Substitute anaphor with antecedent in mapping table Exit Else Identification of NP with ‘dk’ Do the steps (1.1-1.5) Else Identification of NP with ‘ls’ Do the steps (1.1-1.5) Else Identification of NP with ‘null’ Do the steps (1.1-1.5) End for End for

223

224

CHAPTER 11 REAH: Resolution Engine for Anaphora in Hindi dialogue

11.3.2.2.2 Algorithm for resolving second-person pronouns SPP includes “rqe” and its inflected form “rqeus,” “rq>,s ” and “rq>dks,” which refers to an object that can be a listener or recipient of the action taken by the subject.

Algorithm: For i 5 (token_id(anaphor)
1) to 1 For j 5 sid(anaphor) to 1 1. Identification of NP with ‘ls’ 1.1. Check for number agreement 1.2. Consider NP as antecedent. 1.3. Substitute anaphor with antecedent in mapping table 1.4. Exit 2. Else Identification of NP with ‘dks’ 2.1. Do the steps (1.1-1.4) 3. Else Identification of NP with ‘dk’ 3.1. Do the steps (1.1-1.4) 4. Else Identification of NP with ‘null’ 4.1. Do the steps (1.1-1.4) End for End for

11.3.2.2.3 Algorithm for resolving third-person pronouns TPP includes the root form “og,” and its inflected form “oks,” “mlus,” and “mUgksau”s and their antecedent may reside in same or another sentence and could be singular or plural.

Algorithm: For i 5 (token_id(anaphor)
1) to 1 For j 5 sid(anaphor) to 1 Identification of number of VM[j] If number(VM[j]) 5 5 ‘sg’ and anaphor 5 5 {‘og’, ‘oks’, ‘mlus’, ‘os’, ‘mUgksau’s , ‘mUgs’a } Identification of NP with ‘us’ Check for number agreement Consider NP as antecedent. Substitute the anaphor with antecedent in mapping table Exit Else Identification of NP with ‘dks’ Do the steps (2.1.1-2.1.4) Else If number(VM[j]) 5 5 ‘sg’ and anaphor 5 5 {‘mls’, ‘mldks’, ‘mlls’, ‘muls’, ‘mudh’, ‘mudk’, ‘mudks’, ‘mlds’, ‘mldk’, ‘mldh’} Identification of NP with ‘dks’ Do the steps (2.1.1-2.1.4) Identification of NP with ‘dk’ (Continued)

11.3 The resolution engine

Algorithm: (Continued) Do the steps (2.1.1-2.1.4) Identification of NP with ‘ds’ Do the steps (2.1.1-2.1.4) Identification of NP with ‘ls ‘ Do the steps (2.1.1-2.1.4) Else If number(VM[j]) 55 ‘pl’ and anaphor 5 5 {‘os’, ‘mUgksau’s , ‘mUgs’a } Identification of NP with ‘us’ Do the steps (2.1.1-2.1.4) Else If number(VM[j]) 55 ‘pl’ and anaphor 5 5 {‘muls’, ‘mudh’, ‘mudk’, ‘mudks’} Identification of NP with ‘dks’ Do the steps (2.1.1-2.1.4) Identification of NP with ‘dk’ Do the steps (2.1.1-2.1.4) Identification of NP with ‘ds’ Do the steps (2.1.1-2.1.4) Identification of NP with ‘ls’ Do the steps (2.1.1-2.1.4) End for End for

11.3.2.2.4 Algorithm for resolving reflexive pronouns Reflexives pronouns in Hindi includes “viuk,” “viuh,” “viusvki,” “[kqn,” and “Lo;a.” The algorithm also resolves distributive (repetitive) reflexive pronouns, which are viukviuk, viuh-viuh, and vius-vius.

Algorithm: Initialize n 5 token_id(anaphor) If word[n-1] 5 5 {FPP/SPP/TPP} Call substitution module Pick the antecedent of word[n-1] Else If word[n-1] 5 5 {NP} For i 5 (token_id(anaphor)
1) to 1 For j 5 sid(anaphor) to j-1 Identification of NP with ‘us’ Consider NP as antecedent. Substitute the anaphor with antecedent in mapping table Exit Else Identification of NP with ‘ls’ Do the steps (3.2.1-3.2.3) Else Identification of NP with ‘dks’ Do the steps (3.2.1-3.2.3) End for End for (Continued)

225

226

CHAPTER 11 REAH: Resolution Engine for Anaphora in Hindi dialogue

Algorithm: (Continued) Else If word[n-1] 5 5 5 (anaphor) 5 5 [RFX] // indicates presence of Distributive Reflexive If word[n-2] 5 5 NP For i 5 (token_id(anaphor)
1) to 1 For j 5 sid(anaphor) to j-1 Identification of NP with ‘us’ Consider NP as antecedent. Substitute the anaphor with antecedent in mapping table Exit Else Identification of NP with ‘null’ Do the steps (4.3.1-4.3.3) End for End for Else If word[n-2] 6¼ NP Substitute the anaphor with ‘abstract’ in mapping table Exit

11.3.2.2.5 Algorithm for resolving locative pronouns LOC pronoun refers to expressions that indicate place, location, position, and point of time of an action. They include and

Algorithm: For i 5 (token_id(anaphor)
1) to 1 For j 5 sid(anaphor) to j-2 Identification of NP with ‘esa’ Check for number agreement Consider NP as antecedent. Substitute anaphor with antecedent in mapping table Exit Else Identification of NP with ‘ij’ Do the steps (1.1-1.4) Else Identification of NP with ‘ds’ Do the steps (1.1-1.4) Else Identification of NP with ‘dh’ Do the steps (1.1-1.4) Else Identification of NP with ‘null’ Do the steps (1.1-1.4) End for End for

11.4 Test datasets

11.3.2.2.6 Algorithm for resolving demonstrative pronouns The algorithm has been defined for DEM pronouns that refer to NP an event clause. The DEM pronouns in Hindi includes “bl,” “ml,” “;g,” and “og”.

Algorithm: Initialize n 5 token_id(anaphor), s 5 sid(anaphor), If word[n 1 1] 5 5 {NP} // matched with term pattern For i 5 (s-1) to (s-2) Search word in previous sentences If found, c 5 cid(word_found) Select clauses having clause id 5 5 c Consider the clauses as antecedent Substitute the anaphor with antecedent in mapping table End for Else If word[n 1 1] 5 5 postposition OR word[n 1 1] 5 5 NN // i.e. DEM succeeded by a postposition or noun without case marker i 5 s-1 Select all the clause in sentence(i) with max(cid) // select recent previous clauses Consider the clauses as antecedent Substitute the anaphor with antecedent in mapping table Exit

After identifying the equivalence class of anaphor, the engine saves the findings in the substitution module, which substitute the actual antecedent of an anaphor. The module helps the engine to construct and automatically generate the final output on completion of resolution as mapping table. The mapping table represents the list of anaphors and the antecedent identified by resolver. The approach relies on machine learning, which is based on heuristic rules for precise interpretation that reduces complexity of the engine. The engine utilizing limited shallow features employed different filtration rules, distance factors and constraints that acted as decision factors and it is beneficial in terms of minimum human interaction.

11.4 Test datasets The authors have conducted experiments on five different types of datasets ([15], p. 159, 2015b, p. 32). The performance of each experiment has been measured in terms of precision, recall, and F-score. Dataset 1, dataset 2, dataset 3, and dataset 4 contain synthetic Hindi corpus that have been collected randomly from different domains such as news articles, blogs, and Wikipedia and annotated using Hindi shallow parser. Dataset 5 contains five different kid’s stories from the website (http://indif.com/kids/hindi_stories/short_stories.aspx). The statistics of anaphors to be resolved in all the dataset have been given in Table 11.8.

227

228

CHAPTER 11 REAH: Resolution Engine for Anaphora in Hindi dialogue

Table 11.8 Anaphors to resolve in datasets.

Datasets

No. of sentences in individual corpus

No. of anaphors to resolve

1 2

1 1
3

98 192

3 4 5

1
5 1
3 10
22

712 50 105

Types of anaphors Personal pronouns Personal and reflexive pronouns Personal pronouns Locative pronouns Personal, reflexive, locative, and demonstrative pronouns

11.5 Experiments and evaluations Each subsequent experiment has been conducted to overcome the challenges faced by the authors in former experiment. Thus the authors noted the significant contribution of features and constraints defined in each experiment and plugged them into in later experiments. The aim of individual experiment differs in terms of type of anaphors, size of discourse, and the scope for searching the antecedent. Experiment 1: Implementation on intrasentential anaphors Ref. [15] conducted the experiment on dataset 1 to study the behavior of CMs and attribute features. The aim of the implementation is to check the performance of algorithm for resolving intrasentential anaphors. The algorithms employed limited CMs such as “us” and “ls” CMs and successfully resolved all the 98 personal pronouns in dataset and result in 100% accuracy. The dataset did not contain TPPs. Experiment 2: Implementation on intersentential anaphors The findings of Ref. [15] (pp. 155-159) facilitated the authors to conducted experiment 2 ([16], p. 32) for resolving intrasentential and intersentential anaphors belong to personal and reflexive pronouns. The algorithms correctly resolved the antecedent of 145 anaphors in dataset 2 by extending the search scope for previously 2
3 sentences. The algorithms included more CMs such as “dks,” “dk,” and “ds” and incorporated number agreement that bestow the performance to discover equivalence class for personal and reflexive anaphors. The algorithms resolved successfully 87.01% of FPPs and 66.07% of SPPs. The algorithm resolved 40% of the root form of TPP “og” correctly and 79.5% of reflexive pronoun “viuh” in the corpus ([16], p. 32). Experiment 3: Implementation on long-distance antecedents In the previous experiments, it has been observed that the algorithm did not resolve the TPPs correctly. Therefore to improve the accuracy, experiment 3 has been conducted with the motive of identifying the referent that resides far from the anaphors. The algorithms introduced filtration rules and based on

11.6 Conclusion

Table 11.9 Anaphors to resolve in dataset 5. Dataset 5

Total sentences

Total anaphors to resolve

FPP

TPP

RFX

Story1 Story2 Story3 Story4 Story5 TOTAL

11 11 23 17 21

13 11 22 20 21 87

1 0 3 7 0 11

9 10 15 11 15 60

3 4 5 5 2 16

FPP, First-person pronouns; RFX, reflexive pronouns; TPP, third-person pronouns.

different features such as syntactical position of NP, grammatical rule of verb, distance factors, priorities of term patterns, and number agreement. Pruning the candidate list and identifying the coreferential link also improved the performance. It successfully identified the referent of 453 anaphors out of 712 anaphors. The resolver correctly resolved 78% of FPPs, 71% of SPPs, and 52% of TPP. Experiment 4: Implementation for spatial antecedent The experiments conducted so far identified the antecedents for personal and reflexive pronouns. Experiment 4 has been conducted to find the equivalence class for LOC pronouns that denote a location and refer inanimate entity. The priority of CMs has been set in the order of “esa” . “ij” . “dh.” The evaluation of dataset 4 was challenging and entirely different as the search to be made for abstract entity. The algorithm correctly recognized the referent of 17 LOC anaphors out of 50 instances and noted the accuracy of 34%. Experiment 5: Implementation for large corpus The number of sentences in the dataset 5 stories ranged from 11 to 23, which is remarkably large as in such discourse the antecedent may reside very far from anaphor and determining them is really a complicated task. The counts of anaphors to be resolved in individual stories in dataset 5 have been shown in Table 11.9, which lacks SPPs. The algorithms successfully identified the referent of 53 anaphors out of 87 and result in 64.4% of accuracy. Apart from these, dataset 5 also contains 4 LOC pronouns and also 14 DEM pronouns. Out of 4 LOC pronouns, 3 have been resolved correctly. And out of 14, the algorithm determined the referent of 6 DEM pronouns correctly.

11.6 Conclusion AR system needs significant amount of linguistic knowledge and time to develop set of rules by ultimately acquiring the internal lexical structural associations between lexical items. The acquired lexical information obtained from shallow

229

230

CHAPTER 11 REAH: Resolution Engine for Anaphora in Hindi dialogue

parser helped in forming algorithms for resolving the anaphors of Hindi language. The authors have discussed the experiments for resolving personal, reflexive, DEM, and LOC pronouns with different types of datasets in Hindi. The task of finding the equivalence class for anaphor in simple and single sentence is much easier than for complex and compound sentences, as the discourse sentences have too many antecedent candidates for an anaphor. The algorithms implemented in the engine include rules that have been drawn from the observation of the discourse, which mainly contain dialogues or conversation between speakers and listeners, where abundance of pronouns is available. The performance of all the experiments instead of limitation of features to be used is highly significant. The substantial accuracy indicates that all the significant features and constraints defined for resolving each category of pronouns contributed very well. The distance factors contributed a lot in terms of optimizing the candidate list and finding the coreferential link. The behavior of TPP algorithm while resolving anaphors was found quite unexpected and shows unpredictability in result. It has been observed that the algorithm to resolve reflexive pronouns did not identify the antecedent when reflexive pronoun succeeds personal pronouns that either have no referent or may be abstract in nature. The association of DEM pronouns with neighboring word has been taken into account to resolve it. As algorithms defined in the engine involved limited syntactic constraints and attributes, which have lower down usage of excessive computational deal and resources. It requires negligible human assistance while inputting the text that made the system fully automatic. Thus the efficiency of system remains uphold and less prone to mistakes. Necessary preprocessing has been done on parsed text without manual intervention to remove linguistic errors from input text and hand over to resolver module, so that efficiency and quality output of entire resolution techniques would increase. As the complexity and sentential phrase increases in the discourse, it affects the performance of the engine. These experiments are the evidences that even with limited morphological data and simple syntactic constraints, promising result can be achieved for Hindi AR. The engine resolved anaphors, antecedent of which resides beyond previous 5 sentences, thus evident that it is suitable for long-distance antecedents.

References [1] R. Prasad, M. Strube, Discourse salience and pronoun resolution in Hindi, Penn Working Pap. Linguist. 6 (3) (2000) 189
208. [2] S.L. Devi, V.S. Ram, P.R. Rao, A generic anaphora resolution engine for Indian languages, in: Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, 2014, pp. 1824
1833. [3] K. Dutta, S. Kaushik, N. Prakash, Information extraction from Hindi texts, Proceedings of the Fourth International Conference on Language Resources and Evaluation, LREC (2004) 1911
1914.

References

[4] S. Agarwal, M. Srivastava, P. Agarwal, R. Sanyal, Anaphora resolution in Hindi documents, 2007 International Conference on Natural Language Processing and Knowledge Engineering, IEEE, 2007, pp. 452
458. [5] K. Dutta, N. Prakash, S. Kaushik, Resolving pronominal anaphora in hindi using hobbs algorithm, Web J. Form. Comput. Cognit. Linguist. 1 (10) (2008) 5607
5611. [6] B. Uppalapu, D.M. Sharma, Pronoun resolution for Hindi, in: Proceedings of Seventh Discourse Anaphora and Anaphor Resolution Colloquium (DAARC 09), 2009, pp. 123
134. [7] K. Dutta, S. Kaushik, N. Prakash, Machine learning approach for the classification of demonstrative pronouns for Indirect Anaphora in Hindi News Items, Prague Bull. Math. Linguist. 95 (2011) 33
50. [8] S. Chatterji, A. Dhar, B. Barik, S. Sarkar, A. Basu, Anaphora resolution for Bengali, Hindi, and Tamil using random tree algorithm in weka, in: Proceedings of the ICON2011, 2011, pp. 7
12. [9] P. Lakhmani, S. Singh, Anaphora resolution in Hindi language, Int. J. Inf. Comput. Technol. 3 (2013) 609
616. [10] P. Dakwale, V. Mujadia, D.M. Sharma, A hybrid approach for anaphora resolution in Hindi, in: Proceedings of the Sixth International Joint Conference on Natural Language Processing, 2013, pp. 977
981. [11] P. Lakhmani, S. Singh, P. Mathur, Gazetteer method for resolving pronominal anaphora In Hindi language, Int. J. Adv. Comput. Sci. Technol. 3 (3) (2014) 173
176. [12] S. Singh, P. Lakhmani, P. Mathur, S. Morwal, Anaphora resolution in hindi language using gazetteer method, Int. J. Comput. Sci. Appl. 4 (2014) 567
569. [13] V. Mujadia, P. Gupta, D.M. Sharma, Pronominal reference type identification and event anaphora resolution for Hindi, Int. J. Comput. Linguist. Appl. 7 (2) (2016) 45
63. [14] B. Saqia, K. Khan, A. Khan, W. Khan, F. Subhan, M. Abid, Impact of anaphora resolution on opinion target identification, Int. J. Adv. Comput. Sci. Appl. 9 (6) (2018) 230
236. [15] S. Mahato, A. Thomas, Machine learning approach for resolving pronominal anaphora using Hindi dependency treebank, in: Proceedings of BITCON-2015 Innovations for National Development. IJAERS, IV(II), 2015a, pp. 155
159. [16] S. Mahato, A. Thomas, Exploring semantic information from Hindi dependency treebank for resolving pronominal anaphora, in: Proceedings of NCKITE-2015. International Journal of Computer Applications, 2015b, pp. 0975
8887. [17] S. Mahato, A. Thomas, N. Sahu, A relative study of factors and approaches for Hindi anaphora resolution, Int. J. Manage. IT Eng. 7 (12) (2017) 176
188. [18] S. Mahato, A. Thomas, N. Sahu, A survey on anaphora resolution toolkits, Int. J. Res. Appl. Sci. Eng. Technol. 5 (XII) (2017) 796
801. [19] S. Mahato, A. Thomas, Lexico-semantic analysis of essays in Hindi language, in: CEUR Workshop Proceedings: Educational Data Mining Practices in Indian Academia, 2017c. [20] A. Bharati, R. Sangal, D.M. Sharma, L. Bai, Anncorra: annotating corpora guidelines for POS and chunk annotation for Indian languages, LTRC-TR31, 2006, pp. 1
38. [21] Shallow Parser, [Online], Available from: ,https://ltrc.iiit.ac.in/showfile.php?filename 5 downloads/shallow_parser.php., 2019 (accessed 10.05.19). [22] R. Mitkov, Anaphora resolution: the state of the art, Sch. Lang. European Studies, University of Wolverhampton, 1999, pp. 1
34.

231

CHAPTER

12

Surveying various effective modes and research trends on cognitive Internet of Things over wireless sensor network

Jaya Mishra, Siddhartha Choubey, Jaspal Bagga and Abha Choubey SSTC-SSGI, Bhilai, India

12.1 Introduction From that point forward, we have seen further insurgency in processing as the machines have turned out to be increasingly insightful step by step. Today, we are nearly the time of the shrewd reality where everything should be done robotized and proactively without or insignificant human mediation. It knows the development that should take us to as Objects with computing devices felicitous. Objects with computing devices implies a related system method for the Internet in which anything is addressable in the mechanized universe. This omniavailability enables gadgets to share information easily and sharing information makes frameworks shrewd. For instance, joining traffic sensor information can give a general image of the traffic status of a city. In addition, if I defer the morning flight, the morning timer will alter it automatic in a like manner, enabling us to sleep somewhat more. Everything considered, these are reachable, anyway not by Objects with computing devices alone. We have to make Objects with computing devices progressively insightful. We have to drive awareness to the Objects with computing devices along these lines; it transforms into the comprehensive version of humans. Believe it or not, without Artificial Intelligence (AI) we cannot achieve the most extreme limit and vision of Internet of Things (IoT); we will not more likely separate the entire heap of common items from the IoT tree. Subjective computing that alludes to the down-to-earth authorization of Intellectual AI through the processing model includes another layer of usefulness to the current Objects with computing devices engineering. So, Intellectual Objects with computing devices will enlarge the current Objects with computing devices with the additional intellectual capacity especially like human cognizance.

Cognitive Informatics, Computer Modelling, and Cognitive Science, Volume 2. DOI: https://doi.org/10.1016/B978-0-12-819445-4.00012-6 © 2020 Elsevier Inc. All rights reserved.

233

234

CHAPTER 12 Surveying various effective modes and research trends

12.2 Objects with computing devices and AI 12.2.1 Internet of Things When some PCs are interconnected in a worldwide range then Internet forms. Objects with computing devices have expanded the vision of Internet further. Here, not just PCs are affiliated, rather, every element on the earth ought to be correlated. The fundamental aim was to catch and contribute to information in a mechanized and inescapable way. Different segments of the Objects with computing devices biological system incorporate the registering assets. The detected information is prepared and dissected to buy data and learning based on which someone might make some authoritative move. All things considered, the substance of Objects with computing devices is to make any data accessible to anybody and whenever over every one hindrance [1]. Objects with computing devices gadgets can do their work independently, for example, with no unequivocal outside order. They autonomously gather data and trade them proactively with other Objects with computing device gadgets inside the system. The assembled information are broken down physically. Someone has found the uses of Objects with computing devices in the wide scope of spaces, for example, fabricating division, strategic, transportation, farming, restorative, and social insurance, home and building computerization, keen network, administration part, and so forth. The headways in innovations that have prompted modest sensors, shabby preparing, and modest data transfer capacity with omnipresent remote inclusion and cell phones have given a viable ground to Objects with computing devices to collect data.

12.2.2 Objects with computing devices and computerized ones Prior, we expressed that the aphorism of Objects with computing devices is to interface everything wherever conceivable. For what reason do we have to interface every single thing? The response is clear and straightforward—to robotize. The advantageous part of the substance with computing devices has driven enterprises/organizations to make Objects with computing devices as another stride toward computerization engaging transfer together controlling and the executives [2]. The IoT-based mechanization can lessen the operational expense, when contrasted with the manual method, through robotized control and the board of segregated and free gadgets by interfacing and causing them to speak with one another [2]. Objects with computing devices mix and follow products for conveyance using GPS and RFID technologies which computerize the inventory network for the executives. They can control strategic administration midway through persistent checking of the area expected that time should disembark, and condition (e.g., warmth, dampness, and so forth), while traveling, guaranteeing quality [3]. Interfacing a gear to the Objects with computing devices will empower the producer. The retail part additionally has enormous focal points in stock administration

12.2 Objects with computing devices and AI

by incorporating IoT—for example, identifying low stock by brilliant racks, advising customers about markdown attempt as they enter the store, following products for smoother inventory network the executives, and so on [2]. Other than business and assembling, Objects with computing devices based computerization has been grasped in different fragments, for example, mechanized home and building, urban communities, shopping center, transport, and so on. For instance, temperature and radiance of lights will normally rely upon the circumstance and people groups’ essence. Objects with computing devices applications are potential deferral to the explorer’s cell phone by testing the continuous traffic condition consequently.

12.2.3 Objects with computing devices is not AI Experiencing all the great tales about “smartness and mechanization,” on the off chance you get befuddled and feel that Objects with computing devices is only AI just wearing a most recent extravagant specialized terminology; you are doubtless not to be panned, especially on the off chance you are an amateur in the realm of IoT. Truly, the guaranteed Objects with computing devices has offered solid, especially AI. I characterize simulated intelligence in MerriamWebster as A territory of software engineering that manages enabling machines to appear as though they have human insight. However, Objects with computing devices is characterized by Gartner: The fundamental item of Objects with computing devices is to associate the items that are joined. We find the ‘insight’ originates from the giving out of information and reshuffle this informations for the correct channel. Along these lines, the accentuation in Objects with computing devices is the ‘association’ while, in AI, it is the ‘insight’.

12.2.4 Need for AI in Internet of Things From the past segment’s dialog, we comprehended Objects with computing devices. The response is automatic-clear—to get the best out of IoT. Introductory Objects with computing devices applications had no basic leadership expertise, which made disappointment to accomplish the ideal execution level. The word “insight” was absent from the IoT. Computer-based intelligence makes Objects with computing devices canny. Unreal intelligence has changed Objects with computing devices to a keen element, equipped for carrying on unequivocally based on past information and occasions. It can naturally prepare, learn, and investigate future issues up to some great degree. Computerizing Objects with computing devices merges AI so as someone can take the choice automatically governing for the administration and controlling of IoT. Objects with computing devices contains an intricate system that incorporates billions of gadgets and sensors.

235

236

CHAPTER 12 Surveying various effective modes and research trends

Dealing with this framework is an expensive and grim undertaking. This requests programed checking, the executives and automatic- (automaticdesigning, automatic-securing, automatic-arranging, independent, automaticmending, mindful, automatic-learning, and automatic-changing) ownership of Objects with computing devices to limit the human mediation and accordingly diminishing operational expense [2]. For that, it is fundamental to pervade AI into IoT. The expanding associated gadgets request for ongoing versatility and astuteness to oversee the planned administration activities. Computer-based intelligence fuses thinking and basic leadership abilities to IoT, prompting keen working. Computer-based intelligence includes another layer of usefulness to the current Objects with computing devices engineering, driving gadgets to basic leadership, thinking, and learning. Changing Objects with computing devices into a savvy substance is equipped for carrying on definitively over past information and occasions. It might consequently prepare, become skilled at, and investigate potential problems for some great degree. Unreal intelligence empowered Objects with computing devices senses different stuff. It develops insight from the example it sees on different information it buys from encompassing and system and settles on wise choices. Using AI in Objects with computing devices applications has complex favorable circumstances:

• enhances client experience, • learns example and conduct a connected framework naturally, and • identifies inconsistencies and clashing circumstances. Gadgets of Objects with computing devices alone are not beneficial, much like our body without a cerebrum. The information that Objects with computing devices gadgets produce is significant. In addition, just to catch the information is not adequate. To use Objects with computing devices to its maximum capacity, the immense measure of information produced out of many Objects with computing devices gadgets is to be used prudently. We have to take out significant information from that. This information contains significant experiences on what is working and so forth. The enormous test is to examine the information too long for the right data at the ideal time. The limit of Objects with computing devices information has quite been put to use. The customary techniques for information examination are unfit to deal with the sheer measure of information produced out of Objects with computing devices. Consolidation of wise calculations/programing in Objects with computing devices information examination joined with information from different business sources breeds factual data and outline, fortifying a future forecast situation. This mitigates new framework plans, bringing about expanded profitability and operational effectiveness, which was unattainable before [2]. Other than detecting information for a circumstance, Objects with computing devices must be very mindful of various inconsistencies such as defective sensors, inaccurate information getting, missing information, and information equivocalness to keep the framework coordinated and continued. Utilization of AI assumes

12.3 Intellectual AI and Intellectual compute

a pivotal job in observing sensors, organizing, and different gadgets using shrewd security calculations. Undeniably, AI prompts automatic-sufficient basic leadership and automaticadministration, yet exposed to the inquiry—to what degree knowledge ought to be connected to IoT? Due to the dispersed system engineering of Objects with computing devices with shifting heterogeneous gadgets having an alternate vitality level, it is very sensible to have qualms about the level of insight given to each Objects with computing devices hubs. The hubs are empowered with constrained AI, bringing about automatic-sufficient basic leadership on what, when, and the amount to process. This will confine gadgets to overprocess and be overenergetic accordingly sparing a lot of vitality, which is extremely critical for Objects with computing devices gadgets that are normally vitality insufficient. Once more, an excess of automatic-governance can make the entire framework eccentric, which may resist the framework structure objective. There is additional need to comprehend the exchange off among concentrated and conveyed insight. Unified insight is modern and less mind boggling while at the same time appropriating intelligence at the edges and hubs of system make Objects with computing devices increasingly automatic-governing and adaptable yet at the expense of intricacy as far as execution. In any case, AI is as of now being utilized in Objects with computing devices for better end result and any future attempt unbelievably opens doors for us to look past skylines. We simply need to make a point to snatch the correct open door that changes our lives. Different zones of Objects with computing devices and AI-empowered IoT are advancing and require further research.

12.3 Intellectual AI and Intellectual compute 12.3.1 Intellectual AI and cognition, AI The idiom “discernment” was obtained from the Latin declaration cognition that signifies “to think.” The word reference characterizes “comprehension” (thing) as “the utilization of a cognizant mental procedure” and “intellectual” (action word) as “associated with considering or cognizant mental procedures.” Comprehension accentuates on the procedure how one learns, recalls, and reasons as opposed to a specific actuality that one has learned. Machine cognition had been started from reasoning and rose bit by bit.

12.3.2 Intellectual computing Intellectual frameworks are on a very basic level diverse in contrast with different types of figuring accessible or rehearsed in past. The psychological framework constantly gains from its cooperation with information (organized or

237

238

CHAPTER 12 Surveying various effective modes and research trends

unstructured), individuals, and circumstance and in this manner in the long run thinking abilities [4]. Subjective computing significantly applies operator based innovation. In circumstances, savvy specialists fathom abnormal state goals and adapt automaticsufficiently to the most proficient method to achieve the destinations. Similarly, Intellectual compute grows profound space skill; along these lines reliance in contrast with the master frameworks could prompt a superior choice whether in social insurance, money, or client administration [5]. Subjective advances expand the capacity of PCs in executing the assignments for the most part performed by people, for example, penmanship acknowledgment, face acknowledgment, and different undertakings that require human psychological abilities, for example, arranging, thinking [2]. Setting incorporates highlights that portray what, when, where, and how an element is occupied with its condition and the particular procedure to which it is included with. Setting gives good fortune to discover appropriate arrangement design in the gigantic and assorted gathering of data, which might be a reasonable reaction to the need existing apart from everything else.

12.3.3 Further than mechanization Is it simply one more innovative contrivance? All things considered, essentially, they are unique, in a general sense, by a wide margin. For example, a sensor gadget can make some move as per the information it detects. This mechanization is however not psychological. Give us a chance to expand. On off chance the light changes its radiance and presumably the shading as indicated by time, climate, and my state of mind. Customary computerized frameworks cannot see or express humanly feeling. Conversely, psychological frameworks join passionate conduct for connection [5]. Mechanized frameworks are great for working in organized information; switch it while intellectual frameworks, notwithstanding these fundamental errands, can deal with unstructured information, learn, and surmise additional learning through each procedure and reuse the learning [5]. They can sift through and center around the significant occasions as it were. Just about a century prior, when Ford presented the mechanical production system, the rudimentary type of early modern robotization, the design was to spare working hours from dreary manual undertakings. From that point forward, mechanization has been embraced in a few enterprises with the goal that the lower level laborers can focus on progressively perplexing and inventive assignments. The utilization of psychological advances is stretching out robotization to new regions that have never been the idea of it. By adding subjective capacities to the current mechanization, it obtains businesses past negligible computerizing forms at the lower level to give huge basic leadership learning to all dimensions of representatives and enable them to secure top of the line aptitudes to take up and take care of issues they recently did not have as the source and time to deal with [6]. The first reason for mechanization was unadulterated business arranged and that was to

12.4 Objects with computing devices and Intellectual computing

spare operational expense. In spite of the fact that CObjects with computing devices is getting critical consideration from the business network, the basic role of CObjects with computing devices is to make individuals’ life more brilliant and relaxed by taking us route past the unadorned and unsophisticated robotization.

12.4 Objects with computing devices and Intellectual computing 12.4.1 The Intellectual Internet of Things The motivation behind the Objects with computing devices is to dispose of the limit among creatures. In any case, inferable from the complexity and the size of IoT, it cannot be acknowledged by the essential type of Objects with computing devices [7]. To receive the total reward from IoT, we have to utilize Intellectual computing as an additional what we call as Intellectual Objects with computing devices CIoT. CObjects with computing devices is gone for improving execution and to accomplish knowledge of Objects with computing devices through helpful components with Intellectual Computing [8]. The present Objects with computing devices for the most part centers around detecting its environment and acts in like manner. The choices taken by the gadgets associated with Objects with computing devices are commonly founded on precustomized models. They can surmise on the basis of detected information accessible. In any case, they are not undeniable automatic-ruling frameworks that can take their choices particularly relying upon the prompt setting. By mixing sense into IoT, Intellectual Computing empowers Objects with computing devices to communicate progressively with other associated objects, just as adjust to the present setting through constant gaining from the earth. They will almost certainly watch, channel, and perceive, fundamentally the same as people, and furthermore acclimatize that data to selection noteworthy learning and significant examples [5]. They will comprehend the setting on the basis of the area where Objects with computing devices is connected and acts in like manner [9]. Psychological Computing can be worked out, basically, in three distinct features of Objects with computing devices as depicted underneath: 1. Circulate perspective: The systems administration part of CObjects with computing devices is fundamentally an all-inclusive idea of intellectual radio [10] and psychological systems [11,12]. Intellectual systems endeavor to accomplish the ideal execution by adjusting themselves to the current state of the system. Utilizing subjective systems, the CObjects with computing devices can settle on intelligent choices through grasping the present system condition and breaking down the apparent learning [8]. Subsequently, CObjects with computing devices can take up fundamental versatile measures to amplify organize execution and limit inactivity. The intellectual system can expand organize limit through astute multiarea participation that will be a major aid

239

240

CHAPTER 12 Surveying various effective modes and research trends

as the volume of Objects with computing devices information is developing inconceivably [8]. 2. Detectable perspective: Like people, the Objects with computing devices gadgets is additionally planned to detect the contributions, in the perfect case, as idea, echo, savor, odor, and contact. The creature personality is normally fit for arranging and working together with these biological data sources, progression them. Implanting psychological advancements for Objects with computing procedure gadgets makes it possible. Notwithstanding their ordinary ownership for example detecting the environment and sharing this data, by impersonating human perception capacity, they will be equipped for picking up, considering, and appreciating, without anyone else. As indicated by gaining from past connection experience, they will most likely modify their future collaborations. CObjects with computing devices not exclusively can detect yet in addition figure out how to foresee, alongside time, various feelings and react in like manner [5]. They can modify their reaction as indicated by the state of mind and the changing feelings of different frameworks they are communicating with. 3. Data investigation viewpoint: The broad utilization of Objects with computing devices is creating colossal information and in the coming years, it will end up being the biggest wellspring of computerized information on the planet. Handling and channelizing this immense measure of information into the correct heading and using them for intentional use will turn into a test. In a large portion of the Objects with computing devices applications, the greatest piece of these information is disposed of as a result of the absence of ability to deal with the information of this size. Regardless of whether at times, the Big Data stages, for example, Hadoop are utilized to store every one of these information. They are not being completely used because of the specialized constraints of these stages and the PCs [2]. The conventional methodology of programing that depends on arrangement of restrictive proclamations is not adequate to deal with this colossal measure of information that Objects with computing devices information can be changed over into wise information through subjective strategies that should help associations in mechanizing undertakings, structuring better items, and enhancing new client driven administrations [13]. CObjects with computing devices will perceive the hierarchical objectives, amass, coordinate, and dissect pertinent information to enable organizations to accomplish those objectives [7].

12.4.2 Ownership of Intellectual Internet of Things CObjects with computing devices is inalienably appropriated just as unavoidable in nature. Henceforth, the greater parts of the parts of these two registering ideal models rely upon to be fascinated by CObjects with computing devices too. Aside from those, some other standard (and attractive) ownership of CObjects with computing devices is referenced beneath [2].

12.4 Objects with computing devices and Intellectual computing

Automatic-learning: CObjects with computing devices ought to require least unequivocal programing. It will consistently gain from nature about managing different substances, and from the occasions, constantly improve it automatically. In a perfect world, the learning ought to be unsupervised instead of getting administered, for example, they ought to learn independently from anyone else with no preset parameters. Intellectual Computing works are dependent on nonstop theories of development. Each time CObjects with computing devices gains some new useful knowledge it looks for endorsement from the current theories [2]. Contingent upon the result it refreshes from its memory. Contingency: CObjects with computing devices is not organized either deterministic, for example, it cannot be characterized by recognized terminology. In view of the result it might change the past theories and potentially will locate another course of activity. Versatile: It should be likewise figured out how to alter it automatically as data changes just as when new destinations and prerequisites create and update the procured information on account of any alteration [2]. Adaptable: One automatic learning and versatile abilities give CObjects with computing devices a ton of adaptability in ingestion and preparing of info information. In the event that, for instance, while preparing, an undesirable information or variable comes up, CObjects with computing devices will alter its handling model to fuse that aberrance, not at all like the conventional Objects with computing devices where the program may must be modified [2]. This turns out to be exceptionally practical in the flighty conditions, particularly, where the substance it automatically continuous developing. Energetic: CObjects with computing devices most likely handle ongoing (both delicate and hard) information. It ought to have the energetic ability to process and dissect information on-the-fly. In addition, it must determine uncertainty powerfully and furthermore manage eccentrics. Intelligent: CObjects with computing devices is intended to be intuitive with individuals (clients), machines, and other mechanized administrations. CObjects with computing devices is particularly astounding for its capacity to cooperate with individuals in a completely human manner. Collaborating ability utilizing normal terminology makes it exceptionally groundbreaking. It can take contribution from characteristic terminology or unstructured content and give yield in a similar structure [2] and mostly because of this, CObjects with computing devices has increased boundless pertinence individually. Continual: Automatic-learning is dependably a continual procedure. CObjects with computing devices likewise pursues the continual example: cause a theory— to adapt further—update the speculation. Stateful: CObjects with computing devices has remembrance and procured information on each event with the goal that they can reminisce at whatever point required. Unstructured information well disposed: CObjects with computing devices can deal with a large portion of the unstructured information types [14]. This has

241

242

CHAPTER 12 Surveying various effective modes and research trends

enabled it to be incorporated with most of the information sources on the planet in this way, expanding the adaptability and the extent of learning procuring. Exceptionally incorporated: Though every individual gadget in CObjects with computing devices works (sense and adapt) freely and regularly automaticsufficiently, they all group up centering to add to a focal learning framework. In that sense, each gadget is intently clung to one another through persistent connection, sharing data, and refreshing own insight as needs be [15]. Adaptable: Working scope of an Objects with computing devices can be limited to a little room or extended over an entire city. The unavoidable and universal utilization of Objects with computing devices makes it fundamental to help affixing (or expelling) of cell phones to (or from) the framework progressively. In this manner, CObjects with computing devices should bolster exceptionally continuous adaptability. In this perspective, one thing goes for CObjects with computing devices that it makes less repetitive calculations, which is exceptionally urgent for adaptable frameworks [2]. Setting and circumstance mindful: The legitimacy of Objects with computing devices information and derived learning relies upon the specific logical and situational data, for example, time, area, application and authoritative space, guidelines, client’s profile, procedure, errand, and objective [2]. CObjects with computing devices can reperceive, read, and concentrate this data and apply correspondingly. Automatic-administration: The uncommon size of CObjects with computing devices makes it difficult to oversee physically or notwithstanding utilizing the board devices robotized. It should act naturally overseen. There are various components of the board of CIoT. Of them, some significant ones are stated as follows:

• Diagnosis, investigating, and upkeep: Preferably, CObjects with computing

•

•

devices ought to analyze and investigate it automatic other than standard upkeep, with no human between venation. It ought to have the option to watch the conduct of its parts. For planned support, which incorporates battery substitution, arrange examination, other equipment investigating, and so on, CObjects with computing devices will consequently locate an appropriate space for this with the goal that the upkeep vacation is least. Error resistance. For Objects with computing devices gadgets are fragile and blunder inclined and for the most part works over remote systems, there is every probability of continuous flaw event. CObjects with computing devices ought to have the option to recuperate, if conceivable, or cover these flaws. A large portion of the Objects with computing devices applications gives yield as occasions on the basis of some activated activity. Along these lines, it might confound separate between a blunder condition and a trigger occasion. CObjects with computing devices should stamp this qualification effectively by applying its insight: Performance of the executives. CObjects with computing devices will not just have the option to bring up the explanation behind execution debasement.

12.4 Objects with computing devices and Intellectual computing

• Arrangement of the executives. Any issue caused because of progress in •

system design, particularly on account of gear-tooth native systems, ought to be easily settled without anyone else. Security the board. It is no chance deniable that Objects with computing devices is truly powerless against security dangers. CObjects with computing devices ought to envision these dangers and should shield it automatically and appropriately.

12.4.3 The pillars of Intellectual Internet of Things Endowing noetics to nonliving Objects is not straightforward. We have a tendency to need to supply it. Unusual field of study in AI (Fig. 12.1) square measure needs to be consulted thoroughly. CObjects with computing devices is to be controlled by themselves without any supervising. Machine vision is not going to be provided the attention that speech recognition can provide. CObjects with computing devices the ear as beside natural language processing they’ll be ready to perceive human word and conjointly respond within the same. Composition for surprising occasion/information, in lightweight of its learning data milliliter will prepare or clarify the data and might build important move. Machine Learning (ML) includes different methods by taking in calculations. These calculations are classified as administered and unsubstantiated erudition calculations. ML is connected with business and industry, producess precise continuous data and occasions. ML has a critical impending in Objects with computing devices apparatus, all things considered, and business application space. ML in Objects with computing devices pursues a continual procedure where the essential move made by the “things” is reclaimed as criticism to find out more in this manner. ML is utilized to break down the Objects with computing devices information and constructs an expository model for future expectations. Ongoing Pattern recognition

Machine learning Congnitive AI Natural language processing

Driveless car

Smart health

Machine vision

Cognitive IoT

Smart city

FIGURE 12.1 COGNITIVE IOT PILLARS.

CoBots

Social networking

Speech recognition Ontology

Real time analytic

Smart living

Chatbots

243

244

CHAPTER 12 Surveying various effective modes and research trends

information detected by the “things” are examined over the model to give continuous data. The information is changed into machine learning, which gives an error-free knowledge into the Objects with computing devices. PC vision: PC vision includes preparing visual information (video or picture) to extract the important data out of it. One of the tireless difficulties that have pulled in analysts’ consideration is recognizing articles. As a rule, PC vision process includes highlighting extraction of the picture being referred to and coordinating that to put away pictures. Psychological PC visualization explanation is introduced in the following paragraph: Lately, with innovative improvement, cell phones are getting more astute practically with every mobile that has camera and web association. A gigantic populace over the globe does have brilliant cell phones, which has empowered to acknowledge Objects with computing devices in huge scale. The capability of these gadgets can be saddled for coming to Objects with computing devices to every single turn in the substantial humanity. Objects with computing devices related with PC vision can attempt unlimited conceivable outcomes. Outwardly ID increases normal contrast with articles distinguishing proof utilizing marker system (RFID, QR code, Barcode) as far as versatility and unknown item ID. The advances in PC vision have driven Objects with computing devices in the voyage to find as far as possible to perceive an item, for example, and circumstance which empowers the machine to have a progressively accommodating feeling of genuine world [16]. Instances of Objects with computing devices with PC vision resemble traffics the board, security and carefulness, tolerant well-being checking, and programed route. Pools furnished with PC vision can screen and raise alert for an individual battling or suffocating. Natural Language Processing (NLP): Cause PCs to get normal (human) terminology, which is not new. Throughout the years, NLP innovation has advanced with improvement of registering calculation and processing execution. NLP includes handling the human common terminology to extricate its importance and the goal. For understanding the profound basic significance and semantics, punctuation models are built. Literary sentences will in general be unique and have concealed importance inside. Machine of Hidden Markov Model is helpful to comprehend the concealed significance inside literary sentence. NLP related with Objects with computing devices could be very helpful in trading discourse and inquiry in different hunt and computerization practice [2]. Handling normal terminology without human intercession is one of the determined issues; however, many researches had occurred yet driving arrangement is far away.

12.4.4 Challenge of Intellectual Internet of Things Hypothetically, the possibility of CObjects with computing devices may sound splendid; however, reasonable acknowledgment of it is clearly not going to be direct. To make the vision of CObjects with computing devices genuine, various

12.4 Objects with computing devices and Intellectual computing

specialized difficulties ought to be settled. Underneath a portion of these difficulties are abridged. Restricted battery: For any remote gadget, control is a major issue. CObjects with computing devices gadgets are the same. They require consistent power; truth be told, more power than conventional Objects with computing devices in light of the fact that other than common detecting and information exchanging, CObjects with computing devices gadgets capacities include psychological exercises. Different information types: To discover the pertinence, the Objects with computing devices information is essential to analyze. Protection and protection: Solitude and protection are dependent in greatest difficulties in an organized framework. Each gadget turns into the plausible door for the programers. The utilization of cognizance over information may uncover important bits of knowledge; however, it turns out to be powerless against work raising protection. For felicitous reasonable encryption calculations, the test lies in building up a calculation which bolsters circulate as a key instrument, which is quick and vitality productive. The issues that should be tended to are as follows:

• decentralized validation and security model for Objects with computing devices application and

• data assurance calculations and advancements that are quick and asset and vitality productive. General security concerns are related with IoT. For instance, will it be sheltered to share clients close to home data (e.g., area) with other Objects with computing devices gadgets or applications? Extreme protection concerns for viewpoints too. Objects with computing devices framework looks for client standard of conduct (inclination, purchasing, and use) to produce advertise examination figures. Distinctive business arranging and crusading are organized on the basis of these examinations. Associated gadgets with insight have extended the security and protection issues further. Intellectual Objects with computing devices is empowered to predict the individual inclinations, needs, and appropriateness; these information are defenseless against coincidental releases and are worthwhile to fraudsters [5]. Preparing: The adequacy of the CObjects with computing devices relies upon the quality of preparation it gets. In CIoT, the perception of things improves with nonstop continual preparing. Insight enables things to recall to what they have realized. This scaffolds the learning hole among things, yet would they be able to adapt new things independently and besides would they be able to apply their “presence of mind” imaginatively to survey impacts. Discovering foundation material suitable for preparing is one trying issue. Psychological things gain from past associations. In view of the achievement and disappointments, it fastidiously ascertains the best alternative for new issue circumstance. Gadget association and information stream the board: Objects with computing devices gadgets create information in persistent time arrangement and in a large portion of the cases the information must be handled and broken down in a hurry. The result of one gadget might be nourished to other or might be bolstered back

245

246

CHAPTER 12 Surveying various effective modes and research trends

to it automatically for incitation. Henceforth, it is significant that data ought to stream properly starting with one gadget then onto the next. It is a test to oversee and directing information stream among various gadgets [17]. A crowded arrangement like Objects with computing devices with various gadgets associated with it for data trade has difficulties in looking gadget and contriving convention to trade various information types. Right progression of data helps in relentless data age and synchronous utilization that will prompt streamlining the business procedure (if there should arise an occurrence of authoritative IoT). Consequently it is vital to keep the gadgets, in the system, in amicability. However, what ought to be the perfect gadget association for ideal information stream? Being automatic-sufficient, CObjects with computing devices arrange can progressively change, advance, and find automatic and different systems. Programing and calculations: The product is incorporated into “things” at different system levels to convey and evaluate the system information. Programing works in various conditions having heterogeneous kinds of “things” imparting utilizing various conventions. One major test is to create CObjects with computing devices applications that would incorporate distinctive programing module, working under various conditions rationally. The huge issue is the means by which the conveyed programing is turned into one weave, tending to support arranged problem. The desires are tended to including the following:

• distributed automatic-versatile programing fit for automatic-administration, automatic-advancing, automatic-designing, and automatic-recuperating;

• open and vitality effective programing stage fit for incorporating programing intelligent system assets. It is fundamental to repersive and limit the biasness in the projects. The predisposition is transferred to a CObjects with computing devices program either by the preparation information or the manner in which the calculation is structured. The nearness of a particular sort of data (e.g., statistics) in the preparation of information makes the calculation innately curved to determine the particular kind of issues while demonstrating wasteful in the case of other issues. The framework could be one-sided by the manner in which calculation processes that data. The predisposition may likewise be transferred by the designer into the code to take care of a specific sort of issue either deliberately or unexpectedly. To keep up the nonpartisanship and objectivity of a calculation, the inclination of the board is truly tested. It is similarly essential to think about and consider the “abuse” cases as the utilization instances of CObjects with computing devices [2].

12.5 Value of Intellectual Internet of Things Objects with computing devices have just made a colossal promotion among the organizations. Not just huge players, SMEs are additionally detecting rewarding

12.5 Value of Intellectual Internet of Things

potential in receiving IoT. It guarantees to convey an incentive to a wide range of organizations by reexamining the business procedures and musical drama that will in long run upgrade its level and nature of items and administrations according to the client experience [2]. As Objects with computing devices has given it significant element of business, the information obtaining machine gear-piece, associations are liberated to gather any kind of information (e.g., logical and locational) either identified with business procedure or client. Increment profitability: Objects with computing devices distinguishes the need and absence of workforce aptitude and furthermore empowers associations to prepare representatives in the nick of time. This improves specialists’ effectiveness and decreases confusion of abilities, which thus increases hierarchical efficiency. Improved operational efficiencies: The ongoing sensor information from Objects with computing devices gadgets empowers associations to screen business tasks attentively, limiting human intercession. On the off chance that Objects with computing devices information gathered from coordination arrangements, manufacturing plant floors, and store network are used sensibly, stock administration can be improved, and time to advertise just as personal time, because support can be abridged altogether. Upgraded resource usage: Industrial Objects with computing devices empowers following: generation gear, apparatus, and devices. Inspecting the constant status, better resource utilization can be accomplished. Quicker basic leadership: The constant business process and operational information will assist associations by making quicker and more astute business choices. The associated idea of Objects with computing devices encourages administering the knowledge and subsequently leaders can organize all business choices. Cost sparing: All the previously mentioned increases of embracing Objects with computing devices will in the long run lead to sparing the business consumptions. In any case, just huge quantifiable information amounts to nothing; the nectar is the pertinent information [2]. Be that as it may, quantifiable business and money related advantages may not be accomplished even through applicable Objects with computing devices information alone. To transmute the Objects with computing devices information into cash, they are required to be broken down correctly to acquire definitive learning with the goal that the CEOs and the CIOs can take critical and intentional choices. Utilizing investigation devices on information aggregated from numerous different information sources focuses the empowered organizations to figure out the client requests and spot drifts better. As Nick additionally attested, “PCs can settle on refined choices dependent on information and learning.” CObjects with computing devices has enabled organizations to accumulate, watch, and offer a remarkable measure of different information about clients, faculty, items, and business procedures and activities all through the association [5]. Running distinctive business musical show consistently winds up by loving explicit and precise ongoing data produced by the

247

248

CHAPTER 12 Surveying various effective modes and research trends

Objects with computing devices. Basic leadership likewise turns out to be progressively direct and careful as the forecast of things to come occasions should be possible more precisely with the assistance of keen examination. CIoT, with the utilization of visual examination and information representation systems, can outwardly depict the logical results that furnish people with a superior discernment and set them in a superior position to take a choice [5]. Since Objects with computing devices can give customized and centered information, investing it by comprehension capacities will help organizations to investigate patterns and find sudden examples for better basic leadership by narrowing down to the little yet exact arrangements of exceedingly basic information [2]. Organizations will be electrified to discover new business bits of knowledge and accomplish improved profitability and productivity [2]. Subjective innovations can affect association’s workforce either by expanding their capability or by supplanting them. In both the cases, associations will pick up monetarily. Intellectual frameworks can ace the expert semantic of various callings (e.g., fabricating and drug) and furthermore can speak with the clients in normal terminology [7]. Subsequently CObjects with computing devices may profit associations by disposing of the need of putting resources into the workers to move toward becoming specialists. CObjects with computing devices can convey noteworthy improvements to the administration division by offering profoundly individualized administrations. In spite of the fact that the administration area as of now has encountered noteworthy advancements, latest methodologies or improvements, purchaser conduct, utilization sections, latest administration model should also be developed. With the appearance of new advanced stages, vendors are finding new roads to straightforwardly interface with shoppers. To be progressively receptive to the expanding client commitment merchants need to comprehend their clients much better, which prompts better consumer loyalty. They require to gather and investigate shopper information. Objects with computing devices has empowered them to gather relevant information. Various initiated and customized shopping knowledge will be conveyed unavoidably through focused proposals and individualized correspondence. In the meantime, these intellectual things additionally gather the shopping habits/examples of the customer. Intellectual investigation information of Objects with computing devices will help retailers in perceiving changing conduct and desires for the clients and react in like manner [18]. They can predict customer practices so as to envision prerequisites before they are required. CObjects with computing devices will enable organizations to plan creative customized items and administrations that supplement the client’s decision. The CObjects with computing devices will empower organizations to take proactive measures to maintain a strategic distance from client disappointment by pinpointing the genuine reason for an exhibition debasement [5]. Specialist organizations can survey their administrative foundations with the goal that they can tune themselves so as to give the largest amount of value administration to the customers [2]. To condense, organizations presently can investigate unanticipated conceivable outcomes that were already either disjointed or out of reach [7].

12.6 Areas where we used

Actually business associations are as of now utilizing subjective advancements for long to help their items, business procedure, and business investigation [2].

12.6 Areas where we used In spite of the fact that a number of researches are coming up, the acknowledgment of CObjects with computing devices, multitudinous utilizations, and Objects with computing devices will come up in the following 5 years of range [11]. As a rule, intellectual frameworks will be the following real innovation that will essentially affect market, human services, social services, get suggestion and create buys, and so forth [19]. Simulated intelligence will hoist and also reduce current advancements.

12.6.1 Well turned-out livelihood Residence and keen condition will never come again in dream; different items that we go over in our day-by-day existence are brilliant.

12.6.2 Elegant health One of the greatest effects of CObjects with computing devices will be in the medicinal services. Therapeutic consideration gadgets installed with knowledge could screen the well-being status of wiped out individuals [2]. Any detritions found could be examined to turn away hazardous ailment, sparing the life in time. Data could be accumulated to recommend the individual what movement, sustenance, or medication is reasonable for him to turn away any genuine ailment. In the event of medical crisis, the well-being observing gadget connected to the individual may caution the restorative administrations.

12.6.3 Household appliances Psychological Objects with computing devices has empowered the home apparatuses to be shrewd. The auto-learning makes home apparatuses keen. Subjective gadgets must perceive the client by voice, face, contact, or unique mark in this manner giving information to administration on the basis of client’s past connection [5]. Toward this path, business organizations are refreshing their gadgets to make them more brilliant. For instance, Whirlpool is producing shrewd clothes washer that could be constrained by cell phone [2]. Soon, we will be encompassed by the shrewd home machines, which will have their very own comprehension and would help the client by recognizing the client’s expectation and timetable/utilizes design.

249

250

CHAPTER 12 Surveying various effective modes and research trends

12.6.4 Smart cities The apparatus of CObjects with computing devices is there in everyday administration and exercises and improvement and arrangement. Data collection on water utilization, power and other wellspring of vitality use, open transportation, individual’s surge, and parking spot inhabitance can prompt better choices on city foundation and as assets to the executives [20]. The appliance of CObjects with computing devices in city arranging and the board will make practical civil administrations. Moreover, CObjects with computing devices will increase city worker management via robotized correspondence with and among isolated waste receptacles for better rubbish accumulation and so on. The machine of CObjects with computing devices alongside stacked data will give a strong stage to data handling and correspondence, in this manner conveying quality administrations and data.

12.6.5 Wiki City The idea of Wiki City is obtained, the onsite publicly supported learning archive. A Wiki City is the information storehouse of Objects with computing devices information for a particular area. Much the same as Wikipedia, individuals can alter and get to this data through straightforward website pages. For instance, contamination molecule level in the climate said that, “Day by day Pollution.” Essentially, climaterelated information, for example, temperature, stickiness, daylight, snowfall, downpour, and so forth, and city traffic related data. The conceivable arrangement likewise may be discovered relying upon accessible information.

12.6.6 Synchronized analytics Associated Objects with computing devices gadgets frequently make a huge complex system. These gadgets continuously produce information that should be investigated continuously for making move in opportune time. The machine of AI attempts continuous examination abilities into Objects with computing devices framework [17]. Constant investigation is about information examination in real time. Ongoing investigation is a period basic procedure that in wording relies upon elements like system inactivity, information handling speed, design acknowledgment, data derivation from past information, stockpiling and recovery of stream information, and so forth. By utilizing CIoT, business divisions can induce bits of knowledge from the detected information and convey to another business divisions for continuous and setting explicit basic leadership [5].

12.7 Usecase CObjects with computing devices is still in its neonatal stage. Thus very few real executions of CObjects with computing devices can be found by and by. In any

12.8 Conclusion

case, individuals have begun seeing it and understanding the potential. In spite of the fact that in little scale, CObjects with computing devices has been actualized effectively in different applications. A couple of most discussed business use of CObjects with computing devices has been referenced beneath. Bluemix is the spearheading advances transferred about by IBM with a point of view of giving insight regarding business information customer centric, clientbased administrations, and IoT-based administrations. Plan and usage of utilization explicit AI empowered Objects with computing devices is exceptionally tested as far as information applying machine knowledge over it. Along with this, it gives psychological administrations such as discussions, revelation, and smart virtual operator. It has various capacities, for example, understanding a different terminology, normal terminology characterization, content combination, voice to content and the other way around change, identity knowledge, tone breaking down, and so forth. In addition, it is fit for finding understanding/design in information. Watson virtual specialists understand individual’s need and convey back with explicit administration reasonably felicitous to the individual’s need. The psychological innovation utilized in the Watson permits it communicating with things and related individuals utilizing regular terminology and voice directions. This has basically drastically improved the versatility of the framework to individuals. The whole arrangement of administrations could be obtained by Objects with computing devices gadgets. Bluemix is an arrangement that guarantees figuring crosswise over gadgets unavoidably. Watson Intellectual Computing supports the probability of Objects with computing devices impacting a solid incorporation of public activities. Utilizing keen innovations and remote consideration benefits, the venture goes for expanding the personal satisfaction, autonomy, social association, and diminishing the expense of well-being and care. It particular, centers around improve the everyday lives of the old and crippled individuals by subsiding their reliance on others. Utilizing CIoT, AAL has had the option to remunerate a portion of the inabilities by the methods for the shrewd gadgets. Surrounding insight enables utilization of things with all usefulness without anyone else, consequently strengthening their autonomy. Use of encompassing insight has turned our encompassing article wise, prompting less human movement. Individuals are expanded with programed help at correct time and opportune spot dependent on close to home necessity. It is useful particularly to the older individuals who are utilizing savvy innovations and remote consideration administration can remain longer at home easily.

12.8 Conclusion Objects with computing devices have unquestionably been instrumental in taking robotization to another dimension. Be that as it may, the essential Objects with

251

252

CHAPTER 12 Surveying various effective modes and research trends

computing devices needs insight. By including insight as humanlike perception, the maximum capacity of Objects with computing devices can be transferred out as CObjects with computing devices will make the gadgets sufficiently shrewd to gain powerfully. Comprehension in Objects with computing devices can be seen in three characters: (1) circulate, where Objects with computing devices endeavors to adjusts as per the system condition to boost the correspondence execution and (2) detectable, where Objects with computing devices means to learn, think and perceive alone, and (3) data investigation, where Objects with computing devices information is prepared and broken down to get information that can be utilized for augmenting business. CObjects with computing devices is normally an automatic-educated and automatic-guided framework. It likewise acts different ownership such as contingency, versatile, adaptable, energetic, intelligent, incorporated, continual, stateful, and so forth. Savvy frameworks are difficult to create. In a similar manner, to acknowledge CIoT, different difficulties, for instance, constrained battery, assorted information types, train the framework precisely, social and moral concerns, and so forth are to be dealt with. Subjective AI and CObjects with computing devices can possibly go past essential computerization to convey business advantages, for example, better business examination and choices, expanded business tasks, more consumer loyalty, and expanded incomes. Preferably, the future CObjects with computing devices will most likely make an issue articulation dependent on its gaining from a current issue that it is encountering and will turn out with a most ideal arrangement by applying its AIQ (manmade reasoning remainder) earned through unreal cognizance. While CObjects with computing devices attempt a great deal of guarantees, there is dependably a high probability of sick outcomes when a framework that appertains pretty much every gadget on the planet procures intellectual capacities and do things they are not intended to. The evil dealing with and inaccurate usage of savvy “things” may nullify the relevance of the CIoT. We ought to be extremely cautious in structuring, actualizing, and utilizing CObjects with computing devices to effectively acknowledgment of progressive visualization.

References [1] B. Lydon, Internet of Things: industrial automation industry exploring and implementing IoT, InTech Mag. 2 (2014). [2] P.K.D. Pramanik, S. Pal, P. Choudhury, Beyond automation: the cognitive IoT. Artificial intelligence brings sense to the Internet of Things, Cognitive Computing for Big Data Systems Over IoT, Springer, Cham, 2018, pp. 1 37. [3] Z. Michaelides, Big data for logistics and supply chain management, in: Production and Operations Management Society (POMS) Conference Proceedings in Orlando, FL, 2016. [4] S. Machines, in: John E. Kelly III, Steve Hamm (Eds.), IBM’s Watson and the Era of Cognitive Computing, Columbia University Press, 2013, p. 160.

Further reading

[5] A. Sathi, Cognitive (Internet of) Things: Collaboration to Optimize Action, Springer, 2016. [6] M. Zhang, H. Zhao, R. Zheng, Q. Wu, W. Wei, Cognitive internet of things: concepts and application example, Int. J. Comput. Sci. Issues (IJCSI) 9 (6) (2012) 151. [7] J. Mitola, Q.M. Gerald, Cognitive radio: making software radios more personal, IEEE Pers. Commun. 6 (4) (1999) 13 18. [8] R.W. Thomas, D.H. Friend, L.A. Dasilva, A.B. Mackenzie, Cognitive networks: adaptation and learning to achieve end-to-end performance objectives, IEEE Commun. Mag. 44 (12) (2006) 51 57. [9] M.A. Garrett, Big Data analytics and cognitive computing future opportunities for astronomical research, in: IOP Conference Series: Materials Science and Engineering, vol. 67, no. 1, IOP Publishing, 2014, p. 012017. [10] C. Fortuna, M. Mohorcic, Trends in the development of communication networks: cognitive networks, Comput. Netw. 53 (9) (2009) 1354 1376. [11] Q. Wu, G. Ding, Y. Xu, S. Feng, Z. Du, J. Wang, et al., Cognitive internet of things: a new paradigm beyond connection, IEEE Internet of Things J. 1 (2) (2014) 129 143. [12] W. Li, J. Taheri, A.Y. Zomaya, F. Seredynski, B. Landfeldt, Nature-inspired computing for autonomic wireless sensor networks, Large Scale Netw.-Centric Distrib. Syst. (2013) 219 254. [13] Y. Xu, Recent machine learning applications to internet of things (IoT), in: Recent Advances in Networking, [online]. Available from: ,http://www.cse.wustl.edu/ Bjain/cse570-15/index.html., 2015 (last accessed 03.08.16). [14] J.E. Kelly, Computing, Cognition and the Future of Knowing, 2, Whitepaper, IBM Reseach, 2015. B. Wai-Ling, N. Demeyere, D. Francis, V. Kumar, M. Remoundou, A. Balani, et al., The BCoS cognitive profile screen: utility and predictive value for stroke, Neuropsychology 29 (4) (2015) 638. [15] D. Vernon, Cognitive vision—the development of a discipline, in: Proc. IST 2004 Event “Participate in Your Future”, 2004. [16] B. Sheppard, Warming up to inscrutability: how technology could challenge our concept of law, Univ. Toronto Law J. 68 (Suppl. 1) (2018) 36 62. [17] D.P. Benjamin, D. Lyons, D. Lonsdale, ADAPT: a cognitive architecture for robotics, in: ICCM, 2004, pp. 337 338. [18] A. Sathi, What is a cognitive device? Cognitive (Internet of) Things, Palgrave Macmillan, New York, 2016, pp. 13 27. [19] T. Saber, M. Shafie-Khah, P. Siano, V. Loia, A. Tommasetti, J. Catala˜o, A review of smart cities based on the internet of things concept, Energies 10 (4) (2017) 421. [20] A. Bicchi, M.A. Peshkin, J.E. Colgate, Safety for physical human robot interaction, in: Springer Handbook of Robotics, 2008, pp. 1335 1348.

Further reading G. Banavar, What It Will Take for Us to Trust AI. Available from: ,https.hbr.org/2016/ 11/what-it-will-take-for-us-to-trust-ai., 2016 (accessed 16.02.17). J. Garcia, Machine Learning and Cognitive Systems: The Next Evolution of Enterprise Intelligence (Part I), Wired Innovation Insights, 2014.

253

254

CHAPTER 12 Surveying various effective modes and research trends

A. Ibrahim, A study about using a cognitive agent in replacing level 1 and 2 service desk activities, Third International Congress on Information and Communication Technology, Springer, Singapore, 2019, pp. 307 316. H. Judith, M. Kaufman, A. Bowles, A. Nugent, J.G. Kobielus, M.D. Kowolenko, Cognitive Computing and Big Data Analytics, John Wiley & Sons, Hoboken, NJ, 2015. C. Marie, D. Este`ve, C. Escriba, E. Campo, A review of smart homes—present state and future challenges, Comput. Methods Programs Biomed. 91 (1) (2008) 55 81. Computers helping people with special needs, in: K. Miesenberger, C. Bu¨hler, P. Penaz (Eds.), 15th International Conference, ICCHP 2016, Linz, Austria, July 13 15, 2016, Proceedings, vol. 9758, Springer, 2016. A.K. Noor, Potential of cognitive computing and cognitive systems, Open Eng. 5 (1) (2015) 75 88. Available from: https://doi.org/10.1515/eng-2015-0008. T. Quack, H. Bay, L. Van Gool, Object recognition for the internet of things, The Internet of Things, Springer, Berlin, Heidelberg, 2008, pp. 230 246. D. Schatsky, C. Muraskin, R. Gurumurthy, Cognitive technologies: the real opportunities for business, Deloitte Rev. 16 (2015) 115 129. S. Tarkoma, A. Katasonov, Internet of Things Strategic Research Agenda, Finnish Strategic Centre for Science, Technology and Innovation, 2011. D. Vesset, C.W. Olofson, A. Nadkarni, A. Zaidi, B. McDonough, D. Schubmehl, et al., IDC FutureScape: Worldwide Big Data and Analytics 2016 Predictions, International Data Corporation, 2015. O. Yasumitsu, A. Horibe, K. Matsumoto, T. Aoki, K. Sueoka, S. Kohara, et al., Advanced interconnect technologies in the era of cognitive computing, in: 2016 Pan Pacific Microelectronics Symposium (Pan Pacific), IEEE, 2016, pp. 1 6.

CHAPTER

Time and feature specific sentiment analysis of product reviews

13

Aakanksha Sharaff and Asma Soni Department of Computer Science and Engineering, National Institute of Technology, Raipur, Chhattisgarh, India

13.1 Introduction E-commerce companies usually ask their customers to leave their feedbacks about their products and/or services they have purchased. Such customer reviews are very important for buyers as well as sellers. They help the potential buyers in making decisions on whether the product fulfills their expectations and whether they should purchase the product or not. As for the selling companies, they get feedbacks about their products, what are the expectations of the customers and where they need to make improvements. These reviews highlight the problems and issues related to the product, and the benefits and improvements that attracted the customers. When the customers leave their reviews about the product on a certain e-commerce website, those websites become a great source of firsthand experience, along with being the online retailers. Product reviews help in building trust and loyalty and underline the features that set that specific product apart from others. For the companies, reviews help them to better understand their products. They can give them extensive feedbacks and ideas and areas of improvement. But going through all the reviews and analyzing them as per requirements manually could be a really monotonous and tiresome task. Lots of human errors might creep in, including missing out important data. Manually summarizing the reviews will bring in its own different set of challenges. Such manual summarizing may differ from person to person. Even after one has summarized everything and has come up with an overall idea about the product, it becomes very difficult to analyze how much that review or the overall idea fits in the current scenario, or how useful is the product still according to current trends, advancements, and market demands. Many times, one is not even interested in the whole of the product and, instead, is interested only in some of the features of the product. In such cases it is to be analyzed how good that feature is in the product. Thus sentiment analysis has been employed into this aspect. Sentiment analysis (opinion mining) is concerned with analyzing sentiments, emotions, opinions, Cognitive Informatics, Computer Modelling, and Cognitive Science, Volume 2. DOI: https://doi.org/10.1016/B978-0-12-819445-4.00013-8 © 2020 Elsevier Inc. All rights reserved.

255

256

CHAPTER 13 Time and feature specific sentiment analysis

attitudes, and thought of people from written language. The study/survey of opinion mining has not been confined only to computer science but is now being applied in various other fields also. In this work, feature- and time-specific sentiment analyses of product reviews have been taken up for research. Rather than being concerned about the all features of a product, reviews about the different features of the product have been extracted separately, and then sentiment analysis has been applied on them. This has helped in coming up with an idea about each feature of the product, rather than about the product as a whole. Also, the time specificity of the reviews has been considered, that is, the reviews which were written long back have been given less weightage than those reviews that were written recently. This has been achieved by defining and using an aging factor. The aging factor takes care of the aging of the reviews along with the time. For example, when a certain product was launched, at that time, the product might have been a totally new innovation and might have created a great boom in the market. The users might have all been impressed by the features of the product and would have given all the good reviews. But, as the time goes on, and as new technology advancements take place, it is quite possible that the product might have become totally useless. In such cases, if any prospective buyer reads those old reviews, he will get misguided. Thus it is necessary to have a gradual decrease in the importance of the review along with time. The newer reviews should have more importance than the reviews that were written long back. This aspect has also been dealt in this work. The experiments have been done on the dataset of mobile phone reviews. The rest of the work is organized as follows: an insight to the work already been done in the field of sentiment analysis and opinion mining is presented in Section 13.2. In Section 13.3, a detailed description of each step taken, significance of each step, etc. are discussed. The detailed approaches, techniques, and algorithms used at each step and other setup required have been described in Section 13.6. Section 13.7 highlights all the results obtained during the experimental analysis. The last section concludes the paper and presents further direction of research.

13.2 Related work There has been much work related to sentiment analysis on product reviews. A rating-based prediction technique for identifying new users based on the user attributes has been proposed by Seroussi et al. [1] by using matrix factorization with user attributes (MFUA). This MFUA model uses both the demographic attributes and the attributes deduced from user-generated texts. Gamon [2] experimented upon highly noisy feedback data obtained from the Global Services Survey and have performed sentiment analysis on it. They have performed extensive

13.2 Related work

feature analysis and feature selection by using large initial vectors combined with feature reduction based on log likelihood ratio. Kim and Zhang [3] provided a supervised weighting scheme, called credibility-adjusted term frequency. This adjusts term frequency in Term Frequency Inverse Document Frequency (TF-IDF) to analyze the sentiments. This scheme was robust for both snippets and longer documents. Sathe and Mali [4] proposed a hybrid algorithm for sentiment classification. They combined neural network and fuzzy logic and used Gaussian membership function to fuzzify the input reviews and built a fuzzification matrix. They have then transported this matrix to the multilayer (ML) perceptron back propagation-network. Ray and Chakrabarti [5] combined rule-based methods with deep learning for aspect-level sentiment analysis. ML algorithms such as dependency parsing and parts-of-speech (POS) tagging were employed to identify the opinions of user, and a convolution neural network (CNN) architecture of seven layers deep was built. Dey and Haque [6] used noisy text data for opinion mining. The authors developed a framework in which they first used domain knowledge and cleaned the text and then did opinion mining on it. Then the model was authenticated with knowledge base, and the systems automatically gather more instances for authentication on the basis of training samples provided to it. Then the system analyzes the opinions as expressions containing opinion words, opinion modifiers, and opinionated features. Qiu et al. [7] described a predictive framework for forecasting the ratings of the reviews. They have developed a SentiCRF model, which extracts the aspects of the review and their contexts and maps them to the term pairs collected by the SentiCRF. Araque et al. [8] proposed a sentiment classification model utilizing semantic similarity. This proposed work uses both sentiment models and semantic lexicons. They have compared lexicon vocabulary with text words and built a semantic similarity metric and used this metric as features for sentiment analysis. Tsytsarau and Palpanas [9] designed a theoretical framework which clones sentiment diversity by introducing innovative areas that bag sentiment variance from assembled sentiment census. A potent and expandable indexing and storage method for assorted sentiments has also been developed and proposed a flexible algorithm for identifying conflicts at different time ranges. Potdar et al. [10] come up with a review bot, Samiksha. Samiksha produces a factual summarization of product reviews. This review bot has been defined in four phases. In phase one, it will fetch the user reviews of a particular product. In phase two, it will preprocess the fetched reviews. In phase three, it will analyze the preprocessed reviews on various factors. In phase four, it will generate a factual summarized review of the product. Sun et al. [11] explored eWOW for sentiment classification of product reviews at a fine-grained level. They have used social analytics methods, Apriori algorithms, and sliding window methods at various levels to build a fine-grained sentiment analysis computation method. eWOW is an exploration system, which performs deep analytics for the target products. Liu et al. [12] used intuitionistic fuzzy set theory for product ranking. They have weighed the reviews and formed an intuitionistic fuzzy number using the

257

258

CHAPTER 13 Time and feature specific sentiment analysis

intuitionistic fuzzy weighted averaging (IFWA) operator. Further, the dominance degrees have been calculated and their ranking has been determined using the PROMETHEE II method. Malhotra and Rishi [13] built the intelligent metasearch system for advanced e-commerce (IMSS-AE) using page ranking algorithm and Hadoop MapReduce framework. This IMSS-AE is a website ranking tool in e-commerce and can be used by end users, retailers, and search engine developers. Ahuja et al. [14] did a comparison between TF-IDF and n-grams and measured their impact on sentiment analysis and found that TF-IDF performed better as compared to n-grams. They have also done a comparison among algorithms of sentiment analysis, including KNN, decision tree, support vector machine (SVM), Naı¨ve Bayes, logistic regression, and random forest, and found that logistic regression performed the best among all. Vinodhini and Chandrasekaran did a comparative analysis among probabilistic neural network (PNN), homogeneous ensemble of PNN (HEN), and back-propagation neural network for classifying sentiments [15]. They utilized feature-level sentiment analysis by varying the word granularity. They concluded that HEN outperforms the rest two. Various researchers have also proposed various recommendation system models in their works. Xu et al. [16] proposed topic model
based collaborative filtering. This is a unique personalized recommendation model, which utilizes users’ reviews and ratings. They have generated topic allocations for each review, using extended Latent Dirichlet Allocation (LDA) model, and obtain users’ preferences. They have designed a new metric that measures similarities among users, thus easing the sparsity problems by a large extent. Gao et al. [17] used LDA model to identify topics and built the relationship network. Through the topic relationship, they found a neighbor collection in order to find the largest similarity with target and use collaborative filtering recommendation algorithm. Agarwal et al. [18] developed POS-specific feature-based approach for identifying prior polarity features. They have done feature analysis and revealed that the combination of the prior polarity features with POS-specific features generates important features. Much work has also been done on social network evaluation using texts from social networking sites. Wang et al. [19] proposed SentiView, an interactive visualization system, which analyzes public sentiment in order to find out trending topics on the Internet. The authors have also proposed two new visualization concepts: visualization of varied attributes and identifying complex relationships among them. These are the helix blended with astrolabe and relationship map methods. Jiang et al. [20] investigated social recommendation on the basis of individual preference and interpersonal influence. They blend these two factors in latent spaces using a novel probabilistic matrix factorization method. Gui et al. [21] constructed a heterogeneous network linking products, users, and words with the polarities of the words. A network-embedding method learns the representations of nodes and integrates them into a CNN that does the sentiment classification. Qian et al. [22] introduced personalized recommendation model considering

13.3 Proposed model

personal interest, interpersonal interest similarity, and interpersonal influence. This method improves the accuracy and pertinence of the existing recommender system. Zhao et al. [23] conducted a service quality evaluation mechanism by using the concept of user ratings’ confidence. They have first used entropy to calculate user ratings’ confidence and to constrain their confidence, they have explored spatial
temporal features. Finally, an overall confidence is calculated by fusing them into consolidated model. Some researchers have also used lexicon-based model. Neviarouskaya et al. [24] developed a lexicon-based SentiFul database, which consists of sentimentportraying terms, functional words, modifiers, and modal operators. They have also described the algorithm for automatically extracting the new sentimentrelated compounds from WordNet based on words from SentiFul. Lu et al. [25] proposed a model for self-driven building of a context-aware lexicon. This sentiment lexicon is both domain and context aspect specific in an unlabeled opinionated text collection. Cernian et al. [26] used SentiWordNet lexical resource as a base for developing an approach for sentiment analysis. Through the model, they were able to achieve up to 61% accuracy. Some researchers have also used thesaurus-based approaches. Bollegala et al. [27] proposed a cross-domain sentiment classifier by using sentiment-sensitive thesaurus. Sentiment sensitivity has been achieved by assimilating document-level sentiment labels in the context vectors, which were used for measuring the distributional similarity between words. Mohammad et al. [28] used individual words and multiword expressions to produce a high-coverage semantic orientation lexicon. Yang et al. [29] used SVM with conditional random field learners to construct web-blogs corpora for sentiment analysis. Hussein [30] discussed the effects of challenges in sentiment analysis and concluded that domain orientation and negation are essential factors relevant to the review structure. In another comparison to find out how to improve the results, lexicon-based techniques and POS tagging were the most used, and then bag-of-words.

13.3 Proposed model This research work proposes a model for the feature- and time-specific sentiment analysis of product reviews. The overall methodology of proposed work is shown in Fig. 13.1. First of all, the best classification algorithm for the dataset is found out. For this, random sampling has been used, in which small datasets were extracted from the whole dataset multiple times, and the various classification algorithms were tested upon these dataset sets. The results were then analyzed and the best classification algorithm was determined. This step is of utmost importance to increase the overall accuracy of the model. The classifier that performs the best maximum number of times was chosen to be used for further steps.

259

260

CHAPTER 13 Time and feature specific sentiment analysis

Collect the reviews

Define feature dictionary for the product

Preprocess, tokenize, and vectorize the dataset

Classify the review tokens under each feature

Find the sentiments of the review tokens of each feature

Multiply each of the sentiment score with the aging factor to get the sentiment score

Sum up the results for each feature

Visualize the results as per needs

FIGURE 13.1 Overall research plan.

13.4 Need of feature specificity

Next, a feature dictionary is constructed for the collection of product features based on review comments. The defined feature dictionary consists of all the features to be analyzed about the product and its related attributes. Such a dictionary will differ according to the product for which the model is being used. For example, this work uses the dataset of mobile phones. Thus the defined feature dictionary contains features such as charger, battery, camera, buttons, light, system, display, hardware, connections, headphone, and processor. If the company provides other services too, other than the product features, such as home delivery services, such features can also be included while defining the feature dictionary. This dictionary, more or the less, was defined manually. Then preprocessing techniques, namely, the tokenizing and vectorizing steps were performed on the dataset, as per the needs. These steps are necessary for improving the accuracy of the model. Next, the reviews about the defined features were extracted from the dataset and were classified into having positive or negative or neutral sentiment opinion, using the best performing classification algorithm. Each of these classified reviews terms was then multiplied by the aging factor. The aging factor is calculated using the following formula: f5

tR 2 tL tC 2 tL

where f is the aging factor, tR is the time stamp of writing the review, tL is the time stamp of product launch, tC is the current time stamp. To evaluate the parameters of the formulae, tR is taken from the dataset, tL is known for the product, and tC is calculated as a function of the current time. The sentiment polarity that was found out using the classification algorithm is then multiplied with the aging factor to get the sentiment score, using the formula: ssr 5 f Up

where ssr is the sentiment score of the review, f is the aging factor of the review, and p is the polarity of the review. The sentiment scores for each review term of each feature are then added up, thus giving the sentiment score of the particular feature of the product, ssf. ssf 5 Σ ssr

13.4 Need of feature specificity Feature specificity is required when one is not interested in the product as a whole, but rather, is interested on only one or few of the features of the product. For example, in the case of mobile phones, a photographer or a selfie-addict might be greatly interested in the camera, while being less concerned about the

261

262

CHAPTER 13 Time and feature specific sentiment analysis

volume or the display feature. Similarly, a singer would be interested in the volume of the phone, so that they could practice their vocals with the phone itself. A student would be interested in the earphones, so that they could listen music while studying. Some general features such as long battery are generally desirable by all. Such demands give rise to a need to extract the sentiments from distinct features of the product, rather than the complete product as one. In order to obtain feature-specific sentiment analysis of product reviews, a feature dictionary is defined. This dictionary contains a list of all the features of the product and their attributes. The reviews will be searched for these features and their sentiments will be analyzed. Such a dictionary can be defined manually as per needs and will differ from product to product. In this work, experiments have been done on a mobile phone
review dataset. The feature dictionary contained the following features: charger, battery, camera, processor, light, system, display, hardware, headphone, back, button, box, home, and connection. The reviews were tagged by using unigram, bigram, trigram, and n-gram taggers, and the tagged terms were scanned for these features. These tagged terms were then finally analyzed for the sentiments, and the observed result was presented.

13.5 The aging factor An aging factor has been included in the proposed model. This aging factor has been used to scale the weight of the review as per the timescale. The products or the features that were new or revolutionary at the time when the review was written might not be useful at the time of analysis, as a result of unceasing development and ever-changing requirements. The reviews that were written long back should have less privilege than those that were written recently. Such time-dependent validity of the reviews is incorporated in the model by weighting them based on the time stamp when they were written by using an aging factor. In the model, a linear formula for the aging factor has been proposed. It can be given as follows: f5

tR 2 tL tC 2 tL

where f is the aging factor, tR is the time stamp of writing the review, tL is the time stamp of product launch, and tC is the current time stamp. Furthermore, the sentiment score of the review ssr will be given by ssr 5 f Up

where f is the aging factor of the review and p is the polarity of the review. Finally, the sentiment score of the particular feature of the product, ssf, is obtained by adding up all the sentiment scores of its reviews: ssf 5 Σ ssr

13.5 The aging factor

For example, some product was launched in the year 2000. Two reviews are considered, one written in 2001 and the other in 2015. Current time stamp is 2018. Let us analyze the aging factor for these two reviews: For review written in 2001, f2001 5

2001 2 2000 5 0:056 2018 2 2000

For the review written in 2015, f2015 5

2015 2 2000 5 0:833 2018 2 2000

where f2001 is the aging factor of the product launched in 2001 and f2015 is the aging factor of the product launched in 2015. Thus, it is observed that the review written in 2001 will have less weightage in the summation, as compared to the review written in 2015. Aging factor can also be used to analyze how trendy the product is in the current market. For example, if it is analyzed how many reviews written in 2001 will have a weightage equal to a review written in 2015, n  f2001 5 f2015

where n is the number of reviews. n  0:056 5 0:833 n 5 15

Thus 15 reviews written in 2001 will have a total weightage equal to a review written in 2015. The aging factor can also be used to compare reviews of two products written in different time stamps. For example, a product released in 2000 has five reviews written in 2001 and one review written in 2007. Another product released in 2005 has two reviews written in 2006 and one written in 2010. Comparing their scores, ss2000 5 5 

2001 2 2000 2007 2 2000 11  2018 2 2000 2018 2 2000

5 5  0:056 1 1  0:389 5 0:669 ss2005 5 2 

2006 2 2005 2010 2 2005 11  2018 2 2005 2018 2 2005

5 2  0:076 1 1  0:385 5 0:537

where ss2000 is the sentiment score of the product launched in 2000, and ss2005 is the sentiment score of the product launched in 2005. Thus the product released in 2000 has better score than the product released in 2005, as they have more positive reviews than the product launched in 2015. Similarly, the aging factor can also be used for accessing the validity of the product in present time. Like, in the earlier examples, it is seen that for a product

263

264

CHAPTER 13 Time and feature specific sentiment analysis

launched in 2000, 15 reviews written in 2001 had a weightage equal to the 1 review written in 2015. Now if the one review written in 2015 was a negative review; in such a case, the results would differ as ss2015 5 15 

2001 2 2000 2015 2 2000  11  ð 2 1Þ 2018 2 2000 2018 2 2000

5 15  0:056  1 1 0:833  ð 21Þ 5 0

Thus the sentiment score of the product is 0. In these ways the aging factor plays an important role in deciding the validity and importance of the reviews while doing the sentiment analysis.

13.6 Experimental setup 13.6.1 Collection and preparing of dataset Mobile reviews dataset has been taken from Kaggle. This dataset has three fields: the serial number, the review text, and the date when the review was written. This dataset had around 1887 reviews. The data has been manually differentiated into positive/neutral/negative class, as per their polarity, and form the training and testing data. The train and test dataset has two fields: the review text and the manually classified polarity value (i.e., 21, 0, or 11). Three values were given for the sentiments: (21) for negative opinions, (0) for neutral opinion, and (11) for positive opinions.

13.6.2 Define feature dictionary for product A dictionary of features has been constructed from the reviews obtained from the product. This dictionary contains the features of the mobile phone or the product that is being analyzed. The following features of the mobile phones have been used in this work: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14.

charger battery camera processor light system display hardware headphone back button box home connection

13.6 Experimental setup

13.6.3 Preprocess, tokenize, and vectorize the dataset Preprocessing is an umbrella term that covers all the operations that are used to convert the data in a more appropriate form, as per the needs. The proposed model uses Porter stemmer as stemming algorithm for preprocessing. The terms have been tagged using a combined tagger technique for POS tagging. This way addresses the trade-off between accuracy and coverage. The work combines the results of a trigram tagger, a bigram tagger, a unigram tagger, and a default tagger for better performance. The model also uses some forced tags wherever it was necessary in order to improve the overall performance of the model. TF-IDF vectorization has been implemented with a word analyzer, lowercase as True, min_df 1, ngram_range (1, 3), English stop_words, and max_features 1000.

13.6.4 Classify the review tokens under the features in the feature dictionary The tagged review terms have been classified under each feature in the feature dictionary. The tagged review term has been scanned and it has been searched if the review had the mention of the feature it is being scanned for. The reviews that contained the feature term have been then added to the dictionary. Finally, the obtained review dictionary has been then converted into a Pandas DataFrame.

13.6.5 Find the sentiments of the review tokens for each feature The next step was to find out the sentiments of the review tokens for each feature. For this step, various machine-learning classification algorithms were applied on the initial dataset. Initial dataset was tested by the SVM, the maximum entropy, the Naı¨ve Bayes, and the random forest. For training and testing, the reviews were manually classified into neutral, positive, or negative, and then the dataset was fed into the classifiers. As per the results obtained, the random forest model showed the best accuracy, at 0.859. Second was the SVM, which gave an accuracy of 0.851 as shown in Fig. 13.2. Naı¨ve Bayes and maximum entropy gave accuracies of 0.7745 and 0.712, respectively. Random forest also gave the highest precision of 0.902 shown in Fig. 13.3 followed by SVM, which gave a precision of 0.850. Recall was highest for maximum entropy and SVM shown in Fig. 13.4, which gave recall values of 0.720 each, followed by Naı¨ve Bayes at 0.746 and random forest at 0.690. F-Measure was the highest for random forest at 0.782 shown in Fig. 13.5, followed by Naı¨ve Bayes, at 0.727. Thus it was concluded that random forest has given the best results shown in Fig. 13.6 as per our requirements, and thus was chosen for the further steps. The graphs comparing the various measures has been plotted in Figs. 13.2
13.5. A combined graph has been plotted in Fig. 13.6.

265

266

CHAPTER 13 Time and feature specific sentiment analysis

FIGURE 13.2 Comparing accuracy of different classifiers.

FIGURE 13.3 Comparison of precision values.

FIGURE 13.4 Comparison of recall values.

13.6 Experimental setup

FIGURE 13.5 Comparison of F-measure values.

FIGURE 13.6 Comparison of classifiers.

The random forest has been used as classifier and the predictions were performed by using n-estimators. These n-estimators represent the count of trees in the forest. The number of n-estimators to be used depends precisely on the data. Using more estimators increases the accuracy but also increases the processing time. For accuracy too, it increases up to a certain plateau, after which 2000 or 10,000 estimators do not make a difference, and the accuracy may even show a dip.

13.6.6 Multiply the polarity with the aging factor to get the sentiment score of the review term The polarity of the review is then multiplied by the aging factor to obtain a sentiment score of the review term. The third column of the dataset gives the time stamp when the review was written. The time stamp of product launching was known. The current time stamp is the time stamp when the algorithm is being

267

268

CHAPTER 13 Time and feature specific sentiment analysis

run, or when the experiment is being conducted. The aging factor was calculated using the previous formula, that is, f5

tR 2 tL tC 2 tL

where f is the aging factor, tR is the time stamp of writing the review, tL is the time stamp of product launch, and tC is the current time stamp. The sentiment score of the review ssr was then evaluated using the formula, ssr 5 f Up

where f is the aging factor of the review and p is the polarity of the review.

13.6.7 Sum up the results for each feature Finally, the sentiment score of the particular feature of the product, ssf is obtained by adding up all the sentiment scores of its reviews. ssf 5 Σ ssr

13.6.8 Visualize the results In this work, based on features, the classification of reviews has been done, and result is shown in Table 13.1. In the graph, the X-axis represents the product features and the Y-axis sentiment score of the specific features. Table 13.1 Results of feature classification. Aspect

Negative sentiment score

Neutral sentiment score

Positive sentiment score

Button Back Battery Camera Charger Connection Hardware System Display Processor Box Light Home Headphone

22.000 3.833 56.389 24.889 5.833 17.111 0.500 17.111 0.500 7.778 7.222 0.000 3.556 0.000

10.000 5.167 0.278 30.722 4.500 0.000 4.500 0.000 4.500 6.222 13.889 0.000 6.667 0.667

6.000 1.500 13.333 31.111 0.167 0.778 2.000 12.444 8.889 0.000 4.000 27.556 3.667 0.000

13.7 Result and discussion

13.7 Result and discussion Once the dataset has been gathered, adequate preprocessing has been done to make it good for the further steps. In preprocessing, tokens have been identified from the reviews by using POS tagging, and Porter stemmer has been used for performing stemming operations. The tagged review terms were then classified for each feature. A series of classification algorithms has been applied to find the polarity of review comments. This polarity was then multiplied by the aging factor to find the sentiment score of the review. Also, the time specificity of the reviews has been considered, that is, the reviews that were written long back have been given less weightage than those reviews that were written recently. This has been achieved by defining and using an aging factor. Aging factor takes care of the aging of the reviews along with the time. The aging factor was calculated using a linear function of the time stamp of writing of the review, the time stamp of launch of the product, and the current time stamp. Finally, the sentiment scores of all the reviews related to the particular feature were added up to get an overall idea about the feature shown in Fig. 13.7. The results can be used as per the needs. In this work, the results have been visualized via a graph, showing the sentiment score of the features of the product.

FIGURE 13.7 Sentiment scores of features.

269

270

CHAPTER 13 Time and feature specific sentiment analysis

13.8 Conclusion and future work A model for feature-specific and time-specific sentiment analysis of product reviews has been proposed. This model takes in the reviews about the product, the time stamps of putting those reviews, and a feature dictionary and produces an overall sentiment analysis result of the product feature- and time specifically. An aging factor has been introduced to scale the importance of the review as per the time. The feature specificity of the model will be very useful in the cases where the product as a whole is not that important, rather, only about some of its features are important and are to be analyzed. In order to evaluate the usefulness of the product, time specificity of the model needs to be analyzed. This research can be boosted further by extracting the features automatically rather than creating feature dictionary. The proposed model is product specific. This research can be extended to make a model that is product independent. This research work can also be extended to include a reviewer-nature factor. This will hold the regular character of the reviewer. For example, there may be certain happy-easy-go reviewers who are usually satisfied by the products. On the other hand, there might be some critics, who will always ask for more improvements. The reviewer-nature factor will handle all such variations.

References [1] Y. Seroussi, F. Bohnert, I. Zukerman, Personalised rating prediction for new users using latent factor models, Proceedings of the 22nd ACM Conference on Hypertext and Hypermedia, ACM, 2011. [2] M. Gamon, Sentiment classification on customer feedback data: noisy data, large feature vectors, and the role of linguistic analysis, Proceedings of the 20th International Conference on Computational Linguistics, Association for Computational Linguistics, 2004. [3] Y. Kim, O. Zhang, Credibility adjusted term frequency: a supervised term weighting scheme for sentiment analysis and text classification, in: Proceedings of the Fifth Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, 2014, pp. 79—83, arXiv Prepr. arXiv:1405.3518. [4] J.B. Sathe, M.P. Mali, A hybrid sentiment classification method using neural network and fuzzy logic, Intelligent Systems and Control (ISCO), 2017 11th International Conference on, IEEE, 2017. [5] P. Ray, A. Chakrabarti, A mixed approach of deep learning method and rule-based method to improve aspect level sentiment analysis, Appl. Comput. Inform. (2019). [6] L. Dey, S.K.M. Haque, Opinion mining from noisy text data, Int. J. Doc. Anal. Recogn. (IJDAR) 12 (3) (2009) 205
226. [7] J. Qiu, et al., Leveraging sentiment analysis at the aspects level to predict ratings of reviews, Inf. Sci. 451 (2018) 295
309. [8] O. Araque, G. Zhu, C.A. Iglesias, A semantic similarity-based perspective of affect lexicons for sentiment analysis, Knowl.-Based Syst. 165 (2019) 346
359.

References

[9] M. Tsytsarau, T. Palpanas, Managing diverse sentiments at large scale, IEEE Trans. Knowl. Data Eng. 28 (11) (2016) 3028
3040. [10] A. Potdar, et al., SAMIKSHA—sentiment based product review analysis system, Procedia Comput. Sci. 78 (2016) 513
520. [11] Q. Sun, et al., Exploring eWOM in online customer reviews: sentiment analysis at a fine-grained level, Eng. Appl. Artif. Intell. 81 (2019) 68
78. [12] Y. Liu, J.-W. Bi, Z.-P. Fan, Ranking products through online reviews: a method based on sentiment analysis technique and intuitionistic fuzzy set theory, Inf. Fusion 36 (2017) 149
161. [13] D. Malhotra, O.P.O.P. Rishi, An intelligent approach to design of E-commerce metasearch and ranking system using next-generation big data analytics, J. King Saud Univ.
Comput. Inf. Sci. (2018). [14] R. Ahuja, et al., The impact of features extraction on the sentiment analysis, Procedia Comput. Sci. 152 (2019) 341
348. [15] G.G. Vinodhini, R.M.R.M. Chandrasekaran, A comparative performance evaluation of neural network based approach for sentiment classification of online reviews, J. King Saud Univ.
Comput. Inf. Sci. 28 (1) (2016) 2
12. [16] J. Xu, X. Zheng, W. Ding, Personalized recommendation based on reviews and ratings alleviating the sparsity problem of collaborative filtering, in: e-Business Engineering (ICEBE), 2012 IEEE Ninth International Conference on, IEEE, 2012. [17] S. Gao, et al., Review expert collaborative recommendation algorithm based on topic relationship, IEEE/CAA J. Autom. Sin. 2 (4) (2015) 403
411. [18] A. Agarwal, et al., Sentiment analysis of twitter data, in: Proceedings of the Workshop on Languages in Social Media, Association for Computational Linguistics, 2011. [19] C. Wang, et al., SentiView: sentiment analysis and visualization for internet popular topics, IEEE Trans. Hum.
Mach. Syst. 43 (6) (2013) 620
630. [20] M. Jiang, et al., Social contextual recommendation, Proceedings of the 21st ACM International Conference on Information and Knowledge Management, ACM, 2012. [21] L. Gui, et al., Learning representations from heterogeneous network for sentiment classification of product reviews, Knowl.-Based Syst. 124 (2017) 34
45. [22] X. Qian, et al., Personalized recommendation combining user interest and social circle, IEEE Trans. Knowl. Data Eng. 26 (7) (2014) 1763
1777. [23] G. Zhao, et al., Service quality evaluation by exploring social users’ contextual information, IEEE Trans. Knowl. Data Eng. 28 (12) (2016) 3382
3394. [24] A. Neviarouskaya, H. Prendinger, M. Ishizuka, SentiFul: a lexicon for sentiment analysis, IEEE Trans. Affective Comput. 2 (1) (2011) 22
36. [25] Y. Lu, et al., Automatic construction of a context-aware sentiment lexicon: an optimization approach, Proceedings of the 20th International Conference on World Wide Web, ACM, 2011. [26] A. Cernian, V. Sgarciu, B. Martin, Sentiment analysis from product reviews using SentiWordNet as lexical resource, Electronics, Computers and Artificial Intelligence (ECAI), 2015 Seventh International Conference on, IEEE, 2015. [27] D. Bollegala, D. Weir, J. Carroll, Cross-domain sentiment classification using a sentiment sensitive thesaurus, IEEE Trans. Knowl. Data Eng. 25 (8) (2013) 1719
1731.

271

272

CHAPTER 13 Time and feature specific sentiment analysis

[28] S. Mohammad, C. Dunne, B. Dorr, Generating high-coverage semantic orientation lexicons from overtly marked words and a thesaurus, in: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, 2009. [29] C. Yang, K.H.-Y. Lin, H.-H. Chen, Emotion classification using web blog corpora, Web Intelligence, IEEE/WIC/ACM International Conference on, IEEE, 2007. [30] D.M.E.-D.M. Hussein, A survey on sentiment analysis challenges, J. King Saud Univ.
Eng. Sci. 30 (4) (2018) 330
338.

CHAPTER

Language learnability analysis of Hindi: a comparison with ideal and constrained learning approaches

14

Sandeep Saini1 and Vineet Sahula2 1

Department of Electronics and Communication Engineering, Myanmar Institute of Information Technology, Mandalay, Myanmar 2 Department of Electronics and Communication Engineering, Malaviya National Institute of Technology, Jaipur, India

Glossary NMT SMT SLA

neural machine translation statistical machine translation second language acquisition

14.1 Introduction The human brain has an innate power to hear sound signals in the environment and to get meaningful words from these speech signals. These words, when spoken or written with predefined rules (grammar), make a language. Infants, in their early stage of life, start grabbing the different sounds being produced in the environment around them [1]. These sounds are human-generated language words as well as different kinds of noises present in the environment. Infant’s brain has to differentiate between useful and nonuseful sounds to build its vocabulary in native language [2,3]. When we discuss specifically the native language learning, then parents or guardians are the principal trainers for most of the infants. They speak words to get the attention of an infant or direct him or her to do some actions. Infant’s brain has to divide these speech signals into meaningful words to trigger the actions from different parts of the body, to respond properly to these speech signals. This creates several questions for linguistics and psychologists. How does speech segmentation take place inside an infant’s brain? How does an infant’s brain decode these sounds and the brain processes these signals to trigger Cognitive Informatics, Computer Modelling, and Cognitive Science, Volume 2. DOI: https://doi.org/10.1016/B978-0-12-819445-4.00014-X © 2020 Elsevier Inc. All rights reserved.

273

274

CHAPTER 14 Language learnability analysis of Hindi

actions in different parts of the body? Which words or phrases are learned faster or better by the infants? Can he or she learn all the words with equal easiness? Can any child learn any language? All these questions have been the topics of research for linguistics and recently computer scientists as well. There has been a lot of research on each of the topics in the computational world as well. Speech segmentation, language acquisition models, probabilistic learners, and universal learners are the fields to support the theory presented by linguistics. In this work, we would focus on evaluating language learning strategies in Indian languages, specifically in Hindi. Hindi has more than 300 million native speakers, and more than 800 million people have one or more form of knowledge about the language [4]. Hindi is mostly spoken in the northern part of India and mostly understood in the majority of the Indian territory. Hindi finds its presence in countries having Indian immigrants as well. We have evaluated the learnability of Hindi based on the learner’s recall and precision skills. Pearl et al. [5] have explored and evaluated the language learning strategies in seven languages, namely, English, German, Spanish, Italian, Farsi, Hungarian, and Japanese. Learnability of any language is measured in terms of F-score. Precision and recall capability of the learner model provides the value of F-score. In this analysis, Fscore is a measure of precision and recall ability of word segmentation process, which shows how accurately the words are comprehended and learned by the learner. Section 14.5 describes the F-score for further details.

14.2 Language acquisition theories In this section, we have discussed a few of the major language acquisition theories and strategies proposed in the literature. These theories are initially developed based on the observation of a group of infants under supervised environment. Hyams [6] state that “The supreme issue in linguistic theory is to explain how a child can acquire any human language.” An infant, coming from any social or economic background of the world, shows almost the same skills to sit, grip, walk, and talk. There are no serious differences in acquiring these skills in the early stages of human life. It is observed that the style of sitting, walking, grabbing, and displaying emotions is almost the same in any infant across the globe. Age is almost the same for the development of all these capabilities among infants. While the only differentiating feature among the initial stages of life is the acquisition of a native language. This process of language learning is a continuous process in which a person goes throughout his life. We focus on the early stages of life in the process of language acquisition. At this stage, we can divide the language acquisition process into two segments, namely, the perception of the language and production of the language. Taha [7] and Felser and Drummer [8] have also worked on the language learnability for native and nonnative languages.

14.2 Language acquisition theories

The process of language comprehension in the infant’s brain can be divided into three processes:

• Segments: In this process, the infant brain can grab some of the vowel and

• •

consonant sounds in the age of 5
7 months. From 9 to 11 months, the infant develops phonetic skills and the ability to distinguish between different sounds. The brain develops to filter out nonnative language sounds from 12 months onward. Super segments: At this stage, infants can learn the syllables that are stressed more by the source (parents in most of the cases). Lexicons: In this process, infants identify proper names in the age group of 4
6 months. They develop the capability to segment words from the sentences in the age group of 9
12 months. A typical infant is having a memory of around 100 words by the age of 15 months, and it grows to 200 words by 18 months.

The second important aspect of language learning is the generation of language. Children try to reproduce the sounds that they hear. In the initial stages from 3 to 5 months, there is an onset of babbling. In this stage, infants show the first manifestation of phonology. They can recognize and respond to particular sounds. In the initial stage, only vowel-related sounds are generated and then slowly some consonants are spoken by the age of 8 months. From 10 months onward, the child starts speaking (generating) native language words. Most of these words comprise one or two syllables. These may not be proper words belonging to the dictionary of the language, but somewhat close to the actual words. This stage is also critical to examine language generation process. After regular rectification from the parents, the child produces meaningful words from 12 months onward. Later infants build a vocabulary of the native language and are able to interact with short answers. A summary of all the stages in perception about the language and generation process is shown in Fig. 14.1. Macwhinney [9] suggests that the principle of contrast plays an important role in language acquisition in children. The principle of Contrast states that any difference in the form of a language marks a difference in meaning. The term dog, for instance, which differs in form from the horse, also differs from it in meaning. Every two forms contrast in meaning. This principle is a general one for speakers of a language. It is one that has been stated or assumed by virtually every linguist over the years. Harris [10] focuses on classroom-based language learning. He compared the existing strategies [11
13] used by the instructors for better language acquisition. He described some of the dilemmas faced in designing a handbook for strategy instructors aimed at distance learners in a range of countries. This strategy works for a different level of competence of learners and a large variety of language. Chamot [14] raises her concerns about issues in language learning strategy research and teaching. In her extensive study, she compares three models for language learning strategy instruction. All three models SSBI Model [15], CALLA

275

Perception about the language

CHAPTER 14 Language learnability analysis of Hindi

9–11 Segmentation of words

4–6 Proper name

Lexicons

Super segments

6–9 Segmentation in IP

2–5 Language prosody

4–5

Segments

9–11 Stress words

5–7 Consonents

Vowels

12–14 100 Photowords

9–11 Phonotactics

12–15 Loss of non– native segments

Age in months

0 Language production

276

1

2

3

4

3–5 Vowels

5

6

7

8

9

7–9 Canonical babbling

10

11

10–12 Language specific sounds

12

13

14

15

12–16 Meaningful words

FIGURE 14.1 Various stages in infant life to understand and generate the native language.

Model [16], and Grenfell and Harris model [17] are developed for students in the early stages of their school life. All three models involve identifying the student’s current learning strategies. These are identified through activities such as engaging in discussions about familiar tasks and completing questionnaires. Conclusions from these models suggested that the instructor should adopt new strategies for different languages. Recently, the research on Indian language translation and learnability has started in some major research centers. Balyan and Chatterjee [18] have worked on a hybrid machine translation approach. They have worked on the integration of linguistics and statistics. Another significant work reported by Sakti et al. [19]. This paper also presented speech-translation results, including subjective evaluation, from the first A-STAR field testing, which was carried out in July 2009. Budwig et al. [20] have worked on the acquisition of early verb constructions in Hindi. They have experimented with children in the age group of 27
36 months and studied how verbs are acquired by Hindi speakers. Most of these strategies are proposed and developed by linguistics or psychologists working closely with language learnability. With the evolution of computers, Natural language processing became one of the most researched topics, and there have been various ways of machine translation [21] and simplified approaches for better machine translation [22]. Research groups are working toward Natural language programing. To achieve this, language translation from natural language to machine language is necessary. Therefore a lot of work is underway to map the human language learning strategies with machine learning

14.3 Evaluation models

strategies to develop the systems that would process the instructions in a way our brain in processing. In the next section, we provide an overview of some computational models and methods to evaluate language learning principles adopted by the infant’s brain.

14.3 Evaluation models Language learning strategies mentioned in the above section are proposed and developed, with the assumption that they are universal in nature and would work in multiple languages. These strategies are mainly implied to human beings in some language learning centers. The results are generated from data obtained from these evaluations. These evaluation results are dependent on human data, and the whole process takes a lot of time and efforts in setting up the experiment and evaluation of the results. In this section, we explore some of the computational models and methods that can be used to evaluate language learning and acquisition capabilities of a person. We explore these language learning strategies and how they work in different languages. Most of these strategies are tested on a single language, and very rarely, cross-language analysis is performed. Demonstration of the crosslinguistic success of the strategies is very important in the infant stages of human life most of the theories presumed that the human being has innate properties to acquire any language as a newborn infant. Most of the computational models to evaluate language learning strategies are Bayesian theory based. The Bayesian theory provides a very robust probability
based learning approach based on the existing belief of the learner and the incoming inputs. The complete process involves generating data from the childdirected speech and gets the inference from this data about the learnability of a particular language based on the proposed model. The speech signal is a continuous signal with a lot of overlapping frequencies in a tight bandwidth. This signal for a complete sentence should be synthesized and segmented into words before we start the processing. Speech segmentation can be performed by Bayesian segmentation approaches explained later.

14.3.1 Bayesian segmentation We have investigated one version of the Bayesian segmentation strategy that is universal in nature and thus has language-independent properties. The sound generated in any language comprises a morpheme. A morpheme is the smallest possible grammatical unit in a language. In other words, it is the smallest meaningful unit of a language. These are common for most of the languages. The reason for choosing Bayesian segmentation is its ability to adapt to an incremental gain in the belief and self-updating of new values of existing belief. This model would be

277

278

CHAPTER 14 Language learnability analysis of Hindi

best suited for our target audience as well, which are infants in the age group of 6
12 months. At this age, the child is not having any prior knowledge of the language, and he learns from the sounds being produced around him. This information is combined using Bayesian theorem (14.1) to generate the updated beliefs of the learner [the posterior: P(h|d)]. Pðhd ÞαPðdhÞ  PðhÞ

(14.1)

The Bayesian segmentation strategy was originally described by Goldwater et al. [23]. Considering the constraint that infants possess limited knowledge of language structure at the relevant age, GGJ described two simple generative models: 1. a Unigram assumption 2. a Bigram assumption Unigram model assumes independence between different words and thus effectively believes that word tokens are randomly selected. To encode this assumption in the model, GGJ assumes that the observed sequence of words w1. . .wn is generated sequentially using a probabilistic generative process. To choose the identity of an ith word in the Unigram case, Goldwater et al. use the following equations:
ni21 ðwi Þ 1 αP0 wi P wi jw1 . . .wi21 5 i211α

(14.2)

P0 5 Pðw 5 xi . . .xm Þ 5 Lj Pxj

(14.3)

P0 is a base distribution specifying the probability that a novel word will consist of particular units x1. . .xm. ni21(wi) is the number of times word wi appears in the previous i 2 1 words, α is a free parameter of the model that encodes how likely the novel word is to be generated. P0 provides the model with a preference for shorter words. This can be understood from the fact that the total probability depends on the number of units (syllables) in a word. More units in a word would reduce the probability of choosing that particular word. Thus a word with smaller length will be the better learned. Since children are more familiar and comfortable with smaller words, hence learnability is better. Therefore this method can be useful for language learning at earlier stages of human life. Bigram model is slightly complex in its basic structure. It does not assume independence of consecutive words, but it assumes that consecutive words are related to each other, and if they appear once, then there is a higher probability that they will appear again in the corpus. Thus a word is generated based on the identity of the word that immediately precedes it.
ni21 ðw0 ; wi Þ 1 βP1 wi 0 P wi jw1 5w ; w1 . . .wi22 5 ni22 ðw0 Þi 1 β

(14.4)

14.3 Evaluation models

Pi ðwi Þ 5

bi21 ðwi Þ 1 γP0 wi b211γ

(14.5)

where ni21(w0 , wi) defines the number of times a particular Bigram (w0 , wi) has occurred in the first i 2 1 words, ni22(w0) provides the information on the number of times the word w0 occurs in the first i 2 2 words, bi21(wi) is the number of Bigram types that contain wi as the second word, b is the total number of Bigram types that have existed before. P0 was already defined in Eq. (14.3), and β and γ are free model parameters. The roles of β and γ are very similar to the role of α in the Unigram model. They control the introduced bias toward fewer Biagrams (β) toward fewer unique lexical items as the second word in a Bigram (γ). It is evident from the above equations and definitions that both Unigram and Bigram models prefer smaller lexicons because those words appear more frequently. A learner designed using any of these models would infer based on the data and model parameters, which words appear more in the corpus and showing better learnability for smaller words.

14.3.2 Bayesian inference To infer the learnability of a language based on the data generated from the child-oriented speech, different types of learners are designed. We have embedded Bayesian learners in our study. There are two categories of Bayesian learners adopted for this study, which are explained later [24]. Ideal learner processes data in a batch, and thus it assumes that there is a perfect memory available for the learner. This learner has enough processing resources to search for potential segmentations exhaustively. After an exhaustive search, it selects optimal segmentation. This learner is heavily relying on the availability of a huge corpus and works well if the learner has prior knowledge of the language. BatchOpt is the notation used for this ideal learner. Constraint learner In the practical world, infinite memory (corpus) is not available, and infants are also not having prior knowledge of the language. Thus a lot of constraints can be put on the learner as well. Three major types of constraint learners implemented in this work are as follows: 1. Online optimal: OnlineOpt. This learner processes data incrementally. The learner has enough processing resources to search for potential segmentations exhaustively, and it selects optimal segmentation. 2. Online suboptimal: OnlineSubOpt. This constraint learner also processes data incrementally and has enough processing resources to search for potential segmentations exhaustively. It differs from the OnlineOpt learner in its segmentation selection mechanism, and it selects segmentation probabilistically.

279

280

CHAPTER 14 Language learnability analysis of Hindi

Table 14.1 Summary of Bayesian learners. Learner

BatchOpt OnlineOpt OnlinSubOpt OnlineMem

Parameters

Iterations 5 20,000 NA NA Utterance 5 20,000

Learner assumptions Online processing No Yes Yes Yes

Suboptimal decisions No No Yes No

Recency effects No No No Yes

3. Online limited working memory: OnlineMem. This learner also processes data incrementally like both constraint learners. But it has a limited working memory, so it cannot do an exhaustive search. This learner focuses more on recent data (recency bias). It also selects optimal segmentation. A summary of the properties of all ideal and constraint learners is compiled in Table 14.1.

14.4 Data preparation for learnability analysis We have analyzed language learning models in Hindi. We have considered two types of dataset for this analysis. Initially, we have tested the evaluation models on speech dataset as analyzed by Pearl et al. [5]. Later we have also considered text corpus as well. Speech and text corpus for Hindi are obtained from the following datasets: 1. Electro Medical and Speech Technology (EMST) Laboratory, Department of Electronics and Electrical Engineering at the Indian Institute of Technology Guwahati [25]. This data is collected in four phases, namely, IITG MV Phases I
IV. The IITG-MV Phase-I dataset is collected from 100 subjects over two sessions in an office environment involving multiple sensors, multiple languages, and different speaking styles. We have used this phase only for our experimentation. 2. Hindi Speech Corpus from Technology Development for Indian Languages (TDIL): TDIL Programme, India [26]. 3. English
Hindi parallel corpus from the Institute for Language, Cognition, and Computation, the University of Edinburgh [27]. 4. Institute of Formal and Applied Linguistics (UFAL) at the Computer Science School, Faculty of Mathematics and Physics, Charles University in Prague, Czech Republic [28]. 5. Center for Indian Language Technology (CFILT), IIT Bombay [29].

14.4 Data preparation for learnability analysis

Table 14.2 Details of Electro Medical and Speech Technology (EMST), Technology Development for Indian Languages (TDIL), (ILCC), Institute of Formal and Applied Linguistics (UFAL), and Center for Indian Language Technology (CFILT) Hindi datasets. Number of sentences Number of words Number of unique words

EMST

TDIL

ILCC

UFAL

CFILT

945 3675 1342

7885 48,297 1673

41,396 245,675 8342

237,885 1048,297 21,673

1492,827 20,601,012 250,619

Hindi speech corpus

Speech to text conversion using HTK toolkit

Hindi text corpus

Transliteration

Transliteration and syllabification

Transliteration and phonemization

F score calculations using words as units

F score calculations as syllables as units

F score calculations using phonemes as units

FIGURE 14.2 Preprocessing and evaluation of different kinds of dataset.

All these datasets are exhaustive with an abundant variety of words Table 14.2 provides information regarding the number of words and sentences in each dataset. We have preprocessed and tested the evaluation model on two categories of datasets in different ways. The experimental setup and process are explained in Fig. 14.2. Speech data is initially processed through Hidden Markov Model Toolkit [30]. The toolkit provides satisfactory results for Hindi as well [31]. The resultant

281

282

CHAPTER 14 Language learnability analysis of Hindi

words are stored in text files. Once we have text corpus, then we have considered three different approaches to evaluate the learnability of the language. We have assumed the following assumptions: 1. Words are basic units of language, and learnability analysis is performed on the text corpus directly by considering Unigram and Bigram models for text corpus. 2. Syllables are considered as basic units of language acquisition. For this experiment, we have converted words into syllables, and learnability analysis is performed on the text corpus directly by considering Unigram and Bigram models for the syllabified corpus. 3. Phonemes are considered as the unit of language acquisition. For this experiment, we have converted words into phonemes and by considering Unigram and Bigram models for the phonemic corpus. This data is represented in Hindi language and converted to Devanagari script. We faced difficulties in applying Bayesian learner on Devanagari script. Thus the first step in the preprocessing of data is to transliterate the files into Roman script.

14.4.1 Transliteration Transliteration is the practice of transcribing a word or text written in one writing system into another writing system. We needed to transliterate Devanagari words into Roman script because the further preprocessing steps have been developed to work for Roman scripts. For transliteration, algorithms used in Ref. [32] are involved. The algorithm has two major tasks to perform: 1. Provides single correct transliterated word in English script (Roman) if there is only a single word in Hindi script that could match with the original word. 2. Provides two
three most probable options of the transliterated text, in the order of highest to lowest probability, if there are more than one word in Hindi script that could match with the original word. Transliteration from Devanagari to Roman using this algorithm is 97.24% accurate and quite reliable for our study. We have transliterated both the dataset for further processing and computation.

14.4.2 Syllabification Initial learning results by considering Unigram and Bigram learner models on words did not show the expected results (these are discussed in Section 14.5). It is suggested [33] that syllables are basic units of learning in infant’s language acquisition process. So the complete database is syllabified after transliteration. For syllabification, we have used a rule-based approach [34,35]. A typical syllable contains three components, Nucleus, Onset, and Coda. In the syllable, the Nucleus has always been present while Onset and Coda may be absent. Syllables

14.5 Results and discussions

Syllable Body

C*V+

Syllable

Coda

C*

Coda

Body

Onset

Nucleus

C*

V+

C*

FIGURE 14.3 Structure of a syllable.

consist of vowels (V) and consonants (C). Possible structures for various syllables are V, CV, VC, and CVC. Two different syllable structures are shown in Fig. 14.3. Syllabification is based on sonority of Hindi and English. In addition to the English Sound segments, Vowels, Liquids, Nasals, Fricatives, Affricates, and Stops, Hindi also has the Matras. A complete list of Hindi and English is shown in Fig. 14.4. Based on these Sonority segments and syllable structures, we obtain the syllabified corpus of Hindi.

14.4.3 Phonemization We have phonemicized our data using The Festival Speech Synthesis System [36]. We have converted our words to Phonemes using the Festival TTS system. Final results obtained after all preprocessing techniques are shown in Fig. 14.5.

14.5 Results and discussions Language acquisition evaluation depends on many metrics proposed by different authors. These metrics should compare the adult orthographic representation, that is, the way the words would appear when transcribed in the language by an adult speaker. In this process, we focus on how precisely the learner is able to recall the learned words/syllables. The precision of the learner is defined as Precision 5

number of identified true word tokens number of all identified word tokens

(14.6)

While the recall is the capability of the learner to recollect the learned words, which is identified as

283

284

CHAPTER 14 Language learnability analysis of Hindi

FIGURE 14.4 Sonority hierarchy of Hindi and English.

FIGURE 14.5 Devanagari input, Transliterated output, syllabified representation, and phoneme representation of sample words.

14.5 Results and discussions

Recall 5

Number of identified true word tokens Number of all true word tokens

(14.7)

These two scores, which range between 0 and 1, are typically combined into a single summary statistic via the harmonic mean, referred to as the F-score F523

Precision 3 Recall Precision 1 Recall

(14.8)

F-Score would range between 0 and 1. This can be multiplied by 100 to get the evaluation score in percentage. Higher the value better would be the learnability of the learner model for a particular language. This metric is very simple to compute and to assume an orthographic transcript of the data that is available, and so is a convenient way to compare the results of modeled segmentation strategies across studies. Still, a known disadvantage is that the target segmentation is assumed to be the adult orthographic segmentation, which is unlikely to be true for 6
7-month-old infants using early segmentation strategies. As an example, if the correct sequence of words is “Rahul has entered the classroom,” and instead, we find “Rahul has enter ed the class room” We have five words in true segmentation and seven in the found segmentation. Out of these three words matches exactly in both the sequences. Therefor Precision would be 24.85% (3/7), the recall would be 50% (3/6), and F-score would be 46.2%. We have tested the learnability of Hindi using both the datasets, that is, from EMST and TDIL. Both these datasets are tested for Unigram and Bigram models. For each of the model, all four learner techniques (BatchOpt, OnlineOpt, OnlineSubOpt, and OnlineMem) are applied. F-Score calculations for English and Hindi languages are shown in Table 14.3. These scores show that English is showing best learnability for Bigram model used by constraint learner. The results are the same for both the datasets. We get better learnability using EMST dataset as it comprises stories and the words are closer to infant vocabulary. Thus the results show that English can be better learned if the learner has more recent knowledge, and the next words are more dependent on the previous words. Hindi, on the other hand, provides the best F-score for Bigram model combined with BatchOpt Bayesian learner for both datasets. This implies that prior knowledge of the language and relation between consecutive words increases the learnability of the languages. The results described in Table 14.3 are obtained by considering words as a unit of language. Since words are not the units of speech, we examined the same learner mechanism on syllable level and phonemes level dataset. Werker and Tees [37], Jusczyk and Derrah [38], and Eimas [39] have considered phonemes as the basic units of language, while Swingley [40], Lignos and Yang [41], and Gambell and Yang [42] consider syllables as the units of the language in their respective works. We have considered both the approaches and demonstrated that both the approaches could be considered

285

286

CHAPTER 14 Language learnability analysis of Hindi

Table 14.3 F-Score calculated for English and Hindi languages based on Unigram- and Bigram-based ideal and constraint learners. Model

Unigram

Bigram

F-Score

Learner

BatchOpt OnlineOpt OnlineSubOpt OnlineMem BatchOpt OnlineOpt OnlineSubOpt OnlineMem

English CHILDES [29] 0.531 0.588 0.637 0.551 0.771 0.751 0.778 0.879

Hindi—EMST [20] 0.459 0.586 0.589 0.605 0.814 0.759 0.785 0.741

Hindi (TDIL) [24] 0.429 0.559 0.534 0.591 0.801 0.734 0.642 0.714

EMST, Electro Medical and Speech Technology; TDIL, Technology Development for Indian Languages.

Table 14.4 F-Scores calculated for English and Hindi languages using syllables as a unit of language. Model

Unigram

Bigram

F-Score

Learner

BatchOpt OnlineOpt OnlineSubOpt OnlineMem BatchOpt OnlineOpt OnlineSubOpt OnlineMem

English CHILDES [29] 0.535 0.573 0.598 0.563 0.782 0.734 0.728 0.888

Hindi—EMST [20] 0.665 0.625 0.689 0.617 0.854 0.728 0.756 0.750

Hindi (TDIL) [24] 0.695 0.715 0.723 0.671 0.845 0.759 0.742 0.759

EMST, Electro Medical and Speech Technology; TDIL, Technology Development for Indian Languages.

for language learnability. The same dataset is converted to the syllabified format and phonemicized formats respectively for analysis. Tables 14.4 and 14.5 provide the F-score calculated using these two modifications in the provided datasets. We have also analyzed text corpus for with the same evaluation models. The results are presented in Table 14.6 [43].

14.6 Conclusion and future work

Table 14.5 F-Scores calculated for English and Hindi languages using phonemes as a unit of language. Model

Unigram

Bigram

F-Score

Learner

BatchOpt OnlineOpt OnlineSubOpt OnlineMem BatchOpt OnlineOpt OnlineSubOpt OnlineMem

English CHILDES [29] 0.571 0.580 0.584 0.588 0.731 0.745 0.748 0.894

Hindi—EMST [20] 0.659 0.662 0.679 0.686 0.834 0.767 0.787 0.783

Hindi (TDIL) [24] 0.672 0.687 0.689 0.685 0.830 0.784 0.757 0.754

EMST, Electro Medical and Speech Technology; TDIL, Technology Development for Indian Languages.

Table 14.6 F-Scores calculated for different text corpus of Hindi languages using words as a unit of language. Model

Unigram

Bigram

F-Score

Learner

BatchOpt OnlineOpt OnlineSubOpt OnlineMem BatchOpt OnlineOpt OnlineSubOpt OnlineMem

English CHILDES [29] 0.453 0.487 0.472 0.498 0.831 0.675 0.756 0.794

Hindi—EMST [20] 0.543 0.554 0.565 0.537 0.839 0.697 0.737 0.753

Hindi (TDIL) [24] 0.543 0.554 0.556 0.565 0.840 0.678 0.772 0.720

EMST, Electro Medical and Speech Technology; TDIL, Technology Development for Indian Languages.

14.6 Conclusion and future work In this work, we have studied the learnability of Hindi based on language learning models based on Bayesian inference. Bayesian models, along with different ngram models, are employed to calculate the learnability metrics for the languages. Hindi learnability is analyzed with two standard datasets. It is observed that

287

288

CHAPTER 14 Language learnability analysis of Hindi

unlike English, Hindi is the best learnable with the ideal learner approach. The first set of results was obtained by considering words as a unit of spoken language. These results were in sync with the Bayesian learning mechanism and the structure of the Hindi language. Further investigations were done by syllabification and phonemization of the dataset. These results prove that syllables and phonemes are better units to learn the language, especially at the early stage of human life.

Acknowledgments This research is partially supported by Malaviya National Institute of Technology (MNIT), Jaipur and The LNM Institute of Information Technology, Jaipur, India. We thank MNIT’s computer labs for setting up the experiment and also the LNMIIT’s GPU services in simulations to obtain the results.

References [1] C. Floccia, T. Keren-Portnoy, R. DePaolis, H. Duffy, C.D. Luche, S. Durrant, et al., British English infants segment words only with exaggerated infant-directed speech stimuli, Cognition 148 (2016) 1
9. [2] S.K. Gural, I. Kecskes, D. Gillespie, G.C.W. Rijlaarsdam, S.G. TerMinasova, V.I. Karasik, et al., Word collocations as language knowledge patterns: a study of infant speech, Procedia Soc. Behav. Sci. 200 (2015) 353
358. [3] M. Halpern, How children learn their mother tongue: they don’t, J. Psycholinguist. Res. 45 (5) (2016) 1173
1181. [4] D. Graddol, Future Lang. vol. 303, American Association for the Advancement of Science, 2004. [5] L. Pearl, Evaluating learning-strategy components: being fair (commentary on Cambridge, Pine, and Lieven), Language 90 (3) (2014) e107
e114. [6] N. Hyams, Language Acquisition and the Theory of Parameters, vol. 3, Springer Science & Business Media, 2012. [7] H. Taha, How does the linguistic distance between spoken and standard language in Arabic affect recall and recognition performances during verbal memory examination? J. Psycholinguist. Res. 46 (3) (2017) 551
566. [8] C. Felser, J.-D. Drummer, Sensitivity to crossover constraints during native and nonnative pronoun resolution, J. Psycholinguist. Res. 46 (3) (2017) 771
789. [9] B. MacWhinney, Mechanisms of Language Acquisition: The 20th Annual Carnegie Mellon Symposium on Cognition, Psychology Press, 2014. [10] V. Harris, Adapting classroom-based strategy instruction to a distance learning context, TESL-EJ 7 (2) (2003) 1
19. [11] A.L. Wenden, An introduction to metacognitive knowledge and beliefs in language learning: beyond the basics, System 27 (4) (1999) 435
441. [12] J.M. O’Malley, A.U. Chamot, Learning Strategies in Second Language Acquisition, Cambridge University Press, 1990.

References

[13] J.H. Hulstijn, B. Laufer, Some empirical evidence for the involvement load hypothesis in vocabulary acquisition, Lang. Learn. 51 (3) (2001) 539
558. [14] A.U. Chamot, Issues in language learning strategy research and teaching, Electron. J. Foreign Lang. Teach. 1 (1) (2004) 14
26. [15] A.D. Cohen, Strategies in Learning and Using a Second Language, Routledge, 2014. [16] A.U. Chamot, The cognitive academic language learning approach (calla): an update, in: Academic Success for English Language Learners: Strategies for K-12 Mainstream Teachers, 2005, pp. 87
101. [17] M. Grenfell, V. Harris, Modern Languages and Learning Strategies: In Theory and Practice, Psychology Press, 1999. [18] R. Balyan, N. Chatterjee, Translating noun compounds using semantic relations, Comput. Speech Lang. 32 (1) (2015) 91
108. Hybrid Machine Translation: integration of linguistics and statistics. [19] S. Sakti, M. Paul, A. Finch, S. Sakai, T.T. Vu, N. Kimura, et al., A-star: toward translating Asian spoken languages, Comput. Speech Lang. 27 (2) (2013) 509
527. Special Issue on Speech-speech translation. [20] N. Budwig, B. Narasimhan, S. Srivastava, Interim solutions: the acquisition of early constructions in Hindi, Constr. Acquis. (2006) 163
185. [21] S. Saini, V. Sahula, A survey of machine translation techniques and systems for Indian languages, in: Computational Intelligence & Communication Technology (CICT), 2015 IEEE International Conference on, IEEE, 2015, pp. 676
681. [22] S. Saini, U. Sehgal, V. Sahula, Relative clause based text simplification for improved English to Hindi translation, in: Advances in Computing, Communications, and Informatics (ICACCI), 2015 International Conference on, IEEE, 2015, pp. 1479
1484. [23] S. Goldwater, T.L. Griffiths, M. Johnson, A Bayesian framework for word segmentation: exploring the effects of context, Cognition 112 (1) (2009) 21
54. [24] L. Phillips, L. Pearl, The Bayesian inference as a cross-linguistic word segmentation strategy: always learning useful things, in: Proceedings of the Computational and Cognitive Models of Language Acquisition and Language Processing Workshop, 2014, pp. 9
13. [25] B.C. Haris, G. Pradhan, A. Misra, S.R.M. Prasanna, R.K. Das, R. Sinha, Multivariability speaker recognition database in the Indian scenario, Int. J. Speech Technol. 15 (4) (2012) 441
453. [26] India Hindi Speech Corpus, TDIL: Technology Development for Indian Languages Programme, 2010. Available from: ,http://tdildc.in/index.php?option 5 com/ download&task 5 showresourceDetails&toolid 5 268&lang 5 en.. [27] Cognition Institute for Language and Indic Multi-Parallel Corpus Computation, University of Edinburgh, 2011. Available from: ,http://homepages.inf.ed.ac.uk/ miles/babel.html.. [28] O. Bojar, V. Diatka, P. Rychly´, P. Straˇna´k, V. Suchomel, A. Tamchyna, et al., Hindencorp—Hindi-English and Hindi-only corpus for machine translation, LREC, 2014, pp. 3550
3555. [29] IIT-Bombay Hindi Corpus, 2010. Available from: ,http://www.cfilt.iitb.ac.in/downloads.html.. [30] S. Young, G. Evermann, M. Gales, T. Hain, D. Kershaw, X. Liu, et al., The Book, 3, Cambridge University Engineering Department, 2002, p. 175.

289

290

CHAPTER 14 Language learnability analysis of Hindi

[31] A. Kumar, M. Dua, T. Choudhary, Continuous Hindi speech recognition using Gaussian mixture HMM, in: Electrical, Electronics and Computer Science (SCEECS), 2014 IEEE Students’ Conference on, March 2014, pp. 1
5. [32] K. Gupta, M. Choudhury, K. Bali, Mining Hindi-English transliteration pairs from online Hindi lyrics, LREC, 2012, pp. 2459
2465. [33] L. Phillips, L. Pearl, The utility of cognitive plausibility in language acquisition modeling: evidence from word segmentation, Cognit. Sci. 39 (8) (2015) 1824
1854. [34] D. Eddington, R. Treiman, D. Elzinga, Syllabification of American English: evidence from a large-scale experiment. Part II, J. Quant. Linguist. 20 (2) (2013) 75
93. [35] R. Weerasinghe, A. Wasala, K. Gamage, A rule-based syllabification algorithm for Sinhala, Natural Language Processing—IJCNLP 2005, Springer, 2005, pp. 438
449. [36] R.A.J. Clark, K. Richmond, S. King, Multisync: open-domain unit selection for the Festival speech synthesis system, Speech Commun. 49 (4) (2007) 317
330. [37] J.F. Werker, R.C. Tees, Cross-language speech perception: evidence for perceptual reorganization during the first year of life, Infant Behav. Dev. 7 (1) (1984) 49
63. [38] P.W. Jusczyk, C. Derrah, Representation of speech sounds by young infants, Dev. Psychol. 23 (5) (1987) 648. [39] P.D. Eimas, Segmental and syllabic representations in the perception of speech by young infants, J. Acoust. Soc. Am. 105 (3) (1999) 1901
1911. [40] D. Swingley, Statistical clustering and the contents of the infant vocabulary, Cognit. Psychol. 50 (1) (2005) 86
132. [41] C. Lignos, C. Yang. Recession segmentation: simpler online word segmentation using limited resources, in: Proceedings of the Fourteenth Conference on Computational Natural Language Learning, Association for Computational Linguistics, 2010, pp. 88
97. [42] T. Gambell, C. Yang, Word segmentation: quick, but not dirty, in: Unpublished Manuscript, 2006. [43] S. Saini, V. Sahula, Language learnability analysis of Hindi: a comparison with ideal and constrained learning approaches, J. Psycholinguist. Res. (2019).

Further reading Brian MacWhinney, The CHILDES Project: The Database, vol. 2, Psychology Press, 2000.

CHAPTER

A special report on changing trends in preventive stroke/ cardiovascular risk assessment via B-mode ultrasonography

15

Ankush Jamthikar1, Deep Gupta1, Narendra N. Khanna2, Tadashi Araki3, Luca Saba4, Andrew Nicolaides5, Aditya Sharma6, Tomaz Omerzu7, Harman S. Suri8, Ajay Gupta9, Sophie Mavrogeni10, Monika Turk7, John R. Laird11, Athanasios Protogerou12, Petros P. Sfikakis13, George D. Kitas14, Vijay Viswanathan15, Gyan Pareek16, Martin Miner17 and Jasjit S. Suri18 1

Department of ECE, Visvesvaraya National Institute of Technology, Nagpur, India 2 Department of Cardiology, Indraprastha APOLLO Hospitals, New Delhi, India 3 Division of Cardiovascular Medicine, Toho University, Tokyo, Japan 4 Department of Radiology, University of Cagliari, Cagliari, Italy 5 Vascular Screening and Diagnostic Centre, University of Cyprus, Nicosia, Cyprus 6 Cardiovascular Medicine, University of Virginia, Charlottesville, VA, United States 7 Department of Neurology, University Medical Centre Maribor, Maribor, Slovenia 8 Brown University, Providence, RI, United States 9 Department of Radiology, Cornell Medical Center, New York, NY, United States 10 Cardiology Clinic, Onassis Cardiac Surgery Center, Athens, Greece 11 Heart and Vascular Institute, Adventist Health St. Helena, St Helena, CA, United States 12 Department of Cardiovascular Prevention & Research Unit Clinic & Laboratory of Pathophysiology, National and Kapodistrian University of Athens, Athens, Greece 13 Rheumatology Unit, National Kapodistrian University of Athens, Athens, Greece 14 R&D Academic Affairs, Dudley Group NHS Foundation Trust, Dudley, United Kingdom 15 MV Hospital for Diabetes and Professor M Viswanathan Diabetes Research Centre, Chennai, India 16 Minimally Invasive Urology Institute, Brown University, Providence, RI, United States 17 Men’s Health Center, Miriam Hospital, Providence, RI, United States 18 Stroke Monitoring and Diagnostic Division, AtheroPointt, Roseville, CA, United States

Cognitive Informatics, Computer Modelling, and Cognitive Science, Volume 2. DOI: https://doi.org/10.1016/B978-0-12-819445-4.00015-1 © 2020 Elsevier Inc. All rights reserved.

291

292

CHAPTER 15 A special report on changing trends

15.1 Introduction In 2017 cardiovascular (CV) diseases (CVD) such as heart attack and stroke killed 17.9 million people (31% of global mortality) around the world [1]. Plaque-buildup in carotid and coronary arteries (also called atherosclerotic disease, see Fig. 15.1) is one of the major causes of such mortalities. Recent findings indicate that traditional risk factors, including ethnicity, age, hyperlipidemia, hypertension, diabetes mellitus, smoking, family history of coronary artery diseases (CADs), obesity, and physical inactivity are responsible for the majority of CV mortalities [2
4]. Optimal management of CVD requires (1) the in-depth understanding of various CV risk factors that are associated with the disease, (2) early prediction of the CVD/stroke risk, and (3) the initiation of preventive methods such as usage of statins to treat the disease prior to the occurrence of the vascular events. In the last decade, various efforts were made for the optimal management of CVD/stroke by developing computational risk-prediction models [5
15]. All such clinically well-established risk-prediction models provide long-term risk prediction by taking into consideration the traditional and nontraditional CV risk factors to create a tool for primary prevention. However, the majority of these risk-prediction models are not generalizable due to specific cohort characteristics from which they were derived. As a result, these models either underestimate or overestimate the

FIGURE 15.1 General schematic diagram depicting the role of noninvasive carotid ultrasound and intravascular ultrasound in the detection of vascular atherosclerosis. Courtesy AtheroPointt, Roseville, CA, United States.

15.1 Introduction

CVD risk when applied to a cohort with a different baseline risk profile [16
18]. Another important fact about traditional risk scores is that they do not incorporate the morphological variations of the atherosclerotic plaque detectable during imaging tests. Such image-based data is increasingly being recognized as a key biomarker responsible for the onset of the stroke or CV events [19
21]. Recent advancements in imaging techniques have facilitated a clear visualization of the atherosclerotic plaque morphology and the measurement of several image-based phenotypes. The visualization can be in different shapes and sizes depending upon the nature of imaging (cross-sectional vs longitudinal). For example, imaging coronary artery using intravascular ultrasound (IVUS) provides the cross-sectional images of plaque morphology, while imaging carotid artery using B-mode ultrasound provides longitudinal images of plaque morphology. In general, some commonly used image-based phenotypes are based on (1) wall area (using B-mode ultrasound) and volume of calcium (using IVUS) [22,23], (2) thickness of microstructural high-risk atherosclerotic plaque components such as thin-capped fibroatheroma (using intravascular optical coherence tomography) [24,25], (3) coronary artery calcium scores (using cardiac computed tomography or CT) [26], (4) identification of calcified and noncalcified plaques using coronary CT [27], and (5) carotid image features that include carotid intima-media thickness (cIMT), carotid plaque (CP) area [28,29], wall variability [30], and composite risk scores [31]. In this review, we will focus on the carotid and coronary ultrasound image
based risk factors using advanced machine learning (ML) algorithms. Carotid arteries are considered to be a surrogate indicator of CVD risk. This is because both carotid and coronary arteries have a similar genetic makeup and can be affected by the atherosclerotic plaque buildup in their vascular beds as depicted in Fig. 15.1. In the last two decades, the use of B-mode carotid ultrasound imaging modality for the assessment of atherosclerotic vascular diseases has gained popularity due to its noninvasive, ergonomic, and economic nature [32
38]. cIMT and CP extracted from B-mode carotid ultrasound images are considered to be the two most vital image-based risk factors of myocardial infarction and stroke events [19,20,39
41]. Several longitudinal studies have reported the use of cIMT and CP for the prevention of vascular disease [33
37]. This has resulted in the framing of guidelines that report the use of cIMT and CP for the CVD risk assessment [29,42
44]. Carotid ultrasound image-based phenotypes are also associated with conventional CV risk factors (CCVRF) such as age, increased blood pressure, hyperlipidemia, diabetes mellitus, smoking, and body mass index. Nambi et al. [29] reported an improvement in the CVD risk prediction when both the cIMT and CP were taken into consideration along with the traditional risk factors. Thus scanning the carotid arteries using noninvasive B-mode ultrasound can improve CVD and stroke risk assessment. At present, more than 100 CVD risk prediction models are available, but the selection of the most suitable risk prediction model is still being debated [45]. The majority of the conventional risk-prediction models are based on traditional statistical methods such as multivariate linear regression, logistic regression, and

293

294

CHAPTER 15 A special report on changing trends

Cox regression models [5,7,9,12,13]. Such methods allow the inclusion of a small number of risk factors (or covariates) in the risk model. Furthermore, all such models are good at indicating an association of risk predictors with CVD [5,7,9,12,13]. However, in the case of CV or stroke event prediction, they provide little generalizable benefit [46] because the statistically derived risk prediction models are based on the cohort characteristics that may vary from one cohort to another. Furthermore, local patterns in the clinical, demographics-based, and traditional risk factors from different cohorts are prone to noise (such as data fluctuations or missing values or sudden large deviations) and biases that are not accurately captured by such risk-predicting models [46]. Lastly, the missing piece of the model is the exclusion of image-based morphological characteristics. This makes the overall risk evaluation system weak and unreliable. Thus in order to provide accurate and reliable CVD/stroke risk prediction, it is essential to look beyond the scope of traditional statistically derived risk calculators [46]. Innovations in intelligence
based paradigms such as ML and deep learning (DL) have performed exceptionally well in almost all medical domains [47
51]. The roots of ML systems lie in big data analytics. The term big data indicates the large storage of datasets with multiple clinical demographics (taken at the same time such as coronary, carotid, and renal) collected from multiple sources. In Fig. 15.2, big data has been represented as a collection of data-driven risk factors from multiple sources that include the patients’ demographics, conventional risk factors such as diabetes mellitus, hyperlipidemia, smoking, hypertension, and the image-based phenotypes. ML algorithms can make use of all these stored data and try to analyze different patterns that represent the data. ML systems learn to recognize the different patterns in the dataset and produce CVD- and strokeprevention prediction models. ML-based models follow the data-driven approach, meaning that these models automatically learn their coefficients from the different global and local patterns available in the big data, which may vary among Big data Demographics-based risk factors

Image-based phenotypes

Covariate collection

Conventional risk factors

10-year image-based phenotypes

Machine/deep learning

Artificial neural network

Supervised learning Unsupervised learning Reinforcement learning

Support vector machine

Cognitive computing

Deep learning

Random forest

Machine learning

Convolutional neural network

Ensemble learning

Learning methods

Precision stroke/CVD medicine

Classifier types

FIGURE 15.2 Machine learning and deep learning system framework using big data.

15.2 Risk assessment using traditional methods

different population cohorts. In recent years, studies have shown the potential of various ML techniques in CV and stroke risk prediction [52,53]. Similarly, ML algorithms have also been adopted for stroke risk assessment by characterizing carotid atherosclerotic plaques tissues from the B-mode ultrasound images [54
59]. In comparison to traditional risk calculators, recent findings indicate better performance of ML techniques for accurate CVD risk estimation [53]. Kakadiaris et al. [60] recently published a 13-year follow-up study to show that an ML-based risk calculator outperformed the well-established pooled cohort risk score (PCRS), a risk calculator that is based on the recent guidelines of American Heart Association and American College of Cardiology (also called ACC/AHA risk score or atherosclerosis CVD risk score). The breakthrough results presented by Weng et al. [53] and Kakadiaris et al. [60] have started a new quest to compare the automated ML (also broadly referred to as artificial intelligence) risk prediction models to well-established conventional risk calculators. Motivated by these results, this review provides a more informative understanding of the different ML techniques and various other approaches utilized for CVD and stroke risk prediction. The main focus of this review is to investigate the various ML-based systems used for CVD/stroke risk assessment in particular to carotid and coronary atherosclerosis diseases and more specifically using ultrasonography. Furthermore, the role of different conventional and image-based features has also been discussed in this review.

15.1.1 Article search strategy This review is the outcome of rigorous searches on PubMed, Cochrane Library, and Web of Science to obtain the articles that were published in high impact factor peer-reviewed journals. We took a search window spanning the most recent 10 years for selecting the matching publications for this review. The keywords used for searching articles in the most recent 10 years were “cardiovascular risk assessment,” “stroke risk assessment,” “10-year CVD risk calculator,” “machine learning-based CVD risk calculator,” “carotid ultrasound-based stroke risk,” “automated CVD risk estimation,” “carotid atherosclerotic plaque and machine learning,” “coronary atherosclerotic risk assessment.” Furthermore, a list of references from shortlisted research publications was also shortlisted for this review. Topics discussed in this article were initially discussed with experts in the field of cardiology, neurology, biomedical imaging, computer science, artificial intelligence covering ML, and DL-based risk assessment.

15.2 Risk assessment using traditional methods Traditional methods of CVD and stroke risk assessment are based upon the statistically derived risk calculators [5
7,14,61
65]. Nearly all these conventional

295

296

CHAPTER 15 A special report on changing trends

risk prediction models provide the risk estimation based on conventional regression techniques [66]. For example, the well-established Framingham Risk Score (FRS) [5], the United Kingdom Prospective Diabetes Study (UKPDS56) [14], the Reynolds Risk Score [9], the NIPPON score [14], and the PCRS [6] were all developed by using a Cox regression model (i.e., a proportional hazard model). Similarly, the systematic coronary risk evaluation calculator is an adapted Weibull regression model [7]. As discussed in Section 15.1, such statistical models are well suited when the application is to find the association between risk predictors (so-called risk factors or covariates) and the outcome of interest. Recently, Goldstein et al. [46] pointed out three major challenges associated with these regression-based risk prediction models. (1) These models do not represent the true nonlinear relationships between risk predictors and the outcome of interest. This means that the regression-based models assume that the predictor is linearly associated with the clinical outcome. (2) The risk predictors are sometimes interdependent on each other. Thus their effect on the outcome may not truly be captured by such regression-based models. (3) When the number of risk predictors obtained from the dataset is large, it becomes difficult to decide which risk factors to include in the regression-based models. This may be because of the small but significant associations between some of the risk predictors and the outcome of interest, which may otherwise make the model unstable. More research is required to understand and validate these calculators and further to explain their behaviors with the diverse risk factors. Before we dwell into ML-based CVD/ stroke risk calculators, we will briefly review ML fundamentals and the architectures.

15.3 Fundamentals of machine learning 15.3.1 Types of machine learning techniques Primarily, ML-based models are divided into three categories (Fig. 15.2, labeled as “Learning methods”): (1) supervised learning, (2) unsupervised learning, and (3) reinforcement learning. In supervised learning, the predefined binary labels (high-risk or low-risk; event or nonevent) are obtained from the physicians or from the longitudinal trials as inputs that are used to train the ML system on how to correctly predict the risk outcome. For example, an automated CV event prediction system is usually provided with predefined labels corresponding to an event or no-event category [52,67]. In risk assessment systems the input labels (or response variable) can be obtained from the expert physicians or from results of the longitudinal follow-up studies, or by designing a response variable using a combination of risk factors. The unsupervised learning ML system performs risk stratification without any prior user inputs or labels by identifying and then clustering similar local patterns from the source data [68]. Once trained, each of the clusters corresponds to the one output category, for instance, in the above

15.3 Fundamentals of machine learning

example, the output clusters can either be an event or no-event category. Reinforcement learning is another ML technique that provides its predictions based on rewards. Reinforcement learning is widely adapted in gaming and robotic applications [69]. This review is focused on the supervised ML-based algorithms used in CVD/stroke risk assessment adapting ultrasonography.

15.3.2 General framework of machine learning ML combines the knowledge of computer science and mathematical and statistical models to self-train the systems to provide the desired outcome. The outcome can be the prediction of absolute real numbers or it can be a classification of the input data into the set of desired output classes. Fig. 15.3 shows the generalized ML-based framework that is divided into five stages and discussed next in brief in the following order: (1) feature engineering, (2) data partitioning, (3) model building (or offline system), (4) prediction (or online system), and (5) performance evaluation (PE).

15.3.2.1 Feature engineering: extraction and selection Feature engineering is the most crucial part of any ML-based system that helps in interpreting the input dataset. In CVD/stroke risk assessment, these features can be either CCVRFs such as patients’ demographics, serum biomarkers, and clinical variable or image-based phenotypes such as grayscale features [58,70,71], texturebased features [72], discrete wavelet transform (DWT)-based features [73], Rieszbased features, [74] higher order spectra (HOS)-based features [75,76], fractal Data partitioning

Feature engineering

1

2

Training model

3

Input dataset

Feature extraction

Feature selection

Training classifier

Offline system

Training features

Ground truth for training data

Training

Trained model

Testing features

Feature extraction

Feature selection

Prediction

Testing classifier

Prediction class

4

Evaluation

Prediction model

Ground truth for testing data

Online system

Testing

ML-based risk & AUC Performance evaluation model

5

FIGURE 15.3 The architecture of the ML algorithm showing four stages. ML, Machine learning.

297

298

CHAPTER 15 A special report on changing trends

features, and local binary patterns [77]. Quantitative carotid wall
based features extracted from B-mode ultrasound images such as cIMT and CP are also considered to be reliable for CVD/stroke risk assessment. According to the consensus report of the American Society of Echocardiography, cIMT measurements are generally performed in 1 cm region of common carotid artery at a proximal distance of 1 cm from the bulb [42]. However, studies have shown that measuring the distance between lumen-intima and media-adventitia throughout the length of the carotid artery, including CP thickness, provides an additional benefit in the risk assessment [78,79]. AtheroEdge (AtheroPoint, Roseville, CA, United States) has exclusively published and established a system that can measure fully and automatically cIMT throughout the carotid artery in just a few seconds [80]. The system outputs’ image-based phenotypes include average cIMT (cIMTave), maximum cIMT (cIMTmax), minimum cIMT (cIMTmin), variability in cIMT (cIMTV), and morphologic total plaque area (mTPA). The system underwent inter- and intraoperator variability analysis recently [81
83]. Under wall
based features, one can also measure lumen diameter (LD) [83], stenosis severity index [84,85], and interadventitial diameter (IAD) [86]. Excellent inter- and intra-operator variability for LD and IAD was recently shown in the AtheroEdge model (DL for LD) [87]. The CCVRFs and image-based phenotypes provide a detailed understanding of the severity of CVD/stroke disease and can be used to train the ML-based systems. Recently, Khanna et al. [88] presented a study that estimated the 10-year CUS image-based phenotypes by integrating the five types of current image
based phenotypes (cIMTave, cIMTmax, cIMTmin, IMTV, mTPA) with conventional risk factors. Such 10-year old features can also be used in the ML-based system to provide CVD/stroke risk stratification. Besides feature extraction, dominant feature selection is another important technique that captures the most relevant features and then trains the ML systems. However, feature selection can only benefit the ML-system if a large number of features are captured from the input data. Some commonly used feature selection methods are random forest (RF), logistic regression, mutual information, principal component analysis (PCA), analysis of variance, and Fisher discriminant ratio [89,90].

15.3.2.2 Data partitioning Data partitioning involves dividing the input dataset into two parts: (1) training dataset and (2) testing dataset. Multiple protocols exist for performing this data partitioning task. The most common protocol is 10-fold cross-validation, where the input dataset set is divided into 10 equal parts and at any time, nine parts are used for training the ML-based system, while the remaining one part is used for validating the predictions of the system. This is also termed K10 protocol where 10 indicates the number of total partitions designed during ML-based training model (typically using 80% of the dataset for training). The similar well-known data partitioning protocols are K2 protocol, K3 protocol, K4 protocol, and jackknife (also called as leave-one-out) cross-validation protocols, depending upon

15.3 Fundamentals of machine learning

the percentage of data used for training as 50%, 66%, 75%, and 99%, respectively. Since in a leave-one-out cross-validation protocol, N 2 1 samples are used for training and one sample is used for testing, it is generally adopted when the sample size is relatively small [91].

15.3.2.3 Training model design Training model design involves teaching the ML-based algorithm to learn from the input training features over several iterations. Since this review is focused on supervised ML-systems, predefined ground truth labels are required during the training phase, which in turn generates the offline coefficients. During each of the training iterations, ML-based algorithms provide the output predictions and compute the probable loss (also called an error) by comparing against the supplied labels (response variable). Based on the loss value, the internal coefficients of the model (so-called as hyperparameters) are adjusted. After updating the hyperparameters, ML-based algorithms will be again trained on the input features. The process of updating hyperparameters continues until the loss is minimum (we also call this instance, when the machine is able to split or partition well and the plane of separation is the hyperplane). This is so-called a state in which the ML model is considered to be trained.

15.3.2.4 Prediction or testing model In the prediction or testing phase the optimized coefficients from the trained model (training parameters) are used to transform the test features (derived from the test data) into the output or predicted class. This is also called an online process since this model accepts training parameters from the offline system and transforms the test features from the online system. In cross-validation protocols, the test data is a dataset completely different from the training data. Typically a good artificial intelligence model in risk stratification is one that trains the machine only one time, while the predictions can be done on several types of test datasets. More sophisticated machines are required to train and test on different types of datasets.

15.3.2.5 Performance evaluation of machine learning systems PE metrics are generally used to test the ability of ML systems to accurately predict the risk categories of patients [92]. In the PE model, the predicted class labels of the test patient are computed using the prediction model and compared against the corresponding ground truth label, which is then used by the PE metric. The choice of a PE metric is of utmost importance because it indicates the degree to which the trained and tested model is accomplishing the desired outcomes. Areaunder-the-curve (AUC) derived from receiver operating characteristic analysis is a widely adopted PE metric in medical applications. It indicates the overall performance of the system in terms of sensitivity and specificity (see the Appendix). Besides the classification accuracy, Brier score [93] and concordance index [94] are commonly used PE metrics in CV risk assessment [12,52,95].

299

300

CHAPTER 15 A special report on changing trends

15.3.3 Machine learning
based algorithms In general, classification and regression are the two primary tasks in ML-based algorithms. Almost all the ML-based algorithms are capable of performing both of these two tasks. Classification basically categorizes the input data into one of the predefined labels or outcomes. For example, in a CVD/stroke event prediction task, the input features are generally classified into either event or no-event category. Regression-based ML algorithms are generally used for predictions of some real-valued output. For example, in a CVD/stroke risk estimation task, regressionbased ML algorithms provide the real-valued percentage risk between 0% and 100%. This review is mainly focused on the ML-based application consisting of classification task. The most common ML algorithms used in CVD risk assessment are support vector machine (SVM) [96], artificial neural networks (ANNs) [97], linear and logistic regression [53], and tree-based algorithms such as RF and decision tree (DT) [53]. Another category of ML is ensemble-learning techniques, in which the outcomes of all ML techniques can be combined to train the ML model to increase the accuracy of risk prediction. In Table 15.1, we have compared multiple studies presented on ML-based CVD/stroke risk assessment. One important observation is that most of the studies have utilized the SVM as a classifier during training and testing of classification tasks.

15.4 Risk assessment in machine learning framework In the last decade, various efforts were made to perform the ML-based CV risk stratification using imaging modalities such as carotid CT [73], coronary CT angiography [106], single photon emission CT (SPECT) [107], echocardiography [70,108], magnetic resonance imaging [109], and optical coherence tomography [25]. However, since this review is largely focused on the ML-based CV risk stratification using ultrasound imaging, the discussion on other imaging modalities is considered to be out-of-scope for this review. This section covers various efforts made in the direction of ML-based CV risk stratification using ultrasound imaging (of both coronary and carotid arteries).

15.4.1 Image-based stroke risk assessment using machine learning Deposition of atherosclerotic plaque leading to restriction of blood flow in the carotid and coronary arteries leads to cerebrovascular (ischemic stroke) and CV (myocardial infarction) events [110,111]. In recent years, ML-based algorithms have been widely adopted for stroke risk assessment using noninvasive imaging modalities such as carotid ultrasound [55
59]. Carotid atherosclerotic plaque burden is a crucial biomarker for stroke events and can be readily assessed using imaging tests [20,112]. The appearance of CP in the B-mode ultrasound image

Table 15.1 Machine learning
based CVD/stroke risk stratification in both carotid and coronary artery. (A) C1

C2

C3

C4

C5

C6

C8

C9

C10

C11

C12

C13

C14

SN

Authors

AT (modality)

Features types

TF

Feature selection

Classifier type

Ground truth

N

TI

Training protocol

Performance evaluation

Benchmarking

R1

Acharya et al. [55]

Carotid (CUS)

DWT, HOS, and texture features

7

Statistical test

SVM

Labels from physicians

99

112

K3

R2

Acharya et al. [56]

Carotid (CUS)

DWT features

54

Statistical test

SVM

R3

Acharya et al. [59]

Carotid (CUS)

Texture

8

Statistical test

SVM

R4

Acharya et al. [58]

Carotid (CUS)

Grayscale features

17

Statistical test

SVM, GMM, RBPNN, DT, kNN, NBC, FC

Labels from physicians Risk labels from physicians Labels from physicians

Se (97%), Sp (80%) ACC (91.7%) AUC (0.885) ACC (83.7%)

R5

Acharya et al. [98]

Carotid (CUS)

7

Statistical test

SVM, RBPNN, kNN, DT

R6

Kyriacou et al. [99] Gastounioti et al. [100]

Carotid (CUS) Carotid (CUS)

Phenotypes and HOS features Image-based texture Kinematics features

27

SVM, LR

Araki et al. [91]

Coronary (IVUS)

Grayscale features

56

Statistical test FDR, WRS, PCA PCA

R7

R8

1236

SVM

SVM

346







71

346




ACC (83%)

Against kNN, RBPNN

445

492

K3




Labels from physicians Follow-up data labels Follow-up data labels

59

118

K10

DB1:ACC (93.1%) DB1:ACC (85.3%) ACC (99.1%)

108







ACC (77%)




56

4200




ACC (88%)

Against kNN, PNN, DT, DA

cIMT

19

4004

ACC (94.95%) AUC (0.98)




(B) C1

C2

C3

C4

C5

C6

C8

C9

C10

C11

C12

C13

C14

SN

Authors

AT (modality)

Features types

TF

Feature selection

Classifier type

Ground truth

N

TI

Training protocol

PE

Benchmarking

R1

Hu et al. [101]

NA

CCVRFs

41

Fisher

Naïve Bayes, MLP, RF

382

382

K3

AUC (0.797), BS (0.085)




R2

Narain et al. [102]

NA

CCVRFs

7

NA

QNN

682

NA



ACC (98.57%)

Against FRS

R3

Unnikrishnan et al. [103]

NA

CCVRFs

9

NA

SVM

Follow-up data labels Risk labels from physicians Follow-up data labels

2406

NA

K5

Against FRS

R4

AmbaleVenkatesh et al. [52]

Coronary MRICarotid US

CCVRFs, image phenotypes, and serum biomarkers

735

MDMST

Follow-up data labels

6814

NA

K3

R5

Zarkogianni et al. [104]

NA

CCVRFs

16

NA

RF, Cox, LASSOcox, AIC-Cox backward regression HWNN and SOM

Se (68.2%) Sp (85.9%) AUC (0.71) C-Index (0.81), BS (0.083)

Follow-up data labels

560

NA




R6

Banchhor et al. [72]

Coronary (IVUS)

Texture- and wall-based features

65

PCA

SVM

Carotid plaque burden

22

4930

K10

R7

Araki et al. [67]

Carotid (CUS)

Image-based texture features

16

Statistical test

SVM

LD-based risk labels

204

407

K5, K10, JK

R8

Saba et al. [105]

Carotid (CUS)

Image-based texture

16

PCA

SVM

LD-based risk labels

204

407

K10

K3

AUC (0.715), ACC (71.79%), BS (0.07) ACC (91.28%) AUC (0.91) ACC (NW: 95.08% and FW: 93.47%) ACC (NW: 98.83% and FW: 98.55%)

Against FRS and PCRS

Against UKPDS










(C) C1

C2

C3

C4

C5

C6

C8

C9

C10

C11

C12

C13

#SN

Authors

AT (modality)

Features types

TF

Feature selection

Classifier type

Ground truth

N

TI

Training protocol

Performance evaluation

Benchmarking

R1

Weng et al. [53] Kakadiaris et al. [60]




CCVRFs

30







K4

AUC: 0.764

Against PCRS

CCVRFs

9




Follow-up data labels Follow-up data labels

378,256




RF, LR, ANN SVM

6459




K2

Se (86%), Sp (95%), AUC (0.92)

Against PCRS

R2

C14

ACC, accuracy; ANN, Artificial neural network; AUC, area-under-the-curve; BS, Brier score; CCVRF, conventional cardiovascular risk factors; cIMT, carotid intima-media thickness; CUS, carotid ultrasound; DA, discriminant analysis; DB, database; DT, decision tree; DWT, discrete wavelet transform; FC, fuzzy classifier; FDR, Fisher discriminant ratio; FRS, Framingham Risk Score; GMM, Gaussian mixture model; HOS, higher order spectra; IGR, information gain ranking; HWNN, Hybrid wavelet neural networks; kNN, k-nearest neighbor; LBP, local binary pattern; LD, lumen diameter; LR, logistic regression; MDMST, minimal depth of maximal subtree; MLP, multilayer perceptron; NBC, Naïve Bays Classifier; PCA, principal component analysis; PCRS, pooled cohort risk score; PE, performance evaluation; QNN, quantum neural network; RBPNN, radial basis probabilistic neural network; RF, random forest; Se, sensitivity; SOM, self-organizing maps; Sp, specificity; SVM, support vector machine; UKPDS, United Kingdom Prospective Diabetes Study; WRS, Wilcoxon rank-sum.

15.4 Risk assessment in machine learning framework

(brighter/hyperechoic/echogenic or darker/hypoechoic/echolucent) adds valuable information about the risk profile of a patient [113]. It has been shown that echolucent (darker) atherosclerotic plaque (darker plaque) is a potential indicator for stroke events [114,115]. It has also been shown that the effect of this echolucent plaque is more pronounced in patients with diabetes [116]. Similarly, echogenic (brighter) plaque due to the presence of calcium within a plaque [117,118] may be a marker of less vulnerable/low-risk plaque compared to the echolucent plaque. Identification of both of these plaque phenotypes is a crucial step in stroke risk assessment and can possibly be useful in treatments of stenting or endarterectomy [54,55,59,119]. This decision is clearly a classification task, for which the ML-based systems are well suited. In the last decade, multiple efforts were made to automatically classify CP phenotypes [54
56,59]. In 2010, Acharya et al. [54] presented a study that classified carotid atherosclerotic plaque using the supervised ML-based algorithm such as SVM and AdaBoost classifiers. Texture patterns captured using carotid ultrasound images along with the statistical features (mean and standard deviation) were used to train these ML systems. Authors reported 82.4% classification accuracy using SVMbased classifiers. In 2013, the same group (Acharya et al. [55]) again classified CP phenotypes by considering the combination of DWT, HOS, and texture features. With the addition of both the DWT and HOS features, classification accuracy increased to 91.7%. Other similar studies of stroke risk assessment using CP phenotypes in ML-based algorithms are presented in Table 15.1.

15.4.2 Cardiovascular diseases risk assessment using machine learning In the last few years, ML-based algorithms have widely penetrated into the domain of primary CVD risk assessment, particularly in (1) CAD using characterization of coronary atherosclerotic plaque tissues, (2) CVD risk using coronary calcium score, (3) CAD based on conventional risk factors, (4) overall CVD risk, and (5) prediction of CV events. Risk assessment using ML in each of these different applications requires the data-specific patterns to train the ML model and then to transform the trained knowledge to predict the risk on the test data. These patterns can be derived from the CCVRFs, serum biomarkers, patients’ demographics, imaging modalities, or from the combination of these. In the last decade, multiple studies have explored the potential advantages of using different combinations of features for ML-based CV risk assessment [52,70,72,91]. Using IVUS coronary imaging modality, Araki et al. [91] demonstrated the CAD risk stratification of 15 Japanese patients using 56 grayscale features. The IVUS image-based gray features were captured from the region between inner elastic lamina and external elastic lamina of the coronary wall. The IVUS examinations scanned the coronary arteries, including the left and right anterior descending, left circumflex, and left main coronary artery. Overall, when using the

303

304

CHAPTER 15 A special report on changing trends

SVM-based classifier, the ML classification accuracy for this automated system was 94.95%. This improvement is likely due to the inclusion of (1) more dominant features along with feature selection methods such as PCA, or Fisher discriminant analysis and (2) plaque motion analysis. Banchhor et al. [72] recently extended Araki’s [91] work to include coronary wall parameters. In total, 65 carotid and coronary wall
based features were used to perform the ML-based CAD risk assessment. Using the PCA-based features selection approach in conjunction with an SVM-based classifier, the authors reported a classification accuracy of 91.28% with AUC of 0.91. Both of these studies (Araki et al. [91] and Banchhor et al. [72]) indicated the link between coronary and carotid atherosclerosis by considering the cIMT and plaque burden as the gold standard to perform the supervised CVD risk stratification. Risk assessment based on longitudinal trials or follow-up studies are generally considered to be accurate and widely adopted in clinical practice. Proportional hazard models and risk calculators are the conventional tools for CVD risk assessment [5
14]. However, a very recently published study by Ambale-Venkatesh et al. [52] demonstrated that the ML-based algorithms are more accurate and better compared to traditional proportional hazard models for CVD risk assessment. Authors included the participants from the well-known Multi-Ethnic Study for Atherosclerosis (MESA), a 12-year longitudinal cohort study. A total of 735 CV risk predictors or features were extracted from diverse sources such as patients’ demographics, traditional risk factors, imaging modalities, questionnaires, and laboratory biomarkers. Authors reported an overall C-index of 0.81 and a Brier score of 0.083 using the RF-based classifier. The authors also benchmarked their results against the traditional CV risk assessment tools such as FRS and PCRS. Compared to these traditional risk scores, authors indicated in an improvement of C-index by B10% and decreased in Brier score by 10
25%. Weng et al. [53] also presented a prospective study with 378,256 participants and reported better risk stratification using ML-based algorithms over the conventional statistically derived risk calculators. A total of 30 CCVRFs were used for training the ML-based algorithms. Authors compared the performance of four ML-based algorithms such as ANN, RF, gradient boosting machine, and logistic regression. The study reported ANN to be the best classifier with an AUC of 0.76. Kakadiaris et al. [60] reported the most recent study based on MESA participants that reported a superior performance of ML-based system compared to the PCRS-based calculator. The PCRS calculator was based on ACC/AHA guidelines that recommended the lipid-lowering statins to the patients whose risk was more than 7.5% to reduce the risk of Atherosclerosis Cardiovascular Disease. Kakadiaris et al. [60] used the same PCRS but the statin eligibility threshold was chosen as 9.75%. This was mainly because of the 13-year follow-up nature of the MESA study, while the PCRS was based on 10-year follow-up dataset. The authors used the same nine CCVRFs that were used for PCRS computation and demonstrated the better performance of an ML-based SVM classifier

15.5 Medical implications of machine learning
based risk assessment

(AUC 5 0.92) compared to PCRS (AUC 5 0.71). In this review, we have investigated some more studies that adapted ML-based algorithms for CVD risk assessment. A summary of all such studies along with their attributes is presented in Table 15.1.

15.4.3 Cardiovscular disease/stroke risk assessment indices Some investigators have recommended the use of the single index that quantifies the CVD/stroke risk [54,55,59,70,73,120]. Acharya et al. [70] recommend the use of the single discriminative index called HeartIndex for assessing the risk of CAD. The HeartIndex was derived from the features to classify the echocardiography images into two risk classes: normal and CAD. In order to assess neurological risk, Pedro et al. [120] proposed the enhanced activity index that was based on the carotid artery plaque morphology and severity of carotid stenosis to classify the occurrence of ipsilateral ischemic symptoms. Acharya et al. [73] proposed the single values Atheromatic index (from AtheroPoint, Roseville, CA, United States) to identify two CP phenotypes: high versus low risk. The design of the Atheromatic index was based on the CT image phenotypes that were derived from local binary patterns, wavelet transform, and the textures of the CP image. The same group [54] also proposed a symptomatic asymptomatic carotid index (SACI) based on the texture and statistical features derived from the carotid ultrasound image. The SACI index has also been tested in a carotid atherosclerotic plaque ultrasound
based tissue characterization study by computing the grayscale features in ML framework [55].

15.5 Medical implications of machine learning
based risk assessment Risk assessment systems are primarily aimed at identifying the risk profiles of patients and to stratify them into one of the several CVD risk classes (e.g., lowrisk, moderate-risk, and high-risk classes). Risk stratification tools inform such patient treatment decisions as the need and strength of statins (i.e., the wellknown lipid-lowering medications) such as atorvastatin, pravastatin, and simvastatin [121] and diabetes-controlling medications such as metformin [122]. Traditional statistically derived risk prediction models have been reported to either underestimate or overestimate the CVD risk [17,18] and therefore have the unintended consequence of inappropriate medication prescription in some patients and inappropriate underutilization in other patients, both of which have potential harmful side effects and outcomes. This also could increase the economic burden on the patients and healthcare systems. ML systems provide better risk assessment and support in avoiding unnecessary over- or undertreatment [53]. A recent follow-up study presented by Kakadiaris et al. [60] investigated the statin

305

306

CHAPTER 15 A special report on changing trends

eligibility for the patients using both (1) PCRS calculator based on ACC/AHA risk prediction guidelines and (2) ML-based risk calculator. ACC/AHA model recommended statins for 46% of the patients, while the ML-based risk calculator identified only 11.7% eligible for statin therapy.

15.6 Deep learning
based cardiovascular risk stratification DL is an extension of classical ANNs and efficient ML techniques to analyze medical images. It consists of multiple convolutional layers that allow extraction of more data-dependent patterns, and thus it helps in improving the accuracy of outcome prediction [123]. Convolutional neural network (CNN) is a DL algorithm that has gained large attention while analyzing medical images (Fig. 15.4). This is because CNN has the ability to extract a large number of image-based features compared to the handcrafted statistical features [124,125]. In CNN algorithm, an input image gets convolved with a number of kernels that are responsible for the deck of feature extraction (convolution operation is shown by a green rectangle using kernel banks-magenta color). The features are selected using polling operation (shown by an inverted green triangle), where the meaningful features are selected. The coefficients of all these kernels are learned during the training process of CNN. A basic architecture of DL using CNN is as depicted in Fig. 15.4. There two challenges associated with the basic CNN architecture: (1) the basic architecture may suffer from overfitting due to the input data and thus results in a reduction in risk stratification accuracy; (2) the basic architecture may have a

FIGURE 15.4 An architectural diagram for the deep learning
based convolutional neural network algorithm. Courtesy AtheroPointt, Roseville, CA, United States.

15.7 Challenges in machine learning design

Convolutional layers and pooling

Inception layers and pooling

Classifier

FIGURE 15.5 Advanced CNN model by incorporating convolution, pooling, inception, softmax layers. CNN, Convolutional neural network. Courtesy AtheroPointt, Roseville, CA, United States.

limited number of convolution layers. A larger number of convolutional layers extract more features from the input images and thus provide the power to the DL system to separate the classes accurately. This, however, increases the complexity of the system. The two challenges can be solved (1) by including the dropout strategy in the basics model and (2) by adding multiple inception layers in the basic model, respectively. The detailed version of DL using CNN has been depicted in Fig. 15.5. The details of dropout and inception layers are out of the scope of this review. However, their functionality has clearly been discussed in our previous paper [48]. The main advantage in CNN is its ability to extract the context-based features and as a result does not require any prior information, as in Supervised ML, before training the system. Recently, Lekadir et al. [126] used CNN to automatically characterize the plaque tissues from carotid ultrasound images. A very recent set of studies by Biswas et al. [47,87] used CNN to measure the cIMT and LD, respectively, based on CUS images. Although the focus of this review is on ultrasonography, it must be noted that DL techniques have also been tried to perform risk assessment using different imaging modalities such as coronary CT angiography [127], optical coherence tomography [25], and MRI [109].

15.7 Challenges in machine learning design There are some important key challenges while adapting ML techniques in medical domains especially in CVD/stroke risk assessment, including the following:

• Black-box nature of the ML techniques ML-based algorithms have proven their potential in providing robust and accurate solution almost in every medical domain, including CVD/stroke risk

307

308

CHAPTER 15 A special report on changing trends

•

•

•

assessment. However, it is somewhat challenging for clinical practitioners and physicians to adapt ML techniques in their clinical practice. This may be because of the so-called black box nature of ML algorithms [128] and lack of external validations. Unlike traditional statistical derived methods that are based on discrete clinical variables (e.g., age, blood pressure, diabetes), the internal working of ML algorithms is not easy to interpret for most physicians. This uncertainty about the logic of ML algorithm may cause some practitioners to be more reluctant to adopt this technology in clinical practice. Further, fewer studies have been published in respectable peer-reviewed journals that show (1) strong scientific validations, (2) well-established gold standards, (3) variation analysis of the datasets, and (4) real-world ML clinical applications that can be adapted on a daily basis. Achieving generalization of ML models It is of paramount importance that the ML model must be generalizable [105]. This means the ML model designs must be tested on almost every relevant dataset, including longitudinal follow-up datasets (or during prospective trials or retrospective datasets in which outcomes were recorded) with potentially different features. This makes the model more robust and clinically reliable. Role of transfer learning Since conducting multiple clinical trials with large sample size to produce highly generalizable and reliable results is not economically viable, an ML technique known as transfer learning may be valuable. Furthermore, if an ML model is trained on a particular dataset with a unique set of features, then it also becomes practically impossible to collect the similar kind of features in multicenter trials due to its lengthy timely collection of data and soaring cost. This puts a limit on generalization of ML systems. Transfer learning, therefore, provides the benefit of training on one set and application of transformed parameters on a different set. Lack of access to data

The biggest and the most important challenge for most ML-based data scientists and developers is the lack of access to the well-established patient and population-based datasets, such as MESA and ARIC. It is true that conducting such multicenter clinical trials is expensive, but further attempts at collaboration and improving access to such datasets are needed to validate these ML-based designs. Because of these key challenges, medical practitioners are not ready to rely on ML techniques for clinical decision-making [92,129]. It is of utmost importance for both medical practitioners and data scientists to come together and increase the interpretability of computationally complex ML techniques. In order to make the ML models generalizable, more funding sources should come together and conduct longitudinal multicenter clinical trials or observational studies that can benefit both data scientists and system developers to make more robust models.

References

15.8 Conclusion In this review, we reported the changing trend for CVD/stroke risk assessment, ranging from traditional statistically derived calculators to advanced ML-based risk assessment systems. ML-based algorithms show better performance compared to traditional methods. In this review, we found that the supervised machine technique such as SVM is the widely adapted ML-based algorithm for CVD risk assessment followed by RF and ANN. Data-driven features are the most vital part of ML algorithms. Most of the risk assessment techniques (i.e., traditional and ML based) commonly incorporated traditional CV features in their risk assessment model. For better risk assessment, the feature engineering domain needs to be explored more in the near future. Different image-based risk factors can be incorporated to improve automated risk assessment systems. DL is the rapidly developing field for image analysis that extracts more robust features from images. Image phenotypes extracted using DL combined with CCVRFs can provide a better platform for CVD/stroke assessment. Verification and validation of the ML/DL systems is a crucial component for adopting the system designs in routine clinical practice. Finally, with the advancements in big data and artificial intelligence
based paradigms, we are likely to see more sophisticated CVD/stroke risk assessment tools in the future.

Acknowledgments Authors would like to thank the editors and reviewers of the Current Reports of Atherosclerosis for giving valuable suggestions for improving the manuscript.

Funding None.

Disclosure Dr. Jasjit S, Suri is affiliated to AtheroPoint, focused in the area of stroke and cardiovascular imaging.

References [1] Cardiovascular diseases (CVDs): Key facts by WHO, May 2016. Available from: ,http://www.who.int/news-room/fact-sheets/detail/cardiovascular-diseases-(cvds)..

309

310

CHAPTER 15 A special report on changing trends

ˆ unpuu, et al., Association of psychosocial risk factors [2] A. Rosengren, S. Hawken, S. O with risk of acute myocardial infarction in 11 119 cases and 13 648 controls from 52 countries (the INTERHEART study): case-control study, Lancet 364 (9438) (2004) 953
962. [3] M.J. O’Donnell, S.L. Chin, S. Rangarajan, et al., Global and regional effects of potentially modifiable risk factors associated with acute stroke in 32 countries (INTERSTROKE): a case-control study, Lancet 388 (10046) (2016) 761
775. An important logitudinal study that associated the convetional risk factors with risk of stroke events. [4] M.J. O’Donnell, D. Xavier, L. Liu, et al., Risk factors for ischaemic and intracerebral haemorrhagic stroke in 22 countries (the INTERSTROKE study): a case-control study, Lancet 376 (9735) (2010) 112
123. [5] R.J. Stevens, R.L. Coleman, A.I. Adler, I.M. Stratton, D.R. Matthews, R.R. Holman, Risk factors for myocardial infarction case fatality and stroke case fatality in type 2 diabetes: UKPDS 66, Diabetes Care 27 (1) (2004) 201
207. [6] L. Saba, F. Molinari, K. Meiburger, et al., What is the correct distance measurement metric when measuring carotid ultrasound intima-media thickness automatically? Int. Angiol. 31 (5) (2012) 483
489. [7] R. Conroy, K. Pyo¨ra¨la¨, Fitzgerald Ae, et al., Estimation of ten-year risk of fatal cardiovascular disease in Europe: the SCORE project, Eur. Heart J. 24 (11) (2003) 987
1003. [8] R.B. D’agostino, R.S. Vasan, M.J. Pencina, et al., General cardiovascular risk profile for use in primary care: the Framingham Heart Study, Circulation 117 (6) (2008) 743
753. [9] P.M. Ridker, J.E. Buring, N. Rifai, N.R. Cook, Development and validation of improved algorithms for the assessment of global cardiovascular risk in women: the Reynolds Risk Score, JAMA 297 (6) (2007) 611
619. [10] R.J. Stevens, V. Kothari, A.I. Adler, I.M. Stratton, R.R. Holman, The UKPDS risk engine: a model for the risk of coronary heart disease in Type II diabetes (UKPDS 56), Clin. Sci. 101 (6) (2001) 671
679. [11] V. Kothari, R.J. Stevens, A.I. Adler, et al., UKPDS 60: risk of stroke in type 2 diabetes estimated by the UK Prospective Diabetes Study risk engine, Stroke 33 (7) (2002) 1776
1781. [12] J. Hippisley-Cox, C. Coupland, P. Brindle, Development and validation of QRISK3 risk prediction algorithms to estimate future risk of cardiovascular disease: prospective cohort study, BMJ 357 (2017) j2099. [13] D.C. Goff, D.M. Lloyd-Jones, G. Bennett, et al., 2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/ American Heart Association Task Force on Practice Guidelines, J. Am. Coll. Cardiol. 63 (25 Part B) (2014) 2935
2959. [14] N.D.R. Group, Risk assessment chart for death from cardiovascular disease based on a 19-year follow-up study of a Japanese representative population, Circ. J. 70 (10) (2006) 1249
1255. [15] L. Nobel, N.E. Mayo, J. Hanley, L. Nadeau, S.S. Daskalopoulou, MyRisk_stroke calculator: a personalized stroke risk assessment tool for the general population, J. Clin. Neurol. 10 (1) (2014) 1
9. [16] K. Bonek, P. Głuszko, Cardiovascular risk assessment in rheumatoid arthritis
controversies and the new approach, Reumatologia 54 (3) (2016) 128.

References

[17] E. Arts, C. Popa, A. Den Broeder, et al., Performance of four current risk algorithms in predicting cardiovascular events in patients with early rheumatoid arthritis, Ann. Rheum. Dis. (2014). annrheumdis-2013-204024. [18] N. Garg, S.K. Muduli, A. Kapoor, et al., Comparison of different cardiovascular risk score calculators for cardiovascular risk prediction and guideline recommended statin uses, Indian Heart J. 69 (4) (2017) 458
463. [19] B. Mathiesen Ellisiv, H. Johnsen Stein, T. Wilsgaard, H. Bønaa Kaare, M.-L. Løchen, I. Njølstad, Carotid plaque area and intima-media thickness in prediction of first-ever ischemic stroke, Stroke 42 (4) (2011) 972
978. [20] J.D. Spence, M. Eliasziw, M. DiCicco, D.G. Hackam, R. Galil, T. Lohmann, Carotid plaque area: a tool for targeting and evaluating vascular preventive therapy, Stroke 33 (12) (2002) 2916
2922. [21] G. Belcaro, A.N. Nicolaides, G. Ramaswami, et al., Carotid and femoral ultrasound morphology screening and cardiovascular events in low risk subjects: a 10-year follow-up study (the CAFES-CAVE study1CAFES-CAVE 5 Carotid-Femoral morphology and cardiovascular events1), Atherosclerosis 156 (2) (2001) 379
387. [22] H.M. Garcia-Garcia, M.A. Costa, P.W. Serruys, Imaging of coronary atherosclerosis: intravascular ultrasound, Eur. Heart J. 31 (20) (2010) 2456
2469. [23] S.K. Banchhor, T. Araki, N.D. Londhe, et al., Five multiresolution-based calcium volume measurement techniques from coronary IVUS videos: a comparative approach, Comput. Methods Prog. Biomed. 134 (2016) 237
258. ¨ , et al., Atherosclerotic tissue characterization [24] G. Van Soest, E. Regar, S. KoljenoviA in vivo by optical coherence tomography attenuation imaging, J. Biomed. Opt. 15 (1) (2010) 011105
011109. [25] A. Boi, A.D. Jamthikar, L. Saba, et al., A survey on coronary atherosclerotic plaque tissue characterization in intravascular optical coherence tomography, Curr. Atheroscler. Rep. 20 (7) (2018) 33. [26] M.J. Blaha, M.B. Mortensen, S. Kianoush, R. Tota-Maharaj, M. Cainzos-Achirica, Coronary artery calcium scoring: is it time for a change in methodology? JACC: Cardiovasc. Imaging 10 (8) (2017) 923
937. [27] J. Eckert, M. Schmidt, A. Magedanz, T. Voigtla¨nder, A. Schmermund, Coronary CT angiography in managing atherosclerosis, Int. J. Mol. Sci. 16 (2) (2015) 3740
3756. [28] T.Z. Naqvi, M.-S. Lee, Carotid intima-media thickness and plaque in cardiovascular risk assessment, JACC: Cardiovasc. Imaging 7 (10) (2014) 1025
1038. [29] V. Nambi, L. Chambless, A.R. Folsom, et al., Carotid intima-media thickness and presence or absence of plaque improves prediction of coronary heart disease risk: the ARIC (atherosclerosis risk in communities) study, J. Am. Coll. Cardiol. 55 (15) (2010) 1600
1607. [30] L. Saba, G. Mallarini, R. Sanfilippo, G. Zeng, R. Montisci, J. Suri, Intima media thickness variability (IMTV) and its association with cerebrovascular events: a novel marker of carotid therosclerosis? Cardiovasc. Diagn. Ther. 2 (1) (2012) 10. [31] E. Cuadrado-Godia, M. Maniruzzaman, T. Araki, et al., Morphologic TPA (mTPA) and composite risk score for moderate carotid atherosclerotic plaque is strongly associated with HbA1c in diabetes cohort, Comput. Biol. Med. 101 (2018) 128
145. [32] A. Laine, J.M. Sanches, J.S. Suri, Ultrasound Imaging: Advances and Applications, Springer, 2012.

311

312

CHAPTER 15 A special report on changing trends

[33] L.E. Chambless, G. Heiss, A.R. Folsom, et al., Association of coronary heart disease incidence with carotid arterial wall thickness and major risk factors: the atherosclerosis risk in communities (ARIC) Study, 1987
1993, Am. J. Epidemiol. 146 (6) (1997) 483
494. [34] D.H. O’leary, J.F. Polak, R.A. Kronmal, et al., Distribution and correlates of sonographically detected carotid artery disease in the Cardiovascular Health Study. The CHS Collaborative Research Group, Stroke 23 (12) (1992) 1752
1760. [35] M.L. Bots, A.W. Hoes, P.J. Koudstaal, A. Hofman, D.E. Grobbee, Common carotid intima-media thickness and risk of stroke and myocardial infarction: the Rotterdam Study, Circulation 96 (5) (1997) 1432
1437. [36] M. Rosvall, L. Janzon, G. Berglund, G. Engstro¨m, B. Hedblad, Incident coronary events and case fatality in relation to common carotid intima-media thickness, J. Intern. Med. 257 (5) (2005) 430
437. [37] M.W. Lorenz, C. Schaefer, H. Steinmetz, M. Sitzer, Is carotid intima media thickness useful for individual prediction of cardiovascular risk? Ten-year results from the Carotid Atherosclerosis Progression Study (CAPS), Eur. Heart J. 31 (16) (2010) 2041
2048. [38] N.N. Khanna, A.D. Jamthikar, D. Gupta, et al., Rheumatoid arthritis: atherosclerosis imaging and cardiovascular risk assessment using machine and deep learning
based tissue characterization, Curr. Atheroscler. Rep. 21 (2) (2019) 7. [39] J.T. Salonen, R. Salonen, Ultrasonographically assessed carotid morphology and the risk of coronary heart disease, Arterioscler. Thromb. Vasc. Biol. 11 (5) (1991) 1245
1249. [40] T. Hirata, Y. Arai, M. Takayama, Y. Abe, K. Ohkuma, T. Takebayashi, Carotid plaque score and risk of cardiovascular mortality in the oldest old: results from the TOOTH study, J. Atheroscler. Thromb. 25 (1) (2018) 55
64. [41] H.W. Park, W.H. Kim, K.H. Kim, et al., Carotid plaque is associated with increased cardiac mortality in patients with coronary artery disease, Int. J. Cardiol. 166 (3) (2013) 658
663. [42] J.H. Stein, C.E. Korcarz, R.T. Hurst, et al., Use of carotid ultrasound to identify subclinical vascular disease and evaluate cardiovascular disease risk: a consensus statement from the American Society of Echocardiography Carotid Intima-Media Thickness Task Force endorsed by the Society for Vascular Medicine, J. Am. Soc. Echocardiogr. 21 (2) (2008) 93
111. [43] J.H. Stein, H.M. Johnson, Carotid intima-media thickness, plaques, and cardiovascular disease risk: implications for preventive cardiology guidelines, J. Am. Coll. Cardiol. 55 (15) (2010) 1608
1610. [44] J.F. Polak, M.J. Pencina, K.M. Pencina, C.J. O’Donnell, P.A. Wolf, R.B. D’Agostino Sr, Carotid-wall intima-media thickness and cardiovascular events, New Engl. J. Med. 365 (3) (2011) 213
221. [45] G.M. Allan, S. Garrison, J. McCormack, Comparison of cardiovascular disease risk calculators, Curr. Opin. Lipidol. 25 (4) (2014) 254
265. [46] B.A. Goldstein, A.M. Navar, R.E. Carter, Moving beyond regression techniques in cardiovascular risk prediction: applying machine learning to address analytic challenges, Eur. Heart J. 38 (23) (2016) 1805
1814. [47] M. Biswas, V. Kuppili, T. Araki, et al., Deep learning strategy for accurate carotid intima-media thickness measurement: An ultrasound study on Japanese diabetic cohort, Comput. Biol. Med. 98 (2018) 100
117.

References

[48] M. Biswas, V. Kuppili, D.R. Edla, et al., Symtosis: a liver ultrasound tissue characterization and risk stratification in optimized deep learning paradigm, Comput. Methods Prog. Biomed. 55 (2018) 165
177. [49] C. Lam, D. Yi, M. Guo, T. Lindsey, Automated detection of diabetic retinopathy using deep learning, AMIA Jt. Summits Transl. Sci. Proc. 2017 (2018) 147
155. [50] J. Heo, J. Yoon, H.J. Park, Y.D. Kim, H.S. Nam, J.H. Heo, Machine learning-based model can predict stroke outcome, Stroke 50 (2019) 1263
1265. [51] B.J. Erickson, P. Korfiatis, Z. Akkus, T.L. Kline, Machine learning for medical imaging, Radiographics 37 (2) (2017) 505
515. [52] B. Ambale-Venkatesh, C.O. Wu, K. Liu, et al., Cardiovascular event prediction by machine learning: the multi-ethnic study of atherosclerosis, Circ. Res. 121 (2017) 1092
1101. [53] S.F. Weng, J. Reps, J. Kai, J.M. Garibaldi, N. Qureshi, Can machine-learning improve cardiovascular risk prediction using routine clinical data? PLoS One 12 (4) (2017) e0174944. This study compared the ML-based risk startification with convetional risk calculators. [54] R.U. Acharya, O. Faust, A.P.C. Alvin, et al., Symptomatic vs. asymptomatic plaque classification in carotid ultrasound, J. Med. Syst. 36 (3) (2012) 1861
1871. [55] U.R. Acharya, O. Faust, A. Alvin, et al., Understanding symptomatology of atherosclerotic plaque by image-based tissue characterization, Comput. Methods Prog. Biomed. 110 (1) (2013) 66
75. [56] U.R. Acharya, O. Faust, S.V. Sree, et al., An accurate and generalized approach to plaque characterization in 346 carotid ultrasound scans, IEEE Trans. Instrum. Meas. 61 (4) (2012) 1045
1053. This was an important study that perfromed the carotid atheroscleriotic plaque characterization using ML approach. [57] U.R. Acharya, M.M.R. Krishnan, S.V. Sree, et al., Plaque tissue characterization and classification in ultrasound carotid scans: A paradigm for vascular feature amalgamation, IEEE Trans. Instrum. Meas. 62 (2) (2013) 392
400. [58] U.R. Acharya, M.R.K. Mookiah, S.V. Sree, et al., Atherosclerotic plaque tissue characterization in 2D ultrasound longitudinal carotid scans for automated classification: a paradigm for stroke risk assessment, Med. Biol. Eng. Comput. 51 (5) (2013) 513
523. [59] U.R. Acharya, S.V. Sree, M.M.R. Krishnan, et al., Atherosclerotic risk stratification strategy for carotid arteries using texture-based features, Ultrasound Med. Biol. 38 (6) (2012) 899
915. [60] I.A. Kakadiaris, M. Vrigkas, A.A. Yen, T. Kuznetsova, M. Budoff, M. Naghavi, Machine learning outperforms ACC/AHA CVD risk calculator in MESA, J. Am. Heart Assoc. 7 (22) (2018) e009476. This is the first of its kind study which have compared the machine learning-based risk calculator with ACC/AHA risk calculator. This article is very important to usage of ML in CVD risk assessment. [61] A. Ramachandran, C. Snehalatha, Current scenario of diabetes in India, J. Diabetes 1 (1) (2009) 18
28. [62] R. Gupta, R.S. Rao, A. Misra, S.K. Sharma, Recent trends in epidemiology of dyslipidemias in India, Indian Heart J. 69 (3) (2017) 382
392. [63] R. Anchala, N.K. Kannuri, H. Pant, et al., Hypertension in India: a systematic review and meta-analysis of prevalence, awareness, and control of hypertension, J. Hypertens. 32 (6) (2014) 1170.

313

314

CHAPTER 15 A special report on changing trends

[64] I.M. van der Meer, A. Iglesias del Sol, A.E. Hak, M.L. Bots, A. Hofman, J.C. Witteman, Risk factors for progression of atherosclerosis measured at multiple sites in the arterial tree: the Rotterdam Study, Stroke 34 (10) (2003) 2374
2379. [65] H. Øygarden, Carotid intima-media thickness and prediction of cardiovascular disease, J. Am. Heart Assoc. 6(1) (2017) e005313. [66] N.N. Khanna, A.D. Jamthikar, D. Gupta, et al., Performance evaluation of 10-year ultrasound image-based stroke/cardiovascular (CV) risk calculator by comparing against ten conventional CV risk calculators: a diabetic study, Comput. Biol. Med. 105 (2019) 125
143. [67] T. Araki, P.K. Jain, H.S. Suri, et al., Stroke risk stratification and its validation using ultrasonic Echolucent Carotid Wall plaque morphology: a machine learning paradigm, Comput. Biol. Med. 80 (2017) 77
96. [68] C.M. Bishop, Pattern Recognition and Machine Learning, Springer, 2006. [69] R.S. Sutton, A.G. Barto, Reinforcement Learning: An Introduction, MIT Press, 2018. [70] U.R. Acharya, S.V. Sree, M.M.R. Krishnan, et al., Automated classification of patients with coronary artery disease using grayscale features from left ventricle echocardiographic images, Comput. Methods Prog. Biomed. 112 (3) (2013) 624
632. [71] V.K. Shrivastava, N.D. Londhe, R.S. Sonawane, J.S. Suri, A novel and robust Bayesian approach for segmentation of psoriasis lesions and its risk stratification, Comput. Methods Prog. Biomed. 150 (2017) 9
22. [72] S.K. Banchhor, N.D. Londhe, T. Araki, et al., Wall-based measurement features provides an improved IVUS coronary artery risk assessment when fused with plaque texture-based features during machine learning paradigm, Comput. Biol. Med. 91 (2017) 198
212. [73] U. Acharya, S.V. Sree, M. Mookiah, et al., Computed tomography carotid wall plaque characterization using a combination of discrete wavelet transform and texture features: a pilot study, Proc. Inst. Mech. Eng., H: J. Eng. Med. 227 (6) (2013) 643
654. [74] J.C.M. Than, L. Saba, N.M. Noor, et al., Lung disease stratification using amalgamation of Riesz and Gabor transforms in machine learning framework, Comput. Biol. Med. 89 (2017) 197
211. [75] V.K. Shrivastava, N.D. Londhe, R.S. Sonawane, J.S. Suri, Computer-aided diagnosis of psoriasis skin images with HOS, texture and color features: A first comparative study of its kind, Comput. Methods Prog. Biomed. 126 (2016) 98
109. [76] G. Pareek, U.R. Acharya, S.V. Sree, et al., Prostate tissue characterization/classification in 144 patient population using wavelet and higher order spectra features from transrectal ultrasound images, Technol. Cancer Res. Treat. 12 (6) (2013) 545
557. [77] U.R. Acharya, S. Vinitha Sree, M.M. Krishnan, F. Molinari, R. Garberoglio, J.S. Suri, Non-invasive automated 3D thyroid lesion classification in ultrasound: a class of ThyroScan systems, Ultrasonics 52 (4) (2012) 508
520. [78] F. Molinari, K.M. Meiburger, L. Saba, et al., Automated carotid IMT measurement and its validation in low contrast ultrasound database of 885 patient Indian population epidemiological study: results of AtheroEdge® Software, Multi-Modality Atherosclerosis Imaging and Diagnosis, Springer, 2014, pp. 209
219. [79] F. Molinari, G. Zeng, J.S. Suri, Intima-media thickness: setting a standard for a completely automated method of ultrasound measurement, IEEE Trans. Ultrason. Ferroelectr. Freq. Control. 57 (5) (2010) 1112
1124.

References

[80] F. Molinari, K.M. Meiburger, J. Suri, Automated high-performance cIMT measurement techniques using patented AtheroEdget: a screening and home monitoring system, in: Eng. Med. Biol. Society, EMBC, 2011 Annu. Int. Conf. IEEE: 2011, IEEE, 2011, pp. 6651
6654. [81] L. Saba, S.K. Banchhor, T. Araki, et al., Intra-and inter-operator reproducibility of automated cloud-based carotid lumen diameter ultrasound measurement, Indian Heart J. 70 (5) (2018) 649
664. [82] J.H. Stein, M.C. Tattersall, Carotid intima-media thickness and cardiovascular disease risk prediction, J. Am. Coll. Cardiol. 63 (21) (2014) 2301
2302. [83] L. Saba, S.K. Banchhor, N.D. Londhe, et al., Web-based accurate measurements of carotid lumen diameter and stenosis severity: an ultrasound-based clinical tool for stroke risk assessment during multicenter clinical trials, Comput. Biol. Med. 91 (2017) 306
317. [84] P. Krishna Kumar, T. Araki, J. Rajan, et al., Accurate lumen diameter measurement in curved vessels in carotid ultrasound: an iterative scale-space and spatial transformation approach, Med. Biol. Eng. Comput. 55 (8) (2017) 1415
1434. [85] P.K. Kumar, T. Araki, J. Rajan, J.R. Laird, A. Nicolaides, J.S. Suri, State-of-the-art review on automated lumen and adventitial border delineation and its measurements in carotid ultrasound, Comput. Methods Prog. Biomed. 163 (2018) 155
168. [86] L. Saba, T. Araki, P.K. Kumar, et al., Carotid inter-adventitial diameter is more strongly related to plaque score than lumen diameter: an automated tool for stroke analysis, J. Clin. Ultrasound 44 (4) (2016) 210
220. [87] K.-R. Muller, S. Mika, G. Ratsch, K. Tsuda, B. Scholkopf, An introduction to kernel-based learning algorithms, IEEE Trans. Neural Netw. 12 (2) (2001) 181
201. [88] N.N. Khanna, A.D. Jamthikar, T. Araki, et al., Nonlinear model for the carotid artery disease 10-year risk prediction by fusing conventional cardiovascular factors to carotid ultrasound image phenotypes: a Japanese diabetes cohort study, Echocardiography 36 (2) (2019) 345
361. [89] M. Maniruzzaman, M.J. Rahman, M. Al-MehediHasan, et al., Accurate diabetes risk stratification using machine learning: role of missing value and outliers, J. Med. Syst. 42 (5) (2018) 92. [90] M. Maniruzzaman, N. Kumar, M. Menhazul Abedin, et al., Comparative approaches for classification of diabetes mellitus data: Machine learning paradigm, Comput. Methods Prog. Biomed. 152 (2017) 23
34. [91] T. Araki, N. Ikeda, D. Shukla, et al., A new method for IVUS-based coronary artery disease risk stratification: a link between coronary & carotid ultrasound plaque burdens, Comput. Methods Prog. Biomed. 124 (2016) 161
179. [92] S.J. Al’Aref, K. Anchouche, G. Singh, et al., Clinical applications of machine learning in cardiovascular disease and its relevance to cardiac imaging, Eur. Heart J. 40 (24) (2019) 1975
1986. [93] C.-L. Chou, Y.-J. Wu, C.-L. Hung, et al., Segment-specific prevalence of carotid artery plaque and stenosis in middle-aged adults and elders in Taiwan: a communitybased study, J. Formos. Med. Assoc. 118 (1) (2019) 64
71. [94] S. Farkas, S. Molna´r, K. Nagy, T. Hortoba´gyi, L. Csiba, Comparative in vivo and in vitro postmortem ultrasound assessment of intima-media thickness with additional histological analysis in human carotid arteries, Perspect. Med. 1 (1) (2012) 170
176.

315

316

CHAPTER 15 A special report on changing trends

[95] G. Gamble, B. Beaumont, H. Smith, et al., B-mode ultrasound images of the carotid artery wall: correlation of ultrasound with histological measurements, Atherosclerosis 102 (2) (1993) 163
173. [96] C. Cortes, V. Vapnik, Support-vector networks, Mach. Learn. 20 (3) (1995) 273
297. [97] A. Dalbeni, A. Giollo, A. Tagetti, et al., Traditional cardiovascular risk factors or inflammation: Which factors accelerate atherosclerosis in arthritis patients? Int. J. Cardiol. 236 (2017) 488
492. [98] U.R. Acharya, S.V. Sree, F. Molinari, L. Saba, A. Nicolaides, J.S. Suri, An automated technique for carotid far wall classification using grayscale features and wall thickness variability, J. Clin. Ultrasound 43 (5) (2015) 302
311. [99] E.C. Kyriacou, S. Petroudi, C.S. Pattichis, et al., Prediction of high-risk asymptomatic carotid plaques based on ultrasonic image features, IEEE Trans. Inf. Technol. Biomed. 16 (5) (2012) 966
973. [100] A. Gastounioti, S. Makrodimitris, S. Golemati, N.P. Kadoglou, C.D. Liapis, K.S. Nikita, A novel computerized tool to stratify risk in carotid atherosclerosis using kinematic features of the arterial wall, IEEE J. Biomed. Health Inform. 19 (3) (2015) 1137
1145. [101] X. Hu, P.D. Reaven, A. Saremi, et al., Machine learning to predict rapid progression of carotid atherosclerosis in patients with impaired glucose tolerance, EURASIP J. Bioinform. Syst. Biol. 2016 (1) (2016) 14. [102] R. Narain, S. Saxena, A.K. Goyal, Cardiovascular risk prediction: a comparative study of Framingham and quantum neural network based approach, Patient Preference Adherence 10 (2016) 1259. [103] P. Unnikrishnan, D.K. Kumar, S. Poosapadi Arjunan, H. Kumar, P. Mitchell, R. Kawasaki, Development of health parameter model for risk prediction of CVD using SVM, Comput. Math. Methods Med. 2016 (2016) 1
7. [104] K. Zarkogianni, M. Athanasiou, A.C. Thanopoulou, K.S. Nikita, Comparison of machine learning approaches toward assessing the risk of developing cardiovascular disease as a long-term diabetes complication, IEEE J. Biomed. Health Inform. 22 (5) (2018) 1637
1647. [105] L. Saba, P.K. Jain, H.S. Suri, et al., Plaque tissue morphology-based stroke risk stratification using carotid ultrasound: a polling-based PCA learning paradigm, J. Med. Syst. 41 (6) (2017) 98. [106] M. Motwani, D. Dey, D.S. Berman, et al., Machine learning for prediction of allcause mortality in patients with suspected coronary artery disease: a 5-year multicentre prospective registry analysis, Eur. Heart J. 38 (7) (2017) 500
507. [107] V.A. Korshunov, S.M. Schwartz, B.C. Berk, Vascular remodeling: hemodynamic and biochemical mechanisms underlying Glagov’s phenomenon, Arterioscler. Thromb. Vasc. Biol. 27 (8) (2007) 1722
1728. [108] Y. Leskinen, T. Lehtimaki, A. Loimaala, et al., Carotid atherosclerosis in chronic renal failure-the central role of increased plaque burden, Atherosclerosis 171 (2) (2003) 295
302. [109] L. Razzouk, C.B. Rockman, M.R. Patel, et al., Co-existence of vascular disease in different arterial beds: peripheral artery disease and carotid artery stenosis—data from Life Line Screening(®), Atherosclerosis 241 (2) (2015) 687
691. [110] C. Banerjee, M.I. Chimowitz, Stroke caused by atherosclerosis of the major intracranial arteries, J. Vasc. Surg. 65 (6) (2017) 1864
1865.

References

[111] P.C. Chen, J.S. Jeng, H.C. Hsu, T.C. Su, K.L. Chien, Y.T. Lee, Carotid atherosclerosis progression and risk of cardiovascular events in a community in Taiwan, Sci. Rep. 6 (2016) 25733. [112] E. Cuadrado-Godia, S.K. Srivastava, L. Saba, et al., Geometric total plaque area is an equally powerful phenotype compared with carotid intima-media thickness for stroke risk assessment: a deep learning approach, J. Vasc. Ultrasound 42 (4) (2018) 162
188. [113] K.W. Beach, Principles of ultrasonic imaging and instrumentation, in: A. Nicolaides, K.W. Beach, E. Kyriacou, C.S. Pattichis (Eds.), Ultrasound and Carotid Bifurcation Atherosclerosis, Springer London, London, 2012, pp. 67
96. [114] A. Gupta, K. Kesavabhotla, H. Baradaran, et al., Plaque echolucency and stroke risk in asymptomatic carotid stenosis: a systematic review and meta-analysis, Stroke 46 (1) (2015) 91
97. [115] A. Huibers, G.J. de Borst, R. Bulbulia, H. Pan, A. Halliday, Plaque echolucency and the risk of ischaemic stroke in patients with asymptomatic carotid stenosis within the first asymptomatic carotid surgery trial (ACST-1), Eur. J. Vasc. Endovasc. Surg. 51 (5) (2016) 616
621. [116] V. Kotsis, A.D. Jamthikar, T. Araki, et al., Echolucency-based phenotype in carotid atherosclerosis disease for risk stratification of diabetes patients, Diabetes Res. Clin. Pract. 143 (2018) 322
331. [117] T.H. Park, Evaluation of carotid plaque using ultrasound imaging, J. Cardiovasc. Ultrasound 24 (2) (2016) 91
95. [118] E. Picano, M. Paterni, Ultrasound tissue characterization of vulnerable atherosclerotic plaque, Int. J. Mol. Sci. 16 (5) (2015) 10121
10133. [119] A.N. Nicolaides, S.K. Kakkos, E. Kyriacou, et al., Asymptomatic internal carotid artery stenosis and cerebrovascular risk stratification, J. Vasc. Surg. 52 (6) (2010) 1486
1496.e1485. [120] L.M. Pedro, J.M. Sanches, J. Seabra, J.S. Suri, J. Fernandes e Fernandes, Asymptomatic carotid disease—a new tool for assessing neurological risk, Echocardiography 31 (3) (2014) 353
361. [121] K. Pahan, Lipid-lowering drugs, Cell. Mol. Life Sci. 63 (10) (2006) 1165
1178. [122] M. Abramowicz, G. Zuccotti, J.-M. Pflomm, Metformin for prediabetes (Reprinted from The Medical letters on Drugs and Therapeutics, vol 58, pg 141, 2016), JAMA 317 (11) (2017) 1171. [123] Y. LeCun, Y. Bengio, G. Hinton, Deep learning, Nature 521 (2015) 436. [124] G. Litjens, T. Kooi, B.E. Bejnordi, et al., A survey on deep learning in medical image analysis, Med. Image Anal. 42 (2017) 60
88. [125] D. Shen, G. Wu, H.-I. Suk, Deep learning in medical image analysis, Annu. Rev. Biomed. Eng. 19 (2017) 221
248. ` . Betriu, et al., A convolutional neural network for [126] K. Lekadir, A. Galimzianova, A automatic characterization of plaque composition in carotid ultrasound, IEEE J. Biomed. Health Inform. 21 (1) (2017) 48
55. This was an important study that perfromed the carotid atheroscleriotic plaque characterization using DL approach. [127] M. Zreik, R.W. van Hamersvelt, J.M. Wolterink, T. Leiner, M.A. Viergever, I. Iˇsgum, A recurrent CNN for automatic detection and classification of coronary artery plaque and stenosis in coronary CT angiography, IEEE Trans. Med. Imaging 38 (7) (2019) 1588
1598.

317

318

CHAPTER 15 A special report on changing trends

[128] Can We Open the Black Box of AI? (2016) Available from: ,https://www.nature. com/news/can-we-open-the-black-box-of-ai-1.20731.. [129] M. Henglin, G. Stein, P.V. Hushcha, J. Snoek, A.B. Wiltschko, S. Cheng, Machine learning approaches in cardiovascular imaging, Circ. Cardiovasc. Imaging 10 (10) (2017) e005614.

Appendix: performance evaluation parameters Sensitivity and specificity are computed using true positive (TP), true negative (TN), false positive (FP), and false negative (FN). TP indicates the count for which predicted class labels match with ground truth label for high-risk threshold point; FN is defined as the number of times the predicted class labels that are incorrectly classified as low risk, FP is defined as the number of times the predicted class labels that are incorrectly classified as high risk, and TN is defined as the number of times predicted class labels that are correctly matched with lowrisk ground truth label. Sensitivity and specificity are mathematically represented as, sensitivity 5 TP=ðTP 1 FNÞ and specificity 5 TN=ðTN 1 FPÞ. Furthermore, the accuracy of risk stratification is mathematically represented as accuracy 5 ðTP 1 TNÞ=ðTP 1 FN 1 FP 1 FNÞ.

CHAPTER

16

A healthcare text classification system and its performance evaluation: a source of better intelligence by characterizing healthcare text

Saurabh Kumar Srivastava1, Sandeep Kumar Singh1 and Jasjit S. Suri2 1

Department of Computer Science & Engineering, JIIT Noida, Noida, India Advanced Knowledge Engineering Center, Global Biomedical Technologies, Inc., Roseville, CA, United States

2

16.1 Introduction Text classification provides the conceptualized meaning to real-world collections. A text classification system categorizes documents in one or more predefined classes according to the textual contents. This can be further useful for text-based surveillance system especially in social media- and health-related insights [1] for timely and massive information extraction from large datasets [2]. The role of social media for biomedical domain has a significant impact on relevant knowledge extraction using healthcare ontology [3]. The text miner can extract the text information that can be shared between patients and healthcare decision makers for a large-scale text-based disease surveillance system [4]. It can also be used for mining health-related information that can be utilized by both patients and practitioners. Text data mining has predominantly adapted machine learning (ML) algorithms for text classification [5]. The presence of noise in text data can distort text information and can largely impact the classifier’s performance during ML applications [4,5]. It causes legibility of the text by damaging the interpretation of the text and this could have serious consequences in healthcare. This noise can be categorized in the form of misrepresentation of the text information, and can be quantified as misrepresentation ratio (MRR). Further, due to this misrepresentation, ML classifiers are unable to learn and generalize under cross-validation protocols [6,7]. Thus this results in low accuracies when classifying the text information.

Cognitive Informatics, Computer Modelling, and Cognitive Science, Volume 2. DOI: https://doi.org/10.1016/B978-0-12-819445-4.00016-3 © 2020 Elsevier Inc. All rights reserved.

319

CHAPTER 16 A healthcare text classification system

One important area that is being untouched in text classification is characterization of input text and linking this input characterized text to the performance of the ML system (see Fig. 16.1). The figure shows how MRR is linked between the input healthcare text data and the performance of the ML system. The figure shows that different types of data (having different MRR values) can be fed to the ML system to predict the class label for testing data, which can then compute the performance of the ML system. Thus our study explores a unique and powerful mechanism that creates further scope for the design of better

Start

Data type Disease

Twitte rA

SMS

WebKB4

DS1

DS2

DS3

DS4

Protocol type

DS5

K4

CL2

Machine learning system

K5

CL3

K10

Classifier type

MRR

R8

CL1

K2

Misrepresentation ratio

320

CL4 JK CL5

Partition protocol

Classifiers

Predicted class

Performance type

ACC

SEN

SPE

ROC

AUC

PPV

End

FIGURE 16.1 Role of MRR linking input data and performance evaluation via machine learning paradigm. Five datasets: DS1, DS2, DS3, DS4, DS5; five protocols: K2, K4, K5, K10, JK; five classifiers: CL1, CL2, CL3, CL4, and CL5. MRR, Misrepresentation ratio.

16.2 Brief literature survey and our proposed model

algorithms for text classification, an intelligence that is so necessary to have the best impedance match between the type of classifier adapted in ML, and the input text data type having certain noise characteristics. Further, this intelligence can be optimized when the amalgamation of attributes is involved such as ML partition protocol and the type of features used for achieving generalization in ML.

16.2 Brief literature survey and our proposed model Several classification techniques have been presented in the area of text classification. Kautz et al. [8] developed a text classification system where the data type had multiple classes. The author used the “imbalance” dataset for their analysis, where size varied from 21 to 2156. The study used the ANOVA model and showed an accuracy (ACC) of 86%. The study did not use conventional performance measures such as receiver operating characteristic (ROC), area under the curve (AUC), sensitivity (SEN), rather it suggested a scheme named “multiclass performance score,” a generic performance measure that had minimum influence of training and testing conditions over all multiclass problems. Even though the system showed reasonable ACC, the system did not characterize the input data with respect to ML performance. In 2011 Japkowicz and Shah [9] demonstrated ML-based application for text classification and presented several types of feature extraction methods. It was an informative collection for beginners. Not much was emphasized on the characterization of the input text data and its interactive role with classifiers. Sokolova and Lapalme [10] presented systematic analysis of 24 measures based on ML paradigm. The result was based on measure invariance taxonomy with all relevant label distribution. The system did not deliver the performance, rather it illustrated role of statistical consistency and metrics relationship while showing classifier performance. Huang and Ling [11] proposed a greedy search
based evaluation measure and tested system on 20 different datasets using artificial neural network. The average ACC of the system was 77.43%. The authors demonstrated the system in context of classification, but there was no significance of noise characterstics in the proposed model. Thus one could not evaluate the design of their hypothesis. Wong et al. [12] showed a performance enhancement scheme based on hedge (weight updation) algorithm, which was capable of improving the AUC and traditional performance measures. This algorithm considered weight updating classifier for AUC optimizaiton. The results were evaluated on Reuters dataset (21578). The authors showed that AUC improved by 10% over the baseline. There was no hypothesis laid out, and the input data was not characterized to link with the performance measure. Iwata et al. [13] hypothesized that the classes in different taxonomies were correlated with target classes and could participate in classifier performance. Further, author validated experimentally using 20News dataset with approximately 20,000 documents. Naive Bayes algorithm was adapted that achieved the best ACC of 87%.

321

322

CHAPTER 16 A healthcare text classification system

Sriram et al. [14] improved the traditional bag of words (BOW) model by extracting domain-specific features from user profile. They showed that BOW-A method achieves 18.3% improvement over traditional BOW model. Further, the paper had no hypothesis regarding characterizing input datasets. Caragea et al. [15] compared traditional BOW model with rule-based models. The author showed that structure-based features could improve the performance of classification task. The study created his own web crawled dataset of 2000 documents that showed the structural features with Random Forest achieved the best ACC of 92.83%. In summary, we conclude that none of the previous algorithms demonstrated a link between the input data type and the performance measure by creating some kind of hypothesis, which is so necessary for evaluation of the ML systems and the type of classifiers adapted. Our study is the first study to bring the concept of linking the input data type with known noise characteristics in the form of MRR. We therefore link the performance of the ML-based system on five types of text classifiers to the characteristics of the input data. One way to characterize such a data is via computing the MRR that measures the amount of noise present in a dataset. Higher the MRR (noise) of a dataset, poorer will be the performance (ACC) of ML system.

16.2.1 Our model This study hypothesizes the role of MRR and performance evaluation (PE) of the classification systems—a unique contribution toward evaluation of healthcare text classification systems. Our study takes a different approach in which we target and understand the source and the cause of the issue, which focuses on understanding the characterization of input text data. Thus we look a step closer to model the input text data by estimating how worse the text misrepresentation is. Mathematically, one can express this misrepresentation in the form of MRR. By doing this, one can better appreciate the link between the hypothesis and PE in ML paradigm. This hypothesis is streamlined by taking several classes of data with an increasing order of MRR. Thus if the ML system generalizes well on lower MRR values, then one can characterize a particular ML system for a particular text data type: an intelligence that is necessary in evaluating the performance of surveillance systems. Since ML system consists of several attributes such as classifier type and protocol type, it is therefore vital to model the performance of the ML system based on these attributes along with the input data (having a known MRR). The validation of the hypothesis is concluded if our assumption of ML behavior is consistent with the MRR data type, which states “the ACC of the system will fall if the MRR rises.” To model the approach in a comprehensive way, we consider a variety of data types, training partition protocol types, and classifier types. Our system uses a conventional ML approach where the offline training parameters are computed by adapting the combination of observed healthcare text

16.3 Data types

tweets and the corresponding ground truth labels for the healthcare tweets. For example, disease dataset has tweets with five kinds of labels: abdominal pain, cough, conjunctivitis, diarrhea, and nausea. Similarly in TwitterA dataset, the ground truth labels are no-health tweet, sickness of the patient, no-sickness of the patient, and improper English in the tweet. The online testing system consists of transforming the test text data by the offline parameters to predict the multiple classes. If one can model the input data in terms of noise characteristics one can better reason the variations in classifier performance with different datasets. We presented intercomparison work with existing research in the benchmarking Table 16.6. The spirit of our system comes from the recent model proposed by Suri’s group (see Shrivastava et al. [16,17]) where the hypothesis was clearly build, and solid feature selection strategies were adapted for superior classification and PE. Further, the same team demonstrated the design of reliability and stability indices. Current research requires an adaptable and reliable classifier system that could produce accurate results in all the category of text datasets. The rest of the chapter is organized as follows. Section 16.3 presents five kinds of text data along with their MRR characteristics. The methodology based on BOW is presented in Section 16.4 along with the ML system. Section 16.5 demonstrates the experimental protocols, and finally, Section 16.6 shows the results. Section 16.7 explains hypothesis validation and PE, and Section 16.8 shows discussions on evaluated results. The study then presents the conclusions and future work.

16.3 Data types We considered two categories of datasets that belong to different MRR values. First category belongs to more unstructured domain. The unstructured datasets does not organize in predefined manner and it contains links, slang words (common in speech), and repetition of texts and lacks with pattern predictability. Three types of dataset were considered: TwitterA, Disease, and SMS. Twitter and diseased data types were from Twitter containing tweets on the healthcare domain and SMS data type is typically short (small in size) mobile text messages. During preprocessing step for Twitter data, links (such as video and image links) and retweets are removed as they do not have any impact during analysis. Finally, the dataset contains unique information (no duplication). Second category belongs to structured domain, which consisted of WebKB4 and Reuters (R8) datasets. WebKB4 category consists of student, project, course, and facutly related information. Reuters (R8) dataset belongs to news category, it has eight classes corresponding to its instances. In this category, instances are holding the appropriate information of its corresponding ground truth.

323

324

CHAPTER 16 A healthcare text classification system

The limitations in first-category Twitter datasets are tweet size, jargons, and typos. While the second category have longer text messages confined to their assigned labels. We have taken five different MRR-based data types Appendix (Table B.1): TwitterA [18], Disease (prepared corpus), SMS [19], WebKB4 [20], and Reuters (R8) [21]. The dataset has variation in their instance sizes (2010
7674) and ground truth (2
8 classes). All the datasets are considered for exhaustive result evaluation based on classifiers (c), validation protocols (k), and number of trials (t). All the dataset have a common language format (English). In the analysis, we found that the ML performance is directly binded with data-related MRR. All the data-related MRR is calculated by identifying important terms from each datasets. Stopwords are the terms, which are commonly used anddo not have significance in corpus. We calculate important terms by removing stopwords from the dataset and making remaining terms stemmed. Eq. (16.1) calculates the MRR for a particular datset:
 It 3 100 MRRð%Þ 5 1 2 Tt

(16.1)

where It and Tt represent the terms that are not stopwords and total terms, respectively. Here, text perturbation is considered as noise factor which is represented by MRR value. Informative terms are counted when stopwords are removed from the dataset and all the terms are stemmed. We discuss each of the selected data types and their corresponding MRR.

16.3.1 Data type 1: TwitterA dataset TwitterA dataset is manually created dataset, which consists basically tweets on health-related messages. We focus on only textual information, so other irrelevant features (hash tags, links, and retweets) are eliminated. A total of 5128 tweets are labeled into four different categories: sick, health, no sick, and not English. Authors in Ref. [18] also mentioned that a total of 1832 (35.73%) tweets are in health category while others 3296 (64.27%) tweets are in nonhealth category. As the dataset is manually designed so, Twitter’s diversities and noise is not considered during experiment. Authors considered concrete features in dataset that allow classifiers improved rate of learning that would help in validation of proposed method. The dataset description is given in Appendix (Table B.2), and sample data is presented in Appendix A.1.

16.3.2 Data type 2: WebKB4 dataset WebKB4 [20] contains web pages collected from the department of computer science of four universities (Cornell, Texas, Washington, and Wisconsin) in January 1997 under the text learning project at Carnegie Mellon University. These pages are divided into four categories. A total 4199 samples are classified into project

16.3 Data types

(504), course (930), student (1641), and faculty (1124). These samples are organized into directory structure. We considered WebKB4 datasets with four mentioned labels. For example, a particular faculty may be represented by home page, publication list, and curriculum vitae. Only faculty home page is part of faculty class. The publication details, vitae, and research interest pages are placed in other categories. This dataset is more structured and informed than TwitterA dataset as it contains specific labels. The dataset description is given in Appendix (Table B.3), and sample data is presented in Appendix A.2.

16.3.3 Data type 3: Disease dataset For the preparation of Disease dataset, five different symptoms have been selected: abdominal pain, conjunctivitis, cough, diarrhea, and nausea. Moreover, for the same 12,146 raw tweets were collected by using synonyms of mentioned disease keywords in Ref. [22]. To collect the tweet, Python’s tweepy API [23] is used. We have chosen random time of around 3
4 hours in a day for tweets collection, and these random hours are used for continuously 5 days for each symptom tweets. After collection, with the help of domain expert, we categorized all the tweets into its category and finally a refined 2010 dataset is prepared. This dataset have 365 abdominal pain, 501 cough, 407 diarrhea, 491 nausea, and 246 conjunctivitis related tweets. Samples are presented in Appendix A.3, and datarelated description is presented in Appendix (Table B.4).

16.3.4 Data type 4: Reuters (R8) dataset Reuters (R8) [21] dataset is originally collected and labeled by Carnegie Group, Inc. and Reuters Ltd., an international news agency division of Thomson Reuters. It is more structured and widely used collections for text categorization research. Reuters (R8) is part of Reuters-21578 samples. Reuters (R8) contains eight categories partitioned unevenly. A total of 7674 documents are classified into acq (2292), crude (374), earn (3923), grain (51), interest (271), money-fx (293), ship (144), and trade (326) categories. Two categories grain and ship are very small in terms of their samples. The dataset description is given in Appendix (Table B.5), and sample data is presented in Appendix A.4.

16.3.5 Data type 5: SMS dataset SMS spam dataset [19] is a collection of messages tagged with spam and ham. The collection of 3375 SMS ham messages are randomly extracted from the Department of Computer Science, National University Singapore, which consists 10,000 legitimate messages. A total of 425 spam SMS messages are taken from Grumbletext Website: a UK forum in which users make public claim about SMS spam messages. A total of 450 SMS ham messages are collected from PhD thesis. A total of 1002 SMS ham and 322 spam messages are collected from SMS spam

325

326

CHAPTER 16 A healthcare text classification system

corpus. Average no. of words and length are 15.72 and 4.44 character long. A total of 5574 SMS messages are used for our analysis. The dataset holds 747 spam and 4827 ham messages. The dataset description is given in Appendix (Table B.6), and sample dataset is presented in Appendix A.5. All the considered datasets are described in Appendix (Table B.1).

16.4 Methodology The central engine of the system presented in Fig. 16.1 is the ML system. This figure showed the role of MRR linking input data and PE via ML system by characterizing the input dataset. The heart of the system is further expanded in Fig. 16.2 that constitues the architecture of ML. It consists of two phases: training phase and testing phase. In training phase the computed features are passed to the training-based classifier along with the ground truth labels to generate the offline training coefficients. These cofficients are then transformed by the online features, computed using testing datasets to generate the predicted class. This class is then compared against the ground truth lables to evaluate the cross-validation Protocol type

Data type

Training text data

Testing text data

Feature extraction

Feature extraction

Extracted training features

Extracted testing features

Classifier type Offline classifier Ground truth labels

Offline training parameters

Online classification

Predicted class

Offline text classification system

FIGURE 16.2 Architecture of machine learning model.

Online text classification system

16.4 Methodology

performance of the ML system. We use BOW model that considers all the terms in the text and creates a respective vector for the document. It represents all the documents of a dataset in the form of vectors. The core of the ML system is the classifier that helps in training and testing the incoming features. We therefore briefly present these classifiers used in our paradigm.

16.4.1 Brief discussion on classifiers MRR deteriorates the performance of the ML systems. This study incorporates the relationship between the input text data and output performance via the ML layer, while validating the hypothesis. Our hypothesis is validated by considering five set of classifiers: support vector machine (SVM), multilayer perceptron (MLP), AdaBoost (AB), stochastic gradient descent (SGD), and decision tree (DT); five set of data types; and five set of cross-validation protocols. We briefly discuss them, keeping in mind that they are fully plug-and-play subsystems. Readers can look at the references for more details.

16.4.1.1 Support vector machine The SVM [24] is a classifier that maximizes the distance between decision hyperplane [25] and treated as dimensional vector, which is called support vectors. Initially SVM was designed to support two class problem; here, we have extended to support multiclass problem. For our experiment, we consider classification using linear model of the form of: yðxÞ 5 wT φðxÞ 1 b

(16.2)

where φ(x) denotes kernel function that denotes the feature transformation; basically kernel functions are used to transform original feature space to a higher dimensional feature space [26,27]. The feature becomes linearly separable where b is a bias parameter. Vector w is normal to the hyperplane. The training input feature vector is represented by vector x. The test feature vectors are classified and represented by y(x).

16.4.1.2 Multilayer perceptron A MLP [24] is category of neural network. It follows feed forward mechnism that maps input data onto corresponding outputs. MLP consists multiple layers, where layers are fully connected to the next one in the form of directed graph. The nodes in MLP acts as a processing element with a nonlinear activation function. MLP follows standard linear perceptron to distinguish data that are not linearly separable.

16.4.1.3 AdaBoost AB is termed Adaptive Boosting also popular for its metalearning [28] feature. The term “meta” refers to combination of other learning algorithms. It is sensitive to noisy data. The AB works on weighing and combining methodology in learning phase.

327

328

CHAPTER 16 A healthcare text classification system

16.4.1.4 Stochastic gradient descent SGD, also known as incremental gradient descent [29], is a stochastic approximation of the gradient descent optimization for minimizing objective function. In other words, SGD tries to find minima or maxima by iteration. SGD follows discriminative learning of linear classifier under convex loss function, so it is a combination of SVM and logistic regression. The algorithm is popular because of its efficiency and ease of implementation.

16.4.1.5 Decision tree DT [24] is a classifier that maps observations to the form of target values. In DT leaves represent as class label and branches represents conjunctions. The DT highlights some advantages [7] over other classifiers as it uses rules for data classification. These rules are comprehensive and hence allows its end user to confidently accept the classifier result. Two most popular variants are J48 and Random Forest.

16.5 Experiment protocol We use five different kinds of cross-validation protocols (K2, K4, K5, K10, and JK) in our study. These protocols are used with each data types and classifier type. Since we need to study the impact of MRR on the prediction ACC, we therefore use exhaustive set of partition protocols.

16.5.1 Experimental protocol 1: system classifier accuracy computation over all parameters The objective of this protocol is to estimate the system’s classifier ACC ηðcÞ by running all five set of data, all sets of protocols, and all sets of trials per protocol for each type of classifier. This can be me mathmatically represented as ηðcÞ and represented by the following equation: d5D 5T P k5K P tP

ηðcÞ 5

ηðd;c;k;tÞ

d51 k51 t51

D3K 3T

(16.3)

where ηðd; c; k; tÞ represents the ACC of the classifer computed when data type is d, classifier type is c, protocol type is k, and trial number is t. When the total number of data types, classifiers, protocols types, and trials are represented by D, C, K, and T, the mean ACC of the performance of classification algorithms are evaluated in terms of performance measures, that is, ROC, AUC, ACC, positive predictive value (PPV), SEN, and specificity (SPE). If TP, FP, TN, and FN are number of true positives, false positives, true negatives, and false negatives, respectively, then the performance measures can be defined as follows:

16.5 Experiment protocol

16.5.2 Experimental protocol 2: effect of training data size on classification accuracy The objective of this protocol is to understand the learning behavior of the ML system and further to study the effect of the training data on the text classification ACC. Thus for each dataset (DSn), we divided the dataset into 10 parts and selected incrementally 10% more data in successive iterations. For each incremental data size, we compute the system classification ACC using all data types (D), all classifiers (C), all protocols (K), and all trials (T). This is mathematically given as per the following equation: d5DðN 5T P tr Þ c5C P k5K P tP

ηsys ðNtr Þ 5

ηðd; c; k; tÞ

c51 k51 t51

d51

D3C3K 3T

(16.4)

16.5.3 Experimental protocol 3: overall mean performance using all parameters: D, C, K, and T The overall system is computed by considering all the parameters. If ηðd; c; k; tÞ represents the ACC of the classifer computed when data type is d, classifier type is c, protocol type is k, and trial number is t, and total number of data types, classifiers, and protocols types are D, C, K, and T, then the mean ACC of the system ηsys is mathematically expressed as: d5D 5T P c5C P k5K P tP

ηsys 5

ηðd; c; k; tÞ

d51 c51 k51 t51

D3C3K 3T

(16.5)

Sensitivity It is the statistical measure which shows the proportion of actual positive samples that are correctly classified and can be expressed mathematically as:


 TP 3 100 TP 1 FN

SENð%Þ 5

(16.6)

Specificity It is the statistical measure which shows the proportion of actual negative samples that are correctly classified and can be expressed mathematically as:


 TN SPEð%Þ 5 3 100 FP 1 FN

(16.7)

329

330

CHAPTER 16 A healthcare text classification system

Positive predictive value It is the proportion of the true positives against all the positive classification results and can be expressed mathematically as:
PPVð%Þ 5

 TP 3 100 TP 1 FP

(16.8)

Accuracy It is the proportion of true results against all classification results and can be expressed mathematically as:
ACCð%Þ 5

 TP 1 TN 3 100 TP 1 FP 1 TN 1 FN

(16.9)

16.6 Results This section shows the characterization of ML-based systems on the basis of training data size. Our system uses different MRR-based text datasets, different training protocols, and different classifier types for result evaluation. The section shows the results based on the theory discussed in the previous section. The section is divided into three subsections presenting the classifier performance with respect to different datasets and cross-validation protocols.

16.6.1 Results of protocol #1: system accuracy computation over all parameters Keeping the objective for protocol 1 in mind, we plotted the classifiers performance using all the K set of protocols and D sets of data. All performance parameters such as ACC, PPV, SEN, SPE, and AUC are computed. The bar chart showing the comparisons between different classifier outputs is shown in Fig. 16.3, and the corresponding performance parameters is presented in Table 16.1. It can be seen in Fig. 16.3 that neural network category (MLP) performs best among all C classifiers. The corresponding performance parameters can be seen in Fig. 16.4.

16.6.2 Results of protocol #2: effect of the training data size on classification accuracy Our observations show that with an increase in training data size, the system performance increases. This behavior of ML system under this condition of changing

Accuracy (%)

16.6 Results

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

89.96%

SVM-L

91.84%

MLP

86.99%

Adaboost

86.54%

SGD

88.15%

DT

Classifier(s)

FIGURE 16.3 Bar chart representing the mean classifier accuracies for C classifiers over all the data types D, using K protocols, and T trials (D 5 5, K 5 5, and T 5 10).

Table 16.1 Mean and standard deviation of five different classifiers based on statistical attributes over all the datasets. Classifiers (%)

SVM-L

MLP

AdaBoost

SGD

DT

AUC ACC PPV SEN SPE

97.00 6 0.01 89.96 6 0.05 91.80 6 0.04 90.24 6 0.05 75.98 6 0.10

94.74 6 0.03 91.84 6 0.04 91.90 6 0.04 91.84 6 0.04 61.72 6 0.07

96.88 6 0.01 86.99 6 0.07 91.76 6 0.04 89.20 6 0.06 65.51 6 0.07

96.22 6 0.02 86.54 6 0.07 90.84 6 0.05 90.36 6 0.05 64.62 6 0.09

91.75 6 0.05 88.15 6 0.06 88.00 6 0.06 88.00 6 0.07 61.24 6 0.08

Performance (%)

ACC, Accuracy; AUC, area under the curve; DT, decision tree; MLP, multilayer perceptron; PPV, positive predictive value; SEN, sensitivity; SGD, stochastic gradient descent; SPE, specificity; SVM-L, support vector machine with linear kernel.

100 90 80 70 60 50 40 30 20 10 0

AUC Accuracy Precision Sensitivity Specificity

SVM-L

MLP

AdaBoost

SGD

DT

Classifier(s)

FIGURE 16.4 Performance of five different classifiers based on statistical attributes over all the data types D, all protocols K, and all the trials T (D 5 5, K 5 5, and T 5 10).

331

CHAPTER 16 A healthcare text classification system

Accuracy (%)

332

92 91 90 89 88 87 86 85 84 83 82 10

20

30

40

50

60

70

80

90

100

Training data size (N) FIGURE 16.5 Mean accuracy versus change in training data size over all the data types D, all protocols K, and all the trials T (D 5 5, K 5 5, T 5 10).

training data size is shown in Fig. 16.5. With an increase in the training data size, the classification ACC gradually increases and then reaches to the point of diminishing returns. This shows that 55% (shown by the black pointed arrow) of the dataset is required to reach the generalization stage of our ML system. Thus our system starts to learn from 10% of the training datasets to a point close to 55% of the datasets. The corresponding values are shown in the Table 16.2.

16.6.3 Results for the protocol #3: overall mean performance over all D, C, K, and T We here show the performance of the ML system based on ACC, SEN, SPE, and AUC by taking into consideration all the data types (D), classifier types (C), protocol types (K), and total trials (T) in Fig. 16.6 that depicts system performance in the form of bar chart. The mean ACC is 88.7% (B89%). The system showed encouraging results with AUC (95.32%), ACC (88.70%), PPV (90.90%), SEN (89.96%), and SPE (65.81%). The system shows high SEN in comparison to SPE. These measures are inversely proportional to each other. Therefore for a stable and accurate system SPE should be lower that its SEN values. In our study we evaluated SPE value close to (B66%) and SEN as (B91%), which is an indicator or stable system. ACC is evaluated at best cutoff points and AUC is a representation of considering all cutoff points, therefore, values might differ. Overall the system values indicate reliable performance.

16.7 Hypothesis validation and performance evaluation To test the robustness of a system, it is required to validate the hypothesis as per evaluated subsections. Section 16.7.1 explains the formulated hypothesis.

Table 16.2 Effect of training data size on the classification accuracy (ACC). Data size (%)

10

20

30

40

50

60

70

80

90

100

ACC

85.35 6 7.46

87.12 6 6.53

86.72 6 6.65

87.00 6 6.51

88.24 6 5.93

88.20 6 5.90

87.84 6 6.09

88.24 6 5.89

88.09 6 5.98

88.22 6 5.89

CHAPTER 16 A healthcare text classification system

95.32%

Performance (%)

334

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

88.70%

90.90%

89.96% 65.81%

AUC

Accuracy Precision Sensitivity Specificity Measures

FIGURE 16.6 ML system performance (D 5 5, C 5 5, K 5 5, T 5 10). ML, Machine learning.

Section 16.7.2 presents system performance based on ROC and AUC curves. Finally, Section 16.7.3 describes the reliability and stability index of text classification system.

16.7.1 Hypothesis validation We present a prototype for text classification, which discovers mechanism to deal with different MRR (an essential component of text characterization) based datasets. The higher value of performance metric indicates that our system leans toward better performance such as robustness and efficiency.

16.7.1.1 System performance linking misrepresentation ratio with area under the curve of machine learning system AUC of the ROC are statistical measures, which uses all the cutoff points for generalizing system performance. The ROC curve is a plot between SEN versus (1SPE). If the AUC value is closer to unity, the performance of classifier is said to be perfect. For establishing the validation of our hypothesis, we validated our results with AUC. The proposed system shows overall performance as 95%, which is an indicator for a superior generalization and efficiency.

16.7.1.2 Effect of misrepresentation ratio on machine learning classification accuracy This study analyzed the MRR associated with the dataset. MRR defines the misrepresentation characteristics of data types. Hypothesis says that lower MRR always leads higher classification ACC. Our analysis shows that higher MRRbased dataset has low performance among all. Following outcomes validate our hypothesis shown in Table 16.3 and corresponding graph is shown in Fig. 16.7.

16.7 Hypothesis validation and performance evaluation

Table 16.3 Misrepresentation ratio (MRR) (in decreasing order) versus accuracy (ACC). Dataset

MRR (%)

Mean ACC (%)

TwitterA WebKB4 Disease R8 SMS

71.04 68.72 67.38 63.11 61.19

70.13 6 0.15 87.34 6 0.06 93.73 6 0.03 94.45 6 0.03 97.83 6 0.01

MRR versus Mean ACC of datatypes Accuracy (%)

90 70 MRR

50

ACC

30 10 TwitterA

WebKB4

Disease

R8

SMS

Data types

FIGURE 16.7 Characterization of input data types using MRR versus ACC (D 5 5, C 5 5, K 5 5, and T 5 10). ACC, Accuracy; MRR, misrepresentation ratio.

Table 16.4 Misrepresentation ratio (MRR) (decreasing order) versus mean area under the curve (AUC). Dataset

MRR (%)

Mean AUC (%)

TwitterA WebKB4 Disease R8 SMS

71.04 68.72 67.38 63.11 61.19

90.04 6 0.05 92.34 6 0.07 98.36 6 0.02 98.40 6 0.02 98.42 6 0.01

16.7.1.3 Effect of misrepresentation ratio on mean area under the curve for all classifiers and all data types AUC value represents the classifier performance in terms of excellent, good and average category. The results are shown in Table 16.4; here lower MRR-based dataset gives higher AUC that validated our hypothesis. Corresponding figure is shown in Fig. 16.8.

335

CHAPTER 16 A healthcare text classification system

MRR verses Mean AUC of datatypes

AUC (%)

336

100 90 80 70 60 50 40 30 20

MRR AUC

TwitterA

WebKB4

Disease

R8

SMS

Data types FIGURE 16.8 Characterization of input data types using MRR versus AUC (D 5 5, C 5 5, K 5 5, and T 5 10). AUC, Area under the curve; MRR, misrepresentation ratio.

16.7.2 Individual receiver operating characteristic plots for all K protocols, D data types, and C classifiers The study shows reliable performance with respect to different category of classifiers. To validate the text characterization, we measured the classifier performance. Five different categories of classifiers, five different MRR-based datasets, and five different validation protocols are used for hypothesis validation. ROC plot shows the performance index of each classifier type. In ROC analysis, each protocol has five curves that consist of five datasets and five classifiers. Figs. C1.1
C1.5, C2.1
C2.5, C3.1
C3.5, C4.1
C4.5, and C5.1
C5.5 show the performance of K2, K4, K5, K10, and JK protocol, respectively. Our comprehensive data analysis consisted of five types of text datasets [TwitterA, WebKB4, Disease, Reuters (R8), and SMS]; five kinds of classifiers (SVM, MLP-based neural network, AB, SGD, and DT); and five types of training protocols (K2, K4, K5, K10, and JK). Using the decreasing order of MRR, our ML system demonstrates the mean classification AUCs as 90.03%, 92.34%, 98.35%, 98.40%, and 98.42%, respectively, over all the classifiers and protocols. The general behavior of the classifier is consistent with least MRR-based datasets. The consistency has potential to generalize the results for all validation protocols P1
P5, all classifiers CL1
CL5, and all the datasets DS1
DS5. The result shows higher number on AUC values and maximum AUC and this demonstrates our system robustness. The AUC tables are presented in Appendix (Tables D.1
D.5).Appendix E presents positive predictive value tables; Appendix F highlights sensitivity tables; Appendix G shows specificity tables and Appendix H list of abbreviations/symbols.

16.7 Hypothesis validation and performance evaluation

16.7.3 Reliability and stability analysis Reliability and stability indexes are based on sizes of training and testing instances. As we explained that we split a particular dataset into its ten equal partitions. Here each partition is used for analysis with all splitting protocols (K2, K4, K5, K10, and JK).

16.7.3.1 Reliability index Following steps have been adapted for reliability evaluation: Step 1. Compute the ACC for all the values of data size (N) varying from 10% to 100% for all data types, all classifiers and all the data types. Step 2. Consider all accuracies of 10 varying sizes of all datasets and compute mean μN and standard deviation (SD) δN by taking consideration all the accuracies. Step 3. Compute the reliability index ðαN ) using Eq. (16.10) for data size N.
 δN 3 100 αN ð%Þ 5 1 2 μN

(16.10)

where αN is reliability index, μN and δN represents mean and SD of all the accuracies. Step 4. Repeat the Steps 1, 2, and 3 for all dataset with 10 sizes (N) and compute the reliability index of α by taking the mean of all data sizes using the following equation: 0P Nc

αn

1

Bn51 C C αð%Þ 5 B @ Nc A

(16.11)

where Nc shows cardinality of Ds 5 {10, 20, . . ., 100}, which is a set of 10 entries of data size, and n is the index for Ds. The reliability index as presented in Fig. 16.10.

16.7.3.2 Stability index Stability index of any classification system depicts the control theory that shows robust and stable system. A stable system tells the instance size, which is sufficient for memorization process and after that it starts degrading its performance. Stability of any classification system shows the sufficient instance size that lies within a particular tolerance limit. The general tolerance limit is 2% [17]. Stability of our system is computed in following ways: Step 1. Compute the ACC for all the values of data size (N) varying from 10% to 100% for all data types, all classifiers and all the data types.

337

338

CHAPTER 16 A healthcare text classification system

Step 2. Consider all accuracies of 10 varying sizes of all the datasets and compute mean μN . Step 3. Consider all SD from the mean ACC at every data size. Step 4. If the deviation lies under tolerance limit (2%) of mean value, the system will be stable. Step 5. For each data size (N), repeat the Steps 2
4 and if deviation lies under tolerance limit declare stability of the system. To generalize our system performance, we evaluated reliability and stability index of our system. The assessment process is shown in Fig. 16.9. In text classification domain, we present first state-of-the art method, which shows optimized process for text classification, strong choices for train
test instances, and strong reliability and stability index of the system. The system adapts wide range of classifiers and data types one by one with five train
test split criteria. In the individual classifier performance MLP from Neural Network category performs the best among all with 92% ACC. With respect to data if we consider all the classifiers, we find that ensemble category AB with SMS data gives 98% ACC, which is the best among all. Further when we find figure of merits in data, SMS have the highest figure of merit among all the selected datasets. We find that the lowest MRR gives higher AUC values. The ROCs are presented in Appendix C.1
C.5. The experimental protocol showed consistent behavior toward classifiers generalization process. The reliability index of the proposed system is 93%. We demonstrated the system’s stability meeting the tolerance band of 2% of the mean value, thus ensuring the classification system is picking dominant features accurately. The encouraging results on reliability and stability analysis validated the proposed classifier system (Table 16.5). Protocol-based classification results on test data set

Ground truth-based class labels on test data set

Accuracy computation

Accuracy

Reliability assessment

Reliability index

Stability assessment

Stability index

FIGURE 16.9 Flowchart showing the reliability and stability assessment.

16.7 Hypothesis validation and performance evaluation

Table 16.5 Reliability index (αN ) at different data size (N) for K 5 2, 4, 5, 10 and JK and T 5 10. Data size (N)

10

20

30

40

50

60

70

80

90

100

αN (%)

91.26

92.51

92.34

92.52

93.28

93.31

93.07

93.32

93.21

93.33

Reliability index (%)

95 94 93 92 91 90 10

20

30

40

50

60

70

80

90

100

Data size (N)

FIGURE 16.10 Reliability index of classification system (D 5 5, C 5 5, K 5 5, and T 5 10).

Researchers targeted text classification work with different datasets and either one or two train
test split criteria. Hence, we have presented a comprehensive performance of proposed model against existing works. We assess the reliability and stability index of our system by combining all the data types, classifier types and validation protocols. However, it is observed that the combined system gives good performance in text classification category. Our system might give improved performance by improving feature selection in current design. Another extension could be to compare the performance of text classification system using combination of different feature sets such as TF-IDF [30] and n-gram analysis [30]. Third, the MRR in dataset signifies a bad learning semantics of classifiers. In this work, we have considered the quality of data types in terms of MRR, which finally used for PE. To identify informative features in data type, we preprocess the data and removed all the stopwords from data types. In this way, we identified good (informative) terms in each data types and it finally shows figure of merit in entire datasets. Higher MRR will lower the classifier performance is validated by this work extensively. The current study showed a systematic approach to assess the performance of classifier system, which was not presented till date. The reliability index as presented in Fig. 16.10. We use Eq. (16.11) for reliability evaluation on varying size of data. Initially reliability index increase as per increase in size and then gradually achieves the consistent behavior. Our system achieves good reliability for proposed model at 93%.

339

340

CHAPTER 16 A healthcare text classification system

Stability analysis defines the dynamics of control system. Here in our analysis data size can control the dynamics of overall system. We observed that at data size ( . 2458 instances) system is stable within 2% tolerance limit.

16.8 Discussion The study shows robust performance of different category of classifiers while linking with different MRR-based data types. The performance utilizes five different validation protocols for effective generalization over learned data. We demonstrated a unique healthcare text classification system where one can characterize the input text with respect the ML performance. This is the first chapter of its kind, which relates directly the output performance to the input noise level of the text data represented by MRR. We considered noise factor as perturbation, which is represented by MRR value. This MRR is calculated by removing stopwords and making all the terms stemmed in all data types. While this is a new concept, we further performed an exhaustive statistical analysis that consisted of five types of text datasets (TwitterA, WebKB4, Disease, R8, and SMS) with decreasing MRR value; five kinds of classifiers (SVM, MLP-based neural network, AB, SGD, and DT); and five types of training protocols (K2, K4, K5, K10, and JK). With decreasing values of MRR, our ML system demonstrated the mean classification accuracies as 70%, 87%, 93%, 94%, and 98%, respectively. MLP-based neural network showed 92% ACC over all datasets, classifiers, protocols, and trials. This subsystem performed 6% better against the previously published literature. The system was tested for stability and reliability (Section 16.7.3). We demonstrated the system’s variability to be low showing the robustness of the ML system. The current scope of work is limited to MRR only and we have not considered exhaustive nature of structured and unstructured categories of datasets. The scope of this pilot study only links the MRR of data types for ML performance.

16.8.1 Benchmarking A comparative study was performed between the proposed set of techniques against the previously published in the literature. For this, we took eight talking points (attributes) that consisted of (1) type of the data used, (2) features computed during the ML design, (3) process of feature selection, (4) type of the classifier used during the training and testing protocols, and (5) performance metric and the ACC (marked as columns 1
8 in Table 16.6). The rows represent different authors in chronological order. There are two very important points to note in our study: (1) in the last column (column 8), labeled as “hypothesis,” our study is the only study, which was conducted to establish the validity of the hypothesis that characterized the input data with respect to the PE of the system. This was the groundbreaking and novel component and main contribution of our design. (2) Further, we evaluated the

Table 16.6 Classification performance obtained from other approaches from literature. Column 1

Column 2

Column 3

Column 4

Column 5

Column 6

Column 7

Column 8

Reference

Data type/data size

Feature types

Feature selection

Classifier type

Performance metrics

ACC (%)

Hypothesis

Wong et al. [12]

Reuters/21578

Term frequency

Weight updating

Sleeping experts

AUC

AUC: 93.35




Huang and Ling [11]

Binary balanced dataset/2000




New measure (RMS)

ACC AUC

ACC: 78.58 AUC: 84.19




Sriram et al. [14] Iwata et al. [13]

Twitter data/5407

Term frequency

BOW, BOW-A

Greedy search
based algorithm NB

ACC







20News data and web data/20,000







NB, ME, SVM

Weighted errors







Vimal et al. [16]

Psoriasis image/540

Color, texture, HOS

PCA with polling contribution

SVM

ACC SEN SPE AUC




Vimal et al. [17]

Psoriasis image/540

Color, texture, redness, chaotic

Average feature values

SVM

ACC SEN SPE AUC

Caragea et al. [15]

Crawled dataset/2000

TF-IDF

BOW and TFIDF

SVM, DT, NB, and Random Forest

PRE recall

ACC: 99.81 SEN: 99.76 SPE: 99.57 AUC: 1.00 ACC: 100.00 SEN: 100.00 SPE: 100.00 AUC: 1.00 PRE: 88.35 REC: 97.00

Kautz et al. [8]

Numeral/2000, cardiotocography/2156, glass/214, dermatology/366, and skateboard/21




MPS

NB, kNN SVM, C4.5, LR

ACC SEN SPE







Proposed work

SMS/5572 Reuters (R-8)/7674 Disease/2010 WebKB4/4199 TwitterA/5128

Term frequency

BOW

SVM-L, MLP, AdaBoost, SGD, DT

AUCACC PPV SEN SPE

AUC: 95.32 ACC: 88.70 PRE: 90.90 SEN: 89.96 SPE: 65.81

MRR







ACC, Accuracy; AUC, area under the curve; BOW, bag of words; BOW-A, bag of words model when authors profile is used; C4.5, specific algorithm of DT; DT, decision tree; HOS, higher order spectra; kNN, k nearest neighbor; LR, linear regression; ME, maximum entropy; MLP, multilayer perceptron; MPS, multiclass performance score; MRR, misrepresentation ratio; NB, Naive Bayes; PCA, principle component analysis; PPV, positive predictive value; PRE, precision; REC, recall; RMS, root mean square; SEN, sensitivity; SGD, stochastic gradient descent; SPE, specificity; SVM, support vector machine; SVM-L, support vector machine with linear basis function; TF-IDF, term frequency inverse document frequency.

342

CHAPTER 16 A healthcare text classification system

performance of the system with highest ACC (column 7) compared to rest of the authors in the benchmarking table yielding as AUC: 95.32, ACC: 88.70, precision (PRE): 90.90, SEN: 89.96, and SPE: 65.81, respectively, all in percentage. Further, as part of the comprehensive analysis, we had demonstrated our model using all kinds of cross-validation protocols such as K2, K4, K5, K10, and JK yielding to ACC and prediction, unlike other authors. There are several similarities between our study and the work done by other authors. As can be seen from the table, most of the previously published work used “frequency” as criteria (see column 4) for feature extraction, unlike ours, which adapted BOW model. Wong et al. [12] proposed a weight updating strategy as feature selection and achieved an ACC of 78.58%. Huang and Ling [11] developed a new measure that was inspired by root mean square error. Sriram et al. [14] adapted a BOW feature selection technique, which showed an enhanced performance. Iwata et al. [13] proposed the performance metrics in terms of the weighted errors. Our comprehensive data analysis is inspired by the work done by Suri and his team (Shrivastava et al. [16]), where the authors stressed comprehensive PE besides the novel design in feature extraction and feature selection. Caragea et al. [15] has used BOW model and derived better PRE and recall using four different classifiers. Kautz et al. [8] tried to evolve a new generic multiclass performance metric that uniquely evaluated the performance of ML system. We want to emphasize that our hypothesis follows the concept of the real computer vision models where performance always degrades with increase in perturbation in the input data. Work done by Haralick et al. [31] and Suri et al. [32] has shown that robustness of the system with perturbation can bring higher ACC; however, the performance is compromised with the presence of noise in the input data. Our study therefore purely coincide the literature of real-world models. Last but not the least, we want to emphasize that we had an inverse relationship between ACC of the ML system and MRR of dataset (Section 16.7.2). With decreasing order of MRR, our ML system demonstrated the mean classification accuracies as 70.13% 6 0.15%, 87.34% 6 0.06%, 93.73% 6 0.03%, 94.45% 6 0.03%, and 97.83% 6 0.01%, respectively, over all the classifiers and protocols. Further, we not only established the link between MRR and PE of ML system but also comprehensively evaluated our system with five partitioning protocols and five classifiers. The overall system ACC over all datasets, classifiers, and protocols is 89%, thereby showing the entire ML system to be unique. We also observed that higher MRR has lower robustness (increasing order of SD) and as per increase in MRR value the system performance decreases accordingly. The goal of this chapter is not to focus on feature extraction or feature selection technique but to take a simple model BOW to prove the hypothesis. We want to emphasize that MLP showed the best performance. We adapted our ML system with five datasets (DS1
DS5) in which two are tweets collected from Twitter, related to healthcare context. On Twitter people use free handwriting thereby generating more noisy data. Disease category dataset is also Twitter collected tweets but because of preprocessing it has lower MRR compared to TwitterA. Lastly, our ML system undergoes reliability and stability of text classification (Section 16.7.3).

16.9 Conclusion

16.8.2 A special note on classifier, ground truth labels and misrepresentation ratio The classifiers are the backbone of proposed ML system: we have taken five different classifiers (SVM, MLP-based neural network, AB, SGD, and DT) undergoing five types of partitioning protocols (K2, K4, K5, K10, and JK) implementations. Some classifiers do well on ground truth (document size and corresponding labels) of datasets. MLP showed the best ability to learn from neurons and weights. It also creates a network of neuron in its own training, which enhances its ability to learn. Protocols K5 shown higher values in many cases with the classifiers. MLP performs better; SVM with linear kernel and DT performance is in medium category, while SGD and AB are average performer. The role of MRR in characterizing the input healthcare text datasets is important for the success of our model. We started perturbation (MRR) with 71.04% strong scenario and then decreasing the MRR, we find consistent improvement in the ACC. We can see that lower the MRR has a higher learning rate (η). Our experiment demonstrated encouraging results.

16.8.3 Strength weakness and extensions The study has the following strengths: (1) we validated our hypothesis that MRR degrades the ML performance. (2) Comprehensive data modeling and analysis, which consisted of five different datasets with different MRR values, five different training/testing protocols, and five types of classifiers. In spite of throw analysis, we think that by taking larger data bases along with strong feature selection methods can make the system more powerful and extend this pilot study.

16.9 Conclusion In the proposed work a robust and exhaustive text classification system has been discussed. The work shows the text-related MRR degrades the system performance. The comprehensive system, that is, five datasets, five splitting protocols, and five heterogeneous classifiers, is used for measuring its impact in classifier memorization process. As we have considered short and long text messages for this experimental work, we considered all the features (terms) for the experimentation. The performance of the system is measured in terms of ROC, AUC, SEN, SPE, and PPV. Further reliability and stability index of the system is also measured. The system showed good results, that is, 89% and MLP performs best among all, that is, 92% selected categories of classifiers. Such system prototype can help in text categorization in a better way whether it belongs to structured or unstructured category. Our experiment also demonstrates the quality index in dataset and justified that higher informative terms contribute maximum in classification ACC. To the best of my knowledge no one targeted this type of work till date and results can be useful for complex and real-time text surveillance setup.

343

344

CHAPTER 16 A healthcare text classification system

Acknowledgment We are grateful to the publisher Springer Nature for giving the permission to reproduce in part or full the material taken from our source publication (S.K. Srivastava, S.K. Singh, J. S. Suri, Healthcare text classification system and its performance evaluation: a source of better intelligence by characterizing healthcare text, J. Med. Syst. 42 (5) (2018) 1
35. ,https://doi.org/10.1007/s10916-018-0941-6.).

Funding The author(s) received no financial support for the research, authorship, and/or publication of this article.

Conflict of interest The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References [1] T.M. Rico, K. dos Santos Machado, V.P. Fernandes, S.W. Madruga, P.T. Noguez, C. R.G. Barcelos, et al., Text messaging (SMS) helping cancer care in patients undergoing chemotherapy treatment: a pilot study, J. Med. Syst. 41 (11) (2017) 181. [2] K. Lee, A. Agrawal, A. Choudhary, Real-time disease surveillance using twitter data: demonstration on flu and cancer, in: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, August 2013, pp. 1474
1477. [3] A.B. Rios-Alvarado, I. Lopez-Arevalo, E. Tello-Leal, V.J. Sosa-Sosa, An approach for learning expressive ontologies in medical domain, J. Med. Syst. 39 (8) (2015) 75. [4] G.Z. Li, J. Yang, G.P. Liu, L. Xue, Feature selection for multi-class problems using support vector machines, PRICAI, August 2004, pp. 292
300. [5] S. Vahdat, L. Hamzehgardeshi, S. Hessam, Z. Hamzehgardeshi, Patient involvement in health care decision making: a review, Iran. Red Crescent Med. J. 16 (1) (2014) 1
7. [6] U.R. Acharya, O. Faust, S.V. Sree, F. Molinari, L. Saba, A. Nicolaides, et al., An accurate and generalized approach to plaque characterization in 346 carotid ultrasound scans, IEEE Trans. Instrum. Meas. 61 (4) (2012) 1045
1053. [7] U.R. Acharya, S.V. Sree, L. Saba, F. Molinari, S. Guerriero, J.S. Suri, Ovarian tumor characterization and classification using ultrasound—a new online paradigm, J Digit. Imaging 26 (3) (2013) 544
553. [8] T. Kautz, B.M. Eskofier, C.F. Pasluosta, Generic performance measure for multiclassclassifiers, Pattern Recognit. 68 (2017) 111
125.

References

[9] N. Japkowicz, M. Shah, Evaluating Learning Algorithms: A Classification Perspective, Cambridge University Press, 2011. [10] M. Sokolova, G. Lapalme, A systematic analysis of performance measures for classification tasks, Inform. Process. Manag. 45 (4) (2009) 427
437. [11] J. Huang, C.X. Ling, Constructing new and better evaluation measures for machine learning, IJCAI, January 2007, pp. 859
864. [12] A.K. Wong, J.W. Lee, D.S. Yeung, Improving text classifier performance based on AUC, in: 18th International Conference on Pattern Recognition, 2006. ICPR 2006, August 2006, pp. 1
4. [13] T. Iwata, T. Tanaka, T. Yamada, N. Ueda, Improving classifier performance using data with different taxonomies, IEEE Trans. Knowl. Data Eng. 23 (11) (2011) 1668
1677. [14] B. Sriram, D. Fuhry, E. Demir, H. Ferhatosmanoglu, M. Demirbas, Short text classification in twitter to improve information filtering, in: Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, July 2010, pp. 841
842. [15] C. Caragea, J. Wu, S.D. Gollapalli, C.L. Giles, Document type classification in online digital libraries, in: AAAI, February 2016, pp. 3997
4002. [16] V.K. Shrivastava, N.D. Londhe, R.S. Sonawane, J.S. Suri, Reliable and accurate psoriasis disease classification in dermatology images using comprehensive feature space in machine learning paradigm, Expert Syst. Appl. 42 (15) (2015) 6184
6195. [17] V.K. Shrivastava, N.D. Londhe, R.S. Sonawane, J.S. Suri, Computer-aided diagnosis of psoriasis skin images with HOS, texture and color features: a first comparative study of its kind, Comput. Methods Prog. Biomed. 126 (2016) 98
109. [18] S. Tuarob, C.S. Tucker, M. Salathe, N. Ram, An ensemble heterogeneous classification methodology for discovering health-related knowledge in social media messages, J. Biomed. Inform. 49 (2014) 255
268. [19] G.V. Cormack, J.M. Go´mez Hidalgo, E.P. Sa´nz, Spam filtering for short messages, in: Proceedings of the Sixteenth ACM Conference on Conference on Information and Knowledge Management, November 2007, pp. 313
320. [20] C. Lu, X. Zhang, J.R. Park, X. Hu, T. He, Web clustering based on the information of sibling pages, in: Granular Computing, 2008. GrC 2008. IEEE International Conference, August 2008, pp. 480
485. [21] J.G. Liang, X.F. Zhou, P. Liu, L. Guo, S. Bai, An EMM-based approach for text classification, Procedia Comput. Sci. 17 (2013) 506
513. [22] P. Velardi, G. Stilo, A.E. Tozzi, F. Gesualdo, Twitter mining for fine-grained syndromic surveillance, Artif. Intell. Med. 61 (3) (2014) 153
163. [23] J. Roesslein, Tweepy Documentation. ,http://tweepy.readthedocs.io/en/v3.5.0., 2009. [24] S.K. Srivastava, S.K. Singh, Multi-parameter based performance evaluation of classification algorithms, Int. J. Comput. Sci. Inf. Technol. 7 (2015) 115
125. [25] U.R. Acharya, M.R.K. Mookiah, S.V. Sree, D. Afonso, J. Sanches, S. Shafique, et al., Atherosclerotic plaque tissue characterization in 2D ultrasound longitudinal carotid scans for automated classification: a paradigm for stroke risk assessment, Med. Biol. Eng. Comput. 51 (5) (2013) 513
523. [26] A.V.D. Sanchez, Advanced support vector machines and kernel methods, Neurocomputing 55 (1
2) (2003) 5
20. [27] K.R. Muller, S. Mika, G. Ratsch, K. Tsuda, B. Scholkopf, An introduction to kernelbased learning algorithms, IEEE Trans. Neural Netw. 12 (2) (2001) 181
201.

345

346

CHAPTER 16 A healthcare text classification system

[28] R.U. Acharya, O. Faust, A.P.C. Alvin, S.V. Sree, F. Molinari, L. Saba, et al., Symptomatic vs. asymptomatic plaque classification in carotid ultrasound, J. Med. Syst. 36 (3) (2012) 1861
1871. [29] S. Chakravarty, Stochastic Gradient Descent Methods for Large Scale Pattern Classification, 2011. [30] J. Martineau, T. Finin, Delta TFIDF: an improved feature space for sentiment analysis, ICWSM, vol. 9, 2009, p. 106. [31] M.H. Robert, G.S. Linda, Computer and Robot Vision, vol. I, Addison-Wesley, 1992, pp. 28
48. [32] J.S. Suri, R.M. Haralick, F.H. Sheehan, Left ventricle longitudinal axis fitting and its apex estimation using a robust algorithm and its performance: a parametric apex model, Image Processing, 1997. Proceedings., International Conference on, vol. 3, IEEE, October 1997, pp. 118
121.

Appendix A Types of dataset used in the study A.1 TwitterA dataset

Misrepresentation ratio: 71.04%.

Appendix A Types of dataset used in the study

A.2 WebKB4 dataset

Misrepresentation ratio: 68.72%.

A.3 Disease dataset

Misrepresentation ratio: 67.38%.

347

348

CHAPTER 16 A healthcare text classification system

A.4 Reuters (R8) dataset

Misrepresentation ratio: 63.11%.

A.5 SMS dataset

Misrepresentation ratio: 61.19%.

Appendix B Labels used in different text data types

Appendix B Labels used in different text data types Table B.1 Data types. Data type

Name

Classes

Category

Total data size

D1 D2 D3 D4 D5

TwitterA WebKB4 Disease Reuters (R8) SMS

4 4 5 8 2

Tweets Web pages Tweets Movie Messages

5128 4199 2010 7674 5572

Table B.2 TwitterA. Class

Samples

No Health Not English Sick Total

2757 1253 539 579 5128

Table B.3 WebKB4. Class

Samples

Project Course Student Faculty Total

504 930 1641 1124 4199

Table B.4 Disease. Class

Samples

Abdominal pain Cough Diarrhea Nausea Conjunctivitis Total

365 501 407 491 246 2010

Table B.5 Reuters (R8). Class

Samples

acq crude earn

2292 374 3923

(Continued)

349

CHAPTER 16 A healthcare text classification system

Table B.5 Reuters (R8). Continued Class

Samples

grain interest money-fx ship trade Total

51 271 293 144 326 7674

Table B.6 SMS. Class

Samples

Spam Ham Total

747 4827 5574

Appendix C Receiver operating characteristic curves C1 Receiver operating characteristic curves for K2 protocol using five classifiers Protocol type: K2; Classifier type: SVM-L; Data type: DS1-DS5 1 0.9 0.8 0.7 (Sensitivity)

350

0.6 0.5 0.4 DS1: SMS DS2: Disease DS3: TwitterA DS4: WebKB4 DS5: Reuter(R8)

0.3 0.2 0.1 0

0.1

0.2

0.3

0.4

0.5

0.6

(1-Specificity)

FIGURE C1.1

0.7

0.8

0.9

1

Appendix C Receiver operating characteristic curves

Protocol type: K2; Classifier type: MLP; Data type: DS1-DS5 1 0.9 0.8

(Sensitivity)

0.7 0.6 0.5 0.4 DS1: SMS DS2: Disease

0.3

DS3: TwitterA DS4: WebKB4

0.2

DS5: Reuter(R8)

0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

(1-Specificity)

FIGURE C1.2

Protocol type: K2; Classifier type: AdaBoost; Data type: DS1-DS5 1 0.9 0.8

(Sensitivity)

0.7 0.6 0.5 0.4 DS1: SMS DS2: Disease DS3: TwitterA DS4: WebKB4

0.3 0.2

DS5: Reuter(R8)

0.1

FIGURE C1.3

0

0.1

0.2

0.3

0.4 0.5 0.6 (1-Specificity)

0.7

0.8

0.9

1

351

CHAPTER 16 A healthcare text classification system

Protocol type: K2; Classifier type: SGD; Data type: DS1-DS5 1 0.9 0.8

(Sensitivity)

0.7 0.6 0.5 0.4 DS1: SMS DS2: Disease DS3: TwitterA DS4: WebKB4 DS5: Reuter(R8)

0.3 0.2 0.1

0

0.1

0.2

0.3

0.4 0.5 0.6 (1-Specificity)

0.7

0.8

0.9

1

FIGURE C1.4

Protocol type: K2; Classifier type: Decision Tree; Data type: DS1-DS5 1 0.9 0.8 0.7 (Sensitivity)

352

0.6 0.5 0.4

DS1: SMS DS2: Disease DS3: TwitterA DS4: WebKB4 DS5: Reuter(R8)

0.3 0.2 0.1

FIGURE C1.5

0

0.1

0.2

0.3

0.4 0.5 0.6 (1-Specificity)

0.7

0.8

0.9

1

Appendix C Receiver operating characteristic curves

C2 Receiver operating characteristic curves for K4 protocol using five classifiers Protocol type: K4; Classifier type: SVM-L; Data type: DS1-DS5 1 0.9 0.8

(Sensitivity)

0.7 0.6 0.5 0.4 DS1: SMS DS2: Disease DS3: TwitterA DS4: WebKB4 DS5: Reuter(R8)

0.3 0.2 0.1

0

0.1

0.2

0.3

0.4 0.5 0.6 (1-Specificity)

0.7

0.8

0.9

1

FIGURE C2.1

Protocol type: K4; Classifier type: MLP; Data type: DS1-DS5 1 0.9 0.8

(Sensitivity)

0.7 0.6 0.5 0.4 DS1: SMS DS2: Disease DS3: TwitterA DS4: WebKB4 DS5: Reuter(R8)

0.3 0.2 0.1

FIGURE C2.2

0

0.1

0.2

0.3

0.4 0.5 0.6 (1-Specificity)

0.7

0.8

0.9

1

353

CHAPTER 16 A healthcare text classification system

Protocol type: K4; Classifier type: SGD; Data type: DS1-DS5 1 0.9 0.8

(Sensitivity)

0.7 0.6 0.5 0.4 DS1: SMS DS2: Disease DS3: TwitterA DS4: WebKB4 DS5: Reuter(R8)

0.3 0.2 0.1

0

0.1

0.2

0.3

0.4 0.5 0.6 (1-Specificity)

0.7

0.8

0.9

1

FIGURE C2.3

Protocol type: K4; Classifier type: AdaBoost; Data type: DS1-DS5 1 0.9 0.8 0.7 (Sensitivity)

354

0.6 0.5 0.4 DS1: SMS DS2: Disease DS3: TwitterA DS4: WebKB4 DS5: Reuter(R8)

0.3 0.2 0.1

FIGURE C2.4

0

0.1

0.2

0.3

0.4 0.5 0.6 (1-Specificity)

0.7

0.8

0.9

1

Appendix C Receiver operating characteristic curves

Protocol type: K4; Classifier type: Decision Tree; Data type: DS1-DS5 1 0.9 0.8

(Sensitivity)

0.7 0.6 0.5 0.4 DS1: SMS DS2: Disease DS3: TwitterA DS4: WebKB4 DS5: Reuter(R8)

0.3 0.2 0.1 0

0.1

0.2

0.3

0.4 0.5 0.6 (1-Specificity)

0.7

0.8

0.9

1

FIGURE C2.5

C3 Receiver operating characteristic curves for K5 protocol using five classifiers Protocol type: K5; Classifier type: SVM-L; Data type: DS1-DS5 1 0.9 0.8

(Sensitivity)

0.7 0.6 0.5 0.4 DS1: SMS DS2: Disease DS3: TwitterA DS4: WebKB4 DS5: Reuter(R8)

0.3 0.2 0.1

FIGURE C3.1

0

0.1

0.2

0.3

0.4 0.5 0.6 (1-Specificity)

0.7

0.8

0.9

1

355

CHAPTER 16 A healthcare text classification system

Protocol type: K5; Classifier type: MLP; Data type: DS1-DS5 1 0.9 0.8

(Sensitivity)

0.7 0.6 0.5 0.4 DS1: SMS DS2: Disease DS3: TwitterA DS4: WebKB4 DS5: Reuter(R8)

0.3 0.2 0.1

0

0.1

0.2

0.3

0.4 0.5 0.6 (1-Specificity)

0.7

0.8

0.9

1

FIGURE C3.2

Protocol type: K5; Classifier type: AdaBoost; Data type: DS1-DS5 1 0.9 0.8 0.7 (Sensitivity)

356

0.6 0.5 0.4 DS1: SMS DS2: Disease DS3: TwitterA DS4: WebKB4 DS5: Reuter(R8)

0.3 0.2 0.1

FIGURE C3.3

0

0.1

0.2

0.3

0.4 0.5 0.6 (1-Specificity)

0.7

0.8

0.9

1

Appendix C Receiver operating characteristic curves

Protocol type: K5; Classifier type: SGD; Data type: DS1-DS5 1 0.9 0.8

(Sensitivity)

0.7 0.6 0.5 0.4

DS1: SMS DS2: Disease DS3: TwitterA DS4: WebKB4 DS5: Reuter(R8)

0.3 0.2 0.1

0

0.1

0.2

0.3

0.4 0.5 0.6 (1-Specificity)

0.8

0.7

0.9

1

FIGURE C3.4

Protocol type: K5; Classifier type: Decision Tree; Data type: DS1-DS5 1 0.9 0.8 (Sensitivity)

0.7 0.6 0.5 0.4 DS1: SMS DS2: Disease DS3: TwitterA DS4: WebKB4 DS5: Reuter(R8)

0.3 0.2 0.1

FIGURE C3.5

0

0.1

0.2

0.3

0.4 0.5 0.6 (1-Specificity)

0.7

0.8

0.9

1

357

CHAPTER 16 A healthcare text classification system

C4 Receiver operating characteristic curves for K10 protocol using five classifiers Protocol type: K10; Classifier type: SVM-L; Data type: DS1-DS5 1 0.9 0.8

(Sensitivity)

0.7 0.6 0.5 0.4 DS1: SMS DS2: Disease DS3: TwitterA DS4: WebKB4 DS5: Reuter(R8)

0.3 0.2 0.1

0

0.1

0.2

0.3

0.4 0.5 0.6 (1-Specificity)

0.7

0.8

0.9

1

FIGURE C4.1

Protocol type: K10; Classifier type: MLP; Data type: DS1-DS5 1 0.9 0.8 0.7 (Sensitivity)

358

0.6 0.5 0.4 DS1: SMS DS2: Disease DS3: TwitterA DS4: WebKB4 DS5: Reuter(R8)

0.3 0.2 0.1

FIGURE C4.2

0

0.1

0.2

0.3

0.4 0.5 0.6 (1-Specificity)

0.7

0.8

0.9

1

Appendix C Receiver operating characteristic curves

Protocol type: K10; Classifier type: AdaBoost; Data type: DS1-DS5 1 0.9 0.8

(Sensitivity)

0.7 0.6 0.5 0.4 DS1: SMS DS2: Disease DS3: TwitterA DS4: WebKB4 DS5: Reuter(R8)

0.3 0.2 0.1

0

0.1

0.2

0.3

0.4 0.5 0.6 (1-Specificity)

0.8

0.7

0.9

1

FIGURE C4.3

Protocol type: K10; Classifier type: SGD; Data type: DS1-DS5 1 0.9 0.8

(Sensitivity)

0.7 0.6 0.5 0.4 DS1: SMS DS2: Disease DS3: TwitterA DS4: WebKB4 DS5: Reuter(R8)

0.3 0.2 0.1

FIGURE C4.4

0

0.1

0.2

0.3

0.4 0.5 0.6 (1-Specificity)

0.7

0.8

0.9

1

359

CHAPTER 16 A healthcare text classification system

Protocol type: K10; Classifier type: Decision Tree; Data type: DS1-DS5 1 0.9 0.8 (Sensitivity)

0.7 0.6 0.5 0.4 DS1: SMS DS2: Disease DS3: TwitterA DS4: WebKB4 DS5: Reuter(R8)

0.3 0.2 0.1

0

0.1

0.2

0.3

0.4 0.5 0.6 (1-Specificity)

0.8

0.7

0.9

1

FIGURE C4.5

C5 Receiver operating characteristic curves for JK protocol using five classifiers Protocol type: JK; Classifier type: SVM-L; Data type: DS1-DS5 1 0.9 0.8 0.7 (Sensitivity)

360

0.6 0.5 0.4 DS1: SMS DS2: Disease DS3: TwitterA DS4: WebKB4 DS5: Reuter(R8)

0.3 0.2 0.1

FIGURE C5.1

0

0.1

0.2

0.3

0.4 0.5 0.6 (1-Specificity)

0.7

0.8

0.9

1

Appendix C Receiver operating characteristic curves

Protocol type: JK; Classifier type: MLP; Data type: DS1-DS5 1 0.9 0.8

(Sensitivity)

0.7 0.6 0.5 0.4 DS1: SMS DS2: Disease DS3: TwitterA DS4: WebKB4 DS5: Reuter(R8)

0.3 0.2 0.1

0

0.1

0.2

0.3

0.4 0.5 0.6 (1-Specificity)

0.7

0.8

0.9

1

FIGURE C5.2

Protocol type: JK; Classifier type: AdaBoost; Data type: DS1-DS5 1 0.9 0.8

(Sensitivity)

0.7 0.6 0.5 0.4 DS1: SMS DS2: Disease DS3: TwitterA DS4: WebKB4 DS5: Reuter(R8)

0.3 0.2 0.1

FIGURE C5.3

0

0.1

0.2

0.3

0.4 0.5 0.6 (1-Specificity)

0.7

0.8

0.9

1

361

CHAPTER 16 A healthcare text classification system

Protocol type: JK; Classifier type: SGD; Data type: DS1-DS5 1 0.9 0.8

(Sensitivity)

0.7 0.6 0.5 0.4 DS1: SMS DS2: Disease DS3: TwitterA DS4: WebKB4 DS5: Reuter(R8)

0.3 0.2 0.1

0

0.1

0.2

0.3

0.4 0.5 0.6 (1-Specificity)

0.8

0.7

0.9

1

FIGURE C5.4

Protocol type: JK; Classifier type: Decision Tree; Data type: DS1-DS5 1 0.9 0.8 0.7 (Sensitivity)

362

0.6 0.5 0.4 DS1: SMS DS2: Disease DS3: TwitterA DS4: WebKB4 DS5: Reuter(R8)

0.3 0.2 0.1

FIGURE C5.5

0

0.1

0.2

0.3

0.4 0.5 0.6 (1-Specificity)

0.7

0.8

0.9

1

Appendix D Area under the curve tables

Appendix D Area under the curve tables Table D.1 Area under the curve; PT: P1
P5; CT1 (support vector machine with linear kernel); decision tree: DS1
DS5. Protocol type

SMS

Disease

TwitterA

WebKB4

R8

K2 K4 K5 K10 JK

0.9900 0.9926 0.9985 0.9986 1.0000

0.9900 0.9908 0.9949 0.9952 1.0000

0.8839 0.8921 0.8945 0.8900 0.9537

0.9551 0.9478 0.9519 0.9619 1.0000

0.9915 0.9938 0.9950 0.9940 1.0000

Table D.2 Area under the curve; PT: P1
P5; CT2 (multilayer perceptron); decision tree: DS1
DS5. Protocol type

SMS

Disease

TwitterA

WebKB4

R8

K2 K4 K5 K10 JK

0.9600 0.9700 0.9800 0.9900 1.0000

0.9766 0.9779 0.9608 0.9472 1.0000

0.9062 0.9396 0.9301 0.8625 1.0000

0.8819 0.8603 0.8742 0.9100 1.0000

0.9901 0.9722 0.9837 0.9993 1.0000

Table D.3 Area under the curve; PT: P1
P5; CT3 (AdaBoost); decision tree: DS1
DS5. Protocol type

SMS

Disease

TwitterA

WebKB4

R8

K2 K4 K5 K10 JK

0.9800 0.9800 0.9720 0.9800 1.0000

0.9877 0.9909 0.9901 0.9928 1.0000

0.8873 0.8946 0.8995 0.8899 0.9167

0.9716 0.9755 0.9745 0.9678 1.0000

0.9927 0.9962 0.996 0.9976 1.0000

Table D.4 Area under the curve; PT: P1
P5; CT4 (stochastic gradient descent); decision tree: DS1
DS5. Protocol type

SMS

Disease

TwitterA

WebKB4

R8

K2 K4 K5 K10 JK

0.9800 0.9800 0.9800 0.9925 1.0000

0.9877 0.9887 0.9902 0.9943 1.0000

0.8530 0.8678 0.8827 0.8733 0.9907

0.9402 0.942 0.9465 0.9456 1.0000

0.9705 0.9833 0.9895 0.9875 1.0000

363

364

CHAPTER 16 A healthcare text classification system

Table D.5 Area under the curve; PT: P1
P5; CT5 [decision tree (DT)]; DT: DS1
DS5. Protocol type

SMS

Disease

TwitterA

WebKB4

R8

K2 K4 K5 K10 JK

0.9600 0.9717 0.9700 0.9800 1.0000

0.9496 0.9600 0.9660 0.9775 0.9800

0.7976 0.8752 0.8725 0.8554 1.0000

0.7098 0.7635 0.7794 0.8264 1.0000

0.9544 0.9712 0.9372 0.9048 1.0000

Table D.6 Mean area under the curve (AUC). Data type AUC (%)

SMS

Disease

TwitterA

WebKB4

R8

98.42 6 0.01

98.36 6 0.02

90.04 6 0.05

92.34 6 0.07

98.40 6 0.02

Appendix E Postive predictive value tables Table E.1 Positive predictive value; PT: P1
P5; CT1 (support vector machine with linear kernel); decision tree: DS1
DS5. Protocol type

SMS

Disease

TwitterA

WebKB4

R8

K2 K4 K5 K10 JK

0.98 0.97 0.98 0.97 1.00

0.97 0.97 0.97 0.99 1.00

0.7 0.71 0.71 0.71 1.00

0.85 0.86 0.86 0.88 1.00

0.96 0.97 0.97 0.97 1.00

Table E.2 Positive predictive value; PT: P1
P5; CT1 (multilayer perceptron); decision tree: DS1
DS5. Protocol type

SMS

Disease

TwitterA

WebKB4

R8

K2 K4 K5 K10 JK

0.98 0.98 0.98 0.98 1.00

0.94 0.95 0.95 0.97 1.00

0.7 0.72 0.72 0.69 1.00

0.89 0.88 0.89 0.89 1.00

0.96 0.97 0.97 0.97 1.00

Appendix F Sensitivity tables

Table E.3 Positive predictive value; PT: P1
P5; CT1 (AdaBoost); decision tree: DS1
DS5. Protocol type

SMS

Disease

TwitterA

WebKB4

R8

K2 K4 K5 K10 JK

0.97 0.97 0.96 0.97 1.00

0.97 0.96 0.96 1.00 1.00

0.67 0.71 0.71 0.69 1.00

0.89 0.89 0.90 0.90 1.00

0.95 0.96 0.95 0.96 1.00

Table E.4 Positive predictive value; PT: P1
P5; CT1 (stochastic gradient descent); decision tree: DS1
DS5. Protocol type

SMS

Disease

TwitterA

WebKB4

R8

K2 K4 K5 K10 JK

0.98 0.98 0.98 0.98 0.97

0.96 0.97 0.95 0.96 1.00

0.66 0.71 0.68 0.71 1.00

0.83 0.86 0.84 0.83 1.00

0.96 0.97 0.96 0.97 1.00

Table E.5 Positive predictive value; PT: P1
P5; CT1 [decision tree (DT)]; DT: DS1
DS5. Protocol type

SMS

Disease

TwitterA

WebKB4

R8

K2 K4 K5 K10 JK

0.96 0.96 0.95 0.96 1.00

0.94 0.94 0.95 0.96 1.00

0.65 0.65 0.66 0.67 1.00

0.77 0.79 0.78 0.81 1.00

0.90 0.91 0.91 0.93 1.00

Table E.6 Mean positive predictive value (PPV). Data type SMS PPV (%)

Disease

TwitterA

91.80 6 0.04 91.90 6 0.04 91.76 6 0.04

WebKB4

R8

90.84 6 0.05 88.00 6 0.06

Appendix F Sensitivity tables Table F.1 Sensitivity; PT: P1
P5; CT1 (support vector machine with linear kernel); decision tree: DS1
DS5. (Continued)

365

366

CHAPTER 16 A healthcare text classification system

Table F.1 Sensitivity; PT: P1
P5; CT1 (support vector machine with linear kernel); decision tree: DS1
DS5. Continued Protocol type

SMS

Disease

TwitterA

WebKB4

R8

Protocol type

SMS

Disease

TwitterA

WebKB4

R8

K2 K4 K5 K10 JK

0.98 0.97 0.98 0.97 1.00

0.91 0.93 0.93 0.94 1.00

0.67 0.68 0.69 0.69 1.00

0.85 0.85 0.85 0.86 1.00

0.94 0.96 0.96 0.95 1.00

Table F.2 Sensitivity; PT: P1
P5; CT1 (multilayer perceptron); decision tree: DS1
DS5. Protocol type

SMS

Disease

TwitterA

WebKB4

R8

K2 K4 K5 K10 JK

0.98 0.98 0.98 0.98 1.00

0.94 0.95 0.95 0.97 1.00

0.70 0.71 0.72 0.68 1.00

0.89 0.88 0.89 0.89 1.00

0.96 0.97 0.97 0.97 1.00

Table F.3 Sensitivity; PT: P1
P5; CT1 (AdaBoost); decision tree: DS1
DS5. Protocol type

SMS

Disease

TwitterA

WebKB4

R8

K2 K4 K5 K10 JK

0.97 0.97 0.96 0.97 1.00

0.91 0.93 0.93 0.93 1.00

0.66 0.66 0.67 0.64 1.00

0.86 0.85 0.84 0.84 1.00

0.92 0.93 0.93 0.93 1.00

Table F.4 Sensitivity; PT: P1
P5; CT1 (stochastic gradient descent); decision tree: DS1
DS5. Protocol type

SMS

Disease

TwitterA

WebKB4

R8

K2 K4 K5 K10 JK

0.98 0.98 0.98 0.97 1.00

0.91 0.92 0.93 0.93 1.00

0.71 0.69 0.70 0.70 1.00

0.84 0.82 0.87 0.88 1.00

0.92 0.94 0.96 0.96 1.00

Appendix G Specificity tables

Table F.5 Sensitivity; PT: P1
P5; CT1 [decision tree (DT)]; DT: DS1
DS5. Protocol type

SMS

Disease

TwitterA

WebKB4

R8

K2 K4 K5 K10 JK

0.96 0.96 0.96 0.96 1.00

0.94 0.94 0.95 0.96 1.00

0.65 0.65 0.66 0.67 1.00

0.77 0.79 0.78 0.80 1.00

0.90 0.91 0.91 0.92 1.00

Table F.6 Mean sensitivity (SEN). Data type

SMS

Disease

TwitterA

WebKB4

R8

SEN (%)

90.24 6 0.05

91.84 6 0.04

89.20 6 0.06

90.36 6 0.05

88.00 6 0.07

Appendix G Specificity tables Table G.1 Specificity; PT: P1
P5; CT1 (support vector machine with linear kernel); decision tree: DS1
DS5. Protocol type

SMS

Disease

TwitterA

WebKB4

R8

K2 K4 K5 K10 JK

0.6689 0.6544 0.6819 0.7930 0.6666

0.7458 0.5832 0.8252 0.9112 0.6666

0.7950 0.8069 0.8057 0.7950 0.7936

0.8318 0.7996 0.8193 0.8436 0.6666

0.8050 0.8451 0.8464 0.8432 0.5000

Table G.2 Specificity; PT: P1
P5; CT1 (multilayer perceptron); decision tree: DS1
DS5. Protocol type

SMS

Disease

TwitterA

WebKB4

R8

K2 K4 K5 K10 JK

0.6662 0.6663 0.6656 0.6652 0.5000

0.5550 0.5696 0.5713 0.5748 0.5000

0.6642 0.6511 0.6519 0.6573 0.5000

0.6317 0.6306 0.6335 0.6297 0.5000

0.7058 0.7120 0.7120 0.7156 0.5000

367

368

CHAPTER 16 A healthcare text classification system

Table G.3 Specificity; PT: P1
P5; CT1 (AdaBoost); decision tree: DS1
DS5. Protocol type

SMS

Disease

TwitterA

WebKB4

R8

K2 K4 K5 K10 JK

0.7091 0.6503 0.6775 0.5673 0.4666

0.6686 0.6369 0.6448 0.6391 0.5555

0.6060 0.6358 0.6397 0.6478 0.7575

0.7095 0.7550 0.7762 0.7952 0.7083

0.6199 0.6112 0.6202 0.6024 0.6760

Table G.4 Specificity; PT: P1
P5; CT1 (stochastic gradient descent); decision tree: DS1
DS5. Protocol type

SMS

Disease

TwitterA

WebKB4

R8

K2 K4 K5 K10 JK

0.5882 0.5706 0.5474 0.5738 0.5000

0.6045 0.6054 0.6184 0.6379 0.6666

0.6352 0.6362 0.6453 0.6653 0.7777

0.7626 0.7778 0.7923 0.8179 0.5000

0.6570 0.6887 0.6940 0.6904 0.5000

Table G.5 Specificity; PT: P1
P5; CT1 [decision tree (DT)]; DT: DS1
DS5. Protocol type

SMS

Disease

TwitterA

WebKB4

R8

K2 K4 K5 K10 JK

0.6599 0.6609 0.6615 0.6632 0.5000

0.5421 0.5655 0.5680 0.5668 0.5000

0.6627 0.6665 0.6746 0.6649 0.5000

0.6159 0.6156 0.6110 0.6140 0.4500

0.7107 0.7110 0.7143 0.7096 0.5000

Table G.6 Mean specificity (SPE). Data type

SMS

Disease

TwitterA

WebKB4

R8

SPE (%)

75.98 6 0.10

61.72 6 0.07

65.51 6 0.07

64.62 6 0.09

61.24 6 0.08

Appendix H List of abbreviations/symbols

Appendix H List of abbreviations/symbols SN

Abbreviations/ symbols

1 2 3 4 5 6 7 8

D C K T K2 K4 K5 K10

9 10 11 12 13 14 15 16 17 18 19 20 21 22

JK MRR SVM-L MLP SGD DT ACC SEN SPE PRE REC PPV SD ηsys

23

η (d,c,k,t)

24 25

η (c) ηsys ðNtr Þ

26 27 28 29 30 31 32 33 34 35 36

It Tt N Ntr Nte μN δN αN α Nc DSn

Description Total number of data types (5) Total number of classifiers (5) Total number of partition protocols (5) Total number of trials (10) Partition protocol (1/2 samples for training and 1/2 for test) Partition protocol (3/4 sample for training and 1/4 for test) Partition protocol (4/5 sample for training and 1/5 for test) Partition protocol (9/10 sample for training and 1/10 for test) Jack Knife (N 2 1 sample for training and 1 for test) Misrepresentation ratio Support vector machine with linear basis function Multilayer perceptron Stochastic gradient descent Decision tree Accuracy Sensitivity Specificity Precision Recall Positive predictive value Standard deviation System accuracy System accuracy w.r.t. data type d, classifier type c, protocol type k, and trial type t Mean accuracy of classifier (C) System mean accuracy corresponding to varying training size Important terms Total terms Total data size for each dataset Total training data size for each dataset Total testing data size for each dataset Mean accuracy for the dataset of data size N SD for the dataset of data size N Reliability index for each dataset of size N Reliability index of text classification system Cardinality of dataset Generic form of dataset “n 5 1, 2, 3, 4, 5”

369

Index Note: Page numbers followed by “f” and “t” refer to figures and tables, respectively.

A Acetylcholine (ACh), 25 26 Achala swaras, 104 AdaBoost (AB), 327 Aging factor, 261 264 Agmatine, 28 29 Allopregnanolone (ALLO), 31 32 α-melanocyte stimulating hormone (α-MSH), 29, 31 Alzheimer’s disease (AD), 22, 32 35 β-amyloid, extraneuronal plaque deposition of, 33 34 neurofibrillary tangles, intraneuronal accumulation of, 34 35 Amplitude, 169 Amritsar Massacre, 200 201 Anaphora in Hindi, 211 212 Anaphora resolution (AR), 211 boundaries in, 212 213 efficiency of linguistic preprocessor, 212 lack of efficient named entity recognizer, 213 no benchmark for POS tagging, 213 nonavailability of freeware Hindi discourse, 212 Animals, physiological behavior gesture in, 138f ANOVA model, 321 Apriori algorithm, 11 Area under the curve (AUC), 299, 321 AUC tables, 363 364 Arm-push-up angle, 52f, 53 54, 53f, 54t, 55f, 56 57 Arohana swaras, 112 113 Artificial intelligence (AI), 3, 133 Artificial neural networks (ANNs), 300 AtheroEdge, 297 298 Atheromatic index, 305 Atherosclerosis, 294 295 Audava raga, 104 Authoritativeness, 205 207 Automatic-learning, 241 Autonomic nervous system (ANS), 149 Avrohan, 103 104

B Back-propagation neural network, 258 Bag of words (BOW) model, 322 Basal forebrain (BF), 24 Bayesian inference, 279 280

Bayesian rational decision-making, 81 83 Bayesian segmentation, 277 279 Bayesian theory, 277 Bengal partition, 198, 200 201 β-amyloid, extraneuronal plaque deposition of, 33 34 Big data, cognitive science for, 120 121 Bigram model, 278 279, 286t Black-box nature of ML techniques, 307 308 Blocked-input models, 76 Bluemix, 251 B-mode ultrasonography, 291 article search strategy, 295 deep learning-based cardiovascular risk stratification, 306 307 machine learning, general framework of, 297 299 data partitioning, 298 299 feature engineering: extraction and selection, 297 298 performance evaluation of machine learning systems, 299 prediction or testing model, 299 training model design, 299 machine learning-based algorithms, 300 medical implications of, 305 306 machine learning design, challenges in, 307 308 machine learning framework, risk assessment in, 300 305 cardiovascular disease/stroke risk assessment indices, 305 cardiovascular diseases risk assessment using ML, 303 305 image-based stroke risk assessment, 300 303 machine learning techniques, types of, 296 297 risk assessment using traditional methods, 295 296 BOLD (blood-oxygenation level-dependent response), 184 Boosted-fixation model, 76 Boundaries in anaphora resolution, 212 213 Brain, 3 4 Brain computer interface (BCI), 179 Brain-imagining techniques, 180 187 computer tomography (CT), 180 181 computer tomography head, 180 181 cranial ultrasound, 186 187

371

372

Index

Brain-imagining techniques (Continued) magnetic resonance imaging, 183 185 near-infrared spectroscopy based imaging equipment, 182 183 single-photon emission computed tomography, 185 186 Brain machine interface (BMI), 121 Brain regions involved in the different types of memory, 23t Brain-signaling techniques, 187 190 electroencephalography (EEG), 187 188 advantages of, 188 application of, 188 disadvantages of, 188 electromyography, 189 190 advantages of, 190 applications of, 190 limitations of, 190 magnetoencephalography, 189 advantages of, 189 limitations of, 189 Brier score, 299

C Calcitonin gene-related peptide, 29 CALLA Model, 275 276 Cancellable rise-to-threshold (CRTT) model, 65 66, 82f Cardiovascular disease/stroke risk assessment indices, 305 Cardiovascular diseases risk assessment using machine learning, 303 305 Cardiovascular risk stratification, deep learningbased, 306 307 Carnatic music, mathematical structure of, 104 109 Carotid arteries, 293, 297 298, 300 303 Carotid intima-media thickness (cIMT), 293, 297 298 Carotid plaque (CP), 293, 297 298 Case marker (CM), 216 CBT, 197 198 Center for Indian Language Technology (CFILT), 280, 281t Central nervous system (CNS), 3, 125 Chest-open angle, 53f, 56 57 CIoT, 239 240, 245, 247 248, 251 252 Classification accuracy, effect of training data size on, 329 Classifiers, 327 328, 343 AdaBoost, 327 decision tree (DT), 328 multilayer perceptron, 327 stochastic gradient descent (SGD), 328

support vector machine, 327 Classroom-based language learning, 275 CObjects, 239 241, 243, 248 Cocaine- and amphetamine-regulated transcript (CART), 29 30 Coda, 282 283 Cognition, 22, 25 32 classical neurotransmitters, 25 29 acetylcholine (ACh), 25 26 agmatine, 28 29 dopamine (DA), 27 28 γ-aminobutyric acid (GABA), 27 glutamate, 26 27 serotonin (5-hydroxytryptamine), 28 neuropeptides, 29 31 α-melanocyte stimulating hormone, 31 cocaine- and amphetamine-regulated transcript (CART), 29 30 neuropeptide Y, 30 31 neurosteroids, 31 32 Cognition-related diseases, 32 36 Alzheimer’s disease, 32 35 β-amyloid, extraneuronal plaque deposition of, 33 34 neurofibrillary tangles, intraneuronal accumulation of, 34 35 Lewy body diseases (LBDs), 35 36 Cognition social science, 123 Cognitive computing and big data (CCBD) analytics, 120 Cognitive control, 124 Cognitive enhancement, 126 128 via conventional, 127 ethical issues and concerns of, 128 129 via pharmaceutical, 127 via unconventional, 127 Cognitive hearing science, 122 123 Cognitive image processing, 124 125 Cognitive neurology, 197 198 Cognitive neuroscience/physiology, 5 6 Cognitive psychology, 6 12 Cognitive science, 1 3 applications of, 2f Cognitive science, future of, 119 and cognitive enhancement, 125 129 cognitive enhancement, 126 128 ethical issues and concerns of cognitive enhancement, 128 129 scope for neuroscience research and challenges, 126 role in varied domains, 120 125 for big data, 120 121 brain machine interface (BMI), 121 cognition social science, 123

Index

cognitive control, 124 cognitive image processing, 124 125 in linguistics, 123 124 for philosophy, 121 for psychology, 122 Colonialism/nationalism, 200 203 Comparative cognition, 4 7 flowchart of, 8f Complete raga, 104, 108 Computer-based intelligence, 235 236 Computer tomography (CT), 180 181, 181f cranial ultrasound, 186 187 CT head, 180 181 magnetic resonance imaging (MRI), 183 185 near-infrared spectroscopy based imaging equipment, 182 183 single-photon emission computed tomography (SPECT), 185 186 Concordance index, 299 Constraint learner, 279 280 Constraint satisfaction problem (CSP), 110 111 Contrast, principle of, 275 Conventional CV risk factors (CCVRF), 293, 297 298 Convolutional neural network (CNN), 256 257, 306 307 Coronary artery, 292, 300 303 Corticotrophin-releasing hormone, 29 Cosmicism syndrome, 205 207 Countermanding, 76 Cranial ultrasound (CU), 186 187 advantages of, 186 limitations of, 187 Credibility-adjusted term frequency, 256 257 Customer reviews, 255

D Data, lack of access to, 308 Data annotation, 215 Data investigation viewpoint, 240 Data partitioning, 298 299 Data types, 323 326, 349t disease dataset, 325, 347 labels used in different text data types, 349 350 Reuters (R8) dataset, 325, 348 SMS dataset, 325 326, 348 TwitterA dataset, 324, 346 WebKB4 dataset, 324 325, 347 Decision field theory (DFT), 73 74 Decision-making, 66 68, 69f, 72 sequential sampling models of, 69f Decision process as optimal stochastic control, 84 Decision tree (DT), 327 328 Deep learning (DL), 294 295

-based cardiovascular risk stratification, 306 307 Dehydroepiandrosterone (DHEA), 31 32 Demonstrative pronouns, algorithm for resolving, 227 Depth sensor, 62 Description experience gap, 74 75 Detectable perspective, 240 Devanagari script, 282 Diencephalon, 23 24 Diffuse optical imaging (DOI), 182 Diffuse optical tomography (DOT), 182 Diffusion process, 68 71 Digital speech processing, 109 110 Disease dataset, 325, 347 Distance factors, 220 221 Dopamine (DA), 27 28 Drift-diffusion model, 68

E E-commerce companies, 255 Economic decision, models of, 73 75 Elderly exercise evaluation, cognitive system of, 49, 53 57 elderly exercise measurement, 52 53 multiple Kinect sensors, 59 61 frame synchronization, 60 sensing data integration without calibration, 61 robot interface, feedback by, 57 59 system overview, 50 51 Electrodermal activity (EDA), 149 analysis of electrodermal activity signal, 167 170 area measurements, 169 phasic electrodermal activity, 168 170 tonic electrodermal activity, 170 application of, 156 157 artifacts removal from electrodermal activity signal, 167 in different sleep stages, 157 158 different stages in EDA signal processing, 168f electrodermal activity signal collection sites, 164 166 electrodermal indices of emotion and stress, 158 end remarks, 171 experiment design, 158 162 climatic conditions, 163 demographic characteristics, 163 164 external and internal influences, 162 163 hypothesis, 160 161 internal or physiological influences, 163 measure of performance, 161 162 stimulus, 161

373

374

Index

Electrodermal activity (EDA) (Continued) types of experiments, 159 160 as an indicator of general arousal, 157 pretreatment of sites, 166 167 Electrodermal response, 149, 169 Electroencephalography, 125, 187 188 advantages of, 188 application of, 188 disadvantages of, 188 signal capturing process, 188f Electro Medical and Speech Technology (EMST), 280, 281t, 285 Electromyography (EMG), 189 190, 190f advantages of, 190 applications of, 190 limitations of, 190 Elegant health, 249 Enhanced activity index, 305 Epilepsy, 22 Equivalence class, identifying, 223 227 EWOW, 257 258 Exosomatic process, 150 151 Experimental validity, 159 Exposure therapy, 137 External validity, 159 Extremism, 207 Extremist mind, reading through literary language. See Tagore’s The Post Office EZ-diffusion model, 71

F Fast decision-making, 72 Feature engineering, 297 298 Feature specificity, need of, 261 262 The Festival Speech Synthesis System, 283 Formant, 112 113, 114f, 115f Free-description tasks, 161 162 Freeware Hindi discourse, nonavailability of, 212 Frontal eye fields (FEFs), 76 F-score, 274, 285 calculated for different text corpus of Hindi languages using words as a unit of language, 287t calculated for English and Hindi languages based on Unigram- and Bigram-based ideal and constraint learners, 286t using phonemes as a unit of language, 287t using syllables as a unit of language, 286t Functionalist, 123 124 Functional magnetic resonance (fMRI) imaging, 184 advantages of, 184 185 disadvantages of, 185

Further than mechanization, 238 239

G Gadgets of Objects with computing devices, 236 Gaming, 139 140 virtual reality for, 141f γ-aminobutyric acid (GABA), 27 Global Services Survey, 256 257 Glucagon-like peptide 1, 29 Glutamate, 26 27 Goldilocks principle, 26 Google Lens, 135 GPTS (Green Paranoid Thoughts Scale), 137 138 Grenfell and Harris model, 275 276 Ground truth labels, 343

H Hadoop, 240 Healthcare text classification system and its performance evaluation, 319 benchmarking, 340 342 classification accuracy, effect of training data size on, 329 332 classifiers, 327 328, 343 AdaBoost, 327 decision tree (DT), 328 multilayer perceptron, 327 stochastic gradient descent (SGD), 328 support vector machine, 327 data types, 323 326 disease dataset, 325, 347 Reuters (R8) dataset, 325, 348 SMS dataset, 325 326, 348 TwitterA dataset, 324, 346 WebKB4 dataset, 324 325, 347 effect of training data size on classification accuracy, 329 ground truth labels, 343 hypothesis validation, 334 335 individual receiver operating characteristic plots, 336 literature survey and proposed model, 321 323 misrepresentation ratio, 343 overall mean performance, 332 overall mean performance using all parameters, 329 accuracy, 330 positive predictive value, 330 sensitivity, 329 specificity, 329 reliability index, 337 stability index, 337 340 strength, 343

Index

system accuracy computation over all parameters, 330 system classifier accuracy computation over all parameters, 328 330 HeartIndex, 305 Hierarchical drift diffusion model (HDDM), 65, 71 High-density diffuse optical tomography, 182 183, 183f Hindi, language learnability analysis of. See Language learnability analysis of Hindi Hindi dependency Treebank, 213 214 Hippocampus, 23 Hobb’s theory, 213 214 Homogeneous ensemble of PNN (HEN), 258 Household appliances, 249 Human brain, 3 4 Huntington’s disease (HD), 22 Hybrid machine translation approach, 276 5-Hydroxytryptamine (5-HT), 28

I Ideal learner, 279 Image-based stroke risk assessment using machine learning, 300 303 Incremental gradient descent. See Stochastic gradient descent (SGD) Indian music, history of, 103 104 Institute of Formal and Applied Linguistics (UFAL), 280, 281t Intellectual AI and cognition, AI, 237 Intellectual computing, 237 240 Intellectual Internet of Things, 239 240 challenge of, 244 246 ownership of, 240 243 pillars of, 243 244 value of, 246 249 Intellectual Objects, 233 Intelligent metasearch system for advanced ecommerce (IMSS-AE), 258 Intensity, 112 113, 114f, 115f Interadventitial diameter (IAD), 297 298 Interlink, 197 Internet of Things (IoT), 233 234 Intravascular ultrasound (IVUS), 293 Intuitive decision-making, 72 73

J Janak raga, 103 104, 108 109

K Kinect sensor. See Multiple Kinect application for occlusion problem k-means clustering, 6, 136 137 Krama Sampurna raga, 109

L Language, 273 274 Language learnability analysis of Hindi, 273 evaluation models, 277 280 Bayesian inference, 279 280 Bayesian segmentation, 277 279 future work, 287 288 language acquisition theories, 274 277 learnability analysis, data preparation for, 280 283 phonemization, 283 syllabification, 282 283 transliteration, 282 results and discussions, 283 286 LATER (linear rise-to-threshold at ergodic rate) model, 84 85 Learnability of language, 274 Lewy body diseases (LBDs), 35 36 Lexicon-based model, 259 Lexicons, 275 Likelihood ratio (LR), 68 Linear and logistic regression, 300 Linear approach to threshold explaining space and time (LATEST) model, 65 66, 85 Linear ballistic accumulator (LBA) model, 71 72 Linguistic preprocessor, efficiency of, 212 Linguistics, 2 3, 135 cognitive science in, 123 124 Linguistic theory, 274 Literary language reading an extremist mind through. See Tagore’s The Post Office Long-term memory (LTM), 24 25 Lumen diameter (LD), 297 298

M Machine cognition, 237 Machine learning (ML), 136 137, 243 244, 293 295, 319, 322 323 achieving generalization of, 308 area under the curve of system performance linking misrepresentation ratio with, 334 -based algorithms, 300 -based risk assessment, 305 306 black-box nature of, 307 308 cardiovascular disease/stroke risk assessment indices, 305 cardiovascular diseases risk assessment using ML, 303 305 challenges in design of, 307 308 classification accuracy, 334 general framework of, 297 299

375

376

Index

Machine learning (ML) (Continued) data partitioning, 298 299 feature engineering, 297 298 performance evaluation, 299 prediction/testing model, 299 training model design, 299 image-based stroke risk assessment, 300 303 types of machine learning techniques, 296 297 Magnetic resonance imaging (MRI), 183 185, 184f fMRI imaging, 184 185 MRI head, 184 Magnetoencephalography (MEG), 189 advantages of, 189 limitations of, 189 Markov Model Toolkit, 281 282 Matrix factorization with user attributes (MFUA), 256 257 Mean area under the curve, effect of misrepresentation ratio on, 335 Medial temporal lobe (MTL), 23 Melakarta theory, 104, 106 Melanocortin-4 receptor (MC4R), 31 Memory encoding, neuroanatomy of, 22 24 basal forebrain (BF), 24 diencephalon, 23 24 medial temporal lobe (MTL), 23 Memory formation, mechanisms underlying, 24 25 Memory function of brain, 22 Metatasks, 161 162 Mind of extremist, 203 205 Mining algorithms, 137 Mirror neurons, 198 Misrepresentation ratio (MRR), 319, 322, 334, 339, 343 on machine learning classification accuracy, 334 on mean area under the curve for all classifiers and all data types, 335 Morpheme, 277 278 Morris water maze (MWM), 28 Movement inhibition, models of, 75 85 Bayesian rational decision-making, 81 83 decision process as optimal stochastic control, 84 estimation of stopping efficacy, 79 80 linear approach to threshold explaining space and time model, 84 85 optimal Bayesian statistical inference, 83 84 proactive control, 78 79 trigger failures, 80 81 Multi-Ethnic Study for Atherosclerosis (MESA), 304 Multilayer perceptron (MLP), 327

Multiple choice, 161 162 Multiple Kinect application for occlusion problem, 59 61 frame synchronization, 60 sensing data integration without calibration, 61 Music information retrieval (MIR), speech analyses in, 103

N Naive Bayes algorithm, 321 Named entity recognizer (NER), 213 National/colonial “egoism”, 198 National extremism, 200 203 Nationalism, 201 203 “Nation is the greatest evil for the Nation”, 205 207 Native language, acquisition of, 274 Natural languages processing (NLP), 211, 276 277 Near-infrared spectroscopy based imaging equipment, 182 183 diffuse optical imaging (DOI) or diffuse optical tomography (DOT), 182 functional near-infrared spectroscopy (fNIRS), 182 high-density diffuse optical tomography, 182 183 Neuroanatomy of memory encoding, 22 24 basal forebrain (BF), 24 diencephalon, 23 24 medial temporal lobe (MTL), 23 Neurocomputational predictions. See Tagore’s The Post Office Neurodiagnosis techniques sleep-based disorder analysis using, 191 192 Neurofibrillary tangles, intraneuronal accumulation of, 34 35 Neuroimaging, 179 Neuronal etiology of creative works, 198 Neuropeptides, 29 31 α-melanocyte stimulating hormone, 31 cocaine- and amphetamine-regulated transcript (CART), 29 30 neuropeptide Y, 30 31 Neuropeptide Y (NPY), 29 31 Neuroscience, 3 4 Neurosteroids, 31 32 Neurotransmitters, classical, 25 29 acetylcholine (ACh), 25 26 agmatine, 28 29 dopamine (DA), 27 28 γ-aminobutyric acid (GABA), 27 glutamate, 26 27 serotonin (5-hydroxytryptamine), 28

Index

n-grams, 258 NLP innovation, 244 Novel object recognition test (NORT), 28 Nucleus, 282 283

O Objects with computing devices and artificial intelligence (AI), 234 237 Internet of Things (IoT), 233 234 need for AI in Internet of Things, 235 237 objects with computing devices and computerized ones, 234 235 Objects with computing devices and Intellectual Computing intellectual Internet of Things, 239 240 challenge of, 244 246 ownership of, 240 243 pillars of, 243 244 Occlusion problem, multiple Kinect application for, 59 61 frame synchronization, 60 sensing data integration without calibration, 61 OnlineMem, 280 OnlineOpt, 279 OnlineSubOpt, 279 Onset, 282 283 Optimal Bayesian statistical inference, 83 84 Optimal segmentation, 279 280 Optimal stochastic control, decision process as, 84 Orienting response (OR), 156 157 “The Outsiders” (Lovecraft, H.P.), 204 205

P Papez circuit, 158 Parallel processing time (PPT), 81 Parent raga, 104 Parkinson’s disease (PD), 22 Parts-of-speech (POS) tagging, 213, 256 257 PC vision, 244 Perceptual decision, models of, 67 73 fast decision-making, 72 intuitive decision-making, 72 73 Performance evaluation (PE), 322 Personalized recommendation model, 258 259 Philosophy, 2 3, 135 cognitive science for, 121 Phonemes, 282 Phonemization, 283 Photomultiplier tubes (PMT), 185 Physiological behavioral gesture in animals/ humans, 9f Physiology, 2, 5 6, 134 136 Pitch, 112 113, 114f

Pituitary adenylate cyclase activating polypeptide, 29 Polysomnography (PSG), 191 192 Pooled cohort risk score (PCRS), 294 295, 304 305 Positive predictive value, 330 Positive predictive value tables, 364 365 The Post Office (R.N. Tagore). See Tagore’s The Post Office Praat, practical example using, 112 116 Proactive control, 78 79 Probabilistic neural network (PNN), 258 Product reviews, 255 258 PROMETHEE II method, 257 258 Psychological Computing, 239 240 Psychological Objects, 249 Psychology, 1 2, 134 cognitive, 6 12 cognitive science for, 122 Psychotic people, flow chart for, 138f Pulse-height analyzer (PHA), 185 Python, 1, 4

R Raga identification, speech recognition technique for, 101 Carnatic music, mathematical structure of, 104 109 digital speech processing, 109 110 Indian music, history of, 103 104 music information retrieval, speech analyses in, 103 Praat, practical example using, 112 116 proposed methodology for raga classification, 110 112 Rating scales, 161 162 Rational decision-making (RDM) model, 65 66 Receiver operating characteristic (ROC) curves, 321, 350 362 Regression-based ML algorithms, 300 Reinforcement learning, 296 297 Reliability index, 337 Resolution engine for anaphora in Hindi dialogue (REAH), 211 boundaries in anaphora resolution, 212 213 efficiency of linguistic preprocessor, 212 lack of efficient named entity recognizer, 213 no benchmark for POS tagging, 213 nonavailability of freeware Hindi discourse, 212 categorization of Hindi anaphora, 211 212 experiments and evaluations, 228 229 resolution engine, 214 227 anaphora resolution phase, 222 227

377

378

Index

Resolution engine for anaphora in Hindi dialogue (REAH) (Continued) preprocessing phase, 214 221 state-of-the-art, 213 214 background of the authors, 214 test datasets, 227 Reuters (R8) dataset, 325, 348 Robot interface, feedback by, 57 59 Roman script, 282

S Saccadic reaction time, 84 85 Samiksha, 257 258 Sampurna raga, 104 Samveda, 103 104 Sapta swaras, 103 104 SBQs (safety-based questionnaires), 6 7, 137 138 Schizophrenia, 22 Segments, 275 Sensitivity, 321, 329 Sensitivity tables, 365 367 Sensory memory, 24 25 SentiCRF model, 257 Sentiment analysis, 255 256, 258 259 SentiView, 258 259 Sequential sampling models of decision-making, 69f Serotonin (5-hydroxytryptamine), 28 Shakti Standard Format (SSF), 215 Shaudav raga, 104 Short-term memory (STM), 24 25 Simon task, 75 Single-photon emission computed tomography (SPECT), 185 186 advantages of, 185 186 disadvantage of, 186 Skin admittance (SY), 150 151 Skin impedance (SZ), 150 151 Skin potential (SP), 150 151 Sleep-based disorder analysis, 191 192 using polysomnography, 191 192 Smart cities, 250 SMS dataset, 325 326, 348 Somatostatin, 29 Sonority hierarchy of Hindi and English, 283, 284f Space and time model linear approach to threshold explaining, 84 85 Specificity, 329 Specificity tables, 367 368 Spectrogram, 112 113, 114f, 116f Speech-analysis tools, 109 110 Speech recognition, 101 applications of, 102 103

Speech segmentation, 277 Speech signals, 273 274 SSBI Model, 275 276 Stability index, 337 340 Stenosis severity index, 297 298 Sthai swaras, 104 Stochastic gradient descent (SGD), 327 328 Stop-signal response time (SSRT), 79 81 Stroke risk assessment image-based, 300 303 indices, 305 Stroop task, 75 Subjective computing, 238 Substance P, 29 Super segments, 275 Supervised learning, 296 297 Support vector machine (SVM), 258 259, 300, 327 Swadeshi Movement, 200 201 Swadeshi Nationalism and Nonalignment, 203 Syllabification, 282 283 Syllables, 282, 283f Symptomatic asymptomatic carotid index (SACI), 305 Synchronized analytics, 250 System classifier accuracy computation over all parameters, 328 330 accuracy, 330 positive predictive value, 330 sensitivity, 329 specificity, 329 System performance linking misrepresentation ratio with area under the curve of machine learning system, 334

T Tagore’s The Post Office, 197 198 affecting factors to activate mirror neuron in, 198 Amal as a religion under control, 207 208 characters of, 201t, 202f colonialism/nationalism or national extremism, 200 203 hypothesis, 199 200 “Nation is the greatest evil for the Nation”, 205 207 neurological observation, 203 205 Technology Development for Indian Languages (TDIL), 280, 281t, 285 Term patterns, defining, 216 TF-IDF, 258 vectorization, 265 Third-person pronouns (TPP), 213 214 Thyrotrophin-releasing hormone, 29

Index

Time- and feature-specific sentiment analysis of product reviews, 255 aging factor, 262 264 experimental setup, 264 268 classifying the review tokens under the features in the feature dictionary, 265 collection and preparing of dataset, 267 268 defining feature dictionary for product, 264 finding the sentiments of the review tokens for each feature, 265 267 multiplying the polarity with the aging factor, 267 268 preprocessing, tokenizing, and vectorizing the dataset, 265 summing up the results for each feature, 268 visualizing the results, 268 future work, 270 need of feature specificity, 261 262 proposed model, 259 261 related work, 256 259 result and discussion, 269 Tomography, 180 Tonic electrodermal activity, 170 Training model design, 299 Transdisciplinary research, 197 Transfer learning, Role of, 308 Transliteration, 282 Treebank, 215

Tree-based algorithms, 300 Trigger failures (TF), 80 81 TwitterA dataset, 324, 346

U Unigram model, 278 279, 286t Unsupervised learning, 296 297 Usecase, 250 251

V Vasopressin, 29 Video-EEG polysomnography, 191 Virtual reality systems, 5, 9 12, 11f, 135 application of, 133 cognitive neuroscience/physiology, 136 137 cognitive psychology, 137 141 cognitive science, 133 135 Visualization, 133, 136 Voice recognition, 101 Vowel-related sounds, 275

W WebKB4 dataset, 324 325, 347 Weiner noise process, 68 71 Well turned-out livelihood, 249 Whirlpool, 249 Wiki City, 250

379