Biologically Inspired Cognitive Architectures (BICA) for Young Scientists : Proceedings of the First International Early Research Career Enhancement School on BICA and Cybersecurity (FIERCES 2017) 978-3-319-63940-6, 3319639404, 978-3-319-63939-0

Table of contents :
Front Matter ....Pages i-xix
Front Matter ....Pages 1-1
Task Planning in “Block World” with Deep Reinforcement Learning (Edward Ayunts, Alekasndr I. Panov)....Pages 3-9
Discrete Modeling of Multi-transmitter Neural Networks with Neuronal Competition (Nikolay Bazenkov, Varvara Dyakonova, Oleg Kuznetsov, Dmitri Sakharov, Dmitry Vorontsov, Liudmila Zhilyakova)....Pages 10-16
A Simple Virtual Actor Model Supporting Believable Character Reasoning in Virtual Environments (Pavel A. Bortnikov, Alexei V. Samsonovich)....Pages 17-26
Intelligent Search System for Huge Non-structured Data Storages with Domain-Based Natural Language Interface (Artyom Chernyshov, Anita Balandina, Anastasiya Kostkina, Valentin Klimov)....Pages 27-33
Modeling Behavior of Virtual Actors: A Limited Turing Test for Social-Emotional Intelligence (Arthur Chubarov, Daniil Azarnov)....Pages 34-40
Rethinking BICA’s R&D Challenges: Grief Revelations of an Upset Revisionist (Emanuel Diamant)....Pages 41-46
A Roadmap to Emotionally Intelligent Creative Virtual Assistants (Alexander A. Eidlin, Alexei V. Samsonovich)....Pages 47-56
Applying a Neural Network Architecture with Spatio-Temporal Connections to the Maze Exploration (Dmitry Filin, Aleksandr I. Panov)....Pages 57-64
A Hands-on Laboratory Tutorial on Using CST to Build a Cognitive Architecture (Ricardo R. Gudwin)....Pages 65-71
Robot Navigation Based on an Artificial Somatosensorial System (Ignazio Infantino, Adriano Manfré, Umberto Maniscalco)....Pages 72-77
About Realization of Aggressive Behavior Model in Group Robotics (Irina Karpova)....Pages 78-84
Human Brain Structural Organization in Healthy Volunteers and Patients with Schizophrenia (Sergey Kartashov, Vadim Ushakov, Alexandra Maslennikova, Alexander Sboev, Anton Selivanov, Ivan Moloshnikov et al.)....Pages 85-90
No Two Brains Are Alike: Cloning a Hyperdimensional Associative Memory Using Cellular Automata Computations (Denis Kleyko, Evgeny Osipov)....Pages 91-100
Informative Characteristics of Brain Activity to Diagnose Functional Disorders in People with Stuttering (Anastasia Korosteleva, Olga Mishulina, Vadim Ushakov, Olga Skripko)....Pages 101-106
Event-Related fMRI Analysis Based on the Eye Tracking and the Use of Ultrafast Sequences (Anastasia Korosteleva, Vadim Ushakov, Denis Malakhov, Boris Velichkovsky)....Pages 107-112
The Presentation of Evolutionary Concepts (Sergey V. Kosikov, Viacheslav E. Wolfengagen, Larisa Yu. Ismailova)....Pages 113-125
Semantic Comprehension System for F-2 Emotional Robot (Artemy Kotov, Nikita Arinkin, Alexander Filatov, Liudmila Zaidelman, Anna Zinina)....Pages 126-132
Methodology of Learning Curve Analysis for Development of Incoming Material Clustering Neural Network (Boris Onykiy, Evheniy Tretyakov, Larisa Pronicheva, Ilya Galin, Kristina Ionkina, Andrey Cherkasskiy)....Pages 133-138
Modern Views on Visual Attention Mechanisms (Lubov Podladchikova, Anatoly Samarin, Dmitry Shaposhnikov, Mikhail Petrushan)....Pages 139-144
Model of Interaction Between Learning and Evolution (Vladimir G. Red’ko)....Pages 145-150
Intelligent Planning Methods and Features of Their Usage for Development Automation of Dynamic Integrated Expert Systems (Galina V. Rybina, Yuri M. Blokhin, Sergey S. Parondzhanov)....Pages 151-156
Ontological Approach for the Organization of Intelligent Tutoring on the Basis of Tutoring Integrated Expert Systems (Galina V. Rybina, Elena S. Sergienko)....Pages 157-162
A Continuous-Attractor Model of Flip Cell Phenomena (Alexei V. Samsonovich)....Pages 163-172
Neural Network Classification Method for Solution of the Problem of Monitoring Theremoval of the Theranostics Nanocomposites from an Organism (Olga Sarmanova, Sergey Burikov, Sergey Dolenko, Eva von Haartman, Didem Sen Karaman, Igor Isaev et al.)....Pages 173-179
Realization of the Gesture Interface by Multifingered Robot Hand (Pavlovsky Vladimir, Stepanova Elizaveta)....Pages 180-185
A Conscious Robot that Can Venture into an Unknown Environment in Search of Pleasure (Yuichi Takayama, Junichi Takeno)....Pages 186-191
Algorithms for Intelligent Automated Evaluation of Relevance of Search Queries Results (Anna Tikhomirova, Elena Matrosova)....Pages 192-197
“Re:ROS”: Prototyping of Reinforcement Learning Environment for Asynchronous Cognitive Architecture (Sei Ueno, Masahiko Osawa, Michita Imai, Tsuneo Kato, Hiroshi Yamakawa)....Pages 198-203
Adaptive Control of Modular Robots (Alexander V. Demin, Evgenii E. Vityaev)....Pages 204-212
Model of Heterogeneous Interactions Between Complex Agents. From a Neural to a Social Network (Liudmila Zhilyakova)....Pages 213-218
Front Matter ....Pages 219-219
Stochastic Data Transformation Boxes for Information Security Applications (Ahmad Albatsha, Michael A. Ivanov)....Pages 221-227
An Innovative Algorithm for Privacy Protection in a Voice Disorder Detection System (Zulfiqar Ali, Muhammad Imran, Wadood Abdul, Muhammad Shoaib)....Pages 228-233
Handwritten Signature Verification: The State of the Art (Anastasia Beresneva, Anna Epishkina, Sergey Babkin, Alexey Kurnev, Vladimir Lermontov)....Pages 234-238
The Port-in-Use Covert Channel Attack (Dmitry Efanov, Pavel Roschin)....Pages 239-244
Discovering and Clustering Hidden Time Patterns in Blockchain Ledger (Anna Epishkina, Sergey Zapechnikov)....Pages 245-250
On Attribute-Based Encryption for Access Control to Multidimensional Data Structures (Anna Epishkina, Sergey Zapechnikov)....Pages 251-256
Gamma-Probe for Locating the Source of Ionizing Radiation (Jake Hecla, Timur Khabibullin, Andrey Starikovskiy, Anastasia Tolstaya)....Pages 257-269
New Life of Old Standard: Transition from One-Dimensional Version to 3D (Mikhail A. Ivanov, Andrey V. Starikovskiy)....Pages 270-275
Algorithmic Foundation for Benchmarking of Computational Platforms Running Asymmetric Cipher Systems (Mikhail A. Kupriyashin, Georgii I. Borzunov)....Pages 276-281
Analysis of SIEM Systems and Their Usage in Security Operations and Security Intelligence Centers (Natalia Miloslavskaya)....Pages 282-288
Organization’s Business Continuity in Cyberspace (Natalia Miloslavskaya, Svetlana Tolstaya)....Pages 289-295
DLP Systems as a Modern Information Security Control (Victor Morozov, Natalia Miloslavskaya)....Pages 296-301
Cognitive Data Visualization of Chirality-Dependent Carbon Nanotubes Thermal and Electrical Properties (Vadim Shakhnov, Vadim Kazakov, Lyudmila Zinchenko, Vladimir Makarchuk)....Pages 302-307
Security Module Protecting the Privacy of Mobile Communication (Andrey Starikovskiy, Leonid Panfilov, Ilya Chugunkov)....Pages 308-317
Extracting of High-Level Structural Representation from VLSI Circuit Description Using Tangled Logic Structures (Andrey Trukhachev, Natalia Ivanova)....Pages 318-323
Medical Knowledge-Based Decision Support System (Alexey Fomin, Mikhail Turov, Elena Matrosova, Anna Tikhomirova)....Pages 324-328
Copyright Protection for Video Content Based on Digital Watermarking (Ivanenko Vitaliy, Ushakov Nikita)....Pages 329-334
Method for Early Cognition of Unloyal Behaviour by Combining Analysis of Natural and Artificial Detection Methods (Sergey I. Zhurin)....Pages 335-342
Probabilistic Assessment of the Organization of Tournaments and Examinations Using Paired Comparisons (Margarita A. Zaeva, Alexander A. Akhremenkov, Anatoly M. Tsirlin)....Pages 343-348
Criteria for Assessing the Results of Production Activities of Automobile Gas Filling Compressor Stations (Andrew A. Evstifeev, Margarita A. Zaeva)....Pages 349-355
Back Matter ....Pages 357-358

Citation preview

Advances in Intelligent Systems and Computing 636

Alexei V. Samsonovich Valentin V. Klimov Editors

Biologically Inspired Cognitive Architectures (BICA) for Young Scientists Proceedings of the First International Early Research Career Enhancement School on BICA and Cybersecurity (FIERCES 2017)

Advances in Intelligent Systems and Computing Volume 636

Series editor Janusz Kacprzyk, Polish Academy of Sciences, Warsaw, Poland e-mail: [email protected]

About this Series The series “Advances in Intelligent Systems and Computing” contains publications on theory, applications, and design methods of Intelligent Systems and Intelligent Computing. Virtually all disciplines such as engineering, natural sciences, computer and information science, ICT, economics, business, e-commerce, environment, healthcare, life science are covered. The list of topics spans all the areas of modern intelligent systems and computing. The publications within “Advances in Intelligent Systems and Computing” are primarily textbooks and proceedings of important conferences, symposia and congresses. They cover significant recent developments in the field, both of a foundational and applicable character. An important characteristic feature of the series is the short publication time and world-wide distribution. This permits a rapid and broad dissemination of research results.

Advisory Board Chairman Nikhil R. Pal, Indian Statistical Institute, Kolkata, India e-mail: [email protected] Members Rafael Bello Perez, Universidad Central “Marta Abreu” de Las Villas, Santa Clara, Cuba e-mail: [email protected] Emilio S. Corchado, University of Salamanca, Salamanca, Spain e-mail: [email protected] Hani Hagras, University of Essex, Colchester, UK e-mail: [email protected] László T. Kóczy, Széchenyi István University, Győr, Hungary e-mail: [email protected] Vladik Kreinovich, University of Texas at El Paso, El Paso, USA e-mail: [email protected] Chin-Teng Lin, National Chiao Tung University, Hsinchu, Taiwan e-mail: [email protected] Jie Lu, University of Technology, Sydney, Australia e-mail: [email protected] Patricia Melin, Tijuana Institute of Technology, Tijuana, Mexico e-mail: [email protected] Nadia Nedjah, State University of Rio de Janeiro, Rio de Janeiro, Brazil e-mail: [email protected] Ngoc Thanh Nguyen, Wroclaw University of Technology, Wroclaw, Poland e-mail: [email protected] Jun Wang, The Chinese University of Hong Kong, Shatin, Hong Kong e-mail: [email protected]

More information about this series at http://www.springer.com/series/11156

Alexei V. Samsonovich Valentin V. Klimov •

Editors

Biologically Inspired Cognitive Architectures (BICA) for Young Scientists Proceedings of the First International Early Research Career Enhancement School on BICA and Cybersecurity (FIERCES 2017)

123

Editors Alexei V. Samsonovich Krasnow Institute George Mason University Fairfax, VA USA

Valentin V. Klimov National Research Nuclear University MEPhI Moscow Russia

and National Research Nuclear University MEPhI Moscow Russia

ISSN 2194-5357 ISSN 2194-5365 (electronic) Advances in Intelligent Systems and Computing ISBN 978-3-319-63939-0 ISBN 978-3-319-63940-6 (eBook) DOI 10.1007/978-3-319-63940-6 Library of Congress Control Number: 2017946691 © Springer International Publishing AG 2018 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Printed on acid-free paper This Springer imprint is published by Springer Nature The registered company is Springer International Publishing AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Preface

Merging Biologically Inspired Cognitive Architectures and Cybersecurity The challenge to replicate all the key features of the human mind in a digital environment using a biologically inspired approach (the BICA Challenge) is the spirit and the core of the new frontier that every year attracts more and more young scientists. Its counterpart challenge of Cybersecurity acquires priority as we advance deeper and deeper into the uncharted territory. After many decades of progress in the field of artificial intelligence, problems that we are facing today require a fresh, multidisciplinary view. We need to learn from scratch how to achieve goals that could never be taken seriously in the past, with an understanding that a novel approach is necessary, because essential qualities of biological intelligent systems such as robustness, flexibility, adaptability, communicability, and reliability are still unmatched by their artificial counterparts. This volume includes papers from the First International Early Research Career Enhancement School (FIERCES) on Biologically Inspired Cognitive Architectures and Cybersecurity, which is the second meeting of the FIERCES series. The school was held in Baltschug Kempinski hotel in Moscow, Russia, during August 1–6, 2017. Combining two hot topics—BICA and Cybersecurity, its mission was to facilitate interaction and collaboration among top experts in the field (including such names as Agnese Augello, Piotr Boltuc, Peter Gärdenfors, Olivier Georgeon, Ricardo Gudwin, Ignazio Infantino, Frank Krueger, Adriano Manfre’, Giovanni Pilato, Aaron Sloman, Filippo Vella) and young researchers, who devoted themselves to the solution of the BICA Challenge, by bridging cross-disciplinary, cross-generation, and cross-cultural barriers. Biologically Inspired Cognitive Architectures (BICA) is computational blueprints for building intelligent agents, inspired from biological prototypes. During the meeting, they helped us to utilize the vast accumulated knowledge about the brain in order to learn from nature how to build intelligent systems. At the same time, new techniques and concepts in digital security complemented the main focus

v

vi

Preface

of the school and the book. As a consequence, this first school on BICA and Cybersecurity was interdisciplinary in nature and yielded bidirectional flow of understanding between experts in all involved disciplines. Therefore, topics of articles included in this volume extensively cover the most advanced scientific fields relevant to BICA that are traditionally considered at the international level of significance and discussed at many mainstream national and international conferences on artificial intelligence, neuroscience, and cognitive modeling, including conferences organized by BICA Society. The list of the latter is quite long. Beginning with the AAAI Fall Symposia on BICA (2008, 2009), the Annual International Conference on BICA has been held every year since 2010, demonstrating progressively growing popularity. Locations of the conference included Washington, DC (2010); Palermo, Italy (2012); Kiev, Ukraine (2013); Cambridge, Massachusetts (2014); Lyon, France (2015); and New York, USA (2016). The 2017 BICA event in Moscow, however, was unique in its kind, because it brought the conference and the school together. In this year, we received a record number of qualified submissions for a BICA event. Not all papers submitted and not all works presented at the school were selected for publication. In selecting papers, we paid attention to their scientific quality and relevance to the two challenges. All works included in this volume have been carefully peer-reviewed and refereed and reflect the high level of ongoing research and development in participating leading universities and research centers around the world, including those in the USA, France, Germany, Italy, Spain, Japan, Brazil, China, Ukraine, Belarus, and also in Russia (Moscow, St. Petersburg, Novosibirsk, and other Russian cities). The list of our reviewers was equally widely distributed around the globe. Some good papers were rejected, because they were too long, and were redirected to a journal venue. Each accepted paper was reviewed and peer-refereed by at least two independent anonymous reviewers. Overall, all authors, reviewers, and participants did a great job. The result is what you see in this book. Papers included in this volume are a mixture of tutorials, research articles, focused on fundamental and applied areas of cognitive, social, and neurosciences and artificial intelligence, and position papers. Topics include, but are not limited to, cybersecurity, cognitive modeling, automated planning and behavior generation, soft computing, knowledge engineering, semantic search, ontologies and knowledge management, acquisition, representation and processing of knowledge, applied intelligent systems, intelligent tutoring, instrumental systems for artificial intelligence. We are grateful to all authors who contributed their works to this volume. We also would like to express our many thanks to all people who helped us with the organization of the first FIERCES event on BICA+Cybersecurity, first and foremost, Ms. Leontina Di Cecco from Springer, Drs. Aleksandr I. Panov and Vladimir G. Redko from the Institute for System Analysis, and Ms. Anastasia Tolstaya from NRNU MEPhI. The last, but not the least, is our appreciation and acknowledgment of the sponsors of FIERCES on BICA. Financial sponsorship was provided by the Russian Science Foundation (Grant No 15-11-30014 to

Preface

vii

Dr. Alexei V. Samsonovich). Organizational support was provided by the Institute for Cyber Intelligent Systems and Department of Cybernetics of the National Research Nuclear University MEPhI (Moscow Engineering Physics Institute): https://mephi.ru/eng/about/departments/22.php, with the participation of BICA Society (http://bicasociety.org) as a main organizer. Other organizer names include Russian Association for Artificial Intelligence. August 2017

Alexei V. Samsonovich Valentin V. Klimov

Organization

Program Committee Kenji Araki Joscha Bach Paul Baxter Galina A. Beskhlebnova Tarek Besold Jordi Bieger Perrin Bignoli Douglas Blank Pavel Bortnikov Mikhail Burtsev Erik Cambria Leonardo Lana De Carvalho Suhas Chelian Antonio Chella Olga Chernavskaya Christopher Dancy Haris Dindo Sergey A. Dolenko Jim Eilbert Thomas Eskridge Usef Faghihi Stanley Franklin Salvatore Gaglio Olivier Georgeon Ian Horswill Eva Hudlicka

Hokkaido University, Japan MIT Media Lab, USA Plymouth University, USA SRI for System Analysis RAS, Russia University of Osnabruck, Germany Reykjavik University, Iceland Yahoo Labs, USA Bryn Mawr College, USA National Research Nuclear University MEPhI, Russia MIPT, Russia Nanyang Technological University, Singapore UFVJM, Brazil HRL Laboratories LLC, USA University of Palermo, Italy P.N. Lebedev Physical Institute, Russia Penn State University, USA University of Palermo, Italy M.V. Lomonosov Moscow State University, Russia AP Technology, USA Florida Institute of Technology, USA University of Indianapolis, USA University of Memphis, USA University of Palermo, Italy Claude Bernard Lyon 1 University, France Northwestern University, USA Psychometrix Associates, USA

ix

x

Christian Huyck Ignazio Infantino Eduardo Izquierdo Alex James Li Jinhai Magnus Johnsson Darsana Josyula Kamilla Jóhannsdóttir William Kennedy Deepak Khosla Valentin Klimov Giuseppe La Tona Luis Lamb Othalia Larue Christian Lebiere Jürgen Leitner Simon Levy Antonio Lieto Olga Mishulina Steve Morphet Amitabha Mukerjee Valentin A. Nepomnayschih David Noelle Andrea Omicini Aleksandr I. Panov David Peebles Giovanni Pilato Michal Ptaszynski Subramanian Ramamoorthy Thomas Recchia Vladimir Redko James Reggia Frank Ritter Paul Robertson Brandon Rohrer Paul Rosenbloom Christopher Rouff

Organization

Middlesex University, UK Consiglio Nazionale delle Ricerche, Italy Indiana University, USA Kunming University of Science and Technology, China Kunming University of Science and Technology, China Lund University, Sweden Bowie State University, USA Reykjavik University, Iceland George Mason University, USA HRL Laboratories LLC, USA National Research Nuclear University MEPhI, Russia University of Palermo, Italy Federal University of Rio Grande do Sul, Brazil University of Quebec, Canada Carnegie Mellon University, USA Australian Centre of Excellence for Robotic Vision, Australia Washington and Lee University, USA University of Turin, Italy National Research Nuclear University MEPhI, Russia Enabling Tech Foundation, USA Indian Institute of Technology Kanpur, India Institute for Biology of Inland Waters, Russia University of California Merced, USA University of Bologna, Italy Federal Research Center “Computer Science and Control” RAS, Russia University of Huddersfield, UK ICAR-CNR, Italy Kitami Institute of Technology, Japan University of Edinburgh, Scotland US Army ARDEC, USA Scientific Research Institute for System Analysis RAS, Russia University of Maryland, USA Penn State University, USA DOLL Inc., USA Sandia National Laboratories, USA University of Southern California, USA Near Infinity Corporation, USA

Organization

Galina Rybina Rafal Rzepka Ilias Sakellariou Alexei V. Samsonovich Fredrik Sandin Ricardo Sanz Michael Schader Michael Schoelles Valeria Seidita Ignacio Serrano Javier Snaider Donald Sofge Rosario Sorbello Terry Stewart Swathikiran Sudhakaran Sherin Sugathan Junichi Takeno Knud Thomsen Vadim Ushakov Rodrigo Ventura Pei Wang Mark Waser Tom Ziemke

xi

National Research Nuclear University MEPhI, Russia Hokkaido University, Japan University of Macedonia, Greece George Mason University, USA, and NRNU MEPhI, Russia Lulea University of Technology, Sweden Universidad Politecnica de Madrid, Spain Yellow House Associates, USA Rensselaer Polytechnic Institute, USA University of Palermo, Italy Instituto de Automatica Industrial, CSIC, Spain Google, USA Naval Research Laboratory, USA University of Palermo, Italy University of Waterloo, Canada Fondazione Bruno Kessler, Italy Enview Research & Development Labs, India Meiji University, Japan Paul Scherrer Institute, Switzerland National Research Center Kurchatov Institute, Russia Institute for Systems and Robotics, Portugal Temple University, USA Digital Wisdom Institute, USA University of Skovde and Linkoping University, Sweden

Core Organizing Committee of BICA 2017 Alexei V. Samsonovich (General Chair) Valentin V. Klimov (Co-chair) Aleksandr I. Panov (Technical Chair) Antonio Chella Michele Ferrante Olivier Georgeon Kamilla R. Johannsdottir Christian Lebiere Paul Robertson Terry C. Stewart

GMU, USA and NRNU MEPhI, Russia NRNU MEPhI, Russia FRC CSC, Russia University of Palermo, Italy Boston University, USA Claude Bernard Lyon 1 University, France Reykjavik University, Iceland Carnegie Mellon University, USA DOLL, Inc., USA University of Waterloo, Canada

Contents

Biologically Inspired Cognitive Architectures Task Planning in “Block World” with Deep Reinforcement Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Edward Ayunts and Alekasndr I. Panov Discrete Modeling of Multi-transmitter Neural Networks with Neuronal Competition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nikolay Bazenkov, Varvara Dyakonova, Oleg Kuznetsov, Dmitri Sakharov, Dmitry Vorontsov, and Liudmila Zhilyakova A Simple Virtual Actor Model Supporting Believable Character Reasoning in Virtual Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pavel A. Bortnikov and Alexei V. Samsonovich Intelligent Search System for Huge Non-structured Data Storages with Domain-Based Natural Language Interface . . . . . . . . . . . . . . . . . . . Artyom Chernyshov, Anita Balandina, Anastasiya Kostkina, and Valentin Klimov

3

10

17

27

Modeling Behavior of Virtual Actors: A Limited Turing Test for Social-Emotional Intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Arthur Chubarov and Daniil Azarnov

34

Rethinking BICA’s R&D Challenges: Grief Revelations of an Upset Revisionist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Emanuel Diamant

41

A Roadmap to Emotionally Intelligent Creative Virtual Assistants . . . . . Alexander A. Eidlin and Alexei V. Samsonovich Applying a Neural Network Architecture with Spatio-Temporal Connections to the Maze Exploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dmitry Filin and Aleksandr I. Panov

47

57

xiii

xiv

Contents

A Hands-on Laboratory Tutorial on Using CST to Build a Cognitive Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ricardo R. Gudwin Robot Navigation Based on an Artificial Somatosensorial System. . . . . . Ignazio Infantino, Adriano Manfré, and Umberto Maniscalco

65 72

About Realization of Aggressive Behavior Model in Group Robotics . . . . 78 Irina Karpova Human Brain Structural Organization in Healthy Volunteers and Patients with Schizophrenia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sergey Kartashov, Vadim Ushakov, Alexandra Maslennikova, Alexander Sboev, Anton Selivanov, Ivan Moloshnikov, and Boris Velichkovsky No Two Brains Are Alike: Cloning a Hyperdimensional Associative Memory Using Cellular Automata Computations . . . . . . . . . . . . . . . . . . . Denis Kleyko and Evgeny Osipov

85

91

Informative Characteristics of Brain Activity to Diagnose Functional Disorders in People with Stuttering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 Anastasia Korosteleva, Olga Mishulina, Vadim Ushakov, and Olga Skripko Event-Related fMRI Analysis Based on the Eye Tracking and the Use of Ultrafast Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Anastasia Korosteleva, Vadim Ushakov, Denis Malakhov, and Boris Velichkovsky The Presentation of Evolutionary Concepts . . . . . . . . . . . . . . . . . . . . . . . 113 Sergey V. Kosikov, Viacheslav E. Wolfengagen, and Larisa Yu. Ismailova Semantic Comprehension System for F-2 Emotional Robot. . . . . . . . . . . 126 Artemy Kotov, Nikita Arinkin, Alexander Filatov, Liudmila Zaidelman, and Anna Zinina Methodology of Learning Curve Analysis for Development of Incoming Material Clustering Neural Network . . . . . . . . . . . . . . . . . . 133 Boris Onykiy, Evheniy Tretyakov, Larisa Pronicheva, Ilya Galin, Kristina Ionkina, and Andrey Cherkasskiy Modern Views on Visual Attention Mechanisms . . . . . . . . . . . . . . . . . . . 139 Lubov Podladchikova, Anatoly Samarin, Dmitry Shaposhnikov, and Mikhail Petrushan Model of Interaction Between Learning and Evolution . . . . . . . . . . . . . . 145 Vladimir G. Red’ko

Contents

xv

Intelligent Planning Methods and Features of Their Usage for Development Automation of Dynamic Integrated Expert Systems . . . . . 151 Galina V. Rybina, Yuri M. Blokhin, and Sergey S. Parondzhanov Ontological Approach for the Organization of Intelligent Tutoring on the Basis of Tutoring Integrated Expert Systems . . . . . . . . . . . . . . . . 157 Galina V. Rybina and Elena S. Sergienko A Continuous-Attractor Model of Flip Cell Phenomena . . . . . . . . . . . . . 163 Alexei V. Samsonovich Neural Network Classification Method for Solution of the Problem of Monitoring Theremoval of the Theranostics Nanocomposites from an Organism. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 Olga Sarmanova, Sergey Burikov, Sergey Dolenko, Eva von Haartman, Didem Sen Karaman, Igor Isaev, Kirill Laptinskiy, Jessica M. Rosenholm, and Tatiana Dolenko Realization of the Gesture Interface by Multifingered Robot Hand . . . . 180 Pavlovsky Vladimir and Stepanova Elizaveta A Conscious Robot that Can Venture into an Unknown Environment in Search of Pleasure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 Yuichi Takayama and Junichi Takeno Algorithms for Intelligent Automated Evaluation of Relevance of Search Queries Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 Anna Tikhomirova and Elena Matrosova “Re:ROS”: Prototyping of Reinforcement Learning Environment for Asynchronous Cognitive Architecture . . . . . . . . . . . . . . . . . . . . . . . . . 198 Sei Ueno, Masahiko Osawa, Michita Imai, Tsuneo Kato, and Hiroshi Yamakawa Adaptive Control of Modular Robots . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204 Alexander V. Demin and Evgenii E. Vityaev Model of Heterogeneous Interactions Between Complex Agents. From a Neural to a Social Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213 Liudmila Zhilyakova Methods of Artificial Intelligence in Cybersecurity Stochastic Data Transformation Boxes for Information Security Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221 Ahmad Albatsha and Michael A. Ivanov An Innovative Algorithm for Privacy Protection in a Voice Disorder Detection System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228 Zulfiqar Ali, Muhammad Imran, Wadood Abdul, and Muhammad Shoaib

xvi

Contents

Handwritten Signature Verification: The State of the Art . . . . . . . . . . . . 234 Anastasia Beresneva, Anna Epishkina, Sergey Babkin, Alexey Kurnev, and Vladimir Lermontov The Port-in-Use Covert Channel Attack . . . . . . . . . . . . . . . . . . . . . . . . . . 239 Dmitry Efanov and Pavel Roschin Discovering and Clustering Hidden Time Patterns in Blockchain Ledger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245 Anna Epishkina and Sergey Zapechnikov On Attribute-Based Encryption for Access Control to Multidimensional Data Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 Anna Epishkina and Sergey Zapechnikov Gamma-Probe for Locating the Source of Ionizing Radiation . . . . . . . . . 257 Jake Hecla, Timur Khabibullin, Andrey Starikovskiy, and Anastasia Tolstaya New Life of Old Standard: Transition from One-Dimensional Version to 3D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270 Mikhail A. Ivanov and Andrey V. Starikovskiy Algorithmic Foundation for Benchmarking of Computational Platforms Running Asymmetric Cipher Systems . . . . . . . . . . . . . . . . . . . 276 Mikhail A. Kupriyashin and Georgii I. Borzunov Analysis of SIEM Systems and Their Usage in Security Operations and Security Intelligence Centers . . . . . . . . . . . . . . . . . . . . . . 282 Natalia Miloslavskaya Organization’s Business Continuity in Cyberspace . . . . . . . . . . . . . . . . . . 289 Natalia Miloslavskaya and Svetlana Tolstaya DLP Systems as a Modern Information Security Control . . . . . . . . . . . . 296 Victor Morozov and Natalia Miloslavskaya Cognitive Data Visualization of Chirality-Dependent Carbon Nanotubes Thermal and Electrical Properties . . . . . . . . . . . . . . . . . . . . . 302 Vadim Shakhnov, Vadim Kazakov, Lyudmila Zinchenko, and Vladimir Makarchuk Security Module Protecting the Privacy of Mobile Communication . . . . 308 Andrey Starikovskiy, Leonid Panfilov, and Ilya Chugunkov Extracting of High-Level Structural Representation from VLSI Circuit Description Using Tangled Logic Structures . . . . . . . . . . . . . . . . 318 Andrey Trukhachev and Natalia Ivanova Medical Knowledge-Based Decision Support System . . . . . . . . . . . . . . . . 324 Alexey Fomin, Mikhail Turov, Elena Matrosova, and Anna Tikhomirova

Contents

xvii

Copyright Protection for Video Content Based on Digital Watermarking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329 Ivanenko Vitaliy and Ushakov Nikita Method for Early Cognition of Unloyal Behaviour by Combining Analysis of Natural and Artificial Detection Methods . . . . . . . . . . . . . . . 335 Sergey I. Zhurin Probabilistic Assessment of the Organization of Tournaments and Examinations Using Paired Comparisons . . . . . . . . . . . . . . . . . . . . . 343 Margarita A. Zaeva, Alexander A. Akhremenkov, and Anatoly M. Tsirlin Criteria for Assessing the Results of Production Activities of Automobile Gas Filling Compressor Stations . . . . . . . . . . . . . . . . . . . . 349 Andrew A. Evstifeev and Margarita A. Zaeva Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357

Abstract

Included here are papers from the second year of the prestigious FIERCES series: a new, successful format that puts a school in direct connection with a conference and a social program, all dedicated to young scientists. The friendly social atmosphere of excitement and opportunity is imprinted between the lines of this book. Papers in it represent a good mixture of cutting-edge research focused on advances toward the most inspiring challenges of our time and first ambitious attempts at major challenges by yet unknown, talented young scientists. In this second year of FIERCES, the BICA Challenge (to replicate all the essential aspects of the human mind in the digital environment) meets the Cybersecurity Challenge (to protect all the essential assets of the human mind in the digital environment), that is equally important in our age. As a result, this book is expected to foster lively discussions on today’s hot topics in science and technology and to stimulate the emergence of new cross-disciplinary, cross-generation, and cross-cultural collaboration. FIERCES 2017, or the First International Early Research Career Enhancement School on Biologically Inspired Cognitive Architectures and Cybersecurity, was held on August 1–5 in Baltschug Kempinski, Moscow, Russia.

xix

Biologically Inspired Cognitive Architectures

Task Planning in “Block World” with Deep Reinforcement Learning Edward Ayunts1 and Alekasndr I. Panov1,2(B) 1

2

National Research University Higher School of Economics, Moscow, Russia [email protected], [email protected] Federal Research Center “Computer Science and Control” of Russian Academy of Sciences, Moscow, Russia

Abstract. At the moment reinforcement learning have advanced significantly with discovering new techniques and instruments for training. This paper is devoted to the application convolutional and recurrent neural networks in the task of planning with reinforcement learning problem. The aim of the work is to check whether the neural networks are fit for this problem. During the experiments in a block environment the task was to move blocks to obtain the final arrangement which was the target. Significant part of the problem is connected with the determining on the reward function and how the results are depending in reward’s calculation. The current results show that without modifying the initial problem into more straightforward ones neural networks didn’t demonstrate stable learning process. In the paper a modified reward function with sub-targets and euclidian reward calculation was used for more precise reward determination. Results have shown that none of the tested architectures were not able to achieve goal.

1

Introduction

In the robotics it is usually assumed the concept that robot is able to perform only the actions that it is programmed for. It enables them to do their tasks well, but limit their capacity to perform new actions. That is why the concept of learning robot is of greater interest, because in will potentially let it perform new complex actions, using a few basic ones. The key part in developing such robot is its training, and the novice of this paper that authors use neural networks with various architectures for training the robot to perform complex multi-step action. The work in this area was carried out for a last two decades and especially intensified in recent years. It’s worth mention the research [1], in which humanoid robot was taught to play air-hockey using visual processing of the game-field, sub-targets and primitives - simple actions, that “can be combined to complete the task”. c Springer International Publishing AG 2018  A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6 1

4

E. Ayunts and A.I. Panov

In a last few years neural networks became the catalysts for rapid and widespread popularity for the area. In the one of the key papers [2] in this field author managed to train agent to play Atari using Deep reinforcement learning. More precisely, agent received image at every step, preprocess it using 2D convolutions. The Q-Learning was modelled as neural network with observation as input and vector of separate units for each possible actions as output. The results show that the model was able to perform in some games better than the human. In the [3] authors developed a model which lets robot to foresight consequences of its actions, analyse them and determine the most preferable action. Robot gets as input video frames, and what is important - not necessarily from the same point of view. After model training, the agent is able to move object in new environments. The authors claim that the agent will able to perform more complex actions if the more detailed environment picture will be processed for training. The novice of the work is that authors did not use a special environment. Instead, they feed the net of a couple LSTM layers with photographs, and predict the new pixels by maximal likelihood method. The agent copes with the problem of determining the centre of masses of the object and is able to perform rotation. The only problem is that agent cannot distinguish its manipulator from the objects. In another paper [4] the main breakthrough is the training of the agent to reveal the parts of the picture worth paying attention and process them in high resolution. It gives a big advantage in using convolutional nets as there is no need to use them for processing the whole image, so it allows to economy on computations. The idea is adopted from human vision - the agent as a human, is focused only on a point from the whole picture. LSTM and Dense layers with ReLU were used for training the model. In [5] the author formulated the problem as following: agent should by visual information rotate by fingers an object in hand. The uniqueness of the work is that there are no assumptions on configurations of the hand, or its physical parameters, only pixel data is used. The aim was to provide rotating in only one axis, such that the rotating in all other axises was minimal. The convolutional and fully connected layers were trained and eventually the target was designed so that provide movement only in vertical axis. After 11 mln iterations stable performance was obtained. Value Iteration Networks, introduced in the [4] are of great interest, as they let a neural network not just train in the learning process, but to plan agent’s track. The algorithm “learns an explicit planning computations”, which enables it to generalize better to new domains. In the article authors compare their model with the fully-connected and convolutional networks and show the training results in three different environments and problems, during which VIN shows stable higher performance, especially when the volume of data increases.

Task Planning in “Block World” with Deep Reinforcement Learning

2

5

Problem Statement

The concept used for solution the stated problem is called Q-learning, which idea is the modelling the future cumulative reward for each action at every step. Initially, a zero matrix of dimension nS x nA is created, where nS and nA are the numbers of states and actions respectfully. At each step the agent chooses the action, such that the expected reward from it is maximal among all actions. After each step, the value of the expected reward is modifying by multiplying initial one by coefficient γ, usually equal to 0.99, and adding to it the received reward. The process of such iterations is called Q-learning. The key formula is Q(s, a) = E(R|s, a, π), where Q is expected total payoff for choosing action a in the state s and if in the future, the same strategy will befollowed.  T At every step the future payoff is calculated as Rt = t =t γ t −1 rt , where r is the reward for the current step. The decision function for the agent is Q ∗ (st , at ) = Est [rt + γmax]. Drawback of the algorithm is that there are situations in which proper action is depending on the environment, but the Q-learning does not take into account this point. That is why the hypothesis of the research is that using neural networks which take as input the whole vector of current state and of the target, the problem can be solved. Under the consideration three types of neural networks: multilayer perceptron, recurrent and convolutional networks.

3

Experiments

For conducting experiments the environment based on the OpenAI Gym was developed, available in https://github.com/cog-isa/deep-blocks.git. It consists of 30 × 30 matrix which illustrates the input signal on visual sensor, front-side view on the environment. There are 11 3 × 3 blocks in the environment, marked as ‘1’, all other elements are zeros. The agent also gets target matrix, which also contains the same 11 blocks, but all collected in the centre. The agent has 8 actions - by 4 for movements with cubes and without them. The manipulator is also represented by ‘1’s as reversed letter T in the 3 × 3 cube. The robot is able to move cube only when the manipulator is right above the cube. All restrictions, concerning physics of the movements, were imposed. Also every episode is limited to 100 moves. If the robot is aiming to perform prohibited movements, the state does not change and agent gets zero reward. Otherwise, reward is calculated as element-wise product of vectors of current observations and target divided by the total number of cubes. Multilayer perceptron with such rewardfunction demonstrated low performance, it hadn’t succeed in achieving target, but at least, it was able to move almost all cubes towards their final locations, the problems arise with the cubes at corners. Also agent often get locked in the upper areas of the environment where are no cubes, so, such reward function was not informative. The experiments were also conducted using ordinary Q-learning algorithm.

6

4

E. Ayunts and A.I. Panov

Reward Function

Instead of calculating it as proportion of blocks which are at their correct location at the current state, the euclidian distance is calculated between the manipulator and the block, which should be moved to its final place at current stage (their priorities are taken as given from initially given sub-targets). If the the manipulator is already has took the block, then the reward  is the distance to block’s final location. Reward is normed by dividing by 30 (2), to be less than 1. As soon as the block is at its place, the agent starts to calculate reward with use of the next subtarget. As can be seen from the algorithm below, the initial task is divided into 4 more simple ones. There 4 sub-targets because in the target blocks arrangement only 4 blocks are to be moved. So for each block to be moved a subtarget is designed. The reward is designed in such way to encourage agent to carry the block and don’t do prohibited movements. Data: Current State and Subtarget / Target Result: Reward Done1, Done2, Done3, Done4 = False, False, Fasle, False; dones = [Done1, Done2, Done3, Done4]; For i in dones:; if i is False then if The manipulator is not above the target block then Return 10 * ( 1- distance from manipulator to target block in current subtarget); else Return 20 * (1- distance from target block in current subtarget to its final location); end else end Algorithm 1. Reward Calculation

5

Implemented Neural Architectures

The architecture of the used network consisted of 1800 × 1 input vector, two hidden layers with 1000 and 100 layers respectfully and a output layer of 8 × 1 dimension. Input is constructed from 900 element vector for current state and 900 for target, states are represented as reshaped from 30 × 30 to 900 × 1 vectors. 5.1

Convolutional Network

The architecture for convolutional network was the following: 1D Convolution layer with 14 filter size and kernel = 1 was followed by two Dense layers with Dropout between them, and output dense layer which returns (1, 8) vector. The results show that at current stage the performance of model is very low: agent didn’t manage to achieve at least first subtarget (Fig. 1).

Task Planning in “Block World” with Deep Reinforcement Learning

7

Fig. 1. A model’s loss trained on 100 episodes, replayed after each 50 steps.

5.2

Recurrent Net

After unsatisfactory results with convolution the agent was trained on recurrent network with one LSTM layer. The architecture consisted of input vector with dimension (1,1800), LSTM with 1000 output elements, 1000 elements output Dense, 25% Dropout, Dense with 100 output elements and final layer with eight elements for each action. The agent was trained on 10 000 episodes and the results are very unstable, and it didn’t achieved goal. The same with two LSTM layers (second was added just after the first one). It was trained on the less number of episodes, and although it shows the more apparent downward trend, it also has not won the game (Figs. 2 and 3).

Fig. 2. Model with 1 LSTM layers

Fig. 3. Model with 2 LSTM layers

8

5.3

E. Ayunts and A.I. Panov

Q-Learning

Using regular Q-learning table without neural network didn’t gave more encouraging results. Using the initial reward function (fraction of cubes in their initial point) the mean reward was stable and equal roughly 0.5, it is less than initially and it means that the algorithm is less performative the Deep Q network. The reason for such result is that at every state the optimal movement depends in the context and Q-learning table does not consider other cubes location.

6

Testing the Hypothesis with Simple Model

As was stated before, used approach didn’t showed adoptable performance. In aim to check whether is it operates at all, the simple version of the game was designed. It consisted only of one block in the left corner of the 30 × 30 field and agent’s task was to move it to the right corner. The same reward function was used, with the following adjustments: 1. Reward is equal to Stage Coefficient multiplied by 1 – Distance to target; 2. Stage Coefficient is equal 20 if block is under the manipulator and 10 otherwise; 3. Reward is equal zero if the state has not changed since the last action. Conducted experiments using LSTM and 1D Convolution together showed that agent is able to move the block from left corner to the right one. It was made in 150 steps since the start of training. This result proves the initial hypothesis, that proposed approach is able to solve the task (Fig. 4).

Fig. 4. Loss function of model which managed to perform task

Task Planning in “Block World” with Deep Reinforcement Learning

7

9

Conclusions

In the designed environment agents was trained on Deep Q-Learning model implementation based on neural network with the used reward function proved to be unsuccessful for. Among all tested architectures the most promising was the one with two sequential LSTM and LSTM with 1D convolution layers. Nevertheless experiments with the same agent in the one block environment with the proposed approach showed good performance and so it has a potential on a more complex tasks. Conducted tests lead to a conclusion that the target would be attainable if more advanced reward policy will be used. The hypothesis of the causes of such results is that the model is underfitted because of relatively large size of the environment and plenty of unsuccessful actions which don’t provide information for the network. Further more advanced agent configurations and network architectures are planned to be tested and other planning tasks are to be worked on. Acknowledgements. The reported study was supported by RFBR, research Projects No. 16-37-60055 and No. 17-07-00281.

References 1. Bentivegna, D.C., Ude, A., Atkenson, C.G., Gordon, C.: Humanoid robot learning and game playing using PC-based vision, Switzerland (2002) 2. Mnih, V.: Playing atari with deep reinforcement learning. In: NIPS 2013 (2013) 3. Finn, C., Levine, S.: Deep visual foresight for planning robot motion. In: ICRA 2017 (2017) 4. Mnih, V., Heess, N., Graves, A., Kavukcuoglu, K.: Recurrent models of visual attention (2014) 5. Katyal, K.D., Staley, E.W., Johannes, M.S., Wang, I.-J., Reiter, A., Burlina, P.: In-hand robotic manipulation via deep reinforcement learning (2017)

Discrete Modeling of Multi-transmitter Neural Networks with Neuronal Competition Nikolay Bazenkov1(&), Varvara Dyakonova2, Oleg Kuznetsov1, Dmitri Sakharov2, Dmitry Vorontsov2, and Liudmila Zhilyakova1 1

2

Trapeznikov Institute of Control Sciences of RAS, Moscow, Russian Federation [email protected] Koltzov Institute of Developmental Biology of RAS, Moscow, Russian Federation

Abstract. We propose a novel discrete model of central pattern generators (CPG), neuronal ensembles generating rhythmic activity. The model emphasizes the role of nonsynaptic interactions and the diversity of electrical properties in nervous systems. Neurons in the model release different neurotransmitters into the shared extracellular space (ECS) so each neuron with the appropriate set of receptors can receive signals from other neurons. We consider neurons, differing in their electrical activity, represented as finite-state machines functioning in discrete time steps. Discrete modeling is aimed to provide a computationally tractable and compact explanation of rhythmic pattern generation in nervous systems. The important feature of the model is the introduced mechanism of neuronal competition which is shown to be responsible for the generation of proper rhythms. The model is illustrated with an example of the well-studied feeding network of a pond snail. Future research will focus on the neuromodulatory effects ubiquitous in CPG networks and the whole nervous systems. Keywords: Discrete dynamics
Multitransmitter neuronal Neurotransmitters
Neuromodulation
Central Pattern Generator

system




1 Introduction The neurotransmitter diversity is common to nearly all nervous systems, including the most primitive ones. This similarity indicates a fundamental role played by the neuronal heterogeneity. Functional significance and evolutionary origins of multiple neuronal phenotypes have been discussed in a number of papers [4, 11, 13]. We propose a formal model called a multitransmitter neuronal system. The model introduces several key features. First, the neurons produce endogenous activity like tonic spiking or oscillatory bursting. Second, the neurons interact not by synaptic wirings but via the extracellular space (ECS), which is shared by all neurons present in the circuit. One of the main objectives of the study is to show that a model based on pure non-synaptic interactions could produce the same patterns of neuronal activity as synaptic, “wired”, models. As a proof-of-concept example we provide a simplified model of a snail feeding network [15]. The extended paper can be found in [1]. © Springer International Publishing AG 2018 A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6_2

Discrete Modeling of Multi-transmitter Neural Networks

11

2 Related Work Continuous models. These models describe biological neurons and their membrane processes by differential equations [14]. In [15] a two-compartment neuron-axon model based on Hodgkin–Huxley equations [7] is created for modeling of the snail feeding circuit. The model of bursting neurons, was used in [5] to construct the interaction of interneurons in CPG and motor neurons in the locomotor system of insects. The main advantage of continuous models is their high degree of accuracy. But this leads to some drawbacks: lack of robustness to slight perturbations of parameters, significant computational complexity and poor scalability. Therefore, continuous models were applied only to networks with small numbers of neurons. Discrete models. In contrast to continuous models, discrete models tend to formalize neuronal processes as simple as possible. The most common type of discrete models is the formal neurons [10] and artificial neural networks (ANN) [6]. Another class of discrete models, called “complex networks” [8], considers the nervous system as a large complex network – “connectome” [3]. However, it is shown that a complete connectome is not sufficient for understanding the behavior [2]. One more type of models is represented in [12] where an automata-based language is used for the description of neurons with several types of endogenous activity. Biological background. The empirical generalizations which the model is based on are mostly derived from small neural networks that produce motor outputs, so called Central Pattern Generators [2, 9]. The given paper focuses mostly on heterogeneity of neuronal phenotypes and non-synaptic organization of phasic activity, leaving other properties for further research.

3 Multitransmitter Neural System 3.1

Main Definitions

A multi-transmitter neuronal system is a triple S ¼ \N; X; C [ , where N is a set of neurons, X – extracellular space (ECS) and C is a set of neurotransmitters. Neural inputs. Each neuronal input is characterized by a weight wij 2 R where i 2 N; j 2 C. If wij [ 0 then transmitter j excites neuron i and wij \0 denotes that transmitter
 j inhibits neuron i. Neuronal inputs are represented as a matrix W ¼ wij nm . Neural outputs. The model functions in discrete times t. Neuronal activity is denoted as yi ðtÞ 2 f0; 1g; yi ðtÞ ¼ 1 if neuron i is active at time t. After an activation, a neuron

12

N. Bazenkov et al.

releases
some amount of one or several neurotransmitters represented in a matrix D ¼ dij nm where dij  0 is the amount of transmitter j released by neuron i. Extracellular space. Neurons in the model communicate over the common extracellular space which contains the transmitters that have been released at time t. A state of ECS is represented as a vector X ðtÞ ¼ ðx1 ðtÞ;


; xm ðtÞÞ, where xj ðtÞ denotes the amount of neurotransmitter j present in the ECS at time t. 3.2

Excitation and Inhibition

Every neuron is influenced by all transmitters to which it possesses receptors. Each neuron has excitation and inhibition thresholds P1i and P0i ; P0i \0\P1i . The excitation function z1i ðtÞ indicates that neuron i is excited at time t: ! m X wij xj ðtÞ  P1i : z1i ðtÞ ¼ I ð1Þ j¼1

The inhibition function z0i ðtÞ, which indicates that a neuron is inhibited, is similar: z0i ðtÞ ¼ I

m X

! wij xj ðtÞ  P0i :

ð2Þ

j¼1

Here I(
) is the indicator function, xj ðtÞ – the components of the ECS state at time t. 3.3

Neuronal Types

We consider three types of neurons which represent different firing behavior properties: oscillatory, tonic and passive follower. Each neuron is represented as a finite automaton with two inputs. The activity at time t is described by the output function: yi ðtÞ ¼ FhðiÞ ðz1i ðt  1Þ; z0i ðtÞ; si ðt  1ÞÞ:

ð3Þ

Here h(i) is a type of neuron i, z1i(t − 1) is the excitation at the previous time, z0i(t) is the inhibition at time t and si(t − 1) is the internal state at the previous time. Endogenous oscillator. An endogenous oscillator produces bursts of spikes every Ti times if not inhibited by other neurons. If an oscillator is inhibited it will be active after the inhibition disappears. If an oscillatory neuron is excited then it will become active at the next time immediately. The internal structure and the output function of an oscillatory neuron are provided in the Table 1. Tonic neuron. Neurons of this type are active as long as they are not inhibited: yi ðtÞ ¼ :z0i ðtÞ:

ð4Þ

Discrete Modeling of Multi-transmitter Neural Networks

13

Table 1. State transitions and outputs of an oscillatory neuron State Inputs (z0, z1) (z0 = 0, z1 = 0) s0 s1, y = 0 ... ... sk sk+1, y = 0 ... ... sT s0, y = 1

(z0 = 0, z1 = 1) (z0 = 1, z1 2 {0,1}) s0 , y = 1 s1, y = 0 ... ... s0 , y = 1 sk+1, y = 0 ... ... s0 , y = 1 sT, y = 0

Follower neuron is active only after being excited by others so the output function takes the following form: yi ðtÞ ¼ :z0i ðtÞz1i ðt  1Þ:

ð5Þ

Post-inhibitory rebound (PIR). It is a gain coefficient that increases the output of a neuron that was inhibited at the previous time by the PIR gain coefficient kPIR  1. i

3.4

Neuronal Competition and Model Dynamics

The main principle of the model’s dynamics is a competition between neurons for the opportunity to be active during the next time step. The competition algorithm doesn’t mimic functioning of biological CPGs but provides a conflict resolution rule so the model can generate rhythms similar to those observed in biological circuits. Figure 1 shows an example from [15] where the competition plays a crucial. This algorithm determines which neurons will be active during the next time t.

14

N. Bazenkov et al.

4 Example: Snail Feeding Rhythm The feeding network of a pond snail Lymnaea stagnalis is a well-studied example of a CPG. As shown in the Fig. 1, the network consists of three interneurons responsible for separate phases of the feeding rhythm: protraction, rasp and swallow. A model proposed in [15] consists of 38-dimensional system of differential equations. Here we propose a discrete model emphasizing the logic of the interactions and neuronal roles in the CPG. The network consists of three neurons N1, N2, N3, each produces its own transmitter: ach, glu and xxx because the third transmitter in the CPG remains unknown. The neuronal properties are shown in the Table 2.

Fig. 1. A competition between oscillatory N1M and tonic N3t neurons in the feeding CPG [15]. The dominant neuron defines the current phase of the feeding cycle

Fig. 2. Feeding CPG model: (left) structure of neuronal interactions; (right) the produced rhythm: neuronal activity (up) and concentrations of neurotransmitters (bottom)

The produced rhythm is shown in Fig. 2. The default output of N3 is lower than that of N1 so the first phase is won by N1. Then N2 wins the competition because of its high output. After being inhibited N3 is able to win and drives the third phase of the rhythm.

Discrete Modeling of Multi-transmitter Neural Networks

15

Then the effect of PIR disappears and N1 wins the competition again. There are several combinations of model parameters that can produce the same rhythm. The question of how to choose the most efficient combination is left for further studies.

Table 2. Model of the feeding CPG Neuron

Type

PIR

Thresholds

Output, D

glu 1 0 0

xxx 0 0 0.5

Input, W P0i N1 N2 N3

P1i Oscillator Follower Tonic

ach 1 1 2

−1 −1 −1

1 -

0 2 0

ach 0 1 −1

glu −1 0 −1

xxx −1 0 0

5 Conclusion We propose a formalized model of a multi-transmitter neural network where neurons interact via shared extracellular space without synaptic connections. Each neuron receives signals from the rest of the network by an individual set of receptors to a subset of the neurotransmitters which are released by other neurons. The model is intended to be a proof-of-concept example that some functional patterns of neural activity can in principle be implemented without synaptic wiring. In the model, we consider three various types of neurons differing in their electrical activity: tonic neurons, oscillators and followers. Oscillators and tonic neurons generate endogenous activity unless they are inhibited by other neurons. An algorithm of neuronal competition is introduced to resolve conflicts between those neurons that inhibits each other and are not allowed to be simultaneously active. To illustrate the key features of the model we considered the well-known central pattern generator that is responsible for feeding behavior of a pond snail Lymnaea stagnalis. The model is able to produce rhythms similar to those observed experimentally and in continuous modeling and, despite its simplicity, proved to be capable of simulation and explanation of the phenomena taking place in living neural ensembles.

References 1. Bazenkov, N., Dyakonova, V., Kuznetsov, O., Sakharov, D., Vorontsov, D., Zhilyakova, L.: Discrete Modeling of Multi-Transmitter Neural Networks with Neuron Competition (2017). https://arxiv.org/abs/1705.02176 2. Bargmann, C.: Beyond the connectome: how neuromodulators shape neural circuits. BioEssays 34(6), 458–465 (2012) 3. Baronchelli, A., Ferrer-i-Cancho, R., Pastor-Satorras, R., Chater, N., Christiansen, M.: Networks in cognitive science. Trends Cogn. Sci. 17(7), 348–360 (2013) 4. Dyakonova, V.: Neurotransmitter mechanisms of context-dependent behavior. Zh. Vyssh. Nerv. Deyat. 62(6), 1–17 (2012)

16

N. Bazenkov et al.

5. Ghigliazza, R., Holmes, P.: A minimal model of a central pattern generator and motoneurons for insect locomotion. SIAM J. Appl. Dyn. Syst. 3(4), 671–700 (2004) 6. Haykin, S.: Neural Networks and Learning Machines, 3rd edn. Prentice-Hall, Upper Saddle River (2009) 7. Hodgkin, A.L., Huxley, A.F.: A quantitative description of membrane current and its applications to conduction and excitation in nerve. J. Physiol. 116, 500–544 (1952) 8. Kuznetsov, O.: Complex networks and activity spreading. Autom. Remote Control 76(12), 2091–2109 (2015) 9. Marder, E., Goeritz, M., Otopalik, A.: Robust circuit rhythms in small circuits arise from variable circuit components and mechanisms. Curr. Opin. Neurobiol. 31, 156–163 (2015) 10. McCulloch, W., Pitts, W.: A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 5, 115–133 (1943) 11. Moroz, L., Kohn, A.: Independent origins of neurons and synapses: insights from ctenophores. Phil. Trans. R. Soc. Lond. B. Biol. Sci. 371(1685), 1–14 (2016) 12. Roberts, P.: Classification of Temporal Patterns in Dynamic Biological Networks. Neural Comput. 10(7), 1831–1846 (1998) 13. Sakharov, D.: The multiplicity of neurotransmitters: the functional significance. Zh. Evol. Biokhim. Fiziol. 26(5), 733–741 (1990) 14. Sterratt, D., Graham, B., Gillies, A., Willshaw, D.S.: Principles of Computational Modelling in Neuroscience. Cambridge University Press, Cambridge (2011) 15. Vavoulis, D., Straub, V., Kemenes, I., Kemenes, G., Feng, J., Benjamin, P.: Dynamic control of a central pattern generator circuit: a computational model of the snail feeding network. Eur. J. Neurosci. 25, 2805–2818 (2007)

A Simple Virtual Actor Model Supporting Believable Character Reasoning in Virtual Environments Pavel A. Bortnikov1(&) and Alexei V. Samsonovich1,2 1 National Research Nuclear University “Moscow Engineering Physics Institute”, Kashirskoe Shosse 31, Moscow 115409, Russian Federation [email protected], [email protected] 2 George Mason University, 4400 University Drive 2A1, Fairfax, VA 22030, USA

Abstract. An artifact needs to possess a human-level social-emotional intelligence in order to be accepted as a team member and to be productive in the team. A general theoretical model describing this kind of artificial intelligence is currently missing. This work makes one step toward its development, using a simplistic virtual environment paradigm and an emotional biologically inspired cognitive architecture as the basis. We describe a Virtual Actor model supporting believable character reasoning. The model was implemented and tested in a pilot experiment in a virtual environment involving human participants. Preliminary results indicate that a Virtual Actor of this sort can be believable and socially acceptable in a small heterogeneous group. Keywords: Narrative intelligence
Social-emotional intelligence architectures
Believable virtual characters
BICA challenge


Cognitive

1 Introduction Future robots and virtual agents will work side by side with humans as team members and personal assistants. Their productive collaboration with humans is impossible without the ability to understand the developing narrative in which they participate, and to react adequately to human emotions, that arguably are expressed in every collaborative action. Principles of social and emotional interaction of actors of different nature (people, robots, virtual agents), based on the relationships of trust, leadership and mutual appraisal, are today in the focus of attention in artificial intelligence [1–5]. However, there is no generally accepted understanding of them based on one universal framework. Here we look for a generally applicable scheme of designing a model of actor behavior in social settings, specifically, in small groups. Our approach is cognitive-architecture-based [6]. A cognitive architecture is a framework or blueprint for designing intelligent agents. It integrates a complete set of individual cognitive capabilities in an intelligent agent embedded in a real or virtual environment, allowing © Springer International Publishing AG 2018 A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6_3

18

P.A. Bortnikov and A.V. Samsonovich

the agent to perform sensory perception, cognition, decision making, and action. Biologically inspired cognitive architectures (BICA) are based on principles borrowed from cognitive psychology or neuroscience. One of such architectures that possesses elements of emotional intelligence is Ebica [6]. Here we select the general framework of Ebica to design and test a simple model of a Virtual Actor, that we hope can be characterized by a human level of believability and social acceptability. These two expectations are tested experimentally in a simple virtual environment paradigm, for which the expectations are confirmed.

2 Virtual Actor Model Here we describe a general model based on the cognitive architecture Ebica [6] and the believable character reasoning (BCR) formalism [13], applied to a virtual environment game paradigm. The limited volume of this article does not allow us to explain the two frameworks in detail. Main points are the following. A Virtual Actor can select and change the Character it performs, depending on its experience. Here Character is an abstraction that specifies a rational agent with its own drives, motives, goals, feelings, relationships, capabilities, knowledge and a recent history [14, 15]. The basis for this concept comes from the field of narrative reasoning [16–18], including narrative planning [19]. Actions of actors in narrative planning are motivated. In our case, the most important attribute of Character is its motive. In addition, the formalism of BCR is based on a higher structure, called a hierarchical narrative network (HNN), which is defined as a tuple: HNN ¼ \S; E; C; A; P [ :

ð1Þ

Where S is the set of nodes representing situations, E is the set of directed edges representing events, C is the set of characters {c} involved in events and situations, A is the set of character arcs, and P is the set of actors and avatars associated with the characters. In the BCR formalism [13], each character c is represented by a tuple: c ¼ \p; m; A [ ;

ð2Þ

where p is the Character’s perspective, including the actor identity, the viewpoint, etc.; m is the set of motives, and A is the character arc [13]. In the present work, however, a modified version of the original BCR model is used, designed to describe social interactions of Virtual Actor with human participants. The main idea in this new version is that all actions of Virtual Actor must be plausibly motivated and understandable from a human-like perspective to participants interacting with Virtual Actor in the given class of paradigms. Therefore, Character in this model is defined as follows: C ¼ \M; R; G [ ;

ð3Þ

where C is Character, M is its Motive, R is the dynamically changing set of relationships between interacting actors (possibly including humans and Virtual Actors), and

A Simple Virtual Actor Model Supporting Believable Character

19

G is the set of Goals, which Virtual Actor is currently setting and pursuing, together with their priorities. These Goals depend on the environment, in which Virtual Actor is embedded, the current Motive guiding Virtual Actor, and its current Relationships with other actors. Motive can change when significant events occur during the interaction of actors; the list of these significant events is given below in Sect. 4.2.

3 The Teleport Paradigm The following virtual reality paradigm is a modification of The Russian Elevator Story [7]. Three players, previously unfamiliar with each other, are trapped on a collapsing platform. It has two teleports (circles on the floor in Fig. 1) that lead to a secure platform. You can get out of the platform to a safe tower by teleportation; however, your teleport needs to be activated by another player. After reaching a secure platform, a player can (but does not have to) save one of the remaining two players. When two are rescued, the platform collapses, “killing” the third player.

Fig. 1. Screenshot of the virtual environment interface (its earlier version is described in [12]). A, B, C are avatars, two of which are controlled by human participants, and one by Virtual Actor.

On the platform, actors can move freely, can express their own positive attitude to others by greeting each other, can kick each other, etc. Otherwise they do not communicate with each other. Each experimental session consists of a series of identical rounds, each of which begins with a starting situation and ends when two players reach the tower, or the time limit expires (in the latter case all loose). The score of each player (displayed in the bottom left corner of the screen, Fig. 1) is computed as the number of times the player reached the tower. Fixed letters A, B, C (Fig. 1) are assigned to players randomly at the beginning of each session and are kept fixed for the players throughout the session, while the players remain anonymous. This paradigm was implemented using a virtual reality platform. The platform, the interface (Fig. 1) and the paradigm were implemented by Mr. Arthur Chubarov [12].

20

P.A. Bortnikov and A.V. Samsonovich

4 Virtual Actor Embedded in the Teleport Paradigm 4.1

General Description of the Model and Its Parameters

Here and below, instead of referring to a Character, we refer to the associated with it Motive. Motive in the present context is the key element of the model determining the behavior of Virtual Actor, including the order and priority of its goals and the actions it performs. Three basic Motives were selected for behavior control in this paradigm. They are named as follows: Friendship, Altruism, and Success (Table 1). Table 1. Appraisals of actions. Kick Greet Distrust −10 10 Neutral −25 10 Trust −45 5

NotSave Save −30 130 −80 100 −120 60

The rules for Motive changing, as well as rules for selection of goals and actions to perform, are based on current relationships between Virtual Actor and other players. The main criterion for decision making is the variable named Trust, introduced as a parameter characterizing separately each pair of players. The value of Trust is measured as the appraisal A of one player by another based on the history of all past actions committed by them with respect to each other, and the phase of trust f, calculated as a function of Trust. The appraisal A is calculated using the following formula, which is a modified version of its analog in [6] (we dropped the imaginary part): Aðt þ 1Þ ¼ ð1  r Þ  AðtÞ þ r  a:actionðf ðTrustÞÞ;

ð4Þ

where f (Trust) is a variable, the value of which is the current phase of trust. The range includes the following values: “Distrust”, “Neutral” or “Trust”. Then, a.action (f (Trust)) is the appraisal of the action, which depends on the phase of trust. Again, Trust here is a variable, which measures the amount of trust to the player at the current time (Table 2). Table 2. Thresholds for transitions between phases of trust. Phase before transition Phase after transition Threshold Distrust Neutral −200 Neutral Distrust −250 Neutral Trust 180 Trust Distrust −50

4.2

Algorithms and Rules of Dynamics

Depending on the current Motives and values of Trust, the list and priority of current goals that the Virtual Actor sets for itself are determined. Most relevant goals in this

A Simple Virtual Actor Model Supporting Believable Character

21

paradigm are: establish mutual trust, maintain partnership, hinder a particular actor’s escape, protect and save a particular actor (possibly self). Selection of goals and corresponding actions depend on the current motive of the actor. We consider three cases (Table 3). Table 3. Algorithm of action selection. Function Update (action, agent, target, position[]) Input: action, agent, target, position[]; Output: action, agent, target, position[]; Switch (Motive) Case (Friendship): If HavePartner == true then If Save (actor)== true then Save (partner) elif Greet (actor)== true then Greet (partner) elif NearActor == notPartner then Kick (notPartner) elif position (actor) == position (teleport) & position (partner) == position (otherTeleport) then Activate (teleport) else MoveTo (teleport) If StageTrustToPlayer1 == Trust & TrustToPlayer1 > TrustToPlayer2 then Partner = Player1 HavePartner = true else If Save (actor)== true then Save (agent) elif Greet (actor) == true then Greet (agent) elif position (actor) == position (teleport) then Activate (teleport) else MoveTo(teleport) If StageTrustToPlayer1 == Trust & TrustToPlayer1 > TrustToPlayer2 then Partner = Player1 HavePartner = true If StageTrustToPlayer2 == Trust & TrustToPlayer2 > TrustToPlayer1 then Partner = Player2 HavePartner = true Case (Altruism): If Save(actor) == true then Save(agent) elif Greet (actor) == true then Greet (agent) elif position (actor) == position (teleport) then activate (teleport) else MoveTo (teleport) Case (Success): If Save (actor) == true then Save (agent) elif NearActor == player then Kick (player) elif Greet (actor) == true then Greet (agent) elif position (actor) == position (teleport) & position (partner) == position (otherTeleport) then Activate (teleport) else MoveTo (teleport)

Motive “Friendship”. When the character is driven by this Motive, the basic goal is to find a reliable partner and maintain relations with him during the whole session. Accordingly, the choice of actions depends on whether the trusted partner has been already selected or not.

22

P.A. Bortnikov and A.V. Samsonovich

If the partner is not selected, the dominating goal is to establish trust with any player, which forces the actor to avoid aggressive actions toward others. The partner of choice will be the most successful and the most generous one, who is disposed to cooperate with the Virtual Actor. After establishing a partnership with one of the players, the list of goals and behaviors changes: now the key factor is the partner’s rescue and protection of teleports from the third player. If, however, the selected partner commits negative actions with respect to Virtual Actor, then his status must change. This is achieved again through a change of Motive, as described by the Algorithm in Table 4. Then attempts are made to establish mutual trust relations with the third player.

Table 4. Algorithm of motive change. Function ChangeMotive (Motive, Trust[]) Input: Motive, Trust[]; Output: Motive; Switch (Motive) Case (Friendship): If StageTrustToPlayer1 == Trust & StageTrustToPlayer2 == Trust & PlayersPartners == true then Motive = Altruism If StageTrustToPlayer1 == Disrust & StageTrustToPlayer2 == Disrust then Motive = Success Case (Altruism): If BetrayPlayer == true & StageTrustToBetrayed == Trust then Motive = Friendship If StageTrustToPlayer1 == Disrust & StageTrustToPlayer2 == Disrust then Motive = Success Case (Success): If StageTrustToPlayer1 == Trust & StageTrustToPlayer2 == Trust & PlayersPartners == true then Motive = Altruism If BetrayPlayer == true & StageTrustToBetrayed == Trust then Motive = Friendship

Motive “Altruism”. For a character driven by this Motive, the goal is to save both players, even at the cost of self-sacrifice. Behavior is characterized by the absence of negative actions towards other players and by the priority of saving others. Motive “Success”. For a character driven by this motive, the goal is to win by any possible means. The key aspects of behavior are the priority of own rescue, and as a means only, helping others to establish trust relationships with them, if this does not contradict the first goal.

A Simple Virtual Actor Model Supporting Believable Character

23

Motive Change As discussed in Sect. 2, motive alteration is possible for Virtual Actor. Motive changes are triggered by key events, according to the rules specified by the algorithm of Table 4 below. The list of key events includes the following. 1. 2. 3. 4. 5.

“Betrayal” of the partner. Achieving a high level of trust with an actor. Establishment of positive relations with a distrusted actor. Establishment of negative relations with a trusted actor. Establishment of a partnership relation.

5 Experimental Testing 5.1

Settings, Participants and Procedures

This Virtual Actor model was implemented as an embedded agent in the virtual reality platform created by Mr. Arthur Chubarov [12] and tested in a series of experiments involving human participants (further “the subjects”). Testing was conducted during the training semester during a series of experiments. All subjects were Undergraduate Students of MEPhI, residing in Moscow. Among the subjects were both women and men. The age of the subjects was within 22 to 28. The experiment was conducted using four identical HP computers running Windows 10, all connected to the broadband Internet. To prevent subjects from seeing each other, partitions were installed between seats. During the experiment, subjects used individual headphones, in which music played during the experiment. Thus, participants could not see or hear each other’s movements. This was done to make participation anonymous. In the virtual environment, identical avatars were used labeled by letters (Fig. 1). A letter attached to a particular participant or the Virtual Actor was randomly generated in each session, while remained fixed during the session. At the end of each experiment, subjects completed a survey, answering, e.g., which avatar they believe was controlled by an automaton.

5.2

Results and Discussion

The following outcomes were recorded in this experiment. The total number of times when any actor (a human participant or the Virtual Actor) was rescued was 189, of which in 118 cases the savior was a human. Among those 118 cases, Virtual Actor was rescued 64 times. Therefore, the confidence interval for the probability that a human selects to save a virtual actor, as opposed to another human, is (0.44, 0.64). In other words, there is no statistical significance in the difference between a human and a virtual actor. Results are summarized in Fig. 2. Similarly, there is no significant difference in the frequency of correct and wrong guesses by human participants as to which of the two avatars was controlled by an automaton. In the survey, participants answered the question “Among the two other avatars, who was the automaton?” Among all available answers, the number of correct guesses was 11, while the number of wrong guesses was 5. On the basis of these data,

24

P.A. Bortnikov and A.V. Samsonovich

Fig. 2. Comparison of Virtual Actor to a human in the Teleport paradigm. Turing guesses: The confidence interval and the estimated probability of a correct guess by a participant, answering the question, which of the two avatars was controlled by Virtual Actor. Chance to be rescued: The confidence interval and the estimated probability of Virtual Actor to be selected for rescue by a human participant, as opposed to selecting another human participant. The dashed line represents the chance level. The null hypotheses cannot be rejected, which is a good outcome.

we calculate the estimated probability of a correct guess as 0.69, while the confidence interval at the significance level of 5% is (0.41, 0.89): see Fig. 2. Therefore, based on this pilot study, we found no significant difference between a human and Virtual Actor, both in terms of believability (measured by the frequency of correct guesses) and social acceptability, measured by the relative frequency of rescues. Indeed, the frequency of rescues measures social acceptability, because the only sensible reason for the rescuer to select an avatar to be rescued is when the rescuer attributes human-like qualities to the actor controlling that avatar.

6 Conclusions In this study, we used a simplistic virtual environment paradigm and an emotional biologically inspired cognitive architecture Ebica as a basis for design and implementation of a Virtual Actor that was expected to be believable and socially acceptable in the selected paradigm. Preliminary results indicate that the implemented Virtual Actor is not significantly different from a human participant in believability as well as in social acceptability. In fact the believability test that we conducted can be considered as a limited version of a Turing test [20]. Despite the criticism of the Turing test (e.g., [21]) of which we are aware, yet cannot discuss it here, so far there is no more simple, universal and reliable criterion of a proximity of artificial and natural intelligence.

A Simple Virtual Actor Model Supporting Believable Character

25

The second part of the outcome, the social acceptability measure (how frequently participants selected Virtual Actor for rescue, as opposed to selecting a human, without knowing who is who), is even more impressive, because it suggests that an Ebica-based Virtual Actor can deserve no less sympathy and trust than a human in this paradigm. Our future steps will include generalization and expansion of the model to other paradigm. The hope is that results are transferrable across paradigms, and a Virtual Actor proven believable and socially acceptable in one paradigm will have similar characteristics in other paradigms. Our primary interest is in practically useful applications, such as creative virtual assistants for digital artists [22]. Acknowledgments. We are grateful to NRNU MEPhI Students who contributed to this study as participants or developers, in particular, to Mr. Arthur Chubarov for creating the virtual environment platform used in experiments, and to Mr. Daniil Azarnov for valuable discussions. This work was supported by the RSF Grant # 15-11-30014.

References 1. Meyer, J.-J.C., van der Hoek, W., van Linder, B.: A logical approach to the dynamics of commitments. Artif. Intell. 113, 1–40 (1999) 2. Meyer, J.-J.C.: Reasoning about emotional agents. In: Proceedings of ECAI 2004, pp. 129–133. IOS Press (2004) 3. Gratch, J., Marsella, S.: A domain-independent framework for modeling emotions. J. Cogn. Syst. Res. 5(4), 269–306 (2004) 4. Steunebrink, B.R., Dastani, M., Meyer, J.-J.C.: A logic of emotions for intelligent agents. In: Proceedings of AAAI 2007. AAAI Press, Menlo Park (2007) 5. Jung, Y., Kuijper, A., Fellner, D.W., Kipp, M., Miksatko, J., Gratch, J., Thalmann, D.: Believable virtual characters in human-computer dialogs. Eurographics (STARs) 2011, 75–100 (2011) 6. Samsonovich, A.V.: Emotional biologically inspired cognitive architecture. Biol. Inspired Cogn. Archit. 6, 109–125 (2013) 7. Samsonovich, A.V., Tolstikhina, A., Bortnikov, P.A.: A test for believable social emotionality in Virtual Actors. Procedia Comput. Sci. 88, 450–458 (2016) 8. Heider, F., Simmel, M.: An experimental study of apparent behavior. Am. J. Psychol. 57, 243–259 (1944). doi:10.2307/1416950 9. Ortony, A., Clore, G.L., Collins, A.: The Cognitive Structure of Emotions. Cambridge University Press, Cambridge (1988) 10. Picard, R.W.: Affective Computing. MIT Press, Cambridge (1997) 11. Sloman, A.: Beyond shallow models of emotion. Cogn. Process. 2(1), 177–198 (2001) 12. Chubarov, A., Azarnov, D.: Modeling behavior of virtual actors: a limited turing test for social-emotional intelligence. In: Samsonovich, A.V. and Klimov, V.V. (eds.) Biologically Inspired Cognitive Architectures for Young Scientists. Advances in Intelligent Systems and Computing, pp. 32–39. Springer, Berlin (2017) 13. Samsonovich, A.V., Aha, D.W.: Character-oriented narrative goal reasoning in autonomous actors. In: Aha, D.W. (ed.) Goal Reasoning: Papers from the ACS Workshop. Technical Report GT-IRIM-CR-2015-001, pp. 166–181 (2015). https://smartech.gatech.edu/bitstream/ handle/1853/53646/Technical%20Report%20GT-IRIM-CR-2015-001.pdf

26

P.A. Bortnikov and A.V. Samsonovich

14. Haven, K.: Story Proof: The Science Behind the Startling Power of Story. Libraries Unlimited, Westport (2007). ISBN 978-1-59158-546-6 15. Haven, K.: Story Smart: Using the Science of Story to Persuade, Influence, Inspire, and Teach. ABC-CLIO, LLC, Santa Barbara (2014). ISBN 9781610698115 16. Abell, P.: A Case for cases: comparative narratives in sociological explanation. Sociol. Methods Res. 38(1), 38–70 (2009) 17. Schmid, W.: Narratology: An Introduction. Walter de Gruyter GmbH & Co. KG, Berlin/New York (2010). ISBN 978-3-11-022631-7 18. Finlayson, M.A., Corman, S.R.: The military interest in narrative. Sprache und Datenverarbeitung (Int. J. Lang. Data Process.) 37(1–2), 173–191 (2013) 19. Riedl, M.O., Young, R.M.: Narrative planning: Balancing plot and character. J. Artif. Intell. Res. 39, 217–268 (2010) 20. Turing, A.M.: Computing machinery and intelligence. Mind LIX 59, 433–460 (1950) 21. Korukonda, A.R.: Taking stock of Turing test: a review, analysis, and appraisal of issues surrounding thinking machines. Int. J. Hum Comput Stud. 58, 240–257 (2003) 22. Eidlin, A.A., Samsonovich, A.V.: A roadmap to emotionally intelligent creative virtual assistants. In: Samsonovich, A.V., Klimov, V.V.: Biologically Inspired Cognitive Architectures for Young Scientists. Advances in Intelligent Systems and Computing, pp. 46–56. Springer, Berlin (2017)

Intelligent Search System for Huge Non-structured Data Storages with Domain-Based Natural Language Interface Artyom Chernyshov(&), Anita Balandina, Anastasiya Kostkina, and Valentin Klimov National Research Nuclear University “MEPhI”, Moscow, Russian Federation [email protected], [email protected], [email protected], [email protected]

Abstract. Nowadays the number of huge companies and corporations has in their disposition various non-structured texts, documents and other data. The absence of clearly defined structure of the data makes the implementation of searching queries complicated and even impossible depending on the storage size. The other problem connected with staff, which may face the problem with misunderstanding of the special query languages, knowledge of which is necessary for the execution of searching queries. To solve these problems, we propose the semantic search system, the possibilities of which include the searching index construction, for queries execution and the semantic map, which would help to clarify the queries. In this paper we are going to describe our algorithms and the architecture of the system, and also to give a comparison to analogues. Keywords: Semantic search
Semantic map language
Domain-based natural languages


Non-structured data
Natural

1 Introduction Natural Language Understanding (NLU) is a set of tasks considered from the point of view of the semantic knowledge, which occurs in the natural language. NLU is one of the main problems in the area of natural language processing (NLP). NLP is a scientific direction in the area of artificial intelligence, which deals with the automated processing and understanding of the natural language. It is unofficially considered that the task of NLU AI is a complete task, which means that the complexity of solving this problem is as high as the solution of other central problems of artificial intelligence or how the creation of computers capable of thinking like a human. The goals of the NLU are still far from reaching. One of the main tasks in modern NLP systems is the consideration of features and the ambiguous nature of the natural language. Despite this, recent studies have made significant progress in solving

© Springer International Publishing AG 2018 A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6_4

28

A. Chernyshov et al.

important NLU subtasks, such as syntactic analysis of dependencies and weak semantic analysis. A wide range of applications, such as question-answer type systems, automatic abstraction or retrieval of information, can potentially benefit from the NLU study. In this paper, we propose the usage of syntactic and semantic analysis methods for natural language texts to construct a query structure that will be translated directly into query languages of specific database management system. Generally, the question-answering system (QA-systems) may be divided into two parts: the closed-domain systems and open-domain systems. The first ones give answers into a specific domain (for example, nuclear engineering), and can be considered as an easier task because NLP systems can exploit domain-specific knowledge frequently formalized in ontologies. Alternatively, closed-domain might refer to a situation where only a limited type of questions is accepted. The other ones work with questions about almost anything, and can only rely on general ontologies and world knowledge. On the other hand, these systems usually have much more data available from which to extract the answer. But on the other hand the size of general ontology may negatively affect the accuracy of the answer. In this case we can only talk about closed-domain QA-system, because the goal of our system is giving answers in a specific domain as relevant as possible. However, the domain may vary depending on the customer needs by modification of one of the components of the system. In this way the NLU task may be simplified, as we only work with domain-based natural language. Further, there will be explained the methods of the construction and rectification of searching queries which we are going to use in our system.

2 Natural Language Understanding and Processing For the representation of the internal structure of the input request it is proposed to use the syntactical dependencies tree. In this case we use the part-of speech tags or POS-tags to mark the words of input sentence. The main word, normally verb, which is also called predicate is mark as the root of the tree. The other words of the input request are connected to the root with POS-tags. In this case, we can consider the nodes of the tree define the constraints of the query, and the leafs are parameters of the query which specify the certain conditions. Formally, the input request model can be represented as follows: I ¼ \W; C [ ; where

• W ¼ fw1 ; . . .; wn g – is the set of words of the input request; • C ¼ fc1 ; . . .; cn1 g – is the set of links between the words of the input query.

ð1Þ

Intelligent Search System for Huge Non-structured Data Storages

29

In turn, the sets W and C can be represented as follows: wi ¼ \nf ; pos; gr cat [ ; where

ð2Þ

• Nf - normal form (infinitive for verbs, singular number of nominative case for nouns, etc.); • Pos – part-of-speech tag (subj, dobj, etc.); • Gram_category - grammatical categories or parameters (gender, number, case for nouns, form and time for verbs, etc.).

3 Rectification of Queries with Semantic Mapping Rectification of Queries (RoQ) is the process of reformulating a seed query to improve retrieval performance in information retrieval operations [1]. Rectification of queries involves techniques such as: (i) Finding synonyms of words, and searching for the synonyms as well; (ii) Finding all the various morphological forms of words by stemming each word in the search query; (iii) Fixing spelling errors and automatically searching for the corrected form or suggesting it in the results; (iv) Re-weighting the terms in the original query [2]. The most part of the existing approaches, algorithms, techniques for rectification of queries are based on using WordNet. WordNet has a database that groups the words into sets of synonyms called synsets and provides definitions, comments, examples of usage of these words, and the actual meaning in each case. Therefore, it combines the elements of a dictionary (definitions and some examples) with those of a thesaurus (synonyms), resulting in an important support for the automatic analysis of text and words. However, in Voorhees [3] the use of WordNet for performing rectification of queries did not actually increase the effectiveness of the information retrieval process without specific expansions and limitations. The numerous attempts to improve approach of thesaurus-based query expansion are mentioned in [4, 5]. In [5] the improvement is the concept of the vector space model under WordNet and the computation of similarity of documents as terms. It is considered that the vector space model gives the significant improvements in the case if we have a deal with analysis of documents. Here we should introduce the concept of a semantic map. In general, a semantic map can be defined as a topological or metric space, the topology and/or the geometry of which reflect semantic characteristics and relations among a set of representations (such as words or word senses) embedded in this space [6, 7]. In the present time there are two types of semantic map. The first type named “strong semantic map” is based on the vector space model and the concept of dissimilarity (LSA, LDA) and we’ve mentioned it above. The second type is “weak semantic cognitive mapping” and consist in using such notion as “opposite relations”. Here it doesn’t take into account individual semantic characteristics of representations

30

A. Chernyshov et al.

given a priori. Only relations, but not semantic features, are given as input. As a result, semantic dimensions of the map that are not predefined to emerge naturally, starting from a randomly generated initial distribution of words in an abstract space with no a priori given semantics and following [7]. The main relation that authors use in this approach is the relation “synonymy-antonymy” for representations. The most known approach of this type is Antomap developed by A. Samsonovich. The disadvantages of the mentioned approaches based on “strong semantic map” are that they are not applied for some goals, for example, if it’s necessary to work with the narrow domain areas and graph databases, not documents. Our approach implements the synonym-based query expansion with the help of weak semantic map Antomap, which uses Microsoft Russian Thesaurus Core as a part of WordNet. The ontology that will consist of concepts defined by the domain area. The Microsoft Russian Thesaurus Core (MRTC), obtained in [7, 8], represents a dictionary cluster of the Microsoft Russian Thesaurus with the major number of connections between words. There are no contradictory connections and duplicates inside it. In fact, the MRTC represents a cluster of the “strongest” relations of a synonimy and an antonymy. It contains the words which have not less than 2 and no more than 11 synonyms. The Antomap is a superstructure over MRTC that allocates words with their connections on the map. The formal definition of Antomap consists in considering of that cognitive semantic map is represented as the dynamic system which is formed by N points in the space or vectors xi 2 ℜ or, in other words, is a distribution of words in an abstract vector space (with no semantics preassociated with its elements or dimensions) that minimizes the following energy function [7, 8]: HðXÞ ¼ 

1 XN 1 XN Wij xi
xj þ kxi k4 ; where i;j¼1 i¼1 2 4

ð3Þ

xi is a 26-dimensional vector that is representing the ith word (out of N). The Wij entries of the symmetric relation matrix equal +1 for pairs of synonyms, –1 for pairs of antonyms, and zero otherwise. The energy function (1) follows the principle of parsimony: it is the simplest analytical expression that creates balanced forces of desired signs between synonyms and antonyms, preserves symmetries of semantic relations, and increases indefinitely at the infinity, keeping the resultant distribution localized near the origin of coordinates. More details of this approach are described in [7, 8]. The structure of the current map allows to compute a semantic similarity (SS) between words. The computation of SS is connected with the “the contextual quality” of the chosen synonyms and helps to dispose from synonyms which aren’t close in the context with the original word. The rest of synonyms expands the query by forming new and similar queries. As for necessary components for SS, the main semantic dimensions of this map are used for computation of it and defined by the principal components of the emergent distribution of words on the map. For example, semantics associated with the first three PCs can be characterized as “good” versus “bad” (PC1), “calming, easy” versus “exciting, hard” (PC2), and “free, open” versus “dominated, closed” (PC3) [7, 8]. These and next three semantic PCs are the most important for

Intelligent Search System for Huge Non-structured Data Storages

31

finding of “the strongest” synonyms for one or another word, despite the common number to equal 26 PCs. The basis of our approach of computation of a semantic similarity is Weighted Euclidean distance in six dimensional space. The weight is the relation 1 to the squared standard deviation of every PC. The common formula is (n = 6): simðx; yÞ ¼

Xn

w ðx  yi Þ2 i¼1 i i

12

ð4Þ

In the present time the algorithm implies usage of the Microsoft Russian Thesaurus for finding all synonyms and only after it the list of synonyms is filtered by the Antomap. The algorithm is as follows: Step 1. The input is the Token (word) X. The synonyms in the thesaurus are searched until the end is reached. In the process of searching all existing synonyms of the current word are extracted and entered in the list of synonyms. The size of the list is L, i.e. the list consists of L-synonyms. Step 2. On the semantic map the Token and its synonyms based on the MTRC are randomly allocated. The Token and its synonyms represent 26-dimensional vectors in space. The first six coordinates are selected. The list of pairs is formed (word, vector coordinates). Step 3. In turn a semantic similarity of the Token is calculated with each of the synonyms for each six components. The value of the semantic similarity is obtained with the common formula represented above. The negative values and zero values are not taken into account here, and the lower threshold of semantic similarity is set, i.e. sim (x, y)  0.5. The upper threshold is set to 1. If the semantic similarity of the Token with a synonym lies in the interval (0, 0.5), i.e. tt is less than the defined value for lower threshold, then the current synonym is “put aside”. Step 4. A list of all pairs (synonym, semantic similarity) sorted by descending semantic similarity is formed.   ci ¼ \ wi ; wj [ ; where ð5Þ wi ; wj - pairs of words related to each other by an oriented relationship, the beginning of which is in the first word, and the end in the second. This method of representing the input request is extremely convenient, since the structure stores information about each word of the sentence. Thus, a good opportunity to find synonyms will be provided, as well as a format for issuing the results of the query.

4 System Description The authors propose to use the described above algorithm in project related to semantic search over huge unstructured documents and other data.

32

A. Chernyshov et al.

Intelligent search engines are the natural continuation of the development of conventional search engines. In comparison with their predecessors, their capabilities can include many different features and functions, such as: • It is necessary to obtain documents in some way associated with a specific considered document. • Search for documents, which exact attributes are unknown, but can be formulated a query in natural language that can characterize them. • It is necessary to compare two or more documents by their meaning. Authors propose the developing an intelligent search system with an interface in a domain-specific natural language. Such an interface will help to get rid of the need to use special query languages like SQL. The user will be able to formulate a search query in natural language using vocabulary adopted in a particular subject area (for example, using terms from nuclear physics). A distinctive feature of the developed system is its flexibility and ability to work in different subject areas, using only the vocabulary and terminology that is adopted in a particular area (or areas) of the customer. Flexibility and ability to work in different subject areas will be achieved through the usage of semantic maps - a tool for comparing commonly accepted terminology and vocabulary adopted in a specific subject area, which in turn is built by using the synonymy relationship on which the semantic map itself is based. The algorithms for processing natural language algorithms are based on the methods of constructing the syntactic trees of dependencies, by which constructs query structure, that is suitable for further translations into specific program queries. The developed system will be designed to facilitate the search process in subject areas with extensive databases, or large volumes of unstructured documents, and has no limitations in the field of application. The system will not depend on the language, so it will be enough to simply enter the international market. To introduce a prototype system into a specific area of activity, certain semantic maps will be created that take into account the specific vocabulary of this particular area. The need to modify one single component for the full-fledged operation of the system in any subject area makes the developed complex flexible and multipurpose.

5 Conclusion In the area of semantic search at the moment there are a lot of gaps and unsolved problems, such as highlighting the meaning of words and sentences, searching in different subject areas, communicating with the user in a language that is close to natural (using specialized vocabulary). These tasks can be considered practically solved for certain industries and spheres of activity. For example, in the field of medicine, there are lots of powerful tools and knowledge bases that allow you to receive answers to various questions based on knowledge stored in the database. An example is the IBM Watson supercomputer, which works well in the field of medicine, but it is rather weakly applicable in other subject areas.

Intelligent Search System for Huge Non-structured Data Storages

33

Our solution is to use a combination of such technologies as machine learning, semantic mapping and ontologies. One of the important tasks is to improve the accuracy of query processing, in which a term-specific terminology is used. For the processing of such terminology and the synonyms highlighting both at the level of words and at the level of whole phrases it is proposed to use semantic maps. This technology has received some distribution abroad, but in Russia the work on this topic began relatively recently and is conducted at a rather slow pace, nevertheless, there are already exist some algorithms that allow building semantic maps for the Russian language. Within this project, the algorithm will be finalized and adapted for working in the area of technical documentation. Described queries rectification algorithm with semantic mapping shows practical application of semantic maps. That will allow to increase the theoretical significance and show practical importance of semantic maps. Acknowledgements. This work was supported by Competitiveness Growth Program of the Federal Autonomous Educational Institution of Higher Professional Education National Research Nuclear University MEPhI (Moscow Engineering Physics Institute). The funding for this research was provided by the Russian Science Foundation, Grant RSF 15-11-30014.

References 1. Abberley, D., Kirby, D., Renals, S., Robinson, T.: The THISL broadcast news retrieval system. In: Proceedings ESCA ETRW Workshop Accessing Information in Spoken Audio, Cambridge, pp. 14–19 (1999) 2. Leung, C.H.C., et al.: Collective evolutionary concept distance based query expansion for effective web document retrieval. In: Proceedings of the 13th International Conference on Computational Science and Its Applications (ICCSA-2013), LNCS, pp. 657–672 (2013) 3. Voorhees, E.M.: Query expansion using lexical-semantic relations. In: Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 61–69 (1994) 4. Navigli, R., Velardi, P.: An analysis of ontology-based query expansion strategies. In: Workshop on Adaptive Text Extraction and Mining, held in conjunction with ECML 2003, Cavtat Dubrovnik, Croatia, 22 September 2003 5. Pinto, F.J., et al.: Joining automatic query expansion based on thesaurus and word sense disambiguation using WordNet. Int. J. Comput. Appl. Technol. 33, 271–279 (2009) 6. Klimov, V., Chernyshov, A., Balandina, A., Kostkina, A.: A new approach for semantic cognitive maps creation and evaluation based on affix relations. In: FIERCES on BICA, pp. 99–105 (2016) 7. Samsonovich, A., Ascoli, G.: Augmenting weak semantic cognitive maps with an ‘‘Abstractness’’ dimension. Hindawi Publishing Corporation Computational Intelligence and Neuroscience (2013). 10 p. 8. Samsonovich, A.V., Ascoli, G.A.: Principal semantic components of language and the measurement of meaning. PLoS One 5(6), 1–17 (2010)

Modeling Behavior of Virtual Actors: A Limited Turing Test for Social-Emotional Intelligence Arthur Chubarov ✉ and Daniil Azarnov ✉ (

)

(

)

National Research Nuclear University “Moscow Engineering Physics Institute”, Kashirskoe Shosse 31, Moscow 115409, Russian Federation [email protected], [email protected]

Abstract. This work presents the design, implementation and study of (1) a videogame-like virtual environment simulator, enabling social interaction of avatars controlled by human participants and by virtual actors; (2) a set of virtual actors with varying forms and degree of social-emotional intelligence, based on the eBICA cognitive architecture; and (3) a limited Turing test for socialemotional intelligence, involving human participants and virtual actors. The virtual envi‐ ronment simulator allows for various forms of emotionally-laden interaction of actors immersed in it in the form of avatars, with data collection characterizing their behavior in detail. The objective here is to compare and evaluate models of social-emotional reasoning based on the Turing test results and other objective behavioral measures, also taking into account subjective judgment of participants. One of the long-term goals is achieving human-level believability of sociallyemotional virtual actors, such as non-player characters in games, personal assis‐ tants, robots, and other intelligent artifacts. Preliminary results indicate impor‐ tance of social-emotional intelligence for believability, and support assumptions of the eBICA architecture. Keywords: Cognitive modeling · Virtual actor · Virtual environment

1

Introduction

Over the past decade, a lot of progress has been made in creation of complex virtual environments. Among the outstanding representative examples are a variety of three dimensional computer games. With each step of the development of technologies to create such environments the developers encounter new challenges to create believable virtual actors controlled by complex systems of artificial intelligence. Since the modern re-emergence of ideas about thinking machine, the main task in the field of artificial intelligence can be considered as the creation of a general purpose reasoning mind to be somewhat similar to that of humans. Modern researchers are developing more and more new ways to achieve this objective. Very important here are useful tests and evaluation criteria, that may allow one to refute a theory that is not worthy of the efforts of developers.

© Springer International Publishing AG 2018 A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6_5

Modeling Behavior of Virtual Actors

35

One of the main human cognitive capacities is emotionality and emotional intelli‐ gence, particularly, emotional reasoning, decision making and behavior control. Human emotions present a vast area for scientific study, including experimental and theoretical challenges. Recently, many studies were aimed to develop an understanding of basic relationships and patterns characterizing human emotions. At the same time, the main practical interest originates from the need for so-called “simulation” of emotions, including their internal processing in artifacts. Integration of the variety of disparate theoretical and modeling approaches to under‐ standing the nature of human emotional intelligence becomes a key challenge in artificial intelligence. A grand-unifying theory of emotional cognition seems to be around the corner. Creating intelligent agents that are capable of effectively and efficiently functioning in a real social environment, including learning and adapting to it, is a task that would be impossible to solve without understanding and using mechanisms of social emotional cognition. Therefore, the present work addresses such mechanisms and principles of their realization in an agent in the context of a social emotional challenge in a virtual environment, as presented below.

2

Materials and Methods

2.1 Virtual Environment Simulator The starting point in designing a virtual actor is always related to selection or creation of an appropriate virtual environment. On the basis of this preliminary analysis, we concluded that the most practical way to address the problem of cognitive modeling is to develop our own virtual environment, where we would have full control of the environment, and the behavior of the various entities in it, including virtual actors. (1) Settings and Procedure In this study, we decided to develop an application using Unity engine for the cogni‐ tive modelling study in the form of a virtual environment, enabling interactions of virtual and human actors with each other in one developing scenario. The experimental setup consists of a virtual stage, in which three avatars are allocated (Fig. 1). Each avatar can be controlled by a human player or a virtual actor. The virtual stage consists of two parts: a rescue zone (further referred to as “the tower”: the left component in Fig. 1) and an action zone (further referred to as “the platform”: the right component in Fig. 1). The platform has two teleported termi‐ nals (Fig. 1). Actors can be moved from the platform to the tower by means of tele‐ portation, as described below. The experiment is conducted as a sequence of logi‐ cally identical epochs, or rounds. Each round has a fixed limited duration, and may terminate earlier, if certain conditions are met. Following the termination, a new round starts automatically. Initially, all actors are placed on the platform at random locations. Each actor located on the platform has the following available behaviors.

36

A. Chubarov and D. Azarnov

Fig. 1. Screenshot of the working virtual environment with three actors in it (avatars A, B, C).

• Greet another actor (always available). Greeting is accompanied by a voice message and a predefined move of the avatar, turning toward the target of greeting. • Kick another actor (available at a short distance; also accompanied by a sound). The kicked actor flies a large distance in the direction of the kick. • Activate another teleporter (available from a teleporter only; cannot activate own teleporter). This action puts the other teleport in the active state, enabling teleporta‐ tion. • De-activate another teleporter (available from a teleporter only; cannot deactivate own teleporter). This action puts the other teleport in the inactive state, disabling teleportation. • Take off: make your own teleportation (available from an active teleport only). Own teleportation is possible from an active teleporter only. This action moves the actor from the platform to the tower. In addition, an actor located on the tower may perform the following two actions, each of which leads to the termination of the round: • Save a selected actor located on the platform. This action transfers the selected actor from the platform to the tower. • Escape alone. This action allows the only player located on the tower to complete the round. The round terminates automatically whenever two actors reach the tower, or an actor on the tower escapes alone, or the time limit expires. Upon termination, all actors located on the tower win, others lose.

Modeling Behavior of Virtual Actors

37

2.2 Experimental Paradigm The experimental paradigm used here is based on the random interaction paradigm described in and on the paradigm of the “Russian elevator story”. The scenario used here is essentially a modified “Russian elevator story” scenario, however, in the present work, the action takes place in space, on the platform, as described above in Sect. 2. The aim of this study is to compare two approaches in modeling of virtual actors, as well as to find patterns in emergent social-emotional relationships among the actors, understood in our case as mutual appraisals of the actors. These appraisals are evaluated using the collection of actors’ interaction data in the virtual environment. Therefore, the definition of appraisal is also given by the cognitive architecture and its parameters, used to implement virtual actors. Thus, the general paradigm is based on a group of three agents interacting with each other, using 6 possible actions. The interacting actors and their actions are appraised by the virtual agents identically, because they all receive identical information and use identical internal cognitive models. 2.3 Data Acquisition and Analysis The profiling server was used to store all the necessary information about behavior of actors. In the experiment, actors can perform actions of various nature. To obtain reliable results, one needs to use large volumes of information. In our experiment, data recording occurs in real time. The server is used primarily for centralized data collection with different types of devices that allows one to receive large amounts of data for statistical processing, without the need for time consuming operations to retrieve data from each of the user devices. This method does not introduce any delay in execution of the application. In the future, on the basis of these data, as well as the comments of the subjects we will produce improvements or complete revision of the existing prototype, in order to improve performance. 2.4 Virtual Actor During the onset of research in virtual actor’s theory and cognitive architectures used in games, a general architecture of a non-player character was designed (Fig. 2). In our implementation, it works as follows. The Selector serves to select actions for execution. Initializer is a block which finds objects in the virtual environment. External sources is part in which the interactions between avatars are registered. Working memory is the unit in which the history of interactions and appraisals is stored. The rules determining how to perform the selection of actions based on the appraisal of actions committed by players with respect to the actor and other players are determined by the eBICA cognitive architecture, introduced by Samsonovich A.V.

38

A. Chubarov and D. Azarnov

Fig. 2. General architecture underlying the implementation of a virtual actor. The cognitive architecture eBICA defines the behavior model implemented using these components

2.5 Subjects and Procedure A total of 8 students participated in the study. One hundred percent of the students were male, and 100% reported that Russian is their native language. The ethnic breakdown was as follows: 100% White. A total of 37.5% of the students were from Moscow while others reported that they were from different parts of Russia. In terms of their student status, they are students of the 4th year MEPhI. All of the students were full time students. All of the students had graduated from public high schools. All subjects gave their consent to participate in the experiment. The experiment was conducted on 4 subject pairs that were randomly created from the participating students. Each experiment lasted for one hour. The first half hour players interacted with a control virtual actor lacking any social emotional intelligence. The remaining half hour players interacted with a virtual actor that possessed elements of social-emotional intelligence. The experiment was carried out using the virtual envi‐ ronment implemented in the form of a cross-platform application, which makes experi‐ ments easily reproducible. The experimental application was preinstalled on three different Windows devices. Two of them were used by the subjects and the third was used by the experimenter. At any time when the test was running, the experimenter was able to observe the session. Instructions and information about the game were given to participants before the experiment. Each of the subjects was asked to determine and report, upon reaching confidence in the judgment, which one of the avatars is controlled by the virtual actor,

Modeling Behavior of Virtual Actors

39

and which one by a human player. Subjects reported their judgments by indicating the ID of the player that was controlled, by their opinion, by a virtual actor, as well as the time of the decision. After the experiment, the subjects were asked to name the reasons due to which they have made their judgments. Analysis of the results of the experiment included comparison of time intervals required for the subjects to make their judgments. It was evaluated time intervals, for which the players make a statement, as well as alle‐ giance to this statement. This way it was produced a similar test as a limited Turing test.

3

Result

3.1 Result and Analysis Based on the outcomes of this experiment, we can conclude that elements of social emotional intelligence have a significant impact on the similarity of behavior of a virtual actor and a human player. Since during experiments we carried profiling of all players using all available data, it was possible to make analysis of the records in order to detect new patterns and to calculate the human response time as a reaction to external actions with respect to both, the actor - and other. Taken together, the conducted Turing test experiments on 8 subjects (8 + 8 sessions) with virtual actors, a statistically significant difference in the frequency of correct and incorrect answers to the main question of the test (which of the two avatars is controlled by a machine) was not found. This applies to both the virtual actors having elements of social and emotional intelligence, and the traditional “heartlessly rational” virtual actors. However, comparison of the time required for the subjects to answer the question of the test, revealed that this time interval was significantly greater (P < 0.042 based on Student’s T-test) in the case where an avatar controlled by a virtual actor, having elements of social and emotional intelligence. This is the first positive result allows us to hope that the correct approach to solving global problem.

4

Discussion

4.1 Summary, Interpretation and Implications of the Outcome Virtual environment with the ability to dive into it the virtual actors of various types have been developed in the course of this work. This virtual environment is crossplat‐ form and have a multiplayer mode. In this virtual environment, a special interface to connect different models based on cognitive architectures was developed. Also, a virtual actor with social features of emotional intelligence was implemented as an example. According to the results of experiments, it was confirmed that a significant difference exists between this type of actor and the type based on a probabilistic approach (lacking emotional intelligence). The data collected during a series of experiments have a big value to researchers of virtual actors. These data allow us to appraise the interaction

40

A. Chubarov and D. Azarnov

between virtual actors and humans. This virtual environment is a universal platform for the study of various models of cognitive architectures in a given scenario. A significant difference between emotional and social model, and model based probabilistic approach is a step towards more and more perfect models. 4.2 Future Work Our future plans include improvement of the virtual environment. During the develop‐ ment of the virtual actor, we found the need to optimize virtual actor’s connection inter‐ faces. When modifying the interface, it would be necessary to change the scenario and logic of the virtual environment accordingly. Future plans also include adding new situations and opportunities for the subjects, as well as comparison of virtual actor models to each other. Our long-term goals include achieving human-level believability of socially emotional virtual actors. Here we primarily imply non-player characters in games, personal assistants, robots, and other intelligent artifacts. 4.3 Conclusions As a result of the work we can say that the designed virtual environment is far from perfect. It is worth noting that the main objectives of this work have been reached. First of all, we were able to connect the virtual actor with social emotional intelligence, which has shown its worthiness as a result of series of experiments. Acknowledgment. This work was supported by the RSF Grant # 15-11-30014. Authors are particularly grateful to Dr. Alexei V. Samsonovich, Professor and Scientific Head of the Institute for Cyber Intelligence Systems (ICIS) of National Research Nuclear University “Moscow Engineering Physics Institute” (MEPhI).

References 1. Parisi, D., Petrosino, G.: Robots that have emotions. Adapt. Behav. 18(6), 453–469 (2010) 2. Phelps, E.A.: Emotion and cognition: insights from studies of the human amygdala. Annu. Rev. Psychol. 57, 27–53 (2006) 3. Russell, J.A.: A circumplex model of affect. J. Pers. Soc. Psychol. 39(6), 1161–1178 (1980) 4. Samsonovich, A.V.: Emotional biologically inspired cognitive architecture. Biol. Inspired Cogn. Archit. 6, 109–125 (2013) 5. Samsonovich, A.V., Tolstikhina, A., Bortnikov, P.A.: A test for believable social emotionality in virtual actors. Procedia Comput. Sci. 88, 450–458 (2016) 6. Samsonovich, A.V., Goldin, R.F., Ascoli, G.A.: Toward a semantic general theory of everything. Complexity 15(4), 12–18 (2010) 7. Mueller, S.T., Jones, M., Minnery, B.S., Hiland, J.M.: The BICA cognitive decathlon: a test suite for biologically-inspired cognitive agents. In: Proceedings of Behavior Representation in Modeling and Simulation Conference, Norfolk (2007) 8. Ortony, A., Clore, G., Collins, A.: The Cognitive Structure of Emotions. Cambridge University Press, Cambridge (1988)

Rethinking BICA’s R&D Challenges: Grief Revelations of an Upset Revisionist Emanuel Diamant ✉ (

)

VIDIA-mant, Kiriat Ono, Israel [email protected]

Abstract. Biologically Inspired Cognitive Architectures (BICA) is a subfield of Artificial Intelligence aimed at creating machines that emulate human cognitive abilities. What distinguish BICA from other AI approaches is that it based on principles drawn from biology and neuroscience. There is a widespread convic‐ tion that nature has a solution for almost all problems we are faced with today. We have only to pick up the solution and replicate it in our design. However, Nature does not easily give up her secrets. Especially, when it is about human brain deciphering. For that reason, large Brain Research Initiatives have been launched around the world. They will provide us with knowledge about brain workflow activity in neuron assemblies and their interconnections. But what is being “flown” (conveyed) via the interconnections the research programme does not disclose. It is implied that what flows in the interconnections is information. But what is information? – that remains undefined. Having in mind BICA’s interest in the matters, the paper will try to clarify the issues. Keywords: Biological inspiration · Brain research programs · Cognitive modeling · Information duality · Cognitive information processing

1

Introduction

Biologically Inspired Cognitive Architectures (BICA) is a loosely defined subfield of Artificial Intelligence (AI) aimed at developing thinking machines with human-like or near-human-like intelligence and cognitive capabilities. What distinguish BICA from other similar AI enterprises is that it based on principles explicitly drawn from biology and neuroscience. There is a long lasting and a widespread belief that nature in its evolution has already encountered most of the problems that we experience today and even has a couple of wonderful and unexpected solutions for any of the challenges that we have to cope with. That is the reason why biologically inspired (or, in short, bioinspired) approaches are so ubiquitous and abundant today when the challenge of creating machines equipped with human-like cognitive capabilities is issued. “Bioinspired” is the most frequently encountered term applied when it comes to discuss the above-mentioned matters. However, other labels are also around: Naturally-inspired, nature-inspired, neuro-inspired, brain-inspired, brain-like, bio-mimicking, and other similar designates. In [1], a broad overview of 195 cognitive architectures is provided © Springer International Publishing AG 2018 A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6_6

42

E. Diamant

covering 700 practical projects implemented over the past 40 years of BICA research and development endeavor. Nevertheless, and despite all these impressive numbers and equally important analytic evaluations, BICA’s R&D is far from being what most observers would regard as a success story. The plurality of solutions that are being devised (and trustworthily observed in [1]), undeniably indicate a fierce lack of appropriate knowledge and commonly accepted theory that would underpin and justify the intentionality of bioinspired approaches. If we know nothing about what is going on in our biological proto‐ types, how could we be inspired by their unknown properties? Indeed, growing understanding of an urgent need to enhance our competence in brain organization and functioning has led to a world-wide range of government-funded research initiatives like the U.S. BRAIN Initiative, the Human Brain Project in Europe, and brain-focused projects in Japan, China, and Korea. The central pillar of almost all these projects is the consensus understanding that the neural basis of human cognition is the primary goal of all these enterprises. A quotation from China Brain Project declaration states this point as follows: “We know very little about how neural circuits are assembled from specific types of neurons in different brain regions and how specific neural circuits perform their signal processing functions during cognitive processes and behaviors. This requires detailed information on the architecture of neural circuits at single-cell resolution and on the spatiotemporal pattern of neuronal activity”, [2]. The Korea Brain Initiative echoes this objective in very similar words: “The Korea Brain Initiative, which is centered on deciphering the brain functions and mechanisms that mediate the integration and control of brain functions that underlie decision-making. The goal of this initiative is the mapping of a functional connectome with searchable, multi-dimensional, and information-integrated features”, [3]. And just a bit later again: “The initiative aims at advancing technologies for a better understanding of the full complexity of the brain, and especially of circuit-function relationship”, [3]. It is expected that it would take from 10 to 20 years before the first results and preliminary understanding of how the brain works would be available. That is too long. Meanwhile, BICA’s R&D has to continue its march towards its proclaimed objectives, and any critical remarks (like mine) will not slow down its impetuous pace. However, it is worth to be mentioned that such critical faultfinding does exist, and it would be wise sometime and somehow to take it into account. What I am speaking about is a 2010 paper [4] that provides an improved scientific perception of the operational principles of the brain as a complexly organized system. Relying on this perception the author tries to build an operational, quantitative model of the brain. He claims that “The scientific disciplines involved in cognitive and brain research are committed to a common method to explain the properties and capacities of complex systems. This method is decompositional analysis, i.e. analysis of the system in terms of its components or subsystems… decomposability of complex systems has been accepted as fundamental for the cognitive and computational neuroscience.” [4]. Indeed, the decompositional principle allows BICA designers to see the brain as composed of “building blocks” which are dedicated for computing certain principally defined cognitive functions. In [1] these building blocks are outlined as different

Rethinking BICA’s R&D Challenges

43

dissimilar modules performing restricted cognitive functions such as perception, atten‐ tion, reasoning, learning, planning, decision-making, formation of memory and action shaping. The author in [4] claims that the decompositional principle is wrong, and that “there is substantial evidence to question this belief. It turns out that this method in fact ignores something fundamental, namely that biological and engineered systems are basically different in nature” and that “…we cannot expect specific functions to be mapped to structurally bounded neuronal structures, and vice versa.” [4]. Therefore, the respective author suggests, “that the decomposability assumption of cognitive science must be abandoned” [4]. Of course, as far as I know, nobody has adopted this proposal. BICA designers, as well as other researchers involved in the flagship Brain projects, know perfectly well that in biology, system inputs are of different modalities and processing these inputs is being performed via different paths thus supporting and realizing different cognitive function. As to my understanding, the most expected and most likely answer to the above raised question should be: the neuronal interconnections convey information. A clear and complete answer. However, this evident and seemingly obvious answer is not so obvious at all. Although the term “information” is the most often and ubiquitously used word today, I am not sure that you will find someone in your surrounding who is skilled enough to explain what the term “information” really means. Therefore, it will be our duty to find out the proper answer to the question “What actually information is?” – the question so imprudently neglected by flagship Brain research projects (in general) and BICA designers (in particular).

2

What is Information?

The notion of “Information” was first introduced by Claude Shannon in his seminal paper “A Mathematical Theory of Communication” in 1948, [5]. Today, Stanford Ency‐ clopedia of Philosophy offers (side by side with Shannon’s definition of information) an extended list of other versions of the term: Fisher information, Kolmogorov complexity, Quantum Information, Information as a state of an agent, and Semantic Information (once developed by Bar-Hillel and Carnap), [6]. Again, as it was mentioned earlier, multiplicity of definitions is not a sign of well-being. Shannon’s Information Theory was about the communication of messages as elec‐ tronic signals via a transmission channel. Only physical properties of the signal and the channel have been taken into account, while the meaning of the message has been ignored totally. Such an approach to information met very well the requirements of a data communication channel. But recent advances in almost all sciences put an urgent demand for meaningful information inclusion into the body of a communicated message. To meet this demand, I have proposed a new definition of information. Contrary to the widespread use of Shannon’s Information Theory, my research relies on the Kolmogorov’s definition of information, [7]. In the mid-60 s of the past century, Kolmogorov has proposed an algorithmic approach to a quantitative information

44

E. Diamant

definition [7]. According to Kolmogorov, a not random binary string (called a sepa‐ rate finite object) can be represented by a compressed description of it (produced by a computer program in an algorithmic fashion) “in such a way that from the descrip‐ tion, the original message can be completely reconstructed” [8]. “The amount of information in the string is then defined as the size of the shortest computer program that outputs the string and then terminates” [8]. (For a really random string such a condensed description cannot be provided and “the shortest program for generating it is as long as the chain itself” [9]). The compressed description of a binary object has been dubbed as “algorithmic information” and its quantitative measure (the length of the descriptive program) has been dubbed as the description “Complexity”. Taking Kolmogorov’s insights as a starting point, I have developed my own defini‐ tion of information that can be articulated in the following way: “Information is a linguistic description of structures observable in a given data set”. To make the scrutiny into this definition more palpable I propose to consider a digital image as a data set. A digital image is a two-dimensional set of data elements called picture elements or pixels. In an image, pixels are distributed not randomly, but, due to the similarity in their physical properties, they are naturally grouped into some clusters or clumps. I propose to call these clusters primary or physical data structures. In the eyes of an external observer, the primary data structures are further arranged into more larger and complex agglomerations, which I propose to call secondary data structures. These secondary structures reflect human observer’s view on the grouping of primary data structures, and therefore they could be called meaningful or semantic data structures. While formation of primary (physical) data structures is guided by objective (natural, physical) properties of the data, the subsequent formation of secondary (semantic) data structures is a subjective process guided by human conventions and habits. As it was said, Description of structures observable in a data set should be called “Information”. In this regard, two types of information must be distin‐ guished – Physical Information and Semantic Information. They are both language-based descriptions; however, physical information can be described with a variety of languages (recall that mathematics is also a language), while semantic information can be described only by means of natural human language. (More details on the subject could be find in [10]). Those, who will go and look in [10], would discover that every information descrip‐ tion is a top-down-evolving coarse-to-fine hierarchy of descriptions representing various levels of description complexity (various levels of description details). Physical infor‐ mation hierarchy is located at the lowest level of the semantic hierarchy. The process of sensor data interpretation is reified as a process of physical information extraction from the input data, followed by an attempt to associate this physical information (about the input data) with physical information already retained at the lowest level of the semantic hierarchy. If such an association is attained, the input physical information becomes related (via the physical information retained in the system) with a relevant linguistic term, with a word that places the physical information in the context of a phrase, which provides the semantic interpretation of it. In such a way, the input physical information becomes

Rethinking BICA’s R&D Challenges

45

named with an appropriate linguistic label and framed into a suitable linguistic phrase (and further – in a story, a tale, a narrative), which provides the desired meaning for the input physical information. (Again, more details can be found on the website).

3

Rethinking BICA

The segregation between physical and semantic information is the most essential insight about the nature of information gained from the new information definition. It puts BICA design challenges in new light and new circumstances. Semantic information processing has nothing to do with raw input data (and its features) – raw data features are dissolved in physical information (which is later processed in the semantic information hierarchy). Data features are meaningless in human world perception (and judgment). We understand the meaning of a written word irrelevant to letters’ font size or style. We recognize equally well a portrait of a known person on a huge size advertising billboard, on a magazine front page, or on a postage stamp – perceptive information is dimensionless. We grasp the meaning of a scene irrelevant to its illumination. We look on the old black-and-white photos and we do not perceive the lack of colors. The same is true for voice perception and spoken utterance understanding – we understand what is being said irrelevantly to who is speaking (a man, women, or a child). Irrelevant to the volume levels of the speech (loudly or as a whisper). Blind people read Brail-style writings irrelevant to the size of the touch-code. And the final bottom line: information is dimensionless, data is dimensional. Reference knowledge base, where human/system previous life experience is accu‐ mulated (to support the system’s cognitive tasks processing) has to be also re-evaluated. The critical issue of continuous autonomous learning is closely related to this subject. As it follows from the preceding discussion, semantics is not a property of the data. Semantics is a property of a human observer that watches and scrutinizes the data. Semantic information is shared among the observer and other members of his community (and that is the common basis of their intelligence). By the way, this community does not have to embrace the whole mankind. This can be even a very small community of several people or so, which, nevertheless, were lucky to establish a common view on a particular subject and a common understanding of its meaning. Therefore, this particular (privet) knowledge cannot be acquired in any other way. (By Machine Learning, for example, by Deep Learning, or other tricks). Semantic informa‐ tion should be only shared or granted! There is no other way to incorporate it as the system’s reference knowledge base (used for processing/interpreting physical informa‐ tion at the system’s input). Therefore, common attempts to formalize semantics and to derive it from input data are definitely wrong. The form in which semantic information has to be reified is a string of words, a piece of text, a story, a narrative. (That follows from semantic information definition already given above). If we accept this assumption, it will be reasonable to suppose that semantic information processing means some sort of language texts processing. (For humans it is, obviously, human natural language texts, but for plants or bacteria it will be a different

46

E. Diamant

kind of language – every living being possess its own intelligence reified as its ability to process semantic information, which is reified in some pra- or proto-language). What implications follow from the statement “semantic information processing means language text processing”? – I do not know (at least at this stage of my research). As to my knowledge, nobody else knows about this not more than I. (Despite there is a wellknown research field of computational linguistics, however, the domain of its studies does not overlap with semantic information processing).

4

Conclusions

The list of amendments waiting to be introduced to BICA’s design practice is long and inspiring. The paper format does not allow its full exhibition. I hope the conference framework will be a proper place for further exchange of views and in depth discussions.

References 1. Kotseruba, I., Gonzalez, O., Tsotsos, J.: A Review of 40 Years of Cognitive Architecture Research (2016). https://arxiv.org/ftp/arxiv/papers/1610/1610.08602.pdf 2. Poo, M., et al.: China brain project: basic neuroscience, brain diseases, and brain-inspired computing. Neuron 92 (2016). Elsevier Inc. http://www.cell.com/neuron/pdf/ S0896-6273(16)30800-5.pdf 3. Jeong, S.-J., et al.: Korea brain initiative: integration and control of brain functions. Neuron 92, 607–611 (2016). Elsevier Inc. http://www.cell.com/neuron/pdf/S0896-6273(16)30805-4.pdf 4. Schierwagen, A.: The Way We Get Bio-Inspired: A Critical Analysis (2010). http:// citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.188.6007&rep=rep1&type=pdf 5. Shannon, C.: A Mathematical Theory of Communication. Published by the Board of Trustees of the University of Illinois, Used with the permission of University of Illinois Press (1948). http://www.mast.queensu.ca/~math474/shannon1948.pdf 6. Information: Stanford Encyclopedia of Philosophy. First published 26 October 2012. http:// plato.stanford.edu/entries/information/ 7. Kolmogorov, A.: Three approaches to the quantitative definition of information. Probl. Inf. Transm. 1(1), 1–7 (1965). http://alexander.shen.free.fr/library/Kolmogorov65_ThreeApproaches-to-Information.pdf 8. Grunwald, P., Vitanyi, P.: Algorithmic information theory (2008). http://arxiv.org/pdf/ 0809.2754.pdf 9. Grunwald, P., Vitanyi, P.: Shannon Information and Kolmogorov Complexity (2004). http:// arxiv.org/pdf/cs/0410002.pdf 10. Diamant, E.: Brain, Vision, Robotics and Artificial Intelligence. http://www.vidia-mant.info

A Roadmap to Emotionally Intelligent Creative Virtual Assistants Alexander A. Eidlin1 and Alexei V. Samsonovich1,2 ✉ (

)

1

2

Department of Cybernetics and BICA Lab, Institute for Cyber Intelligence Systems, National Research Nuclear University “Moscow Engineering Physics Institute”, Kashirskoe Shosse 31, Moscow 115409, Russian Federation [email protected] Krasnow Institute for Advanced Study, George Mason University, Fairfax, VA 22030, USA [email protected]

Abstract. Cognitive psychology has accumulated a vast amount of knowledge about human social emotions, emotional appraisals and their usage in decision making. Can an emotional cognitive architecture injected into an artifact make it more “humane”, and therefore, more productive in a variety of creative collabo‐ ration paradigms? Here, we argue that the answer is positive. A large number of research projects in the field of digital art that are currently underway could benefit from integration of an emotional architecture component into them. An example is the project Robodanza (a robotic dancer), the functioning of which is based on a hidden Markov model trained by a genetic algorithm, yet lacking deep emotional intelligence. Generalizing on this example, we outline a roadmap to building a variety of useful virtual creative assistants to humans based on an emotionally intelligent cognitive architecture. Keywords: Cognitive modeling · Virtual actor · Emotional intelligence · Creative assistant · Co-robots

1

Introduction

How important are socially-emotional interactions, based on a deep understanding of emotions, for creativity? Here, we outline a new approach in the field of digital art based on an emotional cognitive architecture that involves internal processing of representa‐ tions of complex emotions and may serve as a general paradigm of research and devel‐ opment for this direction. On the one hand, it combines many promising innovations, and on the other hand, it fills the ‘gap’ associated with the lack of emotional intelligence in robots. Practically, all existing developments in this area have limitations caused by the absence of deep emotional intelligence in them. An example is Robodanza (a robotic dancer [1]). Future co-robots will work side-by-side with humans who tend to rely on social emotional relationships in collaboration with partners. The emotional aspect of human nature cannot be ignored in teams that will include co-robots [2]. In a human team, every © Springer International Publishing AG 2018 A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6_7

48

A.A. Eidlin and A.V. Samsonovich

collaborative action has an emotional component in it, which includes motivation caused by, and having effect on socially-emotional relationships with partners. Therefore, virtually every action contributes to social relationships in the team, which in turn deter‐ mines the structure and capabilities of the team. The same logic can be applied to heter‐ ogeneous teams including humans and co-robots. When co-robots lack emotional capa‐ bilities, they cannot establish emotional social relations with the team, and therefore cannot integrate into the team at a human level. Although social emotions are the key component of human cognition, the need for their deep processing has been largely dismissed by researchers in artificial intelligence (AI) in favor of the more computationally and logically tractable aspects of human intelligence. Since the advent of AI in the mid-1900s [3], a great deal of work has been done in what now is considered traditional areas of AI [4], with relatively little attention devoted to mechanisms of emotions and emotional intelligence. In modern mainstream research, the role of emotions is typically limited to the reward and punishment function in reinforcement learning [5]. Despite that immense literature is devoted to studies of emotions, the scientific state of the art in understanding emotions at the level of compu‐ tational models is still in the early age [6–8], with studies of computational models of complex social emotions and their role in team relationships being particularly limited. This situation is partially due to the general problem of “conceptual ceiling”, known for decades in brain and behavioral sciences, including computational and systems neuro‐ science and cognitive psychology. Biologically inspired computational models of human emotional cognition are needed to break the ceiling and to build a bridge between neural, cognitive, behavioral studies and computational art [1]. Today there are many reasons for building this bridge. From the perspective of computational science, cognitive systems provide a logical approach to creativity. Here, the state of the art is characterized by a list of topics, that also list opportunities for deploying a general-purpose emotional cognitive architecture, such as Ebica (Emotional Biologically Inspired Cognitive Architecture [9]): • Computational paradigms have been developed for emulation of creativity in arti‐ facts, using heuristic search, analogical and meta-level reasoning. • Metrics, tests, paradigms, frameworks, formalisms and methodologies are created for evaluation of creativity in artifacts. • Specialized computational tools are designed to enhance human creative abilities. • Artificial creative abilities can be transferred using various forms of learning and teaching. • Specialized computer applications demonstrated creativity in a variety of specific domains, such as music, narrative, jokes, poetry, games, fine arts, design, entertain‐ ment, education, innovation, scientific discovery, programming, and more. The paper is organized as follows. Section 2 (next) introduces the main concept and lists three perspective areas for its rapid prototypes. Section 3 describes the suggested approach and gives further detail on the outlined projects. Section 4 gives a discussion of related work. Finally, Sect. 5 presents conclusions of our analysis and describes a possible roadmap to developing and using virtual creative assistants as human partners.

A Roadmap to Emotionally Intelligent Creative Virtual Assistants

2

49

The Concept of an Emotional Creative Assistant

The main object of analysis that we entertain here is a domain-independent cognitive architecture controlling a robot or a virtual assistant working together with a human partner or in a heterogeneous team that is engaged in creative activity. The nature of this activity may be unknown a priori, and may vary in a broad spectrum of intellectual domains, e.g., taken from the above list. The main question is: what are the key features of this architecture and its operation principles, allowing the controlled artifact to collaborate effectively with a person in solving creative tasks? Specifically, do we need to put social and emotional intelligence in this architecture in order to succeed, and if yes, then, should we use the data and inspi‐ rations from human neuropsychology? The hypothesis is that the answers are positive. A cognitive architecture is a digital blueprint of intelligent agents. Especially valuable cognitive architectures (and intelligent agents) are those that have the flexibility in the sense that they can be bolted on top of many systems of various nature, including robots and virtual agents designed for specific application domains. Existing generalpurpose intelligent agents are limited and, when offered as assistants, are often rejected by users (examples: Siri, Cortana, Google Now). This may be due to the lack of socialemotional intelligence in them, among other factors. If social-emotional intelligence in artifacts is the key to their social acceptance by humans, then the development of a cognitive architecture for robots possessing such qualities as social and emotional intelligence is an important task of fundamental science. The aim of the present study is the analysis of a concept of a versatile cognitive architecture supporting socially-emotional functionality, that is put in control of an arti‐ fact that serves as a creative assistant to a human. We call this construct an Emotional Creative Assistant (ECA). There are three perspective areas that can be used for rapid prototyping of an ECA in order to prove the concept. These areas and corresponding prototypes are labeled here eRobodanza, eArtist, and eActor. They are described below. Remarkably, in each of these tasks, one and the same cognitive architecture can be used: it is called Ebica (Emotional Biologically Inspired Cognitive Architecture [9]). 2.1 eRobodanza eRobodanza is an ECA in the form of emotionally-intelligent robots working together with a human in order to generate a creative dance. The Ebica cognitive architecture will facilitate the study of cognitive architecture principles responsible for the effec‐ tiveness of joint work, including ensuring the establishment of an emotional contact with the human, perception, processing and use of social emotions in the process of behavior generation. The size of the effect of social-emotional intelligence of the robot on the efficiency and quality of co-creation will be the measure of success.

50

A.A. Eidlin and A.V. Samsonovich

2.2 eArtist eArtist, e.g., a creative abstractionist, is the robotic assistant (an ECA) in the field of fine art. The Ebica cognitive architecture is used for controlling a robot or virtual actor assisting a human while creating samples of fine art: abstract paintings, abstract shapes composed from specified blocks, etc. 2.3 eActor eActor is a believable actor with behavior potentially indistinguishable from the behavior of a person controlling the same avatar. eActor can function as a video game character or participate in other forms of social interaction in virtual reality, augmented reality, or telepresence. eActor will be able to control a virtual avatar or a robot, and is expected to pass a limited Turing test, thus confirming the credibility of its character in comparison with a human-controlled avatar or robot. All the three areas listed above are unified by the universal emotional cognitive architecture used in each of them, and are based on the same concept of ECA described above.

3

Approach and Methodology

The aim of any given ECA project is to create a cognitive architecture suitable for an intelligent robot, which will allow it to participate in the joint creative process together with a human (e.g., creative dance execution). The robot can be either virtual or real: the particular choice is not important, because the task is to capture the interactive process of co-creation and the influence of the emotional system on it. 3.1 From Robodanza to eRobodanza Dancing is a complex domain that allows one to test ECA models and involves various relevant cognitive and emotional aspects: the perception of sound and music, the execu‐ tion of pre-determined movements, and so on [1]. Challenges here can be separated into higher and lower levels. At the lower level, when one hears a rhythmic sound, a typical reaction is spontaneous body movement synchronized with the musical rhythm. In fact, when we listen to a song, we often unconsciously nod our head, beat our feet, beat our hands and fingers on the table on our legs [10]. Toiviainen, Luck and Thompson [11] investigated the movement induced by music in human beings, focusing on the rela‐ tionship between patterns and metrical levels of music, revealing that they embody musical stimuli at several metrical levels. Xia et al. [12] pursued a similar research; instead, Seo, Kim & Kwon [13] focused their research on simple rhythmic movements like head nodding or hand shaking. Usually, for people, it is very difficult or even impossible to dance on a choreography without any practice, but anyone can keep the rhythm with some simple movements. On the other side, it is difficult for a robot to do planning, generating, executing and

A Roadmap to Emotionally Intelligent Creative Virtual Assistants

51

synchronizing a sequence of postures with any music genre, also because it could be the first time that the robot listens to the particular piece of music. Alternatively, the robot can learn some associations between music and movements through an internal mech‐ anism of evaluation, e.g., using a genetic algorithm (GA), and thereby acquire a creative behavior capability. This results in an original, natural and entertaining performance that is positively evaluated by the audience [1] (Fig. 1).

Fig. 1. A scene from the Robodanza performance at BICA 2015.

Now, moving to the higher-level challenge, further evolution of this approach is to consider the emotional impact in all the creative phases that we need to create for a human-robot joint performance. The robot should learn to respond adequately to human emotions expressed in dance. During the learning phase, a teacher drives the creation of a movement’s repertoire and contributes to defining a style and a behavior of the agent. The execution is not fixed, but improvisation could happen and depends on the dynamic environment. A simple depth camera (RGBD) could give information of the human user movements and facial expressions in the case of the virtual agent. Multiple sensors equipping a real robot enrich the collected sensory input from the environment. After the execution, the agent has to improve its behavior by taking into account the success or failure of the performance, and the emotional impact on the audience. The next step is to shorten this emotional feedback loop, making the robot emotionally responsive to the partner at the scale of individual dance movements. Despite such interest in this applicative field, to the best of our knowledge, there are no works introducing creativity mechanisms that rely on emotional reasoning in a humanoid robotic platform. In our opinion, a realistic dancing behavior does not involve only the perception of beat and emotions, and the choice of the most suitable dancing style. A realistic dancing act also includes the capability to reason intuitively about perceived emotions and to be creative, i.e., to invent or improvise new dancing move‐ ments that express a certain pattern of emotional feelings. Such an innovative behavior involves higher cognitive processes. It requires a motivation for creating something new, the ability to get inspirations from perception, to adequately represent them and compare them to previous experience, to assess the outcome of the creative process and to take into account external judgments. For this reason, we propose to model computational

52

A.A. Eidlin and A.V. Samsonovich

creativity and co-creation tools within the cognitive framework of Ebica [9] (Fig. 2) described below.

Fig. 2. Overview of the Ebica cognitive architecture (reproduced from Samsonovich, 2013).

Here we should point to an alternative approach. Recently a cognitive architecture supporting creativity in robots was introduced [14], involving motivations and emotions [15]. In particular, inspiration was taken from features of the Psi model [16] and its architecture, since it explicitly involves the concepts of emotion and motivation in cognitive processes, which are two important factors in creativity process. A robot equipped with the creative system was previously employed in two different artistic domains: digital paintings, and dance creation. 3.2 Cognitive Architecture The architecture Ebica (Fig. 2) [9] is suitable for the approach used here for implemen‐ tation of emotional intelligence, because it is based on the formalism of schemas and mental states that enable metacognition and human-like episodic memory [17]. The three new elements in Ebica compared to GMU BICA and Constructor [18] are: (1) an emotional state, (2) an appraisal, and (3) a moral schema (Fig. 2). Complex social emotions like humor, jealousy, compassion, shame, pride, regret, etc. can be understood based on moral schemas associated with specific patterns of appraisals. Preliminary simulation studies [9, 19, 20] demonstrate that new elements of Ebica result in the formation of persistent social relationships in small groups of virtual agents. The cognitive architecture is of great importance in this project, because basically, it implements a closed loop between learning, perception, execution and final the eval‐ uation of the dance or in general a creative act. Artificial creativity allows the robot to explore new sequences of movements and assess their aesthetic acceptance (by mean of appropriate distances and expert evaluations). At runtime, the creativity framework in the cognitive architecture also drives the interaction between human-robot finding

A Roadmap to Emotionally Intelligent Creative Virtual Assistants

53

human faces, as well as through interaction with physical contact between humans and robots. Many virtual environment simulators can be used during the development of eRobo‐ danza, including the commercially available Webots [21] that offer realistic copies of commercially available physical robots and suitable environments. 3.3 Toward eArtist A significant advantage of the Ebica-based ECA architecture is that it can be utilized in different artistic domains. For example, after few changes of the cognitive architecture used in the dance, the robot starts producing a collage formed by a mix of photomontage and digital collage. In this case, the artwork is created by a visual and verbal interaction with a human user. The system, through a cognitive architecture, allows the robot to manage the three different phases of a real-time artwork process: (a) taking inspiration from information captured during the postural and verbal inter‐ action with the human user and from the analysis of his/her social web items; (b) performing a creative process to obtain a model of the artwork; (c) executing the creative collage composition and providing a significant title. It should be understood, primarily, how the creativity traits of the robot are imple‐ mented in the proposed architecture: how ideas are generated through an elaboration that is modulated by affective influences; how the personality and the artistic behavior are modeled by learning and guided by external evaluations; the motivation and the confidence evolution as a function of successes or failures.

4

Concluding Outline of the Roadmap

In conclusion, we have articulated the following points. Recent achievements in Neuro‐ science and IT bring to the fore the problem of replicating all the significant aspects of the human mind (primarily emotionality, creativity and social competence) in a computer based on a biologically-inspired approach (the BICA Challenge [2]). Currently developed robots and virtual characters have serious limitations due to the lack of deep emotional intelligence. An emotional cognitive architecture can make robots more “humane” and therefore, more productive in a variety of collaboration paradigms. Many existing examples, however, have limitations due to the lack of deep emotional intelli‐ gence in the architecture. Today, therefore, we are challenged to jump to a qualitatively new level of understanding the principles underlying natural emotional intelligence and a new level in building biologically inspired emotionally-intelligent AI. One possible roadmap of the study along this direction is presented in Fig. 3. It starts with integrating and adapting for the task three main components: (i) the architecture Ebica, (ii) three kinds of semantic maps (lingual, visual and behavioral), and (iii) periph‐ eral tools, including machine vision and motion control. After that, three thrusts are pursued in parallel, followed by testing and evaluation. At the end, we have a useful

54

A.A. Eidlin and A.V. Samsonovich

universal module that can be “bolted” on top of virtually any creative virtual assistant, increasing its efficiency, productivity and quality.

Fig. 3. A possible roadmap to building creative virtual assistants.

The outlined here approach is aimed at answering the question: what are the archi‐ tecture and operation principles of a cognitive robot, or an intelligent agent that allow it to work efficiently on creative problems together with a human partner? We argued that human-like social emotionality is of primary importance in human-robot collabo‐ ration, and therefore, particular attention should be given to the grounding of research in empirical data from human neuropsychology and behavioral experiments, involving social interactions between humans and virtual agents or robots.

5

Discussion of Related Work

Today the main focus in research on affective robotics and socially-emotional engage‐ ment with robots is not in the deep internal processing of social emotions, but in the means of recognition and expression of emotions. Illustrative examples include works of the groups of Dr. Hatice Gunes at the University of Cambridge [22], Dr. Christian W. Becker-Asano at Bosch R&D in Renningen, Stuttgart, and Dr. Hiroshi Ishiguro [23] at the Advanced Telecommunications Research Institute International (ATR) works on several projects related to our proposed research. E.g., the latter group is pursuing such directions as understanding and transmitting human presence, symbiotic human-robot interaction, etc. Similar trends prevail in the field of emotional virtual character research. For example, CADIA at University of Reykjavik led by Prof. Hannes Högni Vilhjálmsson [24] pursues a number of research projects using virtual avatars with social capabilities,

A Roadmap to Emotionally Intelligent Creative Virtual Assistants

55

focusing on the expression and recognition of emotions, rather than their deep under‐ standing. There are a number of schools working on social robots and robots-assistants. E.g., Madeira Interactive Technologies Institute is working on the project “AHA – Augmented Human Assistance”, pursuing the development and deployment of a novel Robotic Assistance Platform designed to support healthy lifestyle, sustain active aging, and support those with motor deficits. Their other project, AuReRo [25] (Human-robot interaction with field robots using augmented reality and interactive mapping) led by Ian Oakley, is based on the use of sturdy robots in unstructured environments. One important example scenario is a Search And Rescue (SAR) operation to seek out victims of catastrophic events in urban environments. Overall, the present state of the art is missing deep processing of complex social emotions in cognitive architectures that controls a robot or a virtual character.

6

Conclusions

In summary, the main object of analysis in this work was a domain-independent cogni‐ tive architecture controlling a robot or a virtual assistant working together with a human partner engaged in a creative activity. The proposed hypothesis is that the vital features of this architecture include socially emotional intelligence; therefore, one needs to use data and inspirations from human neuropsychology in order to succeed. As a result of our analysis, a concept of ECA was introduced as a versatile cognitive architecture supporting socially-emotional functionality, put in control of an artifact that serves as a creative assistant to a human. Further analysis of this concept was conducted using three examples: eRobodanza, eArtist, and eActor. These cases turned out to be similar and allowing for knowledge transfer. In all three cases, socially-emotional interactions appear vital and are expected to be important for creativity and effectiveness of the team. Acknowledgments. The authors are grateful to Dr. Sergey Misyurin, Professor and Director of ICIS of the National Research Nuclear University “MEPhI”, Moscow, Russian Federation, for useful discussions. Our greatest thanks go to Drs. Ignazio Infantino and Umberto Maniscalco, Researchers at ICAR-CNR, section of Palermo, Italy, who provided us with useful background. This work was supported by the RSF Grant # 15-11-30014.

References 1. Manfre, A., Infantino, I., Vella, F., Gaglio, S.: An automatic system for humanoid dance creation. Biol. Inspir. Cogn. Archit. 15, 1–9 (2016) 2. Samsonovich, A.V.: On a roadmap for the BICA Challenge. Biol. Inspir. Cogn. Archit. 1, 100–107 (2012) 3. McCarthy, J., Minsky, M.L., Rochester, N., Shannon, C.E.: A proposal for the Dartmouth summer research project on artificial intelligence. In: Chrisley, R., Begeer, S. (eds.) Artificial Intelligence: Critical Concepts, vol. 2, pp. 44–53. Routledge, London (1955/2000) 4. Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approach. Prentice Hall, Upper Saddle River (1995)

56

A.A. Eidlin and A.V. Samsonovich

5. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998) 6. Gratch, J., Marsella, S.: A domain-independent framework for modeling emotion. Cogn. Syst. Res. 5, 269–306 (2004) 7. Hudlicka, E.: Guidelines for designing computational models of emotions. Int. J. Synth. Emot. 2(1), 26–79 (2011) 8. Marsella, S.C., Gratch, J.: EMA: a process model of appraisal dynamics. Cogn. Syst. Res. 10, 70–90 (2008) 9. Samsonovich, A.V.: Emotional biologically inspired cognitive architecture. Biol. Inspir. Cogn. Archit. 6, 109–125 (2013) 10. Todd, M., Lee, C., O’Boyle, D.: A sensorimotor theory of temporal tracking and beat induction. Psychol. Res. 66, 26–39 (2002) 11. Toiviainen, P., Luck, G., Thompson, M.R.: Embodied meter: Hierarchical eigenmodes in music-induced movement. Music Percept. Interdisc. J. 28, 59–70 (2010) 12. Xia, G., Tay, J., Dannenberg, R., Veloso, M.: Autonomous robot dancing driven by beats and emotions of music. In: Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems, vol. 1, pp. 205–212. International Foundation for Autonomous Agents and Multiagent Systems (2012) 13. Seo, J.-H., Yang, J.-Y., Kim, J., Kwon, D.-S.: Autonomous humanoid robot dance generation system based on real-time music input. In: RO-MAN 2013, pp. 204–209. IEEE (2013) 14. Augello, A., Infantino, I., Pilato, G., Rizzo, R., Vella, F.: Creativity evaluation in a cognitive architecture. Biol. Inspir. Cogn. Archit. 11, 29–37 (2015) 15. Augello, A., Infantino, I., Pilato, G., Rizzo, R., Vella, F.: Binding representational spaces of colors and emotions for creativity. Biol. Inspir. Cogn. Archit. 5, 64–71 (2013) 16. Bartl, C., Dorner, D.: PSI: a theory of the integration of cognition, emotion and motivation. In: Proceedings of the 2nd European Conference on Cognitive Modelling, pp. 66–73. DTIC Document (1998) 17. Samsonovich, A.V., Nadel, L.: Fundamental principles and mechanisms of the conscious self. Cortex 41(5), 669–689 (2005) 18. Samsonovich, A.V.: The constructor metacognitive architecture. In: Samsonovich, A.V. (ed.) Biologically Inspired Cognitive Architectures II: Papers from the AAAI Fall Symposium, AAAI Technical Report, vol. FS-09-01, Menlo Park, CA, pp. 124–134. AAAI Press (2009) 19. Samsonovich, A.V.: An approach to building emotional intelligence in artifacts. In: Burgard, W., Konolige, K., Pagnucco, M., Vassos, S. (eds.) Cognitive Robotics: AAAI Technical Report, vol. WS-12-06, Menlo Park, CA, pp. 109–116. The AAAI Press (2012) 20. Samsonovich, A.V.: Modeling social emotions in intelligent agents based on the mental state formalism. In: Raskin, V., Taylor, J.M., Nijholt, A., Ruch, W. (eds.) Artificial Intelligence of Humor: Papers from the AAAI Fall Symposium, AAAI Technical Report, vol. FS-12-02, Menlo Park, CA. AAAI Press (2012) 21. Webots: robot simulator (2017). http://aha.isr.tecnico.ulisboa.pt/. Accessed 14 Jan 2017 22. Salam, H., Celiktutan, O., Hupont, I., Gunes, H., Chetouani, M.: Fully automatic analysis of engagement and its relationship to personality in human-robot interactions. IEEE Access J. 5, 705–721 (2016) 23. ATR Hiroshi Ishiguro Laboratories, Understanding and Transmitting Human Presence (2017). http://www.geminoid.jp/en/projects.html. Accessed 13 Jan 2017 24. Vilhjálmsson, H.H.: Principal Contributions (2017). http://www.ru.is/~hannes/ru_main_ research.html. Accessed 13 Jan 2017 25. AHA. Augmented Human Assistance (2017). http://aha.isr.tecnico.ulisboa.pt/. Accessed 13 Jan 2017

Applying a Neural Network Architecture with Spatio-Temporal Connections to the Maze Exploration Dmitry Filin1 and Aleksandr I. Panov1,2(B) 1

National Research University Higher School of Economics, Moscow, Russia [email protected], [email protected] 2 Federal Research Center “Computer Science and Control” of Russian Academy of Sciences, Moscow, Russia

Abstract. We present a model of Reinforcement Learning, which consists of modified neural-network architecture with spatio-temporal connections, known as Temporal Hebbian Self-Organizing Map (THSOM). A number of experiments were conducted to test the model on the maze solving problem. The algorithm demonstrates sustainable learning, building a near to optimal routes. This work describes an agents behavior in the mazes of different complexity and also influence of models parameters at the length of formed paths.

1

Introduction

Currently, the problem of increasing a level of robotic systems autonomy due to integration extended knowledge and learning subsystems into their control systems becomes an important direction in artificial intelligence and cognitive architectures [6–8]. Deep Learning with Reinforcement demonstrated impressive results on the so-called “raw data”, i.e. unprocessed images obtained from sensors of a learning system [1]. At present, systems developed for simple experiments in the game simulation environments are beginning to be used in real robotic tasks [3,4]. The idea of using an information received from such sensors as visual and sound as a training data, makes it possible to construct a representation of the environment which an agent interacts with. Integration of neural networks with traditional Q-learning makes it possible to match observed states of environment with reward received from it for certain actions. An automatic generation of characteristics for a better description of environment states using neural networks makes it possible to apply Q-learning to real “raw data” obtained from sensors [5]. In this paper, we present a model combining one of the neural network architectures based on THSOM [2], with Q-learning for the maze solving. Our motivation is the idea that, in general, all the labyrinths (their images), consist of some patterns (see Fig. 1), which can be divided into several categories. If an agent understood in what state of the environment it is, it could determine the c Springer International Publishing AG 2018  A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6 8

58

D. Filin and A.I. Panov

most appropriate action for this state. So, for example, in the case shown in Fig. 2, the agents available actions are “move left” and “move down”. Initially, the THSOM algorithm is used to generate Markov sequences (namely, for recovering the probabilities of transitions between system states) according to the input data stream. Therefore, it seems intuitively that such an approach could be successfully integrated into the concept of Q-learning. Due to the clustering of the input data, we additionally get a reduction in the dimensionality of the environment states space with the minimal loss of information. In contrast to the traditional approach [9], where the Q-table stores all environment states, we assert that only a few states are enough to describe a full motion process of an agent. Using the Q-network architecture developed in DeepMind [1], we denoted the agent’s observed state of the environment, as an image of a maze section from the top view. While the agent is moving, it “sees” the M × M field around itself. It should be noted that initially the agent does not know anything about the environment where he moves. The process of clustering is taking place during the process of an environment exploration by the agent, thus independence from the map of the labyrinth is achieved. Finally, the main difference from the classical Q-learning is that all information about movement is stored within neuronal connections without the use of additional Q-tables, in other words, the connections between sensory and motor parts of the cerebral cortex are modeled [10].

Fig. 1. Examples of maze patterns

Fig. 2. An agent observes a specific environment state

Applying a Neural Network Architecture with Spatio-Temporal Connections

2

59

Formal Problem

Consider a stochastic environment in which an agent can access the following set of actions: A = [L, R, U, D]. Corresponding spatial increments are Δ = [(0, −1); (0, 1); (−1, 0); (1, 0)]. Each maze cell is in one of the following states: S = [0, #, F ]. The first is for free cell, the second is for the wall and the last denotes finish point. When the agent makes a step it receives a reward: R = [SR, W R, F R]. Additionally we give the agent a reward for the finish point approach, we describe it in the next section. Our target is to achieve an aim with the minimum number of steps. An example of an emulated environment is attached below.

3

Neural Network Architecture

As already mentioned, the model is based on the neural network THSOM (Fig. 4), which has two types of inter-neural connections - spatial and temporal. The basic idea is that each input vector is an attractor for neurons, to which some of them are iteratively converge in the learning process, thus clusters of similar patterns are formed. This process is controlled by the standard for learning without a teacher method (for example, Kohonen self-organizing map algorithm). The BMU radius is calculated using the following equation:   −t (1) r = r0 ∗ exp r1

Fig. 3. An emulated maze. The red area is corresponding to the agent’s observation

60

D. Filin and A.I. Panov

but in our experiment it is zero so that only one neuron converges. In this way, the smallest number of necessary states is required, and there is no significant effect on the efficiency of the algorithm. The strength of neural connections adjusted according to the following equation:     dist2 −t ∗ t0 exp ∗ (¯ y−x ¯) (2) x ¯+ = s0 exp s1 t1 where x ¯ represents a neuron, y¯ is an input vector, dist - a distance between the vectors. The calculation of distance deserves special attention here, since in our problem it is a measure of similarity of the labyrinth patterns. Standard metrics are not suitable, since they are based on the difference of the corresponding components of the vector, whereas in the case below, the patterns (see Fig. 5) can be considered the same, as they correspond to the same available agent actions. However, in Fig. 6 patterns though similar in appearance, but differ in the set of actions available to the agent. That means, if position of the wall relative to the agent is not important for him, and he cares only about the form, then, having learned to walk upwards from a horizontal obstacle, he will do this always and there will be no difference from which side of the wall he is. In this regard, a metric was introduced that takes into account both the difference between the structure of the patterns and their location in the area of visibility. Then the distance is considered as: dist(P1 , P2 ) = α ∗ shif t + (1 − α) ∗ dif f

(3)

where dif f is the minimum difference between patterns when overlapping them (in units), shif t the number of shifts to achieve this difference.

Fig. 4. THSOM architecture

Applying a Neural Network Architecture with Spatio-Temporal Connections

61

Fig. 5. An example of similar patterns

Fig. 6. Patterns have the same structure, but in the first case upward action is impossible

The temporal component of the model consists of 4 connections between each pair of neurons that determine the probability distribution in the action space. That is, the stronger the connection, the more likely the action to be taken to move from one state to another. The calculation of temporary weights is according to the formula: t−1 t = min(max(wi,j,a + reward, 0), 1) wi,j,a

(4)

where i and j are environment states, a is an action, reward is an environment feedback. It consists of a constant reward/punishment for each step + reward for approaching to the final point. The last component was added to ensure that the agent is motivated to go exactly to the finish. The choice of an action at the moment of time t corresponds to the strongest outgoing link from the current active neuron. In addition to choosing the optimal action, the model also uses a strategy that plays an extremely important role in the initial stages. When the process is just beginning and the agent does not know anything about the patterns, it acts

62

D. Filin and A.I. Panov

for a while absolutely randomly, accumulating experience. In the general case, the probability of a random action is calculated as:     −t , 0.2 (5) P = max exp M1 3.1

Algorithm

1. Initialize time and space weights with small random values 2. For t=1, T do 2.1. Get an input signal from the sensors (get the current MxM block) 2.2. Find BMU 2.3. Update spatial weights 2.4. if t != 1 then update temporal weights 2.5. Remember current state as prev state 2.6. According to ε−greedy policy choose the best action 2.7. Do the chosen action, remember reward 2.8. If finish point is not achieved yet goto step 2 3. end for

4

Experiments

We tested our algorithm on various sequences of labyrinths 16 × 16, differing in complexity of structure. To begin with, it was decided to run the model 5 times on the same labyrinth (Fig. 3) to understand how well the learning goes. The number of steps is indicated in the following Table 1: Table 1. The number of steps for labyrinth 16 × 16. Iteration 1 Steps

2

3

4

5

119 84 49 33 42

As can be seen, the agent is learning and learning quite quickly. Taking into account the initial location of the agent and the end point location, the results can be considered comparable to human. However, in more complex labyrinths, the agent needs much more steps. For example, if the labyrinth contains dead ends, the agent is spending extra time in order to get out of it, which affects the total time. In this case, as in Fig. 7, the statistics are as follows (Table 2): Table 2. The number of steps for labyrinth with dead ends. Maze 1

2

3

4

5

Steps 784 702 522 284 238

Applying a Neural Network Architecture with Spatio-Temporal Connections

63

Fig. 7. Complex maze Table 3. The number of steps for a sequence of five different labyrinths. Iteration 1 Steps

2

3

4

5

529 758 329 557 205

Next, we tested the algorithm on a sequence of five different labyrinths several times and averaged the results (Table 3): It should be noted that all labyrinths were different relative to each other in terms of the complexity of the patterns and the location of the start and end points. In the general case, during the experiments, the following circumstances that influenced the operation time of the algorithm were elucidated: 1. Constants that control the speed of learning, clustering, as well as the number of neurons and the like. For example, the lower the intensity of clustering, what means, the lower the strength of spatial connections, the longer, but more qualitatively, the algorithm works. Low weights should be considered in the case of labyrinths with a complex structure of patterns, where individual cells affect the movements of the agent (for example, narrow passages, nonstandard wall bends). 2. The location of the start and end points. Since the agent is given a reward for approaching the finish line, he has his own priorities in the movements when passing obstacles. Therefore, if the agent is trained first on a sample of labyrinths, where the finish is at the bottom right, and then given the opportunity to walk through the labyrinth, the end point in which is located on the left from above, this can significantly affect the operation time. However,

64

D. Filin and A.I. Panov

even in this case, the agent will pass such a labyrinth faster than if he had seen it for the first time. 3. The initial location of the agent on the map. The extent to which the experience of the agent during the research will be varied, following the  − greedy rule. The more diverse obstacles that the agent will encounter during this period, what means, the more he learns, the more accurately he will move, based solely on his own experience.

5

Conclusion

In this paper, we present an original neural network architecture of an intelligent agent capable of learning how to build paths in various labyrinths. The architecture is based on the well-known THSOM model, with modifications for use in the learning with reinforcement problems. Based on the results of the conducted experiments, we can conclude that the learning process converges. In general, the agent not only always finds a way, but does it reasonably quickly. In the future, it is planned to more thoroughly analyze each of the model parameters in order to achieve the best time result. Also we are working on a method that will help the agent to deal with complicated situations as dead ends and fake ways and not get stuck in them for a long time. Acknowledgements. The reported study was supported by RFBR, research Projects No. 16-37-60055 and No. 15-07-06214.

References 1. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M.: Playing Atari with Deep Reinforcement Learning (2013) ˇ 2. Koutn´ık, J., Snorek, M.: Temporal Hebbian Self-Organizing Map for Sequences (2008) 3. Gupta, S., et al.: Cognitive Mapping and Planning for Visual Navigation. arXiv:1702.03920 4. Schrodt, F., et al.: Mario Becomes Cognitive. Top. Cogn. Sci. (2017). p. 131 5. Paxton, C., et al.: Combining Neural Networks and Tree Search for Task and Motion Planning in Challenging Environments (2017). arXiv:1703.07887 6. Broy, M.: Software engineering – from auxiliary to key technologies. In: Broy, M., Dener, E. (eds.) Software Pioneers, pp. 10–13. Springer, Heidelberg (2002) 7. Panov, A.I.: Behavior planning of intelligent agent with sign world model. Biol. Inspired Cogn. Archit. 19, 21–31 (2017) 8. Emelyanov, S., et al.: Multilayer cognitive architecture for UAV control. Cogn. Syst. Res. 39, 58–72 (2016) 9. Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: a survey. J. Artif. Intell. Res. 4, 237–285 (1996) 10. Chalita, M.A., Lis, D., Caverzasi, A.: Reinforcement learning in a bio-connectionist model based in the thalamo-cortical neural circuit. Biol. Inspired Cogn. Archit. 16, 45–63 (2016)

A Hands-on Laboratory Tutorial on Using CST to Build a Cognitive Architecture Ricardo R. Gudwin(B) DCA-FEEC-UNICAMP, Av. Albert Einstein 400, Campinas, SP 13083-852, Brazil [email protected] http://cst.fee.unicamp.br

Abstract. In this tutorial laboratory, we provide a step-by-step set of programming experiments illustrating the main foundations of the CST Cognitive Systems Toolkit in building a cognitive architecture to work as an artificial mind for controlling an NPC (non-player character) in a 3D virtual environment computer game. We start by understanding the sensors and actuators available in the NPC and how to control it inside the game. Then, we introduce the main foundations of CST: Codelets and Memories, and how they should be used to integrate a cognitive architecture. Then, we start building specific codelets and memories for a simple instance of the CST Reference Cognitive Architecture and start using it to control the NPC. The lab is a hands-on programming lab, using Java and Netbeans as language/tool. Keywords: Tutorial ment

1

· Cognitive architecture · CST · Virtual Environ-

Introduction

In this tutorial, we cover some core concepts used within CST (Paraense et al. 2016) in order to understand the basic building blocks required for constructing your own Cognitive Architecture using CST. Our emphasis is in understanding the core classes used for building an elementary kind of Cognitive Architecture. Every cognitive architecture built upon CST follows a set of guidelines, using just a few classes as support. More sophisticated architectures using additional CST classes might be learned further, following other available tutorials in our web site. Once you are able to understand these core concepts and how they are meant to be used using CST classes, you will be able to start building your first Cognitive Architecture using CST.

2

Basic Notions

The CST toolkit is designed using some basic notions coming from the work of Hofstadter and Mitchell (1994). In their historical “Copycat Architecture”, c Springer International Publishing AG 2018  A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6 9

66

R.R. Gudwin

Hofstadter & Mitchell defined their view of a cognitive architecture as based on the work of many micro-agents called codelets, which are responsible for all kinds of actions in the architecture. Codelets are small pieces of non-blocking code, each of them executing a well defined and simple task. The idea of a codelet is of a piece of code which ideally shall be executed continuously and cyclically, time after time, being responsible for the behavior of a system’s independent component running in parallel. The notion of codelet was introduced originally by Hofstadter and Mitchell (1994) and further enhanced by Franklin et al. (1998). The CST architecture is codelet oriented, since all main cognitive functions are implemented as codelets. This means that from a conceptual point of view, any CST-implemented system is a fully parallel asynchronous multi-agent system, where each agent is modeled by a codelet. CST’s codelets are implemented much in the same manner as in the LIDA cognitive architecture and largely correspond to the special-purpose processes described in Baar’s Global Workspace Theory (Baars 1988). Nevertheless, for the system to work, a kind of coordination must exist among codelets, forming coalitions, which by means of a coordinated interaction, are able to implement the cognitive functions ascribed to the architecture. This coordination constraint imposes special restrictions while implementing codelets in a serial computer. In a real parallel system, a codelet would simply be called in a loop, being responsible to implement the behavior of a parallel component. In a serial system like a computer, the overall system might have to share the CPU with its many components. In CST, we use multi-thread to implement this implicit parallelism. Each codelet runs in its own thread.

Fig. 1. The CST core

Figure 1 illustrates the general conception of a cognitive architecture, according to CST. A Mind is modeled in terms of a RawMemory , where many MemoryObjects are stored, and a Coderack , containing a set of Codelets.

A Hands-On Laboratory Tutorial

67

Codelets might have MemoryObjects as input, and also other MemoryObjects as outputs. Different Codelets might form Coalitions, by sharing the MemoryObjects they use as input and output.

Fig. 2. The CST overall architecture: codelets

A codelet has two main inputs (which are characterized as LI and GI in Fig. 1), a local input (LI) and a global input (GI). The local input is used for the codelet to get information from memory objects, which are available at the Raw Memory. The global input is used for the codelet to get information from the Global Workspace mechanism (Baars and Franklin 2009). The information coming from the Global Workspace is variable at each instant of time, and usually is related to a summary, an executive filter which selects the most relevant piece of information available in memory at each timestep. The two outputs of a codelet are a standard output, which is used to change or create new information in the Raw Memory, and the activation level, which indicates the relevance of the information provided at the output, and is used by the Global Workspace mechanism in order to select information to be destined to the global workspace. Using this Core, the CST toolkit provides different kinds of codelets to perform most of the cognitive functions available at a cognitive architecture, as indicated in Fig. 2. Also, memory objects are scattered among many different kinds of memories. The Raw Memory is so split into many different memory systems, which are used to store and access different kinds of knowledge. Using the available codelets, different cognitive architectures, using different strategies for perception, attention, learning, planning and behavior generation can be composed in order to perform the role necessary to address a specific control problem. These codelets are constructed according to different techniques in intelligent systems, like neural networks, fuzzy systems, evolutionary computation, rule-based systems, Bayesian networks, etc., which are integrated into a whole control and

68

R.R. Gudwin

monitoring system. The definition and choice of a particular cognitive architecture is constructed using a composition of different kinds of codelets, according to the control problem under analysis. Depending on the problem to be addressed, different strategies might be necessary or useful, depending on the problem constraints.

3

The Main Classes

The main core classes in CST are presented in the UML diagram of Fig. 3. Each of them corresponds to one of the concepts presented earlier.

Fig. 3. The main core classes in CST

In order to build a cognitive architecture, the designer might rely basically on those classes. It is possible to build a complete Cognitive Architecture simply by using these main core classes. Basically, the designer might have to create different classes inheriting from the Codelet class, where each codelet is responsible for a small fragment of processing. These codelets will operate on Memories (which can be MemoryObjects or MemoryContainers), affecting other Memories. The designer might either create an instance of the Mind class, or create a new class inheriting the Mind Class. By doing this, it is creating both a Coderack and a RawMemory . Using the Mind class (or a subclass), the designer might then create all the codelets and Memories for his/her cognitive architecture and insert them within the Mind . After that, the designer might call the start() method from Mind and the Cognitive Architecture should be operational. The source code of CST can be found at: https://github.com/CST-Group/cst

4

The WS3DApp Application

In this tutorial, we will be building WS3DApp, a very simple Cognitive Architecture to control an NPC (a robot) in the WS3D Virtual Environment. Basically, the WS3DApp describes the mind of a robot which needs to look for apples in a 3D environment, pick and eat them. More information about WS3D and

A Hands-On Laboratory Tutorial

69

WS3DApp, and the source code necessary to build these applications can be found at: WS3D: https://github.com/CST-Group/ws3d WS3DApp: https://github.com/CST-Group/WS3DApp This demo architecture is very simple. A sketch of the architecture is given in Fig. 4. We have two sensory codelets: Vision and InnerSense. These sensory codelets produce the VisionMO and the InnerSenseMO, the first a memory object containing a list of objects being seen by the creature in its line of sight, and the second a memory object containing self-referencial information, like the agent position, velocity, and other information. We have then two perceptual codelets processing such information. The first perceptual codelet is the AppleDetector codelet. This codelet is responsible for updating the KnownApplesMO memory object, which is a list containing all apples already seen, but not eaten. The ClosestAppleDetector codelet uses the information from both KnownApplesMO and InnerSenseMO to generate the ClosestAppleMO memory object, containing the information regarding the closest apple. Among the behavioral codelets, we have codelets responsible for three different possible behaviors: the GoToClosestApple codelet, the EatClosestApple codelet and the Forage codelet. The Forage codelet will act only when the list of KnowApplesMO is empty. It will create a random walk, by choosing a random destination and providing the LegsMO with the information with this location. The GoToClosestApple codelet will act only when there is something at the ClosestAppleMO, and the InnerSenseMO detects the Robot is far from the closest apple. It then generates information at the LegsMO in order to the Robot to proceed to the location of the closest apple. The EatClosestApple acts only when there is something at the ClosestAppleMO and the InnerSenseMO detects that the Robots position is in a reachable

Fig. 4. The demo cognitive architecture for the WS3D robot control

70

R.R. Gudwin

vicinity of the closest apple. It then order the HandsMO memory object, to give an order for the Robot to eat the apple. Finally, the motor codelets process the orders given by HandsMO and LegsMO. The HandsAction codelet check if it is there an order to eat an apple, and commands it to the Virtual Environment, and the LegsActionCodelet checks if there is an order to go to some position and sends it to the Virtual Environment. The classes tree for the WS3DApp application is given in Fig. 5, together with a snapshot of the WS3D Virtual Environment in action.

Fig. 5. The WS3DApp application and its classes tree

In the default package, we have the AgentMind class, where all the architecture is defined, the Environment class, responsible for populating the WS3D environment with a robot and some apples (and increasing new apples from time to time), and the ExperimentMain class, which is the application main class. Then we have a hierarchy of codelets packages, where the different codelets are defined, a memory package with auxiliary classes used within MemoryObjects and a support package, with supporting classes (in this case, the MindView class, which prints out what is within each MemoryObject in the architecture.

5

Conclusion

The tutorial requires the reader to study the WS3DApp source code and run the application in order to understand its main mechanisms. By concluding the tutorial, you have studied the Core basics of CST. Most cognitive architectures built with the aid of CST will require just these features shown here. At the CST web site more sophisticated mechanisms are described and made available as advanced tutorials.

A Hands-On Laboratory Tutorial

71

Acknowledgments. The author thanks CEPID/BRAINN for supporting this research (Proc. FAPESP 2013/07559-3).

References Baars, B.J.: A Cognitive Theory of Consciousness. Cambridge University Press, London (1988) Baars, B.J., Franklin, S.: Consciousness is computational: the lida model of global workspace theory. Int. J. Mach. Conscious. 1(01), 23–32 (2009) Franklin, S., Kelemen, A., McCauley, L.: Ida: A cognitive agent architecture. In: 1998 IEEE International Conference on Systems, Man, and Cybernetics, vol. 3, pp. 2646–2651. IEEE (1998) Hofstadter, D., Mitchell, M.: The Copycat Project: A Model of Mental Fluidity and Analogy-Making. BasicBooks, New York (1994) Paraense, A.L., Raizer, K., de Paula, S.M., Rohmer, E., Gudwin, R.R.: The cognitive systems toolkit and the cst reference cognitive architecture. Biol. Inspir. Cogn. Archit. 17, 32–48 (2016)

Robot Navigation Based on an Artificial Somatosensorial System Ignazio Infantino(B) , Adriano Manfr´e, and Umberto Maniscalco Institute of High Performance and Networking (ICAR), National Research Council (CNR), Via Ugo La Malfa,153, 90146 Palermo, Italy {ignazio.infantino,adriano.manfre,umberto.maniscalco}@cnr.it

Abstract. An artificial somatosensory system processes robot’s perceptions by mean of suitable soft sensors. The robot moves in a real and complex environment, and the physical sensing of it causes a positive or negative reaction. A global wellness function drives the robot’s movements and constitutes a basis to compute the motivation of a cognitive architecture. The paper presents preliminary experimentations and explains the influence of the parameters on the robot behavior and personality. Pepper freely moves in an office environment searching for people to engage. The robot searches for a safe path, avoiding obstacles and aiming to explore a significant part of a known space by an approximative map stored in its long term memory (LTM). The short-term memory (STM) stores somatosensory values related to perceptions considered relevant for the navigation task. The collection of previous navigation experiences allows the robot to memorize on the map places that have positive (or negative) effects on robot’s wellness state. The robot could reach the places labeled as negative, but it needs some positive counter effects to contrast its reluctance.

1

Introduction

A great challenge for robotics is to introduce social robots in everyday life by assuring safety and robust sharing of human spaces. We have numerous examples [1], and we find robots in homes, hospitals, care centers, shops, and also in cruise ships. The capabilities of social interaction are growing, and thanks to more sophisticated sensors and algorithms, robots can robustly detect faces, expressions, postures, speech, sounds and other environment entities. But their autonomous navigation is still difficult because human spaces are very complex and change over the time. Usually, the research in this field considers only the physical (geometric) space, investigating how to find optimal paths that avoid obstacles and persons, to reach given locations and places. In this case, the odometry, the lasers, the infrared sensors, and sonars could be used to have a three-dimensional model of the environment with high precision and high c Springer International Publishing AG 2018  A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6 10

Robot Navigation Based on an Artificial Somatosensorial System

73

detail level. Neurobiologically-inspired approaches to the navigation task consider hypothalamus functionalities and reactive mechanisms. For example, by associating an action with a place, and adding topological information [2], the robot can build a cognitive navigation map both to simulate and to plane movements. The robot’s internal sensing (see for example [3]) could influence the planning of navigation task, but in our idea, it has a relevant role if considered with other sensings. If we take inspiration from both human behavior and cognitive processing, the robot has to perceive the environment by an integrated approach, considering sensings, demands, feelings, aims, motivations, movements and the actions. Especially when we talk about a social robot, the navigation task is not only a simple planning of movements, but it is a process that collects environment perceptions with different abstraction levels. It is necessary to switch from the physical level to the cognitive one, introducing emotions, motivations [4], and so on (see for example the approach to the socially aware navigation explained in [5]). The final goal is to define a robust human-aware robot navigation [6], considering comfort, naturalness, sociability from the human point of view. In our idea, the social robot could have similar capabilities arising from a suitable cognitive architecture based on an artificial somatosensory system [7].

2

The Artificial Somatosensory System

The somatic sensations are bodily sensations, such as touch, pain, temperature and proprioception (also involving position and movement). Such sensations play a crucial role in preserving the body from damage or accident and have a strong influence on the behavior. Nowadays, robots have a lot of sensors measuring several internal and external variables such as motors temperatures or currents, battery conditions, the presence of objects and distances, bumps, touches and so on. A challenging task is to make available to high-level cognitive modules controlling the robot the states arising from these measures. In our approach, we adopt the soft sensor paradigm [8] to get sensations and emotions from raw measures. The present work provides new improvements with respect the system presented in [9] and by proposing a suitable cognitive framework capable to include all the artificial somatosensory modules described in detail in [7]. Figure 1 shows the cognitive architecture that integrates the somatosensory system and allows the robot to navigate in the environment taking into account its perceptive feelings. The somatosensory system includes both the internal sensing and external perception of relevant environment entities. The joints temperatures, joints currents, battery level, and so on, have a direct impact on Physiological demands and Certainty (i.e. the capability to accomplish a task or an action). The external sensing of the environment deals with the detection of faces, sounds, objects, but considering only their impact on robot comfort/discomfort. For example, the detection of friendly faces, or the perception of human voice or music, have positive effects on robot comfort, and could have positive influences on its mood; the absence of human faces, or high-intensity sounds, or the presence of an unknown person in private robot’s space have negative effects. We designed suitable soft

74

I. Infantino et al.

sensors to build a somatosensory system and to provide the robot with the capability to have a complex environment perception. The perceptive signals used for it, are also involved in detecting and processing complex external events as human actions, social signs, and so on. In the present work, the focus of attention is on the perception of the environment and its influences on a fundamental task for the robot: the autonomous navigation and exploration of real unstructured spaces.

Fig. 1. The cognitive architecture (on the left). On the right, the somatosensory data for an elementary cell of the map stored in memory.

The cognitive architecture allows the robot to process the somatosensory data to perform a navigation task driven by complex aims involving its feelings and moods. The robot could show a particular behavioral trait (i.e. a personal character) during a navigation task depending the somatosensory features to accomplish (e.g. solitary, sociable, grumpy, and so on). The robot stores a multilayer map of the known environment in its Long Term Memory (LTM). A quadtree decomposition approach allows the system to have a compact representation with different size of the areas depending on the richness of the information that they have associated. The perceptions of the space around the robot, a circular area of 90 cm of diameter, constitutes the relevant data stored in the working memory (Short Term Memory, STM), as reported in Fig. 1. For example, if we refer only to data arising from lasers, the robot has the local map of free spaces and distances of obstacles around it. A particle filter approach allows the robot to localize itself in the map, to move or explore the environment following safety paths. The use of various perceptive devices allows the robot to navigate the environment pursuing different aims. When it is looking for social interactions, the robot tries to reach places where human faces and people has been detected; if it is looking for quite places, the goal is to avoid noisy areas of the map.

Robot Navigation Based on an Artificial Somatosensorial System

3

75

Experiments

Preliminary experiments have been conducted with humanoid robotic platform Pepper1 . The robot has to navigate in an office space constituted by a corridor and 18 rooms, where more than 30 persons work. The environment is not structured, and people can freely move during robot operations. The perceived environment has to be dynamic, and navigation has to be robust to the presence of new obstacles, the possible modification of the furniture, or the state of the doors (opened, closed, or partially opened). A cell detection function uses the robot odometry parameters to establish in which cell the robot is at the time t. At this geometrical layer are linked several content layers whose information are achieved by a set of soft sensors that constitute the artificial somatosensory system. During a robot navigation session, whether it be in exploration mode or a planning mode, the artificial somatosensory system of the robot receive data from the soft sensors. At the same time, the robot’s odometry parameters are also recorded. Thus, each external stimulus perceived and translated in sensation by the soft sensors can be georeferenced in the global frame and then assigned to a particular somatosensory cell (see Fig. 1 on the right). The robot learns the map of the environment through a random exploration based on laser readings and with a simple procedure of obstacle avoidance. In particular, for creating the map, we use the package named GMapping of ROS2 , i.e. a laser-based procedure performing SLAM (Simultaneous Localization and Mapping). Despite the fact that the odometry of the robot is rough, the automatic method for correction of the library allows the robot to have a plausible map. The low accuracy of the map does not constitute a problem given that the navigation and positioning of the robot will still be guided employing a Montecarlo approach (by AMCL package of ROS). From the map obtained by the laser, a skeletoning procedure determines a path that covers the whole navigable space (see Fig. 2 on the left), and which constitutes the basic knowledge of the robot to move quickly from one area to another. A graph represents the skeleton, with relevant points inserted manually. In this way, the navigation can take place at two levels of abstraction. At the higher level, the somatosensory map is used by the planner to indicate a target cell to be reached. The point of the graph that is closest to this cell represents the low-level navigation target. The low-level navigation task allows the robot to move without bumps and following the shortest path made by a sequence of relevant points. During the low-level navigation, the robot uses laser readings to have a current estimate of its location regarding the map, and it tries to reach the current target point by correcting its trajectory at regular intervals. When the robot reaches an intermediate point, it goes to the next point of the graph until reaching the goal (by mean of a potential function [10,11]). In the following, there are two examples of what can happen during the navigation when occurs an event relevant to the somatosensory level.

1 2

http://www.ald.softbankrobotics.com. http://wiki.ros.org/gmapping.

76

I. Infantino et al.

Robot looking for quite places to rest. The robot has a medium-low level of battery power, but it still does not need a refill and is not a particular task to perform. The planner indicates to reach a quiet place where you can stop in stand-by mode waiting for new events. A list of quiet cells is the results of a query on the database, where somatosensory stimulations have been absent or minimal. The robot follows the path on the graph after choosing the target cell as the closest to the current (estimated) position. If the robot reaches the goal without receiving different goals, the standby mode is activated in the desired place of the map. If the area is not reachable (for example because a door is closed), the target cell changes and the robot continues towards another quiet destination. Robot looking for social interactions. Suppose that the robot has not experienced any social interactions for some time. The Affiliation demand (see Architecture figure) causes a lowering of its motivation. Then the planner of the cognitive system tries to increase the motivation by triggering the people search process. A suitable query to the somatosensory database returns the possible cell (or more than one) of the map that has recorded the highest average number of detected human faces (see Fig. 2). The people search process aims to find at least one face but does not give importance to the identity or a particular location on the map. The target cell position is approximated by the closest position between the points that constitute the graph of the low-level navigation. While navigating, the robot updates information of the somatosensory map regarding the visited cells. If the robot finds a face before the target cell, however, its goal is achieved and notifies the planner to have been successful. The planner then can indicate other tasks (e.g. human engagement, face tracking, speech interaction, and so on). If the robot reaches the target cell without having found a person, it notifies the planner of a task failure. In this case, the planner could indicate a new cell that has not yet been visited owning a history of occurrences of the event detected face or people detected.

Fig. 2. On the left, the graph of relevant points used for the low-level navigation task. On the right, the map shows cells with high occurencies of detected faces and people. If robot wish find a human to interact with, it will navigate to location A that has the maximum score. The locations indicated as B, C and D can be other possible target to reach to accomplish the socialization task.

Robot Navigation Based on an Artificial Somatosensorial System

4

77

Conclusions and Future Work

The combination of a cognitive architecture and an artificial somatosensory system allow the robot to enrich its experience of the environment during the navigation. The discussed work offers an efficacy solution to link high-level goals and standard navigation capability of the autonomous robot. We avoid to have precise scene reconstruction, but the navigation is safety and efficient, and robust to environment changes. The somatosensory system provides the robot of a complete set of basic and complex perception useful to planning and reasoning module. In the future work, we will explore the behavior the robot performing a navigation aiming to exploit its social interaction capability, externalizing also its feelings about its internal and external state.

References 1. Broadbent, E.: Interactions with robots: The truths we reveal about ourselves. Ann. Rev. Psychol. 68, 627652 (2017) 2. Cuperlier, N., Quoy, M., Gaussier, P.: Neurobiologically inspired mobile robot navigation and planning. Front. Neurorobotics 1, 3 (2007) 3. Arkin, R.C.: Dynamic replanning for a mobile robot based on internal sensing. In: 1989 Proceedings of the IEEE International Conference on Robotics and Automation, pp. 1416–1421. IEEE (1989) 4. Hasson, C., Boucenna, S., Gaussier, P., Hafemeister, L.: Using emotional interactions for visual navigation task learning. In: International Conference on Kansei Engineering and Emotion Research KEER2010, pp. 1578–1587 (2010) 5. Mead, R., Mataric, M.J.: Autonomous humanrobot proxemics: socially aware navigation based on interaction potential. Auton. Robots 113 (2016) 6. Kruse, T., Pandey, A.K., Alami, R., Kirsch, A.: Human-aware robot navigation: A survey. Robot. Auton. Syst. 61(12), 1726–1743 (2013) 7. Augello, A., Infantino, I., Maniscalco, U., Pilato, G., Vella, F.: The effects of soft somatosensory system on the execution of robotic tasks. In: IEEE International Conference on Robotic Computing 2017, Taiwan, p. 17. IEEE (2017) 8. Maniscalco, U., Pilato, G., Vella, F.: Soft sensor network for environmental monitoring. In: Intelligent Interactive Multimedia Systems and Services 2016, pp. 705–714 (2016) 9. Augello, A., Maniscalco, U., Pilato, G., Vella, F.: Disaster prevention virtual advisors through soft sensor paradigm. In: Intelligent Interactive Multimedia Systems and Services 2016, pp. 619–627 (2016) 10. Koditschek, D.: Exact robot navigation by means of potential functions: Some topological considerations. In: 1987 Proceedings of the IEEE International Conference on Robotics and Automation, vol. 4, p. 16 (1987) 11. Masutani, Y., Miyazaki, F., Arimoto, S.: Sensory feedback control for space manipulators. In: 1989 Proceedings of the International Conference on Robotics and Automation, vol. 3, pp. 1346–1351 (1989)

About Realization of Aggressive Behavior Model in Group Robotics Irina Karpova1,2 ✉ (

1

)

National Research University Higher School of Economics, Moscow, Russia [email protected] 2 National Research Center «Kurchatov Institute», Moscow, Russia

Abstract. One of the actively developing approaches of group robotics systems creation is the use of social behavior models. Aggressive behavior is one of the underlying mechanisms forming social behavior. In this paper, the application of aggressive behavior concepts is considered by analogy with animal aggressive behavior that can be used for solving tasks of group robotics. As a role model, an ant – a true social insect – is proposed. It was shown that in aggressive behavior of ants, the numerical factor and imitative behavior play an important role. Agent’s aggressive behavior model depending on accumulated aggression and the number of other nearby agents is proposed. The results of computer experi‐ ments for territory defense tasks are presented. The results show that aggression is a stabilizing factor for an approximately equal number of agents in different groups. By an increase in group size, aggression becomes a way of capturing foreign territory. Keywords: Group robotics · Social behavior models · Aggressive behavior · Territory defense task

1

Introduction

One of the actively developing approaches to the creation of group robotics systems is the implementation of social behavior models as unified methodological bases [1–3]. The main idea of this approach is to propose a set of methods and mechanisms of their realization based on animal social behavior models. We suppose that these biologically inspired methods will allow us to solve a wide range of tasks in the field of group robotics. Ethologists believe that agonistic or aggressive behavior is one of the main social behavior mechanisms [4]. In the framework of evolutionary theory, aggressive behavior is understood to encourage survival of the fittest, disperse populations, aid adaptation to threatening environments, and generally improve the probability of individual and species survival [5]. Aggressive behavior is an integral part of various social behavior models; therefore, in the present study, the implementation of this mechanism is essential to create higher level models. Aggression is seen as conflict resolution and as a factor for territory divi‐ sion between robot groups. © Springer International Publishing AG 2018 A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6_11

About Realization of Aggressive Behavior Model in Group Robotics

2

79

Aggressive Behavior

Many ethologists consider that aggression is one of the main factors determining the formation of community [4, 6, 7] in particular: (1) the maintenance of population spatial structure; (2) mating behavior; (3) the care of offspring; (4) the maintenance of a domi‐ nance hierarchy of cooperating individuals in a social; (5) the maintenance of group (family) homeostasis. Generally, aggression contributes to the survival of the species. We need to explore the basic mechanisms of aggressive behavior, because we will use it for realization of some social behavior models in group robotics. Agonistic behavior consists of threats, aggression and submission. The threats appear as fixed action patterns [4], which are highly stereotyped models of behavior that are characteristic of a particular species. Aggression usually occurs when a competitor appears. We will consider only aggression against the competitor which is individual to the species because intra-species aggression plays a key role in complex social behav‐ iors. In most cases, contests are settled through the use of non-injurious aggressive behaviors such as demonstration and trials of strength. Actual fighting is rare because of the risk of injury to both participants. Various factors limit the escalation of aggression such as elimination of the causes (competitor, interference, etc.) or demonstration of submission by the other individual. For solving problems of group robotics, researchers often use ants for “role models.” Also, many researchers study features of social behavior with ants as representatives of eusocial (true social) insects. Therefore, ants are a convenient object for study of aggres‐ sion. Aggressive behavior in ants and the role of aggression is examined in detail, for example, in [8]. With the help of aggression, ants defend their territory from ants of another nest (and other species of ants). In this case, the number of ants plays some role. When two ants from different nests encounter each other on the aft area where friends are absent, they prefer to disperse peacefully. But if any forager is carried away and gets too close to a foreign nest, where there are many opponent ants, the forager will be attacked and must flee. Thus, aggression is involved in process of forming social behavior. In a number of works, aggressive behavior is used to find solutions to some problems in robotics (for example, [9, 10]). Also, the term “aggression” is sometimes applied in multi-agent systems [11]. Unfortunately, the authors mentioned in the above works using the concept of aggression and sometimes even referring to the works of ethologists, preferring a formal approach. We are interested in a constructive, biologically based model of aggressive behavior that can be applied to the solution of specific problems in group robotics.

3

Aggressive Behavior Model

We simplify the real situations that take place in nature and do not take into account the features of individuals. Let’s assume aggression occurs only when the competitor appears. Aggression between groups is determined partly by willingness to fight, which

80

I. Karpova

depends on a number of factors including numerical advantage and distance from home territories [12]. The current aggression level of individual A is defined by three param‐ eters: A = (C, Eob , Dother ),

(1)

where C – is “actual aggression,” i.e., the aggressiveness accumulated to this point in time; Eob – the presence of the object of aggression; and Nother – the number of other individuals of the species. The emergence of a competitor definitely leads to the mani‐ festation of aggressive behavior (the sensitivity threshold equals zero). The observation of the Formicidae ant behavior shows that an individual is more likely to become aggressive if other group members are nearby. On the border territories of the two ant species, the density of individuals is higher than in those areas where the territory is not contiguous with the territories of other species. According to Dlusskiy [13], ants can quickly come together to reflect “neighbors” at the slightest danger. Thus, in ants, actual aggressiveness depends on a distance from home territory. Let’s assume that the actual aggression C is inversely proportional to the distance R from its “nest”: C = 1/R. The model should include conflict resolution rules based on a comparison of aggres‐ sion levels. After the collision, the less aggressive ant runs away and the more aggressive ant rushes in pursuit or stays in place. Also, the proximity of other group members plays a big role for ants. Therefore, the following rules of conflict resolution are offered. When the individual detects a competitor, it summarizes the aggression levels of ants of its own species among neighbors (individuals are considered to be neighbors if distance between the individuals is less than some value X):

Aother =

m ∑ j=1

Cj , A = C + Aother ,

(2)

where Cj are actual aggression levels of neighbors; m – the number of the neighbors; and A – current aggression level of the individual. Similarly, the total aggression level of the “foreign” individuals (An) is calculated. The individual wins the conflict with ( ) probability p = A∕ A + An and loses with probability (1 − p).

4

Simulation Experiments

As a model task, a territory defense task is proposed. In collective robotics, this task is usually reduced to the problem of territory division or patrolling. The problem of terri‐ tory division is considered only between individual robots, but not between groups (for example, in [14]). The solution of the patrolling task is to optimize the route which involves robots [15]. Under the territory protection task, we see the problem solved by animals through aggressive behavior: they banish individuals of other families (nests) from their territory to maintain the spatial structure of the population [7]. For an aggressive behavior

About Realization of Aggressive Behavior Model in Group Robotics

81

demonstration using the proposed model for territory protection, place two “nests” (anthills) where individuals of a certain group (type) “live” on the ground. They are able to distinguish their own and “enemies,” and only show aggression towards individuals from another anthill. A series of experiments of two types were carried out. The experiments were conducted with the help of the framework of the ROS-based modeling system. In the first type of experiments, agents didn’t show aggression and moved freely on their own and someone else’s territory. The ratio of the number of agents from different nests (n1:n2) varied from 1:1 to 1:6; ten experiments at 300 cycles of simulation were conducted for each ratio. The results for the series without aggression are shown in Fig. 1. As a test indicator, the total time spent by agents of each group on foreign territory was used.

Fig. 1. The results for no aggression behavior series: (a) total time spent by agents of both groups on foreign territory (t1 – Group 1, t2 – Group 2); (b) ratio t2/(t1 + t2)

The first series (no aggression) shows apparent results: in the absence of aggression, the total time spent by individuals of each group on the “foreign” territory, in fact, is directly proportional to the number of individuals in the group. In the second series, individuals show aggression toward “outsiders” (see Fig. 2).

Fig. 2. Simulation results – series with aggression: (a) total time spent by individuals of both groups on foreign territory (t1 – Group 1, t2 – Group 2); (b) ratio t2/(t1 + t2)

82

I. Karpova

In the second series, rules of aggressive behavior were consistent to the proposed model, and the aggression level was calculated when individuals detected individuals from the other group. The losing individual immediately returned to his anthill, elimi‐ nating the cause of aggression. Other conditions were similar to the conditions of the first series: the ratio of the individuals number from different nests (n1:n2) was also changed from 1:1 to 1:6 and for each ratio 10 experiments at 300 cycles of simulation were conducted. In the results of the second series, emergence of aggressive behavior for equal groups sharply (about 2–2.5 times) reduces the total time spent by individuals of each group on “foreign” territory. Thus, the experimental results confirm that aggression is a stabilizing factor allowing the group to defend its territory. Besides, the increase in number of one of the groups serves to increase the time that individuals of this dominant group spend on “foreign” territory, as well as reduce the number of individuals from the subordinate group that penetrate into “foreign” territory. Therefore, we can assume that aggression is also a factor of foreign territory “capture” by the larger group. This is clearly seen from Fig. 3 which shows graphs of the series with aggression and without.

Fig. 3. Comparison of simulation results – “aggression” (aggr) and “no aggression” (no aggr): (a) total time spent by individuals of both groups on foreign territory; (b) the ratio of time spent by individuals of both groups on foreign territory

It is interesting that the aggressive factor is most significant in the range from 1:1.5 to 1:2. Simulation results show that the time which the dominant group spends on foreign territory increases unevenly. Maximum growth is achieved by the numerical superiority of one group over another from 1.5 to 2 times. This suggests that at this ratio, aggres‐ siveness makes the greatest contribution to the redistribution of territory in favor of larger groups. With a further increase in number of any group, this group just “pushes” the other in the mass; aggressiveness has less influence on the situation.

About Realization of Aggressive Behavior Model in Group Robotics

5

83

Conclusion

In this paper, a territory behavior model based on the proposed aggressive behavior model was implemented. Experiments confirmed that for equal groups of agents, aggres‐ sion is the stability factor that allows each group to defend its territory. If ratio of agents number in groups changes, aggression becomes a way of seizing foreign territory and leads to a redistribution of the territory in favor of larger groups. This mechanism of competition (conflict resolution) should be the same for different cases and should not depend on the resource. Further, we intend to check proposed model to solve other tasks, in particular a foraging task to resolve a conflict for food. And when the number of individuals increases, they often collide with each other and spend time to disperse. This conflict can also be resolved using aggression. Acknowledgements. The project was partially supported by RSF 16-11-00018 grant (review and aggressive behavior model), and RFBR 15-07-07483 grant (simulation experiments).

References 1. Karpov, V.: Modeli social’nogo povedeniya v gruppovoy robototekhnike. (Social behavior models in group robotics). Large-scale Syst. Control 59, 165–232 (2016) 2. Karpova, I.: Psevdoanalogovaya kommunikaciya v gruppe robotov (Pseudo-analog communication in robot group). Mekhatronika, Avtomatizatsiya, Upravlenie 17, 94–101 (2016) 3. Kulinich, A.: A model of agents (robots) command behavior: the cognitive approach. Autom. Remote Control 77(3), 510–522 (2016) 4. Tinbergen, N.: Social Behavior of Animals. Methuen, London (1953) 5. Olivier, B., Young, L.J.: Animal models of aggression. In: Neuropsychopharmacology: The Fifth Generation of Progress, pp. 1699–1708 (2002) 6. Lorenz, K.: On Aggression. Routledge, London (2002) 7. Shilov, I.: Population homeostasis. Zool. Zhurnal 81(9), 1029–1047 (2002) 8. Zakharov, A.: Muravey, sem’ya, koloniya. (Ant, family, colony). Nauka, Moscow (1978) 9. Brown, S., Zuluaga, M., Zhang, Y., Vaughan, R.: Rational aggressive behaviour reduces interference in a mobile robot team. In: 2005 International Conference on Advanced Robotics, ICAR 2005, pp. 741–748 (2005) 10. Zhang, Y., Vaughan, R.: Ganging up: team-based aggression expands the population/ performance envelope in a multi-robot system. In: IEEE International Conference on Robotics and Automation, pp. 589–594 (2006) 11. Scheutz, M., Schermerhorn, P.: The more radical, the better: investigating the utility of aggression in the competition among different agent kinds. From Animals to Animats 8: Proceedings of the 8th International Conference on Simulation of Adaptive Behavior, pp. 445–454 (2004) 12. Frizzi, F., Ciofi, C., Dapporto, L., et al.: The rules of aggression: how genetic, chemical and spatial factors affect intercolony fights in a dominant species, the mediterranean acrobat ant crematogaster scutellaris. PLoS ONE 10(10), 1–15 (2015)

84

I. Karpova

13. Dlusskiy, M.: Muravi roda Formika. (Family Formica ants). Nauka, Moscow (1967) 14. Gunady, M.K., Gomaa, W., Takeuchi, I.: Aggregate reinforcement learning for multi-agent territory division: the hide-and-seek game. Eng. Appl. Artif. Intell. 34, 122–136 (2014) 15. Galceran, E., Carreras, M.: A survey on coverage path planning for robotics. Rob. Auton. Syst. 51(12), 1258–1276 (2013)

Human Brain Structural Organization in Healthy Volunteers and Patients with Schizophrenia ( ) Sergey Kartashov1,3 ✉ , Vadim Ushakov1,3, Alexandra Maslennikova2, 1 Alexander Sboev , Anton Selivanov1, Ivan Moloshnikov1, and Boris Velichkovsky1,4

1

NRC Kurchatov Institute, Moscow, Russia [email protected] 2 Institute of Higher Nervous Activity and Neurophysiology of RAS, Moscow, Russia 3 National Research Nuclear University MEPhI (Moscow Engineering Physics Institute), Moscow, Russia 4 Technische Universität Dresden, Dresden, Germany

Abstract. The purpose of this work was to study and to compare the structural features of the human brain in two groups of people: healthy volunteers and patients with schizophrenia. According to the data of diffusion magnetic reso‐ nance imaging (dMRT), tractography pathways that describe the direction of fibers growth of the white matter of the human brain were reconstructed. Analysis of these paths made it possible to construct maps of the connectivity of all sections of the prepared brain to each other for each subject. With the help of graph theory, so-called rich-club areas were found for each of two groups, that, according to many papers, are the key centers of the brain in the transmission and exchange of information between all areas of the human brain. Keywords: Diffusion · dMRI · Structural connections · Rich-club · Graph theory

1

Introduction

The study of the neurophysiological mechanisms of the rich-club structures functioning of the human brain is closely related to the problems solution of functional systems that provide perception, controlling functions and consciousness. The key solution of these tasks is to determine the architectures of effective connections of the human brain in solving cognitive tasks and at rest, identifying key control centers for rich-club areas for efficient and structural connectivity. Rich-club areas of the brain play an important role in the organization of interaction of different brain areas with each other, as well as providing and maintaining various cognitive states. This paper presents the results of rich-club areas comparison for healthy volunteers and patients with schizophrenia based on diffusion MRI data.

© Springer International Publishing AG 2018 A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6_12

86

2

S. Kartashov et al.

Materials and Methods

Six patients with schizophrenia F 20.0 in ICD-10 (hallucinatory-paranoid syndrome) with the first attack, not receiving neuroleptic therapy, without CCT, aged 20 to 40 years took part in the experiment. The expression of positive symptoms for PANSS was 97.1 ± 3.1. As a control group, subjects (n = 5) without mental disorders, neurological disorders, aged 18 to 35 years were examined. The experiment was partially conducted on the MRI Magnetom Verio 3T residing at the base of Resource Center of Nuclear Physical Methods of Research of the NRC “Kurchatov Institute” (healthy volunteers only). Data on patients with schizophrenia were provided by IHNAN RAS. Experi‐ mental data were obtained using the standard sequence TR = 1900 ms, TE = 2.21 ms and voxel size 1 × 1 × 1 mm3 for structural MRI. Diffusion images were acquired using a diffusion sequence with TE = 101 ms, and TR = 13700 ms. A DTI diffusion scheme was used, and a total of 64 diffusion sampling operations were acquired. The b-value was 1500 s/mm2. The in-plane resolution was 2 mm. The slice thickness was 2 mm. The diffusion tensor was calculated. Ex-ample of white matter tracts, reconstructed from diffusion MRI data see on Fig. 1

Fig. 1. Example of white matter tracts, reconstructed from diffusion MRI data.

During the whole experiment participants were laying motionless and their eyes were closed. No additional stimulation was performed. The analysis of diffusion data was conducted using DSI Studio (http://dsistudio.labsolver.org). Images were normalized to template images (T1-template for structural and EPI template for functional) with 2 × 2 × 2 mm3 voxel size in MNI (Montreal neurological institute) coordinate system. As the result, connectivity matrices were acquired. Example of such matrix is shown at Fig. 2. Each matrix contains positive integers in range from 0 to 10000, that describe the number of white matter tracts between two regions of interests. Connectivity matrices

Human Brain Structural Organization

87

Fig. 2. Example of connectivity matrix acquired using dsi_studio softwere.

were converted to graph format for data analysis purposes. Graph nodes represent brain areas and graph connections represent connections between brain areas. For each node of the graph we compute the following features: • degree – node connections number; • strengths – sum of the node connections weights; • module number – number of the node’s module. Module structure is a subdivision of the network into nonoverlapping groups of nodes in a way that maximizes the number of within-group edges, and minimizes the number of between-group edges. Module structure was obtained with Louvain Method for community detection [1] implemented in brain connectivity toolbox; • hub type – look at the section “Hub Structure Analysis”; • rich-club coefficient – look at the section “Rich-Club Analysis”. • We used python 2.7 for algorithms implementation with the following software libraries: • brain connectivity toolbox for python to compute various graph metrics and coeffi‐ cients; • networkx for graph processing; • plotly for the results visualisation. Hub type of the node shows the prevailing side in ratio of between-modular and intra-modular connections of the node. There are two types of the node hub – provincial hubs and connector hubs. Provincial hub is a node that has more intra-modular connec‐ tions. Provincial hubs are important for connectivity inside the module. Connector hub

88

S. Kartashov et al.

is a node that has more between-modular connections. Connector hubs are important for connectivity between the high-density modules and connectivity of the whole graph. For that propose we used hub selection algorithm described in [2]. For rich-club analysis we used method described in [3]. There are two types of richclub coefficient: unweighted and weighted. Unweighted rich-club coefficient uses brain graph nodes degree. Weighted rich-club coefficient uses brain graph nodes strength. The computation algorithm of the rich-club coefficient on a degree level k is the following: • remove from the graph all nodes with degree below k; • compute unweighted rich-club coefficient as the ratio of remain connection number and maximum possible connections number between the remaining nodes; • compute weighted rich-club coefficient as the ratio between all connections weight sum and the sum of the top weighted connections of the initial graph. For the second sum connections number is equal to the connections number of the first sum. After that, rich-club coefficient was normalized to a set of a comparable random graphs with the same size and connection weights distribution to reduce the effect of the nodes interconnection with the higher degree. Statistical significance of the rich-club organization in current graph assessed only if rich-club coefficient of the degree level k is higher than random rich-club coefficient of this level on a value more than one-sided p-value. The 3D-graphs were built with plotly software library. See example at Fig. 3.

Fig. 3. Example of 3D-graph calculated from connectivity. Red nodes are in the weighted richclub versus yellow ones, which aren’t. Big nodes are connector hubs, while small nodes are provincial.

3

Results

Procedures for data processing and calculation of rich-club regions were carried out for healthy volunteers and for patients with schizophrenia. According to the data of the trac‐ tography, connectivity matrices were reconstructed. Their analysis showed for patients

Human Brain Structural Organization

89

with schizophrenia the absence of rich-club sites on weighted rich-club coefficients. According to unweighted rich-club coefficients from six patients, only three of them had a common zone - Thalamus_R, the remaining rich-club zones varied and their number is very low. For healthy subjects analysis of connectivity matrices of unweighted rich-club coefficients showed the following rich-club areas: Angular_R, Frontal_Inf_Oper_L, Frontal_Inf_Oper_R, Frontal_Inf_Tri_R, Rolandic_Oper_L, SupraMarginal_L, Supra‐ Marginal_R, Temporal_Mid_R, Vermis_9, Amygdala_R, Angular_L, Cerebelum_10_L, Cerebelum_10_R, Cerebelum_7b_R, Frontal_Med_Orb_L, Heschl_L, Occipital_Inf_R, Parietal_Inf_L, Rolandic_Oper_R, Temporal_Pole_Sup_L, Temporal_Sup_L, Temporal_ Sup_R, Vermis_7, for weighted coefficients - Cingulum_Ant_L, Cingulum_Mid_L, Para‐ central_Lobule_R. Perspectively all these areas are supposed to be used in calculation of functional and effective connectivity matrices as key nodes [4, 5].

4

Disscussion

Comparative analysis of two groups of volunteers showed significant differences in the presence of rich-club regions in patients with schizophrenia. This may approve that schizophrenia is associated with structural and functional changes in the gray matter of the brain. These changes results in degradation of the connectivity of different brain regions. In patients with schizophrenia decrease in the volume of gray matter and viola‐ tion of the integrity of the white matter are observed. The functional methods of visu‐ alization also show abnormal neuronal activity in various cognitive tasks associated with the evaluation of short-term memory, long-term memory, decision-making and emotion processing. And these changes are observed in all phases of the disorder [6, 7]. Acknowledgments. This work is supported by the Russian Science Foundation, grant RScF project № 15-11-30014 and by the MEPhI Academic Excellence Project (Contract No. 02.a03.21.0005).

References 1. Blondel, V.D., Guillaume, J., Lambiotte, R., Lefebvre, E.: Fast unfolding of communities in large networks. J. Stat. Mech. Theory Exper. 2008(10), P10008 (2008) 2. Rubinov, M., Sporns, O.: Complex network measures of brain connectivity: uses and interpretations. NeuroImage 53(3), 1059–1069 (2010) 3. Van den Heuvel, M.P., Sporns, O.: Rich-club organization of the human connectome. J. Neurosci. 31(44), 15775–15786 (2011) 4. Zavyalova, V., Knyazeva, I., Ushakov, V., Poyda, A., Makarenko, N., Malakhov, D., Velichkovsky, B.: Dynamic clustering of connections between fMRI resting state networks: a comparison of two methods of data analysis. In: Samsonovich, A.V., Klimov, V.V., Rybina, G.V. (eds.) Biologically Inspired Cognitive Architectures (BICA) for Young Scientists. AISC, vol. 449, pp. 265–271. Springer, Cham (2016). doi:10.1007/978-3-319-32554-5_34

90

S. Kartashov et al.

5 Sharaev, M., Ushakov, V., Velichkovsky, B.: Causal interactions within the default mode network as revealed by low-frequency brain fluctuations and information transfer entropy. In: Samsonovich, A., Klimov, V., Rybina, G. (eds) Biologically Inspired Cognitive Architectures (BICA) for Young Scientists. Advances in Intelligent Systems and Computing, vol. 449, pp. 213–218. Springer, Cham (2016). doi:10.1007/978-3-319-32554-5_27 6. Van den Heuvel, M.P., Sporns, O., Collin, G., Scheewe, T., Mandl, R.C., Cahn, W., Goñi, J., Hulshoff Pol, H.E., Kahn, R.S.: Abnormal richclub organization and functional brain dynamics in schizophrenia. JAMA Psychiatry 70(8), 783–792 (2013) 7. Bohlken, M.M., Brouwer, R.M., Mandl, R.C., Van den Heuvel, M.P., Hedman, A.M., De Hert, M., Cahn, W., Kahn, R.S., Hulshoff Pol, H.E.: Structural Brain Connectivity as a Genetic Marker for Schizophrenia. JAMA Psychiatry 73(1), 11–19 (2016)

No Two Brains Are Alike: Cloning a Hyperdimensional Associative Memory Using Cellular Automata Computations Denis Kleyko(B) and Evgeny Osipov Lule˚ a University of Technology, 971 87 Lule˚ a, Sweden {denis.kleyko,evgeny.osipov}@ltu.se

Abstract. This paper looks beyond of the current focus of research on biologically inspired cognitive systems and considers the problem of replication of its learned functionality. The considered challenge is to replicate the learned knowledge such that uniqueness of the internal symbolic representations is guaranteed. This article takes a neurological argument “no two brains are alike” and suggests an architecture for mapping a content of the trained associative memory built using principles of hyperdimensional computing and Vector Symbolic Architectures into a new and orthogonal basis of atomic symbols. This is done with the help of computations on cellular automata. The results of this article open a way towards a secure usage of cognitive architectures in a variety of practical application domains.

1

Introduction

Alternative computing architectures with autonomous continuously learning and generalization capabilities are highly demanded in many practical domains including robotics, intelligent industries, home automation, and e-health. Biologically Inspired Cognitive Architectures (BICA) by definition satisfy such requirements and as such have a high potential to emerge as a solution for many applications including life- and mission-critical ones. While in the context of BICA the current research focus is on educational processes of cognitive systems or robots this article suggests to look beyond this phase and consider the knowledge replication and transfer functionality. It is exactly this functionality, which will make BICA-based technical systems practically usable in technical applications. Consider a simple scenario where, for example, a BICA system is installed for home automation in a facility with a large number of initially identical rooms. Suppose that one BICA system is installed per room and its purpose is to learn the behavioral pattern of the users in the particular room and adjust the smart functionality accordingly. Obviously, it is unacceptable (or at least impractical) that each installed system is completely blank in the beginning and needs to be trained from the scratch. Instead, it is reasonable to assume that the system is pre-trained on typical behavioral patterns, which then will be replicated in c Springer International Publishing AG 2018  A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6 13

92

D. Kleyko and E. Osipov

several copies, which will continue their independent functionality as the system continues to operate. From the cybersecurity perspective, the challenge associated with the replication of the content is that while the two copies must be functionally identical, they should be based on unique internal representations. “Uniqueness of the copies” has a meaning with respect to technical requirements from potential users of any system: the security of system’s operation. In the presented above practical home automation use-case, a simple copying makes the entire system extremely vulnerable to malicious attacks. Once the BICA-origin is compromised (e.g., the internal representations of atomic concepts become known), all subsequent copies are compromised as well. Thus, addressing the problem of knowledge transfer in BICA-based systems will mitigate the problem all architectures (BICA and non-BICA) suffer from. This article focuses on the challenge of replication of the content of the longterm memory of the BICA system. Specifically it is assumed that the long-term memory is implemented using the principles of hyperdimensional computing and Vector Symbolic Architectures (VSAs). VSAs [1] is a class of connectionist models for representing structured knowledge, implementing analogical-based reasoning [2], and pattern recognition [3–5]. It is argued that VSAs could be a suitable candidate for implementing functionality of general artificial intelligence [6,7] and imitation of functional behavior of living systems [8,9]. VSAs also constitute a significant part of the Semantic Pointer Architecture [10], which exhibits a wide range of cognitive functionality. The core contribution of the article is the pipeline for cloning the content of the learned VSA-based BICA, which satisfy the “no two brains are alike” argument. Technically, the described solution allows mapping the trained memory to a new unique basis while retaining the internal relationships between the symbols represented in hyperdimensional space. The mapping is done with the help of computations on cellular automata (CA). Rule 90 of CA is chosen as it possesses several properties highly relevant for achieving the objectives of the considered mapping task. This paper is structured as follows. Section 2 introduces high-dimensional vectors and operations on them. Cellular automata are briefly described in Sect. 3. Section 4 presents the main contribution of this paper – a pipeline for mapping VSA-based memory into a new unique basis with cellular automata computations. The conclusions are presented in Sect. 5.

2

High-Dimensional Vectors and Related Operations

Hyperdimensional computing (also known as Vector Symbolic Architectures, VSAs) operates with high-dimensional vectors (also referred to as HD vectors or distributed representations). There are several different types of VSAs, each using different representations, e.g. [11–16]. This paper considers only Binary Spatter Code variant of VSAs [17], in which the individual elements only take the binary values “0” or “1”. Information in VSAs is represented in a distributed fashion, where a single, unitary entitity is represented as an HD vector.

No Two Brains Are Alike

93

The values of each element of an HD vector are independent of each other, and “0” and “1” values have approximately the same density, i.e. p1 ≈ p0 ≈ 0.5. The similarity between two HD vectors is characterized by the Hamming distance. It measures the number of positions in the two compared HD vectors in which they have different values. In very high dimensional spaces, the Hamming distance between two arbitrarily chosen HD vectors is concentrated around 0.5. That is, arbitrarily chosen HD vectors are, with overwhelmingly high probability, almost orthogonal (i.e. effectively dissimilar). This is similar to the behaviour of symbolic representations – arbitrarily chosen symbols are generally different. Interested readers are referred to [11,18,19] for comprehensive analysis of probabilistic properties and capacity of the hyperdimensional representation space. There are three common operations with HD vectors: binding, bundling, and permutation. Bundling of vectors. The bundling operation is used to join multiple entities into one structure; it is implemented by a majority rule of the HD vectors representing the entities. A elementwise thresholded sum of n vectors yields “0” when n/2 or more arguments are “0”, and “1” otherwise. If the sum produces an even number, the resulting tie is broken randomly. This is equivalent to adding an extra random HD vectors [11]. The operation is denoted as the sum [a + b + c]. Binding of vectors. Binding, which can be interpreted as assigning a value to a field, is implemented as the elementwise XOR operation (denoted as ⊕) on the corresponding HD vectors. Permutation of vectors. Permutation produces an HD vector dissimilar to the permuted one (i.e. the normalized Hamming distance between them equals approximately 0.5). Several random HD vectors can be generated from a single vector through different permutations. The cyclic shift operation is a special case of the permutation [11]. The result of operation is obtained by cyclically shifting a by i elements and denoted here as Sh(A, i). An example of operation which can be done with HD vectors is called holistic mapping. It can be used to answer non-trivial queries (e.g., “What is the dollar of Mexico?”) by operating on the whole representation [20]. 2.1

VSA-based BICA

Hyperdimensional computing and Vector Symbolic Architectures are used to represent structured knowledge and implement analogical reasoning. Ax example of such usage is the representation of conceptual spaces in order to facilitate neural and symbolic levels of representations [21,22]. The VSAs based memory is normally structured in two parts: a clean-up memory (often referred to as itemmemory) and a compositional memory. The structure is exemplified in Fig. 1. The item memory normally contains atomic HD vectors and is normally used for detailed analysis of VSAs aggregates. For the purposes of future discussion, we also insert results of binding and permutation operations generated during system’s operation. Thus, the HD vectors stored in the item-memory serve as components for all other HD vectors in the system.

94

D. Kleyko and E. Osipov

The second part of the memory is the compositional memory. In terms of symbolic systems, a compositional memory will contain meaningful transformations of the atomic symbols and their bindings. The symbols in this memory are formed by bundling several components from the item memory. VSA structures can be formed from atomic HD vectors only (e.g. [a + b + c]), be a result of bundling of binding or permutation of atomic HD vectors (e.g. [a⊕b+b+c], [a+b+Sh(c, 1)]), and other compositional HD vectors as well (e.g. [b + [a + b + c] + c]).

Fig. 1. An example of an item memory with three atomic HD vectors (a, b, and c), a binding HD vector (a ⊕ b) and a permutated vector Sh(c, 1)). The compositional memory (right-hand side) contains four compositional HD vectors. For simplicity of the presentation, the system is exemplified on 10-dimensional vectors. In the simulations HD vectors with 10,000 elements were used.

Fig. 2. The assignment of new states for a center cell when the cellular automaton uses rule 90. A hollow cell corresponds to zero state while a shaded cell marks one state.

3

Cellular Automata

A cellular automaton (CA) is a discrete computational model consisting of a regular grid of cells [23]. Each cell can be in one of a finite number of states (two - for the binary automaton). States of cells evolve in discrete time steps according to a fixed rule. The state of a cell on the next computational step depends on its current state and states of its neighbors. Notably, amongst the rules of CA there are those (e.g., rule 110), which make CA operate at the edge of chaos and were proven to be Turing complete. Another interesting rule which is employed in this paper is rule 90. The 90th rule of cellular automata is denoted as CA90 . Also, notation CAn90 will be used to denote the n-th computing step of the cellular automaton. Albeit this rule is not Turing complete, it posses several properties relevant for the scope of this paper. The computations performed by cellular automata are local. The new state of a cell is determined by previous states of the cell itself and its two neighboring cells (left and right). Thus only three cells are involved in a computation step,

No Two Brains Are Alike

95

i.e. for binary states, there are in total 23 = 8 combinations. The rule assigns states for each of eight combinations. Figure 2 illustrates all combinations and corresponding states for CA90 . It can be seen from the figure that CA90 assigns the next state of a central cell solely on the previous states of the neighboring cells. In particular, the new state is the result of XOR operation on the states of the neighboring cells. Thus, CA90 has a computational advantage since CA implementation can be vectorized. That is if at time step t vector x(t) describes the states of CA, then the states for the next time step t + 1 are computed as follows: x(t+1) = Sh(x(t), 1)⊕Sh(x(t), −1), where Sh() is the notation for cyclic shift operation 3.1

CA and Hyperdimensional Vectors

In the scope of VSAs and hyperdimensional computing CA (rules 90 and 110) were recently explored in [24–26] for projecting binarized features into hyperdimensional space. Further, in [27] this approach was applied for modality classification of medical images. In the present work, we utilize three major properties of CA rule 90: 1. Random projection; 2. Preservation of the binding operation; 3. Preservation of the cyclic shift. The first property means that when CA90 is initialized with a random HD vector HDIN IT , its evolved state is another HD vector of the same density (p1 ≈ p0 ≈ 0.5). Moreover this vector is dissimilar with the original initialization vector. That is dh (HDIN IT , CAn90 (HDIN IT )) ≈ 0.5. The second property tells that if HD vector c is the result of binding of two other HD vectors: c = a ⊕ b then after n computational steps of CA90 , the evolved HD vector CAn90 (c) is the result of binding of the evolved HD vectors CAn90 (a) and CAn90 (b) for the same number of steps. That is dh (CAn90 (c), CAn90 (a) ⊕ CAn90 (b)) = 0. Finally, computations with CA90 preserve a special case of the permutation operation on HD vectors - cyclic shift by i elements. Suppose d = Sh(a, i). After n computational steps of CA90 , the cyclic shift of the evolved HD vector CAn90 (a) by i elements equals the evolved shifted HD vector CAn90 (d). That is dh (CAn90 (d), Sh(CAn90 (a, i)) = 0. These properties of CA90 constitute the core of the memory cloning pipeline presented in the next section.

4

Cloning of VSA-based Associative Memory with CA90 Computations

The challenge with the replication of the VSA-based memory into a new unique basis comes from the fact that CA90 does not preserve the bundling operation.

96

D. Kleyko and E. Osipov

That is the result of the evolution of the bundled HD vector (computed as the majority sum of several HD vectors) becomes dissimilar to its CA-evolved components. That is why an additional architectural solution is needed. The core of the proposed memory cloning pipeline is the mechanism for describing relationships between HD vectors from the item-memory, which result in the items of the compositional memory. The most straightforward way to describe such relationships is to introduce a relation matrix. This matrix simply marks which items are included in the bundles. The idea is illustrated in Fig. 3. In the left part of the figure the arrows show the contribution of each HD vector from the item-memory to the particular compositional HD vector. The table on the right is the relation matrix representing these relationships in the binary form. Note, that compositional structures are also valid symbols to be used in other VSA-compositions. This is depicted by the lighter arrow originating and ending up in the compositional VSA memory.

Fig. 3. Recomputing compositional HD vectors using connection matrices and mapped atomic HD vectors.

Data: HD vectors from item memory (atomic, bindings and permutations) Passed as arguments Arg1, Arg2, Arg3, Arg4 , Arg5, . . . Result: Computed terminal VSA symbols of the compositional memory tmp1=[Arg2, Arg3] //list of atomic vectors repeating in severalcompositions tmp2=[Arg1]+tmp1 //list of atomic vectors repeating in several compositions A=BUNDLE(tmp2) B=BUNDLE([Arg4]+tmp1) C=BUNDLE([Arg1,Arg2,Arg5]) D=BUNDLE(tmp1+[A]) //this compositional item contains compositional item A ... return [A, B, C, D, . . . ]

Algorithm 1. An example of a program describing the relationships between items of the VSA-based BICA memory.

No Two Brains Are Alike

97

The role of the relation matrix is somewhat similar to a matrix of synaptic weights between two layers in artificial neural networks since elements of the matrix define the contribution from one layer’s activations to the next layer. There are, however, differences. The relation matrix between item and compositional memories is sparse and binary where one in the intersection of ith row and jth column means that ith HD vector is a component of jth HD vector. The sparsity is expected because usually a compositional HD vector is formed only by few components. Another useful way of representing the same information is in the form of an executable program, which takes the atomic items as variables and produces terminal symbols of the compositional memory. For example, in Fig. 3 the program in Algorithm 1 computes the list of items in the compositional memory for Arg1 = a, Arg2 = b, Arg3 = c, Arg4 = a ⊕ b, Arg5 = Sh(c, 1).

Fig. 4. Mapping the first part of the memory with two steps of cellular automata computations.

Now, given the mechanism for describing the relationships between VSAbased memory the process for the replication of the entire memory into a new unique basis is straightforward and consists of two steps: 1. Mapping of the content of the entire item memory to an orthogonal basis with the help of CA90 computations. This is illustrated in Fig. 4. 2. Replicating the compositional memory (either with the help of the relation matrix or executing a program) for new values of items in the item memory. Thus, as the result of the memory replication procedure a new memory of HD vectors is created. All items of this new memory are dissimilar to all items of the original memory, however, the relations between atomic items and within compositional memory are identical. It is important to note that each step of CA90 computations produces a new mapping dissimilar to all other mappings. In other words, multiple replicas with unique basis can be produced by repeating the replication procedure for a different number of CA90 execution steps.

98

4.1

D. Kleyko and E. Osipov

Simulations

Properties of CA90 computations are illustrated in Fig. 5 which plots the results of simulations of a toy memory. The origin item memory consists of four HD vectors: two atomic HD vectors (a and b), one binding (a⊕b), and one permutation (Sh(a, 10)). The source item memory was mapped to 50 dissimilar memories using 50 computational steps of CA90 : one step per mapping. The figure depicts five different curves, plotting normalized Hamming distance versus the number of computational steps. It is clear that the curves are grouped in two regions: in the region of the absolute similarity (dh = 0) and in the region of the dissimilarity (dh ≈ 0.5). Three curves in the dissimilarity region show normalized Hamming distance between three HD vectors in the source memory (a, a⊕b, and Sh(a, 10)) and their mapped versions (denoted as a∗ , (a ⊕ b)∗ , and Sh(a, 10)∗ ) after i steps of computations. Two curves in the absolute similarity region show normalized Hamming distance between two mapped HD vectors ((a ⊕ b)∗ and Sh(a, 10)∗ ) and results of corresponding operations (binding and permutation) performed with mapped atomic HD vectors ((a∗ ⊕ b∗ ) and Sh(a∗ , 10)). The results of the simulation confirm that the mapped memories are dissimilar to the origin item memory (and to each other) but the relationships between HD vector in the mapped memories are preserved.

Fig. 5. Similarity between the source HD vectors and mapped HD vectors for 50 computational steps with rule 90.

5

Conclusions

This paper considered a problem of replicating the associative memory of a technical BICA system into a unique basis of atomic representations. The importance of the problem is motivated from the point of view of security considerations in practical BICA applications. The major challenge is to preserve all previously learned relationships while having all items (symbols) completely unrelated to the memory-origin. For BICA systems employing associative memory based on the principles of Vector Symbolic Architectures, we proposed a pipeline for flexible creation of multiple replicas satisfying the uniqueness property. The core of the replication pipeline is the mechanism for describing relationships between the item and compositional memory (a relation matrix or an executable program) and a procedure for mapping the content of the item memory to the new

No Two Brains Are Alike

99

unique basis using computation of CA90 . This has not been done before and opens a way towards secure practical applications of BICA systems. Acknowledgements. This study is supported in part by the Swedish Research Council (grant no. 2015-04677). The authors thank Ozgur Yilmaz for fruitful discussions during BICA2016 on the usage of cellular automata in the scope of hyperdimensional computing, which inspired the current work and Niklas Karvonen for general discussions on cellular automata.

References 1. Gayler, R.W.: Vector symbolic architectures answer Jackendoff’s challenges for cognitive neuroscience. In: Proceedings of the Joint International Conference on Cognitive Science. ICCS/ASCS, pp. 133–138 (2003) 2. Emruli, B., Sandin, F.: Analogical mapping with sparse distributed memory: a simple model that learns to generalize from examples. Cogn. Comput. 6(1), 74–88 (2014) 3. Kleyko, D., Osipov, E., Papakonstantinou, N., Vyatkin, V., Mousavi, A.: Fault detection in the hyperspace: towards intelligent automation systems. In: IEEE International Conference on Industrial Informatics, INDIN, pp. 1–6 (2015) 4. Rahimi, A., Benatti, S., Kanerva, P., Benini, L., Rabaey, J.M.: Hyperdimensional biosignal processing: a case study for emg-based hand gesture recognition. In: 2016 IEEE International Conference on Rebooting Computing (ICRC), pp. 1–8 (2016) 5. Kleyko, D., Osipov, E., Gayler, R.W.: Recognizing permuted words with vector symbolic architectures: a cambridge test for machines. Procedia Comput. Sci. 88, 169–175 (2016) 6. Levy, S.D., Gayler, R.: Vector symbolic architectures: a new building material for artificial general intelligence. In: Proceedings of the 2008 Conference on Artificial General Intelligence 2008, pp. 414–418 (2008) 7. Rachkovskij, D.A., Kussul, E.M., Baidyk, T.N.: Building a world model with structure-sensitive sparse binary distributed representations. Biol. Inspir. Cogn. Archit. 3, 64–86 (2013) 8. Kleyko, D., Osipov, E., Gayler, R.W., Khan, A.I., Dyer, A.G.: Imitation of honey bees’ concept learning processes using vector symbolic architectures. Biol. Inspir. Cogn. Archit. 14, 57–72 (2015) 9. Kleyko, D., Osipov, E., Bjork, M., Toresson, H., Oberg, A.: Fly-the-bee: a game imitating concept learning in bees. Procedia Comput. Sci. 71, 25–30 (2015) 10. Eliasmith, C.: How to Build a Brain. Oxford University Press, Oxford (2013) 11. Kanerva, P.: Hyperdimensional computing: an introduction to computing in distributed representation with high-dimensional random vectors. Cogn. Comput. 1(2), 139–159 (2009) 12. Gallant, S.I., Okaywe, T.W.: Representing objects, relations, and sequences. Neural Comput. 25(8), 2038–2078 (2013) 13. Plate, T.A.: Holographic reduced representations. IEEE Trans. Neural Netw. 6(3), 623–641 (1995) 14. Kleyko, D., Osipov, E., Rachkovskij, D.A.: Modification of holographic graph neuron using sparse distributed representations. Procedia Comput. Sci. 88, 39–45 (2016)

100

D. Kleyko and E. Osipov

15. Aerts, D., Czachor, M., De Moor, B.: Geometric analogue of holographic reduced representation. J. Math. Psychol. 53, 389–398 (2009) 16. Rachkovskij, D.A.: Representation and processing of structures with binary sparse distributed codes. IEEE Trans. Knowl. Data Eng. 3(2), 261–276 (2001) 17. Kanerva, P.: Fully distributed representation. In: Real World Computing Symposium, pp. 358–365 (1997) 18. Kanerva, P.: Sparse Distributed Memory. The MIT Press, Cambridge (1988) 19. Kleyko, D., Osipov, E., Senior, A., Khan, A.I., Sekercioglu, Y.A.: Holographic graph neuron: a bioinspired architecture for pattern processing. IEEE Trans. Neural Netw. Learn. Syst. 28(6), 1250–1263 (2017) 20. Kanerva, P.: What we mean when we say “What’s the Dollar of Mexico?”. In: AAAI Fall Symposium. Quantum Informatics for Cognitive, Social, and Semantic Processes, pp. 2–6 (2010) 21. Lieto, A., Lebiere, C., Oltramari, A.: The knowledge level in cognitive architectures: current limitations and possible developments. In: Cognitive Systems Research, pp. 1–17 (2017) 22. Lieto, A., Chella, A., Frixione, M.: Conceptual spaces for cognitive architectures. Biol. Inspir. Cogn. Archit. 19, 1–9 (2017) 23. Wolfram, S.: A New Kind of Science. Wolfram Media Inc., Champaign (2002) 24. Yilmaz, O.: Machine learning using cellular automata based feature expansion and reservoir computing. J. Cell. Automata 10(5–6), 435–472 (2015) 25. Yilmaz, O.: Symbolic computation using cellular automata-based hyperdimensional computing. Neural Comput. 27(12), 2661–2692 (2015) 26. Nichele, S., Molund, A.: Deep reservoir computing using cellular automata, pp. 1–9 (2017). arXiv:1703.02806 27. Kleyko, D., Khan, S., Osipov, E., Yong, S.P.: Modality classification of medical images with distributed representations based on cellular automata reservoir computing. In: IEEE International Symposium on Biomedical, Imaging, pp. 1–4 (2017)

Informative Characteristics of Brain Activity to Diagnose Functional Disorders in People with Stuttering Anastasia Korosteleva1,2(B) , Olga Mishulina1 , Vadim Ushakov1,2 , and Olga Skripko1 1

National Research Nuclear University MEPhI (Moscow Engineering Physics Institute), Moscow, Russia [email protected], [email protected], [email protected], [email protected] 2 NRC “Kurchatov Institute”, Moscow, Russia

Abstract. The article presents the results of an experimental study of functional disorders of brain activity in people with stuttering. The experiment was carried out using functional magnetic resonance imaging. The purpose of this study was to identify the characteristics of brain activity in people who stutter and the formation of numerical indicators of the available functional disorders. The results of the comparative analysis of brain activity in the areas of Broca and Wernicke for two participants with stuttering and normal speech.

1

Introduction

Speech is complex operation of the brain. It is controlled by three areas of the brain functioning in different ways: (motor) Broca’s area (Ba) produces speech controlling speech muscles; (sensory) Wernicke’s area (Wa) - recognizes own speech and speech of other people; association (uniting) areas create the structure of phrases and sentences [1,2]. Thus, our speech is a continuous “circular” process. So that the flow of speech is not interrupted, it is necessary that all three speech centers should work synchronously, i.e. in the same rhythm. The stutter is periodic, short break of speech “circle” due to desync of functions of speech centers. In this work, we believe, that people with stuttering have expressed features of brain activity in the Ba and Wa associated with their functional impairment. The main cause of speech convulsions during stuttering is a violation of the internal synchronization of the natural speech cycle, which can be caused by distortions in the transmission or in the processing of feedback signals. People who stutter and people with normal speech participated in the experiments for comparative analysis. The study was performed using functional magnetic resonance imaging (fMRI) based on the use of ultrafast sequences. FMRI is one of the most effective noninvasive methods of studying the connection between c Springer International Publishing AG 2018  A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6 14

102

A. Korosteleva et al.

different brain regions in cognitive processes and identifying functional areas of neuronal activity, including for studying the nature of stammering [3]. For more reliable results the technique of ultrafast fMRI used with the temporal resolution increasing from 2–3 s to 500 ms [4]. The main goal of this research is to identify characteristics of brain activity in certain anatomical areas, on the basis of which it is possible to decide on the degree of functional impairment caused by stuttering. In this work we will focus on a statistical analysis of some characteristics of brain activity such as the ratio of the average signal levels of Wa to Ba when participants performed test tasks, the number of active voxels Wa to Ba when participants performed test tasks, the time alignment of a signal between tasks. The signal equalization time is the time from the beginning of the job to the time when the signal value is within the average signal level when the job is performed taking dispersion into account. In the conducted experiment the proposed characteristics distinguish people with normal speech and stuttering and they can be considered as a basis for further research work.

2

Materials and Methods

Data was recorded using a magnetic resonance tomograph (Magnetom Verio 3T (Siemens, Germany)). Behavioral study participants included a total 4 people (2 men and 2 women) with the leading right hand. There were 2 people who stutter and 2 people with normal speech. The average age of the subjects was 23 years. Each subject was placed in the MRI scanner. The study was conducted using a 32-channel head MR-coil. Scanning consisted of two steps: removal of the anatomical data, functional data record on the basis of the EPI-sequence Ultrafast. Visual stimuli were submitted in the MRI scanner during a functional study using a projector system. The paradigm was developed using Presentation software. The experiment consists of four cycles, in each task is consistently set to the participant. Totally there is a set of 5 tasks: (1) reading aloud; (2) reading to oneself; (3) retelling aloud; (4) retelling to oneself; (5) rest.

Fig. 1. Scheme of one cycle of the experiment

Figure 1 shows the scheme of the first cycle of the experiment, the duration of which is 6 min. Above the time axis, task numbers indicate 30 s intervals assigned to each participant. The duration of each task for 30 s. The experiment contains four cycles and each cycle contains 4 phases. The first cycle includes four phases

Informative Characteristics of Brain Activity

103

in which tasks are performed: “reading aloud,” “read to oneself”, “rest”. The second cycle includes four phases in which tasks are performed: retelling aloud, retelling to oneself, rest. The third and fourth cycles repeat the sequence of tasks of the first and second cycles.

3

Results

The functional fMRI data is preprocessed in the SPM8 for each participant in the experiment. In this work we investigate in detail the nature of the signal activity of the brain when participants in different groups performed the task of reading/retelling aloud. The signal was analyzed in the speech areas. All speech areas are located in both hemispheres, but develop only on one side (righthanders – left, left – right). All members have the right dominant hand, so the area of interest are in the left hemisphere: the area of Brodman area 22 (Wa) and 44 (Ba). Since we were interested in active voxels at the time of tasks 1 and 3, on the map of brain activity, only those voxels that have high significance and are included in the Left Ba (LBa) and the Left Wa (LWa) were singled out for reading/retelling aloud (relative to rest). In this work we analyze the proposed features in the moments of workflow participants complete task 1 and 3. Figure 2 show plots of the magnitude of the signal activity of the LBa and LWa for different groups of participants and for different test tasks. A statistical analysis of the brain activity signal showed that stutterers the Ba is active when there is no talking while performing the “reading/retelling to oneself” or “rest” task, while people with normal speech have sharply decreases in the signal value of the Ba with the completion of the task at “reading/retelling aloud” and increases again when they are speaking aloud. This is consistent with the theory of neural

Fig. 2. Average signal level as a function of time in phases 1–4 of the cycle. (a) the average signal level in the LBa, (b) the average signal level in the LWa. The time axis represents the task numbers that were performed by the participant

104

A. Korosteleva et al.

and cognitive processes of information by people with stuttering, which marks the excitation of the Ba both during the making task aloud and to oneself. Signal of Wa repeats traffic signal of Ba of people without stuttering. Sometimes in case of transition between tasks by stutterers, the signals of the Ba and Wa are in antiphase, and this is not related to the fulfillment of the task of reading/retelling aloud or about oneself. Our goal is to find informative features and numerical indicators to indicate the extent of stuttering and functional disorders of brain activity in the signal activity in Ba and Wa. As shown by statistical analysis of the results of the experiment, there is a certain dependence (Tables 1 and 2) relationship of average signal levels between the anatomic areas in moments of performing tasks on “reading/retelling aloud.” From Table 1 it follows that both groups of people, Ba is stronger excited than the Wa for both tasks. The ratio of average signal level of Wa to the average signal of Ba of people with stuttering in the task of “reading aloud” is higher than when performing the task “retelling aloud”, while people without stuttering inverse relationship is more clearly shown in Table 2. Tables 3 and 4 show the relationship of the number of active voxels between the anatomic areas in moments of performing tasks on “reading/retelling aloud.” The regularity of the values of this characteristic almost completely coincides with the regularity of the ratio of the average signal levels between Ba and Wa and at the time of performing tasks for “reading/retelling aloud” for both groups of participants. That gives foundation to consider these signs interchangeable. From Table 5 it follows that signal alignment occurs in a short time and is practically independent of the anatomical areas and the task being performed for people with normal speech. But for stutterers Ba and Wa are active for some time and slowly lose the intensity of the signal. It depends on the task being performed and the anatomical areas. Table 1. The ratio of average signal level of the LWa to the average signal of LBa in the tasks of “reading aloud” and “retelling aloud” (tasks 1 and 3) for participants with normal speech and stuttering Tasks

The ratio of average signal level in LBa and LWa Normal Stuttering

Reading aloud LWa/LBa

0.1024 0.3645

Retelling aloud LWa/LBa

0.1466 0.2141

Table 2. The ratio of average signal level in LBa and LWa in the tasks of “reading aloud” and “retelling aloud” (tasks 1 and 3) for participants with normal speech and stuttering Tasks

The ratio of average signal level in LBa and LWa

Normal Stuttering

Reading aloud/Retelling aloud LWa/LWa

0.5529

2.7273

Reading aloud/Retelling aloud LBa/LBa

0.7931

1.5588

Informative Characteristics of Brain Activity

105

Table 3. The ratio of the number of active voxels of LWa to the number of active voxels of LBa in the tasks 1 and 3 for participants with normal speech and stuttering Tasks

The ratio of the number of active voxels of LBa and LWa

Normal Stuttering

Reading aloud

LWa/LBa

0.1349

0.7321

Retelling aloud LWa/LBa

0.1637

0.5000

Table 4. The ratio of the number of active voxels of LBa and LWa in the tasks 1 and 3 for participants with normal speech and stuttering Tasks

The ratio of the number of Normal Stuttering active voxels of LBa and LWa

Reading aloud/Retelling aloud LWa/LWa

1.3393

3.4167

Reading aloud/Retelling aloud LBa/LBa

1.6257

2.3333

Table 5. The ratio of average signal level in LBa and LWa in the tasks 2 and 4 for participants with normal speech and stuttering

4

Tasks

Anatomic areas Normal Stuttering

Reading aloud - Reading to oneself

LBa

2.2674

7.7834

Reading aloud - Reading to oneself

LWa

2.1425

7.2413

Retelling aloud - Retelling to oneself LBa

2.8452

8.8612

Retelling aloud - Retelling to oneself LWa

2.7645

8.6732

Discussion

Study of features of brain activity in the Ba and Wa in reading and retelling a text aloud, confirmed the assumption that people with functional abnormality of stuttering have important features of statistical indicators of activity. In the paper we propose that three features of brain activity - the ratio of average signal levels of Wa to Ba in the test task, the number of active voxels of Wa to Ba in the test task, the time alignment of a signal between tasks. The nature of the signal of Ba was significantly different for different groups of participants while performing the task of reading/retelling not only aloud, but to oneself. It is established that the most informative characteristics for identifying stuttering is the time alignment of the signal in the moments of transition between tasks. People who stutter perform process of switching to a new task 3–4 times slower than in people with normal speech, as characteristics of the signal tell. This work has practical significance. A preliminary analysis of the fMRI data shows the possibility of formation of additional features that characterize the degree of functional impairment of stuttering. Such indicators will provide important information for speech-language pathologists for analysis in the dynamics of

106

A. Korosteleva et al.

treatment efficacy and the ability to systematically control the state of a person after completion of rehabilitation. Acknowledgments. This work was supported by the Russian Science Foundation, Grant No 15-11-30014. The authors are grateful to the MEPhI Academic Excellence Project for providing computing resources and facilities to perform experimental data processing (Contract No. 02.a03.21.0005).

References 1. De Nil, L.F., Bosshardt, H.G., Yaruss, J.S., Peters, H.F.M.: Studying stuttering from a neurological and cognitive information processing perspective. In: Fluency Disorders: Theory, Research, Treatment and Self-help, pp. 53–58 (2001) 2. Kent, R.D.: Research on speech motor control and its disorders: A review and prospective. J. Commun. Disord. 33(5), 391–428 (2000) 3. Van Borsel, J., Achten, E., Santens, P., Lahorte, P., Voet, T.: fMRI of developmental stuttering: a pilot study. Brain Lang. 85(3), 369–376 (2003) 4. Golay, X., Pruessmann, K.P., Weiger, M., Crelier, G.R., Folkers, P.J., Kollias, S.S., Boesiger, P.: Presto-sense: An ultrafast whole-brain fMRI technique. Magn. Reson. Med. 43(6), 779–786 (2000)

Event-Related fMRI Analysis Based on the Eye Tracking and the Use of Ultrafast Sequences Anastasia Korosteleva1,2(B) , Vadim Ushakov1,2 , Denis Malakhov2 , and Boris Velichkovsky2,3 1

National Research Nuclear University MEPhI (Moscow Engineering Physics Institute), Moscow, Russia [email protected] 2 NRC “Kurchatov Institute”, Moscow, Russia {tiuq,malahov-denis}@yandex.ru 3 Technische Universitaet Dresden, Dresden, Germany [email protected]

Abstract. The purpose of the study was to investigate the relationship between human cognitive processes and eye movements during inspection of images using methods of ultrafast functional magnetic resonance imaging (fMRI) and eye tracking. We conducted two series of experiments in which participants saw pictures of faces and houses. Statistical processing of the fMRI data showed that visual fixations on different objects in the context of different tasks lead to different patterns of cortical activation, and reconstructed BOLD signal responses show important information about the task context of individual fixations on viewed objects.

1

Introduction

It is known [1–3] that the eye movements, composed primarily of fixations and saccades, are sources of information about the processes of perception and cognition. They are also a visually registered indicator of attention [1,2]. Eye fixations are closely related to processing of visual information and could be considered as “units of information” [4]. It was proposed to use eye fixations as events in functional magnetic resonance imaging (fMRI) to study cortical processing during visual inspection of objects. In our study we repeated the fixation based event-related fMRI analysis [4,5]. To obtain more reliable data, we used the ultrafast fMRI technique increasing time resolution from 2–3 s to 500 ms [6]. So, we expect increased sensitivity of brain activity recording based on changes of blood flow and better results comparing to conventional fMRI sequences. The general task of our research is to prove the methods of combined processing of fMRI and eye tracker data, and to obtain new results through the use of more sensitive method of data capturing. In this work we focus on statistical analysis of eye movement parameters and brain activity. For the eye movement parameters, we used time and duration of eye fixations and the number of fixations on objects (houses and faces). For brain activity analysis, we used activity maps c Springer International Publishing AG 2018  A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6 15

108

A. Korosteleva et al.

to localize regions of interest (ROI) in visual cortex that correspond to perception of objects that represent houses and faces. In the ROI, we reconstructed hemodynamic responses from fixations of different types (houses, faces) in different tasks. Then we compared the reconstructed hemodynamic responses. Some parameters of hemodynamic responses, that we have found, showed statistical significance, so they could be considered as a base for further research.

2

Materials and Methods

During the experiment we simultaneously recorded fMRI data and eye tracking data. Up to now, we studied 15 volunteers (8 males and 7 females) with normal vision (without contact lenses), right handed. The average age of the participants is 25 y.o. Each participant gave an informed agreement for the experiment. To pass visual stimuli into an MRI scanner room during the functional research we used a projection system. For paradigm development we used NBS Presentation software. For the eye tracking we used EyeLink 1000 Plus (SR Research, Canada) at 250 frames per second. Each subject was placed to Magnetom Verio 3T (Siemens, Germany) MRI scanner. 32-channel MRI head coil was used. The scanning process had two stages: capturing anatomical data, recording functional data using Ultrafast EPI-sequence (TR = 720 ms, TE = 33 ms, 56 slices, slice thickness = 2 mm, spatial resolution in each slice = 2 × 2 mm). During the scanning session, we collected functional data in two experimental paradigms: “localizer” and “free-viewing”.

Fig. 1. Scheme representation of the localizer experiment

2.1

Fig. 2. Scheme representation of the free viewing experiment

Localizer and Free Viewing Experiments

Localizer Experiment. The experiment for functional localization (block paradigm) is a standard experiment with passive observation. Subjects viewed pictures of houses and faces in the center of the screen; and then visual cortex areas were localized, that are responsible for perception of houses and faces: Fusiform Face Area (FFA) and Parahippocampal Place Area (PPA) correspondingly. Figure 1 shows a scheme of experiment for functional localization. The experiment consists of 4 cycles. Each cycle contain 2 blocks. The first block of the cycle contains 15 stimuli with pictures of faces and a fixation cross. The second block of the cycle contains 15 stimuli with pictures of houses and a fixation cross. The time of each stimulus is 750 ms, fixation cross time is 10 s, blank interval is 500 ms, total block time is 11.25 s, experiment length is 170 s.

Event-Related fMRI Analysis Based on the Eye Tracking

109

Free Viewing Experiment. In the experiment on active viewing (event-related paradigm), the subjects were in-structed to observe on the stimulus pictures only faces, only houses, or both of them. After each stimulus subjects were asked, whether a control image was presented on the stimulus screen (Sternberg shortterm memory test). At that time subjects should answer by pressing a button. The block consists of 4 screens: Screen 1: Instruction (10 s). Subjects were given 3 types of tasks: “Look at faces” (Face Task, FT), “Look at houses” (House Task, HT) or “Free viewing” (All Task, AT). Screen 2: Stimulus picture consisting of 3 pictures of houses and 3 pictures of faces, placed in a circle in equal distances in 6 fixed positions. Distribution of pictures on these positions was random. Stimulus duration was pseudorandom from 8 to 18 s. Screen 3: Control stimulus (3 s). The stimulus was consisted of a single house or face picture (depending on the task) and was placed at the center of the screen. Screen 4: Fixation cross (10 s). Figure 2 shows a scheme of the experiment with active viewing. The experiment consists of 4 cycles. Each cycle contains 3 blocks: HT, FT, and AT, that denote demonstration of the instruction screen for house viewing (HT), face viewing (FT), and all object viewing (AT) correspondingly. Numbers 2 and 3 correspond to “Screen 2” and “Screen 3” in the block. The block is ended with the fixation cross. 2.2

Data Analysis

For each subject, in the free-viewing experiment, we acquired eye fixations using EyeLink Data Viewer software. Each fixation was marked correspondingly depending on which object was viewed during the fixation and which type of task was presented earlier. We excluded fixations that are outside of regions of interest. Fixations less than 80 ms were also excluded because they did not show useful information [4]. It was suggested to improve an algorithm of fixation selection – to exclude fixations that follow a saccade or precede it. Such fixations give false information because human does not have time to identify a viewing object [1]. fMRI data for Localizer and Free-Viewing experiments for each subject were preprocessed in SPM8 software. From Localizer experiment data, regions of interest (PPA and FFA) were defined. In fMRI data analysis of Free-Viewing experiment, we reconstructed hemodynamic responses to study activity patterns in early visual cortex areas and the areas we defined in Localizer experiment. As event points, we used time of fixations of defined types.

3

Results

Table 1 contains statistics of fixations that were used in fMRI analysis. From these data, it follows that the number of fixations “Face” and “House” is almost the same in corresponding tasks (FT and HT). But in AT, the number of “Face” fixations is bigger than “House” fixations. Table 2 shows distributions of time lengths of fixations in each category. The histograms show that the distributions of fixations during house observations (for all three tasks) are shifted right, to the longer side – 500 ms and more. Fixations on faces usually take less than 500 ms.

110

A. Korosteleva et al. Table 1. Statistics of fixations used in fMRI analysis for all subjects Task

Fixated object

Defined type Mean percentage of fixations for subjects

Mean number of fixations for subjects

All task

House

H-AT

11.9%

730

House task

House

H-HT

22.3%

1370

Face task

House

H-FT

0.7%

40

All task

Face

F-AT

15.6%

960

House task

Face

F-HT

2.1%

130

Face task

Face

F-FT

22.7%

1395

N

24.7%

1520

100%

6145

Excluded fixations Total

Table 2. Histograms of distributions showing length of fixations in each category used in fMRI analysis. HT, FT and AT correspond to tasks to view houses, faces or both Object/Task

House Task

Face Task

All Task

House

Face

In the experiment for localization we defined FFA and PPA areas in the left and right hemispheres. Coordinates of left and right FFA and PPA areas are in accordance with data in the FIBRE article [4]. These areas were localized using “House > Face” and “Face > House” contrasts. Figure 3 shows brain activity maps from Localizer experiment and the ROI that we used in the further analysis. Table 3 contains averaged models of BOLD responses for left and right FFA, PPA, and early visual cortex in three tasks and for two types of viewed objects. Reconstruction is based on hemodynamic response function (HRF), derivative, and the dispersion of the HRF. In HT, maximum levels of all BOLD signals in all areas are practically the same, although there are some differences in configuration – the signal seems to be composed of two peaks with different amplitudes, and the meaning of their value is the subject for further research. In FT the maximum level of BOLD signal for houses (BOLD-H) is low in PPA regions, but in FFA and early visual regions it has practically the same level as BOLD signal for faces (BOLD-F). In AT contrary to the FT BOLD-H level is high in PPA regions. In FFA and early visual areas it is still high. Most short-term fixations

Event-Related fMRI Analysis Based on the Eye Tracking

111

Table 3. Reconstructed hemodynamic responses calculated from fixations on houses (red) and faces (blue) in three different tasks for FFA, PPA and early visual areas (averaged for left and right hemisphere) Task/Area

PPA

FFA

EARLY VISUAL

HT

FT

AT

Fig. 3. Brain activity maps from localizer experiment

correspond to the examination of faces, and BOLD-F has a fast response of 2 to 5 s. BOLD-H had response of 5 to 9 s.

4

Discussion

We successfully proved the method [4] of combined processing of fMRI and eye tracker data. Also, we applied new Ultrafast EPI-sequences with TR = 720 ms which improved temporal resolution of BOLD signal reconstruction. It was possible to detect the relationship between the duration of the fixations and the time of the BOLD signal increase. The differences in PPA region at FT and AT tasks comparing to [4] could be present due to the ultrafast fMRI method and the selection of more informative eye fixations. The form of the BOLD signal curves (Table 3) in all cases is very similar to the curves obtained in [4] up to the smallest details except a few differences. So we can speak about starting new experiments for neurophysiological interpretation of hemodynamic responses obtained with the described method.

112

A. Korosteleva et al.

Acknowledgments. This work was partially supported by a grant from the Russian Science Foundation, RScF Project no. 15-11-30014 (fMRI analysis in neurocognitive research) and by a grant from the Russian Foundation for Basic Research, RFBR Project ofi-m no. 15-29-01344 (FIBER method in research of voluntary attention), by the MEPhI Academic Excellence Project (Contract No. 02.a03.21.0005) (eye tracker analysis in neurocognitive research).

References 1. Yarbus, A.L.: Eye movements during perception of complex objects. Springer, Heidelberg (1967) 2. Velichkovsky, B.M., Joos, M., Helmert, J.R., Pannasch, S.: Two visual systems and their eye movements: Evidence from static and dynamic scene perception. Cogn. Sci. Soc. 27, 2283–2288 (2005) 3. Podladchikova, L., Shaposhnikov, D., Kogan, A., Koltunova, T., Dyachenko, A., Gusakova, V.: Temporal dynamics of fixation duration, saccade amplitude, and viewing trajectory. J. Integr. Neurosci. 8(4), 487–501 (2009) 4. Marsman, J.B.C., et al.: Fixation based event-related fMRI analysis Using eye fixations as events in functional magnetic resonance imaging to reveal cortical processing during the free exploration of visual images. Hum. Brain Mapp. 33(2), 307–318 (2012) 5. Mills, M., et al.: Cerebral hemodynamics during scene viewing: hemispheric lateralization predicts temporal gaze behavior associated with distinct modes of visual processing (2017) 6. Golay, X., et al.: PRESTO-SENSE: An ultrafast whole-brain fMRI technique. Magn. Reson. Med. 43(6), 779–786 (2000)

The Presentation of Evolutionary Concepts Sergey V. Kosikov1 , Viacheslav E. Wolfengagen2 , and Larisa Yu. Ismailova2(B) 1

Institute for Contemporary Education “JurInfoR-MGU”, Moscow 119435, Russian Federation [email protected] 2 National Research Nuclear University “MEPhI” (Moscow Engineering Physics Institute), Moscow 115409, Russian Federation [email protected], [email protected]

Abstract. The paper considers an approach to solving the problem of supporting the semantic stability of information system (IS) objects. A set of IS objects is addressed as a semantic network consisting of concepts and frames. The interpretation that assigns intensional (meaning) and extensional (value) characteristics to network designs is connected to the constructions of the semantic network. The interpretation in the general case depends on the interpreting subject, time, context, which can be considered as parameters. The possibility to preset a consistent interpretation for a given semantic network is regarded as a semantic integrity, and the possibility to control changes in interpretation when the parameter is changed is regarded as semantic stability. Among the tasks related to supporting semantic stability, the problem of modelling evolutionary concepts (EC) is highlighted. It is proposed to construct a computational model of EC based on the theory of categories with a significant use of the concept of variable domain. The model is constructed as a category of functors, and it is shown that the Cartesian closure of the basic category implies Cartesian closure of the category of models. The structure of the exponential object of the category of models has been studied, and it is shown that its correct construction requires taking into account the evolution of concepts. The testing of the model’s constructions was carried out when lining the means of semantic support for the implementation of the best available technologies (BAT). Keywords: Information system · Semantic network · Semantic modeling · Semantic stability · Data model · Computational model · Theory of categories

1

Introduction

The Internet technologies development brings to the change in the principles of working with information. Information systems, the content of which has been c Springer International Publishing AG 2018  A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6 16

114

S.V. Kosikov et al.

previously prepared and verified, are replaced by systems, in which the content is available to everyone for editing. The fact that there is an undefined circle of persons capable of creating and changing information objects within the system leads to a number of problems [1,2], the support of the semantic integrity of the system being the most important among them. The most general representation of the content of the information system involves the identification of objects and their links. Objects (links) within the information system, as a rule, correspond to real or conceivable objects (links) in the real world - the subject domain of the system. Establishing such a correspondence, as a rule, is considered as sensing the information object within the system (we temporarily abstract ourselves from the difference between meaning and value) or setting its semantics. Further steps for detailing the representation of the content of the information system are associated with the identification of individuals that correspond to specific objects of the domain and general notions (concepts) corresponding to the classes of individuals grouped according to some principle. The set of individuals assigned to the concept corresponds to the classical notion of the volume (or meaning) of the concept, and the set of properties and links of the concept defines its meaning or content [3,4]. Semantic integrity in the most general form can be understood as follows. First, the various subjects interacting with the system must give the same object the same meaning. Secondly, the object can appear in different places of the system’s content, i.e., in different contexts. In this case, all occurrences of the object should also be given the same meaning. The above informal considerations can be specified by various ways. The representation of the content of the information system is refined with the help of conceptual modeling methods. The conceptual model is made, as a rule, of concepts corresponding to information objects or entities, and frames corresponding to the connections of entities. Further specification is connected to the adoption of a formal system, which describes the concepts and frames. The informal semantics of entities and connections can be specified by constructing the formal semantics of the considered system, i.e., setting the interpretation of the formal system. The provision of the semantic integrity of the information system requires a model that can reflect both the loss of such integrity and its restoration. If it is necessary to take into account the subjective view on the content of the system, the modeling of the dependence of the system’s constructions interpretation on the subject is needed. This requirement imposes restrictions on the formal systems used. For example, the models of classical logic predicates do not contain tools for modeling such dependencies. Similarly, accounting for the interpretation of various occurrences of the information object requires modeling the dependence of the interpretation on the context. Such models are also difficult to make when using classical logic. As a rule, they are made within the framework of pragmatics, which takes into account the situation of its use when interpreting the expression.

The Presentation of Evolutionary Concepts

115

Specific problems appear when the information system is used for a long time. In this case, it becomes necessary to take into account the dependence of the information objects interpretation on time [5,6]. As time goes, both the volume of entities, represented within the information system, and the content of entities can change. Accordingly, the connections, in which the entities are nested, can also change. Modeling such changes also suggests the use of special logical systems, as a rule, different variants of temporal logic. All considered sources of changes assume that the dependence of the information system objects interpretation on one or another factor that can be addressed as a parameter. Consideration of parameterized interpretations that can be associated both with one way of parametrization and with several different methods is of great interest. In this case, the methods of matching the parametrization on various bases are of interest. Saving the semantic integrity of the information objects system – concepts and frames – when changing the parameters on which the interpretation depends, can be considered as the semantic stability of the system [7–9]. Dependence of the concepts and frames interpretation on the parameter brings to the need to consider evolutionary interpretations [10]. Thus, the task of modeling evolutionary concepts is the central part of the means of supporting the system’s semantic integrity and stability, which makes grounds for its relevance.

2

Task to Develop a Model of Evolutionary Concepts

The need to support semantic integrity at the informal level is taken into account when supporting the practical information systems, especially those systems that are oriented to changing the information by an undefined circle of persons. So, in electronic encyclopedias such as Wikipedia special pages are created, on which the multi-valued terms are given the variants of their interpretation. In case of creating a reference to such a term, the necessary interpretation can be chosen among the proposed ones on the technical page or created anew. It should be noted that the choice is made entirely in manual mode, there are no automation options. Up to the present moment the frame languages are the most popular languages for the representation of the subject domain, and the frame itself is understood as a “hierarchically ordered representation of the standard reality situation”. At the same time, the representation of situations in which an individual changes its former properties and begins to manifest itself as an individual with new properties, becoming indistinguishable from already existing individuals with these latter properties, does not receive a proper solution within the framework of known formalisms. The need to develop consistent methods for describing the changing notions about the subject domain leads to setting the task of developing modeling tools for evolutionary concepts that provide for: – expressions of the classical means for constructing typed descriptions of the subject domain (the Cartesian products, sums, etc.);

116

S.V. Kosikov et al.

– descriptions of the various types of parametrization (according to subject, time, context), as well as a combination of different methods of parametrization; – possibility to integrate the modeling environment with the computing environment. An important requirement for the proposed family of models is the possibility of its integration with a supporting computing environment of applicative type. Such integration provides the possibility of software support for the means of describing evolution through the use of higher order functions and makes an element of novelty of the proposed model.

3

Variable Domains in the Theory of Categories. Development of the Model for Support to Evolutionary Concepts

We give now the basic constructions for modeling the evolutionary concepts in the category. We adopt the usual definition of a category (having objects and arrows), and the usual definition of functor and natural transformation. We denote the category of sets as Set and consider for every category C the op category of contravariant functors from C to Set, which we denote as SetC . In this category functor U maps objects of the category C to sets and arrows f : a → b to functions U f : U b → U a. We often will omit the functor U and denote the value of U f on the element a as a  f . 3.1

Cartesian Closed Category

Cartesian product is an abstraction of the set of ordered pairs. A strict definition looks like this. The cartesian product of objects a and b in category C is the object d with the pair of arrows p : d → a, q : d → b that for any object c and arrows f : c → a, g : c → b there is a unique arrow h : c → d, that: f = p ◦ h,

g = q ◦ h.

Object d is denoted below as a × b. Arrows p and q are called projections for a × b. Since the arrow h, built by f and g, is the unique arrow with the specified property, it can be considered as a function of f and g. This function is denoted as f, g. In new denotations the properties of cartesian product look like f = p ◦ f, g, g = q ◦ f, g, h = p ◦ h, q ◦ h,

The Presentation of Evolutionary Concepts

117

where h : c → a × b. Sometimes it is convenient to consider an arrow f ◦ p, g ◦ q for arrows f : a → c and g : b → d. This arrow is named the product of f and g and is denoted as f × g. Not every category have the cartesian product for any two objects. If it is so, the category is named cartesian. 3.2

Cartesian Product in a Functor Category op

We show now that the category SetC is a cartesian category. Really, cartesian product can be built “pointwise”. We consider now the construction in more details. op Let U and V be two functors from SetC . Then for defining of their cartesian product it is necessary to define a functor U × V and projections (which must be the natural transformations) p : U × V → U and q : U × V → V . Then it is necessary for all natural transformations μ : W → U and ν : W → V define the natural transformation μ, ν and to check the properties of cartesian product. We define the functor U × V as follows: (U × V )A = U A × V A,

(U × V )f = U f × V f.

Then for every A in Set are defined the usual projections p : U A × V A → U A and q : U A × V A → V A. We use these projections as the components of a natural transformation – projection as follows: pA = p,

qA = q.

We have (U × V )f : (U × V )A → (U × V )A, so in the definition of the restriction mapping it is convenient to denote an element of (U × V )A on which we define the value, as (a, b). We apply now (U × V )f to the element (a, b) from (U × V )A. We have (U × V )f (a, b) = (U f × V f )(a, b) = (U f a, V f b), or (a, b)  f = (a  f, b  f ). We verify the correctness of the given definition in the appendix. We define now the arrow of pair evaluation. Let μ : W → U and ν : W → V op be arrows in the category SetC . We define μ, ν as follows: μ, νA = μA, νA. We verify its properties in the appendix. It is significant that the computations in terms of restriction mappings are in general more easy than in terms of functors. The computations in functors, however, have other advantage – they use only properties of cartesian product and do not use the definition of cartesian product in the category Set. So they can be carried out in the general case of cartesian category D. For generalization of the restriction mappings to the arbitrary category D it is necessary to propose a generalization of the “element” notion in the category D. Possibilities of such generalization will be discussed below.

118

S.V. Kosikov et al.

3.3

The Exponential

The exponential in the category is the abstraction of the set of functions. The definition is given below. The exponential of the objects a and b in the category C is such object d together with an arrow ev : d × a → b, that for every object c and arrow f : c × a → b there is the unique arrow h : c → d, for which f = ev ◦ h ◦ p, q. The object d will be denoted as a → b. The arrow h built by f , similar to the case of cartesian product, is the unique arrow with this property, so it can be considered as the function on f . This function will be denoted Λ(f ). The arrow Λ(f ) is named a currying of the arrow f . In the new notations the properties of the exponential can be rewritten so. ev ◦ Λ(f ) ◦ p, q = f, Λ(ev ◦ h ◦ p, q) = h, where h : c → (a → b). The notion of exponential uses the cartesian product of objects. So the category must be cartesian for the existence of exponentials. The cartesian category where for every two objects there is an exponential is named cartesian closed category. Not every category is cartesian closed. The arrow ev is in some case the most general of all arrows of the same type. This observation can be made more precise. The arrow ev is a counit of adjunction given by the exponential construction. This determines its “the most general” character. 3.4

The Exponential in the Functor Category op

We consider now the exponentials in the category SetC . First of all we can notice that the straightforward approach to the building of the exponentials do op not success. It is impossible to build exponentials in SetC pointwise. It is easy to built a counterexample just in the category with two objects. The reason of difficulties is that for construction of the generalization of the function space it is not enough to consider the evaluation mapping evA : (U A → V A) × U A → V A. For the building of the correct functional constructions it is necessary to consider the function values for such objects B that there exists the arrows f : B → A, that is the arrows “on the later stages” of functors U and V . These later stages must be “built in” the functor we have to define. This op way leads to the building of the correct exponential in the category SetC . We describe now this construction in details. Despite of the impossibility of the approach to the construction of the expoop nential by analogy with the cartesian product, the category SetC is still cartesian closed. But successful exponential construction is a little more complicated. op Let U and V be two functors of SetC . We define the functor U → V in the following way. The value of the object mapping of the functor on the object A

The Presentation of Evolutionary Concepts

119

is, according to D.Scott [11,12], the family of mappings U B → V B, indexed by the arrows f : B → A. That is, elements of (U → V )A are mappings ϕ from the set of arrows f : B → A to the corresponding functions U B → V B. The values of the functions corresponding to the different arrows f : B → A must be coordinated. Namely, if we consider the arrow g : C → B, then the composition f ◦ g is an arrow C → A. This composition is also a possible index in the family ϕ and, if ϕf : U B → V B then ϕ(f ◦ g) : U C → V C. The condition of the coordination requires: ϕ(f ◦ g) ◦ U g = V g ◦ ϕf. For representing this condition now in terms of restriction mapping we apply both arrows to the element b ∈ U B. We get ϕ(f ◦ g)(b  g) = ϕf (b)  g. The required condition is a variant of naturality. Below we consider it in detail. The arrow mapping for functor U → V must match each arrow f : B → A to the function transforming the mappings ϕ from arrows C → A to the functions U C → V C to the mappings ψ from arrows C → B to the functions U C → V C. Symbolically: (U → V )A = {ϕ : (h : C → A) → (U C → V C)} (U → V )B = {ψ : (g : C → B) → (U C → V C)} (U → V )f : (U → V )A → (U → V )B where f : B → A. So the input for our task is the arrow f : B → A and the mapping ϕ which transforms each arrow h : C → A to the function U C → V C (it is a set-theoretical function because the functors U and V have their values in the category Set). The output is the mapping ψ, which transforms each arrow g : C → B to the function U C → V C. The natural way to define this mapping is to convert the arrow g : C → B to the arrow h : C → A and then apply the mapping ϕ. For the conversion g to h it is enough to assume h = f ◦ g. Now we formalize the construction. We denote ((U → V )f )(ϕ) = ψ. Then the value of ψ on the arrow g : C → B can be defined as ψg = ϕ(f ◦ g). This definition can be represented formally as ((U → V )f (ϕ))g = ϕ(f ◦ g). This formula defines a restriction mapping for the functor U → V : ϕ  f = ψ where ϕ ∈ (U → V )A. Here ψg = ϕ(f ◦ g), that is (ϕ  f )g = ϕ(f ◦ g).

120

S.V. Kosikov et al.

This is the very construction that can not be generalized to the arbitrary category D. But there is rather wide class of categories allowing such generalization. We check the correctness of the definition in the appendix. Now we define the arrow of the evaluation mapping ε : (U → V ) × U → V . Then εA : (U → V )A × U A → V A and εA(ϕ, a) = ϕ(1A )(a). We check the natuality of the given construction in the appendix. Now we consider currying. Let ψ : U × V → W , then ψA : U A × V A → W A and ψB : U B × V B → W B. So Λψ : U → (V → W ) and (Λψ)A : U A → (V → W )A. We define (Λψ)Aa = ϕ where ϕf : V B → W B where f : B → A. Let b ∈ V B. Then ϕf (b) = ψB(U f a, b) or ((Λψ)A)f b = ψB(a  f, b) We check the properties of the currying in the appendix.

4

The Model of the Evolutionary Concepts

Now we present the model of data object conversion as the special applicative system. Every object of the model can be treated as function. The model is constructed as the projective limit in the category of complete partial orders and their continious maps. In the construction of the model we follow D. Scott [13]. We assume the definition of partial order and the associated notions (upper bound, lower bound etc.) and their elementary properties. A set with partial order will be denoted as D = (D, ). The subset B of D is called directed set, if it is not empty and for all x, y ∈ B there exists z ∈ B such that x  z and y  z. The partial ordered set (poset) is full, if there is a minimal element ⊥ ∈ D (for all x ⊥  x) and for every directed subset B of D there exists a supremum B ∈ D. The map f : D → D is continious if for all directed subsets B of D f ( B) = f (x)|x ∈ B. It is easy to show that full posets and their continious maps form a category, which we call CPO. The category CPO has good properties. We define for posets D and D their cartesian product D × D as sets and the partial order on it: (x, x )  (y, y  ) if and only if x  y and x  y  . It is easy to show that D × D becomes the full poset and the given construction forms the cartesian product in the category CPO.

The Presentation of Evolutionary Concepts

121

Now we consider the set [D → D ] = {f : D → D |f is continious} and define the partial order f  g if and only if for all x

f (x)  g(x).

It is possible to show that [D → D ] is a full poset and there exists application mapping App : [D → D ] × D → D and currying mapping: f → f ∗ : [D × D → D ] → [D → [D → D ]] which are continious. So this construction forms the exponenial in the category CPO and the category CPO is cartesian closed. Now we construct the model as the projective limit in the category CPO. We give now necessary definitions. Let D0 , D1 , . . . be posets and fi ∈ [Di+1 → Di ]. The projective limit lim(Di , fi ) is a poset D = {x0 , x1 , . . .|xi ∈ Di and fi (xi+1 ) = xi } and x0 , x1 , . . .  y0, y1, . . . if and only if for all i xi  yi . It is easy to show that D is a full poset, so the category CPO allows the projective limits. For constructing the objects of the model it is necessary to establish a connection between D and [D → D]. We use the following construction. The pair of maps (ϕ, ψ) is called a projection of D to D, if ϕ : D → D , ψ : D → D are continious; ψ ◦ ϕ = idD ϕ ◦ ψ  idD , where idD is the identity function for D. It is easy to show that if (ϕ, ψ) is a projection of D to D, then (ϕ∗ , ψ ∗ ) is a projection of [D → D ] to [D → D], where for f : D → D, g : D → D ϕ∗ (f ) = ϕ ◦ f ◦ ψ, ψ ∗ (g) = ψ ◦ g ◦ ϕ. It is easy to show that D is a full poset, so the category CPO allows the projective limits. Now we can build the model. Let us start with the projection [D → D] to D: ϕ0 (x)(y) = x,

ψ0 (f ) = f (⊥),

122

S.V. Kosikov et al.

where ⊥ is the minimal element of full poset D. Let us define the projective limit D0 = D Dn+1 = [Dn → Dn ] (ϕn+1 , ψn+1 ) = (ϕ∗n , ψn∗ ) It is possible to show that this construction has the required properties.

5

Testing the Constructions of a Model

The methods of work with evolutionary concepts have been partially tested when developing the software for informational support to the process of BAT implementation. In particular, the methods of supporting the evolutionary trajectory of an object and its accounting when manipulating an object were used in the editor of conceptual descriptions of the subject domain. The conceptual descriptions editor is oriented to the description of concepts of a fairly general structure. The concept description is accompanied by a description of the characteristic frames of the given concept. The arguments of characteristic frames may be either simple concepts or the conceptual operations results. The editor provides for two metalanguages, one of which is intended to describe the structure of the edited concepts, and the other sets the concept representation for users. The evolutionary trajectory of the concept is a part of the concept structure. It is possible to select concepts according to the condition specified at the evolutionary trajectory, as well as to change the concept according to the rule that depends on its evolutionary trajectory. Separate working mechanisms for evolutionary concepts have also been tested when developing the training and methodological complex on the legal basis for the BAT implementation. One of the information elements of the complex was the training program. Since the BAT implementation is a rapidly developing sector of law, the programme underwent evolution during the complex development. As separate stages of evolution caused methodological interest, the data structure was chosen to represent the evolving programme. A set of interface elements providing navigation through the evolutionary structure was also proposed. The testing in the whole demonstrated a significant expansion of the capabilities of applied systems, arising from the introduction of evolutionary objects into them, and at the same time, the insufficient degree of methods development for describing and manipulating such objects, making it difficult to develop the means of visualization, navigation and transformation of evolutionary objects. As expected, a number of problems can be overcome by considering the basic categories with a richer internal structure.

6

Conclusion

The paper considers the task of supporting the semantic stability of information system objects. The solution of the task is proposed basing on the concept of

The Presentation of Evolutionary Concepts

123

an evolutionary concept. There are presented the tools for constructing a model of such concepts, based on category theory means, and using the concept of a variable domain. The solution is sought inside a Cartesian closed category. The category of functors with values in a Cartesian closed category is considered, the functors being used to model the evolution of concepts. It is shown that the Cartesian product in the category of functors can be defined pointwise, and the definition of the exponential requires for the specification of additional information describing the behavior of the simulated concept in earlier and later stages of evolution. The formal correctness of the introduced definitions is shown. The elements of the proposed model have been tested when developing a specialized information system in the field of legal support to the introduction of the best available technologies. The results of the implementation show the practical applicability of the proposed approach. Acknoledgements. Authors acknowledge support from the MEPhI Academic Excellence Project (Contract No. 02.a03.21.0005). The research is supported in part by the RFBR grant 15-07-06898.

Appendix The appendix contains proofs of the properties of the defined constructions. The Cartesian Product in the Category Set. Category Set of sets is cartesian. Cartesian products in it are built as usual as sets of pairs: if A and B are two sets (objects of category Set), their cartesian product is a set {(a, b)|a ∈ A, b ∈ B}. Projections are defined by the natural way: p(a, b) = a,

q(a, b) = b.

The arrow of pair evaluation f, g for f : C → A, g : C → B is defined so: f, g(c) = (f c, gc), where c ∈ C. It is easy to check that the product of arrows is evaluated so: (f × g)(a, b) = (f a, gb). The Properties of the Cartesian Product. 1. U × V preserves the composition. Really, (U × V )(f ◦ g) = U (f ◦ g) × V (f ◦ g) = (U (f ) ◦ U (g)) × (V (f ) ◦ V (g)) = U (f ) ◦ U (g) ◦ p, V (f ) ◦ V (g) ◦ q = = (U f × V f ) ◦ (U g × V g) = (U × V )f ◦ (U × V )g. 2. U × V preserves the unit arrows. Really, (U × V )1 = U 1 × V 1 = 1 × 1 = 1 ◦ p, 1 ◦ q = p, q = 1.

124

S.V. Kosikov et al.

So U × V is really a functor. We verify now that p and q are natural transformations. So we check the condition of naturality pB ◦ (U × V )f = p ◦ (U f × V f ) = p ◦ U f ◦ p, V f ◦ q = U f ◦ p = U f ◦ pA. The condition satisfies so p is really a natural transformation. The check for q is analogous. This check shows that projections in Set are defined really naturally. We can make the evaluations above also in terms of restriction mappings. The indexes of objects can be reconstructed in a unique way, and so they can be omitted because of natural property of the projection. The Properties of the Pairing. We check that defined arrow is a natural transformation, i.e. check the naturality condition: μ, νB ◦ W f = μB ◦ W f, νB ◦ W f  = U f ◦ μA, V f ◦ νA = U f ◦ p, V f ◦ q ◦ μA, νA = ((U × V )f ) ◦ μ, νA. The condition is satisfied, so the natural transformation μ, ν is defined correctly. Now we have to check the characteristic properties of the projection and pairing. We show now that p ◦ μ, ν = μ. We calculate a component (p ◦ μ, ν)A and see that it is equal to ?A. We have (p ◦ μ, ν)A = pA ◦ μ, νA = p ◦ μ, νA = μA. So components of p ◦ μ, ν are really identical with components of μ and so p ◦ μ, ν = μ. We show now that p ◦ η, q ◦ η = η, where η : W → (U × V ). We have p ◦ η, q ◦ ηA = pA ◦ ηA, qA ◦ ηA = p ◦ ηA, q ◦ ηA = ηA. So this characteristic property is also satisfied. The Properties of the Exponential. We have defined the object and arrow mappings for the functor. Now we check that this is really a functor. We must check the composition preservation property (U → V )(f ◦ g) = (U → V )f ◦ (U → V )g and the unit preservation property (U → V )(1A ) = 1(U →V )A . We begin with the composition. Let f : B → A and g : C → B. Then (U → V )(f ◦ g) : (U → V )A → (U → V )C. The elements of (U → V )A are the families ϕ of mappings. For the comparison of these families we apply them to the arrow h : D → C. Then (U → V )(f ◦ g)(ϕ) = ψ, ψh = ϕ(f ?g)?h

The Presentation of Evolutionary Concepts

and

125

((U → V )f ◦ (U → V )g)(ϕ)h = (U → V )f ((U → V )g)(ϕ)h) = (U → V )f (ϕ)(g ◦ h) = ϕ(f ◦ (g ◦ h)).

The property of the unit preservation can be checked in the similar way. The Properties of the Currying. We check the functor character of the defined mapping. We have (Λψ)B ◦ U f = (V → W )f ◦ (Λψ)A ((Λψ)((U f )a))g(b) = ψC(U f a(g), b) = ψC(a(f ◦ g), b) ((V → W )f ◦ (Λψ)Aa)g(b) = (Λψ)Aa)f ◦ g(b) = ψC(a(f ◦ g), b).

References 1. Atzeni, P., et al.: The relational model is dead, SQL is dead, and I don’t feel so good myself. SIGMOD Rec 42(1), 64–68 (2013) 2. Chernyshov, A., Balandina, A., Kostkina, A., Klimov, V.: Intelligence search engine and automatic integration system for web-services and cloud-based data providers based on semantics. Procedia Comput. Sci. doi:10.1016/j.procs.2016.07.449 3. Wolfengagen, V.E., Ismailova, L.Y., et al.: Evolutionary domains for varying individuals. Procedia Computer Science. Doi:10.1016/j.procs.2016.07.447 4. Cuzzocrea, A., Sellis, T.: Semantics-aware approaches to big data engineering. J. Data Semant. 6(2), 55–56 (2017) 5. Ismailova, L.: Criteria for computational thinking in information and computational technologies. Life Sci. J. 11(9s), 415–420 (2014) 6. Castro,G., Costa, B.: Using data provenance to improve software process enactment, monitoring and analysis. In: Proceedings of the 38th International Conference on Software Engineering Companion, ICSE 2016, pp. 875-878. ACM, New York (2016) 7. Wolfengagen, V., et al.: Migration of the Individuals. Procedia Computer Science 88, 359–364 (2016). doi:10.1016/j.procs.2016.07.449 8. Comyn-Wattiau, I. et al.: Conceptual Modeling, Proceedings of 35th International Conference ER 2016, Gifu, Japan, November 14-17, 2016. LNCS, vol. 9974. Springer (2016) 9. Population Modeling Working Group. Population modeling by examples (wip). In: Proceedings of the Symposium on Modeling and Simulation in Medicine, MSM 2015, pp. 61–66. Society for Computer Simulation International, San Diego (2015) 10. Wolfengagen, V.E., Ismailova, L.Y., Kosikov, S.V.: Computational model of the tangled web. Procedia Comput. Sci. doi:10.1016/j.procs.2016.07.440 11. Scott, D.: Advice in modal logic. In: Lambert, K. (ed.) Philosophical Problems in Logic. Reidel (1970) 12. Scott, D.S.: Relating theories of the lambda calculus. In: Hindley, J., Seldin, J. (eds.) To H.B.Curry: Essays on Combinatory Logic, Lambda Calculus and Formalism, pp. 403-450. Academic Press, New York (1980) 13. Scott, D.: The lattice of flow diagrams. In: Symposium on Semantics of Algorithmic Languages, pp. 311-366. Springer (1971)

Semantic Comprehension System for F-2 Emotional Robot Artemy Kotov1 ✉ , Nikita Arinkin1, Alexander Filatov2, Liudmila Zaidelman1, and Anna Zinina1 (

1

)

National Research Center “Kurchatov Institute”, pl. Kurchatova, 1, Moscow, Russia [email protected] 2 Samsung R&D Institute Rus, Dvintsev 12/1, Moscow, Russia

Abstract. Within the project of F-2 personal robot we design a system for auto‐ matic text comprehension (parser). It enables the robot to choose “relevant” emotional reactions (output speech and gestures) to an incoming text – currently in Russian. The system executes morphological and syntactic analysis of the text and further constructs its semantic representation. This is a shallow representation where a set of semantic markers (lexical semantics) is distributed between a set of semantic roles – structure of the situation (fact). This representation may be used as (a) fact description – to search for facts with a given structure and (b) basis to invoke emotional reactions (gestures, facial expressions and utterances) to be performed by the personal robot within a dialogue. We argue that the execu‐ tion of a relevant emotional reaction can be considered as a characteristic of text comprehension by computer systems. Keywords: Natural language comprehension · Syntactic parser · Text analysis

1

The Problem of Text Comprehension

Since the Chinese room argument [1] the problem of automatic text comprehension became one of the cornerstone questions in Computer Linguistics and Artificial Intel‐ ligence. In his original publication Searle has argued that no artificial computer can understand natural text in a way people do. Since then – numerous critics have suggested the architectures of “understanding”, which apply to human comprehension and also can be implemented on a basis of a computing machine. The development of robot companions has opened another view into the problem: human infers, or rather – feels being understood, basing on replies and emotional reactions from the interlocutor. This view has very little to do with internal architecture of the software. As suggested by our communication with dogs – humans feel being understood, basing solely on the behav‐ ioral responses from dogs, without any speech interaction. Following these observation we develop the project of F-2 companion robot with the emotional reactions as the main

Design of the syntactic parser is supported by RFBR grant 16-29-09601 ofi_m, and design of the F-2 robot is executed within the research program of “Kurchatov Institute”. © Springer International Publishing AG 2018 A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6_17

Semantic Comprehension System for F-2 Emotional Robot

127

component to maintain contact with humans. Unlike dogs, F-2 processes natural speech and suggests emotional comments and reactions. To enable this capacity we develop a syntactic parser, which constructs shallow semantic representation, suitable to select a relevant emotional reaction and suggest the distribution of emotional roles in a situation.

2

Approaches to Text Comprehension and Sentiment Analysis

The task of extracting emotional evaluations from text is usually solved using the bagof-words method. The text (as a whole or each sentence separately) is represented as a set of word forms or lexemes; the position of the word in the text or in the sentence is not taken into account. This approach is presented in a number of papers [2–5]. In order to minimize numerous shortcomings of this approach researchers often use the bag-ofn-grams – an unordered set of tuples consisting of n consecutive words [2, 6]. Dialogue systems also often focus on individual words or n-grams to choose their answers. However, intelligent systems for text analysis and user interaction should determine the role of a particular character of the text in a situation, attribute the character to a certain syntactic or semantic valence and recognize the situation frame. Apart from a complete syntactic analysis of the text there are several alternatives that extract fact structure with the help of partial analysis. For example, in [7] text is parsed into so-called T-expressions: three element tuples . Texpressions are used as the basis both for sentiment analysis task and for other applica‐ tions such as automatic question answering. A similar approach is used in [8]. The suggested system divides each fact into four parts: (a) an object from the thesaurus, (b) the type of syntactic relationship between the object and the member of the sentence syntactically associated with the object, (c) member of the sentence associated with the object, and (d) presence of a negation. Syntactic relationships are extracted by Tomita parser – a tool for context-free grammars – equipped with 50 rules. The resulting syntax group of four elements is a subject to further analysis and evaluation. ABBYY Compreno parser [9] for each sentence in the source text constructs a tree, whose nodes have not only grammatical characteristics, but also attributed semantic classes from the ontology. The resulting tree is used as a basis for facts extraction. Once a rule is triggered, the proposition associated with the object is extracted. Thus, the semantics of the sentence is presented as a set of propositions related to given objects. The extracted collection of facts enriches the tree with new characteristics, which in turn triggers other rules, allowing new propositions to be extracted. In Sentilo project [10], a complex linguistic model that includes a variety of linguistic resources and tools is used to perform sentiment analysis. Evaluation of the sentence is computed from RDF graph. The graph nodes are syntactic elements and their ontological classes, the edges – are the relations derived from several of linguistic theories and ontologies. Emotional evaluation is calculated for significant actants and for the whole predicative structure. The averaged positive and negative evaluations are used as the final sentiment score for the whole sentence.

128

A. Kotov et al.

We can see that the main tendency in the field is to construct complete syntactic trees and use extracted facts as reference structures. In our work we separate syntactic and semantic representations, as well as process semantic representations by numerous scripts, responsible for emotional arousal and robot’s reactions.

3

Parser Architecture

Parser is designed to construct semantic representation, suitable to apply an emotional reaction towards a specific participant. In particular, some situations can be considered as ‘terrible’ with possible aggression to be conveyed to A, or as ‘pity’ with possible compassion to be expressed to B. To construct the representation the parser implements morphological, syntactic and semantic processing of the incoming text as suggested by theoretical linguistic models, e.g. [11]. The parser is written in C#, the grammar is in syntXML format and the dictionary is stored in SQL database. On each step of processing the parser may upload the results of analysis to an SQL database or transfer them to other software components, e.g. to F-2 robot (see Fig. 1).

Fig. 1. Architecture of the text analysis and reaction transfer to F-2 robot

3.1 Morphological Processing: Stemmer and Dictionary Stemmer relies on a database, which keeps all wordforms and grammemes for 48,000 lemmas. The dictionary is based on OpenCorpora project [12]. 28,000 words in the dictionary are annotated by semantic markers (from 1 to 18 markers per word, average 2). Markers are assigned (a) basing on hyperonyms – and represent the semantic class of the word, and (b) basing on the sensitive semantic features, for example, ‘intensity’ can be emotionally relevant in phrases like Why do you push me? [13]. Unlike traditional ontologies, a word may keep semantic markers from different classes: bank has the markers for ‘organization’, ‘building’ and ‘abstract container’. This polysemy allows us to simulate “situational effects”, where a word meaning may be shifted depending on the situation or by the emotion to be invoked by semantics – top-down emotional processing [14]. So different reference frames of scripts (units for inferences and emotional reactions) may address different focal markers in the semantics of a word. In addition to polysemy, we describe lexical homonymy: markers can be assigned to several meanings of a word (like bank1 – financial institution, building vs. bank2 – river bank). A script will also select the meaning, which fits better to its reference frame.

Semantic Comprehension System for F-2 Emotional Robot

129

3.2 Syntactic Processing: syntXML We develop a dependency parser with left-to-right approach. After the morphological component assigns morphological hypotheses to each segment (token) – the segment is added to a syntactic stack, and the syntactic component tries to reduce the stack head with the grammar rules. A rule is defined as a possible reduction, where the right-hand side can be reduced to the left-hand head h (1). Head h can also be a member of the right-hand side and subordinate all other right-hand side segments (2). h → < a, b, … n >

(1)

h → < a, b, h, … n > or < a, b, hhead , … n >

(2)

Right-hand side of the rule may have a variable number of segments (1 or more) as well as optional segments. The grammar contains 490 rules, written on a specially designed syntXML language [15]. Application of each rule is evaluated, scores are calculated on the basis of SynTagRus treebank [16] – total score is calculated for a stack. Once a rule is applied, it may assign a semantic role to a segment. We rely on this list of semantic roles, suggested in [17]: ag (agent), pat (patient) etc. The predicate is assigned to p semantic role. This procedure locates clauses in a tree – where each clause consists of a predicate p and a number of its actants. For each type of homonymy appearing within a stack – lexical or morphological ambiguity, ambiguity of rules application – the stack is duplicated. So on each step the parser works with n stacks with highest total scores (for standard tasks we set n = 1000), stacks with lower scores are discarded. 3.3 Semantic Processing: Scripts Each tree is subdivided by syntactic rules into clauses – a predicate and a list of actants. For each actant – semantic markers of the head word (noun, verb) and subordinate words (adjectives, adverbs) are extracted from the dictionary and assigned to the semantic role of the actant. This constructs a semantic frame, representing a single clause (Table 1). Table 1. Semantic representation (frame) for the utterance A real man is always interested in the life of the beloved girl p (predicate) think, pay-attention, frequently

ag (agent) object, somebody, man, positive

pat (patient) abstract, time-period, existence, object, somebody, woman, ofminimal-age, positive

As suggested by M. Minsky [18], artificial agents may have numerous models for drives and emotions – proto-specialists, which compete to control the agent behavior. Further A. Sloman [19] has suggested CogAff architecture, where scripts, responsible for emotional processing, compete with scripts, devoted to rational (deliberative) procedures and meta-management (reflexive thinking). We rely on the list of scripts for emotional processing represented in [20]. It includes 13 scripts for negative situations: DANGER,

130

A. Kotov et al.

APPROPR (“Appropriation”), SUBJV (“Subjectivity” – e.g. ‘all he thinks about is himself!’) etc., and 21 scripts for positive situations: CONTROL, CARE, COMFORT, ATTENTION (e.g. ‘they all adore you!’), APPROVAL (e.g. ‘you did it like a real man!’) etc. The semantic representation in Table 1 activates the following scripts: • PLAN: Somebody plans something frightening against me – ‘man makes some evil plans against woman’ • SUBJV (“Subjectivity”): Somebody is narrow-minded, thinks only about one thing – ‘all men think about are women’. • ATTENTION: Subject is pleased, because somebody pays an attention at him – ‘woman is happy because of the men’s attention’. • APPROVAL: Somebody acts like a hero, does something right – ‘real men do it right to pay attention’. Although APPROVAL and ATTENTION are more relevant, we do not consider the activation of PLAN and SUBJV as false positive. These reactions can be used (a) to generate latent behavioral patterns of the robot (where it is happy but afraid to attract attention), (b) to express mood – where a “depressive” robot prefers negative reactions, (c) by the mechanism of irony to generate sarcastic responses and simulate the sense of humor [21]. Scripts are also helpful to solve syntactic homonymy: if numerous trees are exported by the syntactic analysis, the semantic processor chooses the tree, which has the highest degree of similarity with reference frames of the scripts – which corresponds to a more standard situation or is more likely to invoke emotions.

4

Robot and the Transfer or Behavior

As suggested in [22] the development cycle for a computer agent should include (a) collection of human behavior into a multimodal corpus, (b) design of the behavioral model, (c) implementation of the model in a computer agent, simulation of the observed behavior and (d) test of the simulated behavior. In our studies we collect and annotate records of the multimodal behavior within the project of the Russian Emotional Corpus – REC [23]. We also design software to operate a robot companion and F-2 robot – as a demonstrator of the software. We observe behavioral patterns – gestures and facial expressions, typical to express certain communicative functions [24], draw the patterns in 3D model and save them in a library to be accessed by the robot. Each script is assigned to one or several behavioral reactions: utterance pattern and a BML record – Behavior Markup Language [25]. Sematic analysis of an input text activates one or several scripts, which send their BMLs to robot for execution. BMLs can compete for the robot actua‐ tors, which results in richer and more compound behavior, where numerous reactions are expressed at the same time. As shown, text comprehension can be an important component within the design of a robot companion, which maintains emotional contact with a human. Text under‐ standing here is implemented by the construction of a shallow semantic representation

Semantic Comprehension System for F-2 Emotional Robot

131

and the selection of a relative emotional reaction. This representation can also serve as a basis for knowledge extraction and semantic search.

References 1. Searle, J.: Minds, brains, and programs. Behav. Brain Sci. 3, 417–424 (1980) 2. Pang, B., Lee, L., Vaithyanathan, S.: Thumbs up?: sentiment classification using machine learning techniques. In: Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing, vol. 10, pp. 79–86. Association for Computational Linguistics (2002) 3. Su, F., Markert, K.: From words to senses: a case study of subjectivity recognition. In: Proceedings of the 22nd International Conference on Computational Linguistics, vol. 1, pp. 825–832. Association for Computational Linguistics, Manchester (2008) 4. Chetviorkin, I.I.: Testing the sentiment classification approach in various domains — ROMIP 2011. In: Computational Linguistics and Intellectual Technologies, vol. 2(11), pp. 15–26. RSUH, Moscow (2012) 5. Fang, L., Huang, M.: Fine granular aspect analysis using latent structural models. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers, vol. 2, pp. 333–337. Association for Computational Linguistics, Jeju Island (2012) 6. Poroshin, V.: Proof of concept statistical sentiment classification at ROMIP 2011. Computational Linguistics and Intellectual Technologies, vol. 2(11), pp. 60–65. RSUH, Moscow (2012) 7. Katz, B.: From sentence processing to information access on the world wide web. In: AAAI Spring Symposium on Natural Language Processing for the World Wide Web (1997) 8. Mavljutov, R.R., Ostapuk, N.A.: Using basic syntactic relations for sentiment analysis. In: Computational Linguistics and Intellectual Technologies, vol. 2(12), pp. 91–100. RSUH, Moscow (2013) 9. Anisimovich, K.V., Druzhkin, K.J., Minlos, F.R., Petrova, M.A., Selegey, V.P., Zuev, K.A.: Syntactic and semantic parser based on ABBYY Compreno linguistic technologies. In: Computational Linguistics and Intellectual Technologies, vol. 2(11), pp. 91–103. RSUH, Moscow. (2012) 10. Recupero, D.R., Presutti, V., Consoli, S., Gangemi, A., Nuzzolese, A.G.: Sentilo: frame-based sentiment analysis. Cogn. Comput. 7, 211–225 (2014) 11. Melčuk, I.A.: The experience of the theory of linguistic models “MEANING TEXT” (in Russian). Languages of the Russian Culture, Moscow (1999) 12. Bocharov, V.V., Alexeeva, S.V., Granovsky, D.V., Protopopova, E.V., Stepanova, M.E., Surikov, A.V.: Crowdsourcing morphological annotation. In: Computational Linguistics and Intellectual Technologies, vol. 12(19), pp. 109–114. RSUH, Moscow (2013) 13. Apresian, V.Yu.: Implicite aggression in the language (in Russian). In: Kobozeva, I.M., Laufer, N.I., Selegey, V.P. (eds.) Computational Linguistics and Intellectual Technologies, pp. 32–35. Nauka, Moscow (2003) 14. Clore, G.L., Ortony, A.: Cognition in emotion: always, sometimes, or never? In: Lane, R.D., Nadel, L. (eds.) Cognitive Neuroscience of Emotion, pp. 24–61. Oxford University Press (2000) 15. Kotov, A., Zinina, A., Filatov, A.: Semantic parser for sentiment analysis and the emotional computer agents. In: Proceedings of the AINL-ISMW FRUCT 2015, pp. 167–170 (2015)

132

A. Kotov et al.

16. Boguslavsky, I.M., Iomdin, L.L., Chardin, I.S., Kreidlin, L.G.: Development of a dependency treebank for russian and its possible applications in NLP. In: Proceedings of the Third International Conference on Language Resources and Evaluation (LREC-2002), vol. III, pp. 852–856. European Language Resources Association, Paris (2002) 17. Fillmore, C.J.: The case for case. In: Emmon, B., Harms, R. (eds.) Universals in Linguistic Theory, pp. 1–68. Holt, Rinehart & Winston, New York (1968) 18. Minsky, M.L.: The Society of Mind. Touchstone Book, New-York, London (1988) 19. Sloman, A., Chrisley, R.: Virtual machines and consciousness. J. Conscious. Stud. 10, 133– 172 (2003) 20. Kotov, A.A.: Mechanisms of speech influence (in Russian). Kurchatov Institute, M. (in print) 21. Kotov, A.: Accounting for irony and emotional oscillation in computer architectures. In: Proceedings of International Conference on Affective Computing and Intelligent Interaction ACII 2009, pp. 506–511. IEEE, Amsterdam (2009) 22. Rehm, M., André, E.: From annotated multimodal corpora to simulated human-like behaviors. In: Modeling Communication with Robots and Virtual Humans, pp. 1–17 (2008) 23. Kotov, A., Budyanskaya, E.: The Russian emotional corpus: communication in natural emotional situations. In: Computational Linguistics and Intellectual Technologies, vol. 11(18), pp. 296–306. RSUH, Moscow (2012) 24. Kotov, A.A., Zinina, A.A.: Functional analysis of the nonverbal communicative behavior (in Russian). In: Computational Linguistics and Intellectual Technologies, vol. 1(14), pp. 299– 310. RSUH, Moscow (2015) 25. Kopp, S., Krenn, B., Marsella, S., Marshall, A., Pelachaud, C., Pirker, H., Thórisson, K., Vilhjálmsson, H.: Towards a common framework for multimodal generation: the behavior markup language. In: Intelligent Virtual Agents, pp. 205–217 (2006)

Methodology of Learning Curve Analysis for Development of Incoming Material Clustering Neural Network Boris Onykiy, Evheniy Tretyakov ✉ , Larisa Pronicheva, Ilya Galin, Kristina Ionkina, and Andrey Cherkasskiy (

)

National Research Nuclear University MEPhI (Moscow Engineering Physics Institute), Moscow, Russia {bnonykij,lvpronicheva,aicherkasskij}@mephi.ru, [email protected], [email protected], [email protected]

Abstract. This paper describes the methodology of learning curve analysis for development of incoming material clustering neural network. This methodology helps to understand deeply the learning curve adequate level and to bring learning curve structure to the relevant one of the thematic scope of incoming materials. The methodology is based on visual analysis and comprises the building of directed graphs in order to identify data templates. As the battlefield for material clustering the Nuclear Infrastructure Development Section (NIDS) of the Inter‐ national Atom Energy Agency (IAEA) is selected as the support from NIDS’ experts had been available during the research. Some of the challenges the NIDS faces are data aggregation for Country Nuclear Infrastructure Profiles (CNIP) and data assessment after Nuclear Infrastructure Review Missions (INIR). Keywords: Neural network · Material clustering · Learning curve · Visual analysis

1

Introduction

The Nuclear Infrastructure Development Section (NIDS) works with Member States to improve: understanding of the requirements and obligations essential to implementing nuclear power programmes; and abilities to develop the necessary infrastructure for introducing nuclear power [1]. Member States that are new for nuclear technologies are called newcomers and if newcomers are intended to get nuclear technologies for peaceful purposes, they have to meet special conditions on every phase and relevant issues in order to develop infra‐ structure for nuclear power plant. These phases and issues are described in the “Mile‐ stones in the development of a national infrastructure for nuclear power” (NG-G 3.1), so called the “Milestone document”. The “Milestone document” is also used by newcomers to assess their own development status, and to prioritize their activities towards the development of nuclear infrastructure for nuclear power plant.

© Springer International Publishing AG 2018 A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6_18

134

B. Onykiy et al.

Also, the NIDS provides newcomers with special types of Integrated Nuclear Infra‐ structure Review (INIR) missions to assess infrastructure development and to provide newcomers with guidelines, recommendations and relevant documentation. The assess‐ ment of nuclear infrastructure development is performed in compliance with “Evalua‐ tion of the Status of National Nuclear Infrastructure Development” (NGT-3.2 Rev.1). Besides, the experts of NIDS aggregate information from different sources to the Country Nuclear Infrastructure Profile database (CNIP) to be well informed and to keep tracking of the nuclear infrastructure development, thereby experts meet the challenge of structuring large amounts of information. The development of incoming material clustering neural network is evaluated as one of the solutions towards structuring large amounts of information. Before starting building neural network the development of the ontology was performed in order to prepare learning curve. This paper describes the methodology of learning curve analysis for development of incoming material clustering neural network.

2

Methodology

The methodology enable researcher to identify the completeness of learning curve and completeness of terminology sources. The process of gathering and analyzing data for learning curve comprises several stages: • identification of the NIDS’ working documents, the terminology from the documents are to be included into learning curve. • ontology structure development, the structure should reflect the structure of the data‐ base where incoming materials will be stored that means the database structure has to explain well thematic areas of incoming materials. • the NIDS’ working documents distribution, the working documents have to be distributed among main classes of the developed ontology structure. • the terminology extraction, the specific terminology from identified working docu‐ ments has to be extracted and distributed among relevant ontology structure classes due to complete ontology. • weights calculation, weights reveal the level of relevance of the term to the thematic areas. • building of ontology chart, ontology chart reveals relations between terminology and ontology classes. • analyzation of ontology chart, the completeness of ontology and the completeness of working materials. The IAEA publications were selected as the main source for ontology terminology. These publications are working documents for NIDS and they are used as assessment materials during INIR missions or materials that are used as guidelines for newcomers. The list of working documents is further: • “Milestones in the development of a national infrastructure for nuclear power” (NGG 3.1). • “Managing Human Resources in the Field of Nuclear Energy” (NG-G 2.1).

Methodology of Learning Curve Analysis

135

• “Workforce Planning for New Nuclear Power Programmes” (NG-T 3.10). • “Responsibilities and Capabilities of a Nuclear Energy Programme Implementing Organization” (NG-T-3.6). • “Managing Siting Activities for Nuclear Power Plants” (NG-T-3.7). • “Establishing the Safety Infrastructure for a Nuclear Power Programme” (SSG-16). The list of these documents was provided by NIDS’ experts. The access for their support was available during the research. Also these materials are used in the devel‐ opment of “Competency framework” – the database comprises key activities to be implemented towards development of nuclear infrastructure. The relevant issues for these documents are also provided by NIDS’ experts. The structure for ontology was developed on the basis of “Milestone Approach” that is described in “Milestone document”. The “Milestone Approach” identifies phases that newcomers have to reach due to complete nuclear infrastructure development and issues that describes thematic areas of main obstacles during the development of nuclear infra‐ structure. Totally three phases and nineteen issues identified in the “Milestone Approach” the chart of nodes and edges are presented in the Fig. 1.

Fig. 1. The ontology structure.

The extraction of special terminology from working documents was performed manually by young specialists in this field, such practice minimizes the probability of wrong data extraction. One thousand three hundred terms were extracted that were divided into fifty-seven vocabularies – nineteen vocabularies per phase. W=

∑texts text

F(term, text)

(1)

The weights were calculated after the terminology extraction. The weights represent level of relevance of the term to the specific issue and issue to the specific phase, mind that every issue is different in every phase [2]. Weights for terminology are calculated as the frequency of term usage in every document relevant to every issue and weights

136

B. Onykiy et al.

for issues are calculated as sum of relevant terms’ weights. The weight of term to issue is calculated with the help of Formula 1. The weights of edges between terminology and issues were calculated with help of scripts written in Python 3.5. The provision for building directed graph are nodes, edges and weights of edges [3]. Nodes are recognized as classes of the developed ontology: first class – phases, second class – issues and third class – terminology. Edges represent the linkage between classes, in our directed graph they are: terminology to issues and issues to phases. The graph presented in Fig. 2 was built using the Gephi software and algorithm of Yifan Hu [4].

Fig. 2. The NIDS ontology graph.

The graph shows well the more and the less frequent terminology. The most frequent terms are located in the center and the least ones are on the borders. Terminology that are located on the borders get high biases in neural network as they are relevant to specific issues. Analysis of weights of the most frequent terms that are located in the center shows the adequate level of learning curve, i.e. the term NEPIO

Methodology of Learning Curve Analysis

137

has large relevance to first phase, less to the second phase and the least to the third phase as well as on practice.

3

Results and Discussion

In this subsection we are presenting the experimental results obtained during the devel‐ opment of learning curve for incoming material clustering neural network. The learning curve is represented as ontology with three main classes – phases, nineteen subclasses – issues and terms that take the role of characteristics in the neural network. Totally six specialized documents (listed above) were processed to extract more than 1300 terms. The sample of final data is presented in Table 1. Table 1. The sample of data for weights of terms to issues Term Site Safety Nuclear Nuclear Regulator Nuclear Organization Nuclear Safety NEPIO Programme Training Criteria Nuclear Nuclear Regulatory body

Issue Phase 2 Site and Supporting Facilities Phase 1 Nuclear Safety Phase 2 Human Resource Development Phase 1 National Position Phase 2 Regulatory Framework Phase 1 Nuclear Safety Phase 1 Nuclear Safety Phase 3 Human Resource Development Phase 3 Nuclear Safety Phase 1 National Position Phase 2 Human Resource Development Phase 2 Human Resource Development Phase 2 Site and Supporting Facilities Phase 2 Radioactive Waste Phase 1 Human Resource Development Phase 2 Regulatory Framework

Weight 147 123 85 70 67 64 58 57 55 51 51 49 47 46 45 45

The analysis of the graph showed the terminology that is critical for incoming mate‐ rials neural network. This terminology is located on the borders of the graph and points well on relevant issues and phases. The terminology that is in the center of graph is more common but their weights describes the frequency of terms usage that is helpful to identify the adequate level of learning curve.

138

B. Onykiy et al.

However, some experts highlighted the fact of incompleteness of ontology that can lead to poor accuracy of the neural network. Probably, the incompleteness of working documents could lead to the incompleteness of the ontology. Nevertheless, this issue is not related to the methodology moreover with the help of methodology this issues became obvious. The successful solution of this problem is in the interests of not only developers of various automated search engines that are nowadays used everywhere from small busi‐ nesses and network organizations to science information libraries, but also of average users of various services who are interested in reducing the time and effort consumed in processing the search information that doesn’t meet their expectations.

4

Conclusion

The analysis of learning curve is an important process during the whole building of neural network. It helps developers to understand why a neural network works right or wrong. Very important is to get the support from experts in relevant field as they can point out issues in it. The methodology can be applied for various collections of docu‐ ment analysis in organizations in tasks related to identification of hidden relations between working and incoming documents. In these circumstances, it is highly recommended to use visual analysis for learning curve analysis. The methodology presented in this paper proved its utility with learning curve analysis as it delivered a good picture to identify critical terminology, the incom‐ pleteness of working documents and adequate level of learning curve. Acknowledgments. This work was supported by Competitiveness Growth Program of the Federal Autonomous Educational Institution of Higher Professional Education National Research Nuclear University MEPhI (Moscow Engineering Physics Institute).

References 1. Onykiy, B., Suslina, A., Ionkina, K., Ananieva, A., Pronicheva, L., Artamonov, A., Tretyakov, E.: Agent technologies for polythematic organizations information-analytical support. Procedia Comput. Sci. 88, 336–340 (2016) 2. Ananieva, A.G., Artamonov, A.A., Galin, I.U., Tretyakov, E.S., Kshnyakov, D.O.: Algoritmizatiom of search operations in multiagent information-analytical systems. J. Theor. Appl. Inf. Technol. 81(1), 11–17 (2015) 3. Artamonov, A., Leonov, D., Nikolaev, V., Onykiy, B., Pronicheva, L., Sokolina, K., Ushmarov, I.: Visualization of semantic relations in multi-agent systems. Sci. Vis. 6(3), 68–76 (2014) 4. Hu, Y.: Efficient, high-quality force-directed graph drawing. Math. J. 10(1), 37–71 (2005) 5. Berners-Lee, T., Hendler, J., Lassila, O.: The semantic web. Sci. Am. 304(5), 35–43 (2001)

Modern Views on Visual Attention Mechanisms Lubov Podladchikova(B) , Anatoly Samarin, Dmitry Shaposhnikov, and Mikhail Petrushan Centre of Neurotechnologies, Southern Federal University, Rostov-on-Don, Russia [email protected]

Abstract. The results of our psychophysical tests by eye movement recording and the following stages of research have been considered. Several groups of the known findings in this area were determined, namely: (i) unresolved objectives; (ii) contradictory data; (iii) findings which propose revision of some views; (iv) similar findings obtained in different research centers. The most important results of our psychophysical tests are as follows: (a) bimodal distribution of fixation duration during joint presentation of target objects and distractors; (b) dynamical formation of target images creates conditions for dosed change of perceptual load; (c) decrease of fixation duration and increase of saccade amplitude during the last test stage when volunteer makes the decision about completion of the current visual task; (d) structure of viewing scan path, fixation density and duration, probability of return fixations are specific for each human during viewing of the affective images; (e) return fixations are arranged with reference to areas of interest on image; (f) spatial distribution of fixation duration, velocity and amplitude of saccades are significantly different between tests of viewing of 2D images and navigation in 3D environment. Using of the obtained results in realistic mathematical models of visual attention has been discussed. Keywords: Visual attention · Eye movements · 2D images · 3D environment · Individual peculiarities of viewing · Return fixations

1

Introduction

New stage (Active Vision Era) for research of visual attention (VA) mechanisms by eye movement (EM) parameters started after publishing of Yarbus’ monograph [15,18,20]. In spite of significant progress in studying of VA mechanisms, many aspects of this problem are far from full understanding up to now [2,6,10].

2

Basic Features of the Known Research on Visual Attention Mechanisms

Small area in centre of eye retina (fovea) into that about 2 degrees of vision field are projected has high receptor density; the visual acuity decreases on c Springer International Publishing AG 2018  A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6 19

140

L. Podladchikova et al.

exponential rule from vision field centre to the periphery [20]. Processes based on the mechanisms of overt and covert VA [2,6,10,17,20] are activated to choose areas of interest (AOIs). The overt VA is accompanied by high-amplitude EMs (more 1◦ ) – saccades; the covert VA is basically reflected in gaze fixation duration and microsaccades – EMs with amplitude less 1◦ [6,10]. Choosing of AOIs depends on many factors and is determined by perceptual and cognitive mechanisms interacting with each other [2,6,10,20]. One group of factors is physical image properties. They mainly attract VA during the primary periods of solution of visual task; such VA is called bottom-up. The influence of this form of VA decreases if subject has any motivation. In this case VA is directed to searchable image properties or semantically important objects; such form of VA is based on functioning of hierarchically higher brain structures and is called top-down. These forms of attention did not equivalent to focal and spatial VA in spite of similar consequences of their functioning. It is evident that relationships between these forms of VA should be studied in detail. Another problem for future investigations is temporal dynamics of VA. It should be noted that findings about top-down and bottom-up VA were mainly obtained by simple stimuli such as local linear segments with different orientation, arrows with different direction etc. On the contrary, the findings on focal and spatial VA were basically received by complex images such as scenes, paintings etc.; the tasks in these tests often demand requests to visual memory and participation of other higher cognitive functions. Another feature of the most studies with EM recording is to determine the test time by experimentalist but not by volunteer. The first paradigm did not allow view the fully completed act of visual task solution. It is probable that some volunteers may complete the viewing before finishing of test and subsequent EMs can be performed at increasing activity of non-visual kinds of attention. However, findings about EMs during completion of the current visual task in conditions of self-terminated paradigm are seldom [6,10]. Thus, in spite of big number of EM research, determination of connection between gaze location and VA focus is very complicated objective at modern stage. Evidently, development of new experimental methods can provide estimation of contribution for different VA mechanisms.One method for switching VA is use of unexpected short-term stimuli – distractors [5,6,10] in conditions of self-terminated paradigm. Another method for directed influence on VA is use of different perceptual load [5,10]. The studies in this direction were mainly performed by change of number of simple stimuli. However influence of complicity of stimuli and temporal dynamics of EM parameters did not estimated in works on perceptual load. Several groups of findings can be determined on the base of overview [10] of the known views, theories and experimental data: (i) Unresolved objectives such as quantitative estimations of cognitive load and individual peculiarities of VA [10,11]; relationships between phenomena of Inhibition and Facilitation of Return [7]; difference of AOIs detected by density and duration of fixations [14]; relations between focal and spatial VA

Modern Views on Visual Attention Mechanisms

141

and mechanisms of their control [1,8,17]; peculiarities of EMs in conditions of active and passive navigation in 3D environment [10], etc. (ii) Problems for which contradictory experimental data are known such as temporal dynamics of EMs during visual task solution; distractor effect [5,10]. (iii) Findings which propose revision of some classic views, in particular, difference of brain structures to realize the overt and covert VA [1,3,8]. (iv) Views consistent with experimental data which were obtained in different research centers such as contra-phase temporal dynamics of fixation duration and saccade amplitude; priority of the affective images; connection between viewing scan path structure and visual task [10,20].

3

Towards Development of Biologically Inspired Models of Visual Attention

Mathematical models imitating of neurobiological phenomena are considered as important tool to study the VA mechanisms [3,4,9,12,13,16,19]. It is noted [9,19] that creation of realistic models that allow analyze the contribution of different factors and mechanisms and formulate propositions for experimental verification is actual up to now. At present, understanding of necessity to change the approaches to model the EM control is formed [9,12,13,16]. In particular, the most known models of formation of viewing scan path after [4]is based on primary image features, receiving of saliency maps and analysis of spatial distribution of gaze fixations during image viewing by human. Known approaches to model the fixation duration and cognitive mechanisms [9,13] are based on heuristic algorithms. Receiving of detailed estimations of different factors in experimental studies is necessary to develop the realistic models of image viewing.

4

Some Results of Our Psychophysical Tests

Our psychophysical tests were directed on receiving of quantitative parameters of EMs during solving of different visual tasks such as free viewing of complex affective images, search for modified image fragments, and recognition of dynamically forming images, navigation in complex virtual environment. Tests were mainly carried out in conditions of self-terminated paradigm. The most important results are as follows: 1. Bimodal distribution of fixation duration during solving of search task and joint presentation of target objects and distractors; amplitude of second extremum is increased with growth of complicity of target stimuli in the last case (Fig. 1, I).

142

L. Podladchikova et al.

Fig. 1. (I) Distribution of fixation durations without distractors (1) and with distractors (2) for simple (I, a) and complex (I, b) images. X, X1 , X2 – mean duration of fixations in individual sets of data; (II) Distribution of saccade amplitude in conditions of free viewing of 2D images (II, a) and navigation in 3D environment (II, b).

2. Dynamical formation of target images creates conditions for dosed change of perceptual load level that may be estimated by recognition time of object at given stage of its manifestation. 3. Decrease of fixation duration and increase of saccade amplitude during the last test stage when volunteer makes the decision about completion of the current visual task. 4. A set of EM parameters during viewing of the affective images such as structure of viewing scan path, fixation density and duration, probability of return fixations is specific for each human. In particular, the volunteers (n = 20) had similar probability of return fixations during viewing of images with different valence; Pearson’ coefficient of correlation between probabilities of return fixations during viewing of negative and positive images was equal to 0, 78. 5. Return fixations are arranged with reference to AOIs (Table 1). 6. Spatial distribution of fixation duration, velocity and amplitude of saccades are significantly differed between tests of viewing of 2D images and navigation in 3D environment (Fig. 1, II).

Modern Views on Visual Attention Mechanisms

143

Table 1. Averaged density and duration of return gaze fixations inside and outside AOIs on image. Image types Density (n/deg2 ) Inside AOI Outside AOI

Duration (ms) Inside AOI ∗

Outside AOI

Negative

0,019 ± 0.001 0,006 ± 0.0001

Neutral

0,022 ± 0.001 0,006 ± 0.0001∗ 517 ± 42 (n = 189) 374 ± 81 (n = 19)

426 ± 33 (n = 189) 390 ± 62 (n = 31)

Positive 0.019 ± 0.001 0.007 ± 0,0001∗ 475 ± 46 (n = 155) 400 ± 100 (n = 21) Significant differences (p < 0.001) are marked by *.

5

Conclusion

The most important results of our psychophysical tests are as follows: (a) bimodal distribution of fixation duration during joint presentation of target objects and distractors. Apparently, amplitude of second extremum may be used as quantitative measure of cognitive load; (b) dynamical formation of target images creates conditions for dosed change of perceptual load; (c) decrease of fixation duration and increase of saccade amplitude during the last test stage when volunteer makes the decision about completion of the current visual task; (d) a set of EM parameters is specific for each human during viewing of the affective images; (e) return fixations are arranged with reference to AOIs on the image. This fact is important for estimation of connections between mechanisms of VA, short-term memory and decision making; (f) a set of EM parameters is significantly different between tests of viewing of 2D images and navigation in 3D environment. It is well known [3,12,13,16] that the results of psychophysical tests are important for development of biologically inspired models of formation of scan paths during image viewing. For example, empiric coefficient for inhibition of return is often introduced in such models to prevent the cycles in scan paths. However possibility of facilitation of return did not take into account in these models. Formalization of interactions between phenomena of inhibition and facilitation estimated by return fixations on recently viewed image areas allow create more realistic model of image viewing. Besides individual peculiarities of image viewing must be also formalized in similar models to imitate the results of concrete test for particular volunteer. Acknowledgement. The work is supported in part by projects of Ministry of education and science of RF No 2.955.2017/4.6 and No 6.5961.2017/8.9.

References 1. Awh, E., Belopolsky, A.V., Theeuwes, J.: Top-down versus bottom-up attentional control: a failed theoretical dichotomy. Trends Cogn. Sci. 16(8), 437–443 (2012) 2. Carrasco, M.: Visual attention: the past 25 years. Vis. Res. 51(13), 1484–1525 (2011)

144

L. Podladchikova et al.

3. Henderson, J.M., Brockmole, J.R., Castelhano, M.S., Mack, M.: Visual saliency does not account for eye movements during visual search in real-world scenes. In: Eye Movements: A Window on Mind and Brain, pp. 537–562 (2007) 4. Koch, C., Ullman, S.: Shifts in selective visual attention: towards the underlying neural circuitry. Hum. Neurobiol. 4, 219–227 (1985) 5. Koltunova, T.I., Podladchikova, L.N., Shaposhnikov, D.G., Vladimirskii, B.M., Syrkin, L.D., Kryuchkov, B.I., Usov, V.M.: Dynamics of the duration of gaze fixation and event-related potentials on presentation of fading-in images and distractors. Neurosci. Behav. Physiol. 47(3), 321–327 (2017) 6. Kowler, E.: Eye movements: the past 25 years. Vis. Res. 51(13), 1457–1483 (2011) 7. Lupianez, J., Klein, R.M., Bartolomeo, P.: Inhibition of return: twenty years after. Cogn. Neuropsychol. 23(7), 1003–1014 (2007) 8. Nobre, A.C., Gitelman, D.R., Dias, E.C., Mesulam, M.M.: Covert visual spatial orienting and saccades: overlapping neural systems. Neuroimage 11, 210–216 (2000) 9. Nuthmann, A., Smith, T.J., Engbert, R., Henderson, J.M.: CRISP: a computational model of fixation durations in scene viewing. Psychol. Rev. 117(2), 382–405 (2010) 10. Podladchikova, L.N., Koltunova, T.I., Samarin, A.I., Petrushan, M.V., Shaposhnikov, D.G., Lomakina, O.V.: Modern Views on Visual Attention Mechanisms (in Russian). Publishing and Printing Department of KBI MEDIA CENTER SFedU, Rostov-on-Don (2017) 11. Podladchikova, L.N., Koltunova, T.I., Shaposhnikov, D.G., Lomakina, O.V.: Individual features of emotionally meaningful images viewing (in Russian). Russ. J. Physiol. 102(5), 618–627 (2016) 12. Podladchikova, L.N., et al.: Model-based approach to study the mechanisms of complex image viewing. Opt. Mem. Neural Netw. (Inf. Opt.) 18(2), 114–121 (2009) 13. Samarin, A., Koltunova, T., Osinov, V., Shaposhnikov, D., Podladchikova, L.: Scanpaths of complex image viewing: insights from experimental and modeling studies. Perception 44(8–9), 1064–1076 (2015) 14. Suteliffe, A., Namoune, A.: Investigating user attention and interest in websites. LNCS, vol. 4662, pp. 88–101 (2007) 15. Tatler, B.W., Wade, N.J., Kwan, H., Findlay, J.M., Velichkovsky, B.M.: Yarbus, eye movements, and vision. I-Perception 1(1), 7–27 (2010) 16. Tatler, B.W., Hayhoe, M.M., Land, M.F., Ballard, D.H.: Eye guidance in natural vision: reinterpreting salience. J. Vis. 11(5:5), 1–23 (2011) 17. Velichkovsky, B.M., Joos, M., Helmert, J.R., Pannasch, S.: Two visual systems and their eye movements: evidence from static and dynamic scene perception. In: Proceedings of the XXVII Conference of the Cognitive Science Society, pp. 2283– 2288 (2005) 18. Wade, N.J.: Pioneers of eye movement research. I-Perception 1(1), 33–68 (2010) 19. Walther, D.B., Koch, C.: Attention in hierarchical models of object recognition. Prog. Brain Res. 165, 57–78 (2007) 20. Yarbus, A.L.: Eye Movements and Vision. Plenum Press, New York (1967)

Model of Interaction Between Learning and Evolution Vladimir G. Red’ko1,2(&) 1

Scientific Research Institute for System Analysis, Russian Academy of Sciences, Moscow, Russia [email protected] 2 National Research Nuclear University MEPhI (Moscow Engineering Physics Institute), Moscow, Russia

Abstract. The lecture characterizes the following main properties of interaction between learning and evolution: (1) the mechanism of the genetic assimilation, (2) the hiding effect, (3) the role of the learning load at investigated processes of learning and evolution. During the genetic assimilation, phenotypes of modeled organisms move towards the optimum at learning; after this, genotypes of selected organisms also move towards the optimum. The hiding effect means that strong learning can inhibit the evolutionary search for the optimal genotype. The learning load can lead to a significant acceleration of evolution. Keywords: Interaction between learning and evolution
Genetic assimilation
Hiding effect
Learning load

1 Introduction Our model continues the works by Hinton and Nowlan and Mayley [1, 2], who simulated some features of interaction between learning and evolution. We use also the quasispecies model [3, 4] and our estimations of the evolutionary rate and the efficiency of evolutionary algorithms [5].

2 Description of the Model We consider the evolving population of modeled organisms. Each organism has the genotype and the phenotype. We assume that the genotype and the phenotype of the organism have the same form, namely, they are chains; symbols of both chains are equal to 0 or 1. The length of these chains is equal to N. For example, we can assume that the genotype encodes a modeled DNA chain, symbols of which are equal to 0 or 1, and the phenotype is determined by the neural network of the organism, the synaptic weights of the neural network are equal to 0 or 1 too. These weights are adjusted by means of learning during the organism life. The evolving population consists of n organisms, genotypes of organisms are SGk, k = 1,…,n. The organism genotype SGk is a chain of symbols, SGki, i = 1,…,N. N, n >> 1, 2N >> n. The values N and n do not change during evolution. Symbols SGki are © Springer International Publishing AG 2018 A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6_20

146

V.G. Red’ko

equal to 0 or 1. The evolutionary process is a sequence of generations. The new generation is obtained from the old one by means of selection and mutations. Genotypes of organisms of the initial generation are random. Organisms inherit the genotypes from their parents, these genotypes do not change during the organism life and are transmitted (with small mutations) to their descendants. Mutations are random changes of symbols SGki. The evolutionary process is similar to that of in the quasispecies model [3, 4]. Phenotypes of organisms SPk are chains of symbols SPki, k = 1,…,n, i = 1,…,N; SPki = 0 or 1. The organism receives the genotype SGk at its birth. The initial phenotype of the organism at its birth is equal to the organism genotype: SPk(t = 1) = SGk. The lifetime of any organism is equal to T. The time is discrete: t = 1,…,T. T is the duration of the generation. The phenotype SPk is modified during the organism life by means of learning. It is assumed that there is the certain optimal chain SM, which is searched for in processes of evolution and learning. Symbols SMi of this chain SM are also equal to 0 or 1; the length of the chain SM is N. For a concrete computer simulation, the chain SM is fixed; symbols of this chain are chosen randomly. Learning is performed by means of the following method of trial and error. In every time moment t, each symbol of the phenotype SPk of any organism is randomly changed to 0 or 1, and if this new symbol SPki coincides with the corresponding symbol SMi of the optimal chain SM, then this symbol is fixed in the phenotype SPk, otherwise, the old symbol of the phenotype SPk is restored. So, during learning, the phenotype SPk moves towards the optimal chain SM. At the end of the generation, the selection of organisms in accordance with their fitness takes place. The fitness of the k-th organism is determined by the final phenotype SPk in the time moment t = T. We denote this chain SFk, i.e. we set SFk = SPk(t = T). The fitness of the k-th organism is determined by the Hamming distance q = q(SFk,SM) between the chains SFk and SM: fk ¼ exp½bqðSFk ; SM Þ þ e;

ð1Þ

where b is the positive parameter, which characterizes the intensity of selection, 0 < e is a educational-training tasks (ETT) model in accordance with [4], where Tr = {Tei }, i = 1, ..., c is a set of ETT, at that Tei =< Da , C, V, Vu , Ov , Pa >, where Da is an initial data, C is limitations that must be taken into account when executing ETT, V is correct answers, Vu = {V1 , ..., Vn } is a description of the method of input of the result, where V1 is a numerical value or an interval, V2 is a set of alternative options, V3 is a set of options, V4 is filling in blanks in the text, V5 is selection of solution components from the list, V6 is text labeling, V7 is construction links between elements of the graphical representation; Ov is function of result evaluation Ov (Vs , V ) → R, where R is a set of estimates, Vs is an Input result; RT = {RTi }, i = 1...y is a set of links between the ontology of the course/discipline and the subset of the ETT; H =< Ch , RC > is a model of HT-textbook, where Ch = {Chi }, i = 1...d is a set of chapters of the hypertext textbook (HT-textbook) [5,6], at that Chi = {M 1, M 2}, where 1 is a HTML-model of HT, 2 is a XML-model of H, and RC = {RCi }, i = 1, .., g is a set of links between the element of the course/discipline and the subset of the head of the textbook. Component Se =< P A, F A, SA > is an aggregate of models for elicitation student skills/abilities, where P A is a model of the process of elicitation students abilities to simulate strategies of direct/reverse reasoning, F A is a model of the process of elicitation the students abilities to simulate the simplest situations of the problem domain with frames, SA is a model of the process of elicitation the students’ abilities to simulate the situations of the problem domain with semantic networks. In its turn, P A =< P S, P R >, where P S is a production system in accordance with [6], P R = {P Ri }, i = 1, ..., m is an aggregate of links between the ontology elements of the course/discipline and the components of P S; F A = F, F R, where F is an aggregate of procedures and reference prototype frames in notation FRL [6], F R = {F Rj }, j = 1, ..n is a set of links between the ontology elements of the course/discipline and the components F : SA =< S, SR >, where S is an aggregate of procedures and reference fragments of semantic networks [6].

160

G.V. Rybina and E.S. Sergienko

Accordingly, in the context of the development of tutoring IES, the model of the generalized ontology (Mo ) is represented as: Mo = {Moi }, where Moi = Mei , i = 1, ..., n, n is a number of courses/disciplines of specialization, for which an ontology is constructed Oi . According to [5], each ontology of the course/discipline is represented as: Oi = Me , Fe , where Me is the model ontology of the course/discipline described above; Fe = {Fs , Fq , Fam , Fk , Fke } set of operations (procedures) for construction an ontology (Oi ) of course/discipline, where Fs are procedures for structuring the course/discipline; Fq are procedures for enunciating questions to selected elements of a course/discipline with a single level of hierarchy; Fam are procedures for realizing the adaptive method of repertory grids (RG) [4] for identifying the links between the elements of the course/discipline; Fk are procedures for constructing a model of target competence; Fke are procedures for determining the relationship between attribution and elements of the course/discipline. Basic tools of AT-TECHNOLOGY workbench for supporting the construction of ontologies include tools for construction the ontology of the course/discipline, the means of constructing a generalized ontology and visualization component. With the help of the aforementioned means, ontologies of all basic disciplines are currently implemented and supported generalized ontology [5,6] “Intelligent Systems and Technologies”. The unifying basis for basic intelligent tutoring problems [5,7] is the use of IES of different architectural typology (tutoring IES, ITS on the basis of intellectual agents, etc.) are processes of elicitation knowledge (declarative knowledge of a specific course/discipline) and skills (procedural knowledge that allows to demonstrate how the declarative knowledge of the trainees is used in practice). Generating of test case variants is performed before the beginning of web testing by applying the genetic algorithm to the specific ontology of the course/discipline or to its fragment [5] in accordance with the curriculum for carrying out control activities. Then the current student model is compared with the ontology of the course/discipline, as a result of which so-called “problem zones” are identified in the students knowledge of the individual sections and topics of the course/discipline and the corresponding current competence. Thus, ontologies of courses/disciplines be key in revealing the level of knowledge of students and construction competence-oriented student models. Now consider the place and role of ontologies in the processes of computerbased identification of students abilities to solve tutoring problems. For tutoring of IES and web-IES that operate on the basis of the generalized ontologies “Intelligent systems and technologies” occupy an important place methods elicitation skills to solve tutoring problems is related to modeling the reasoning of the person (student), and other approaches are already required related, in particular, to the methods and means of traditional ES and IES. For example, the learning of special courses/disciplines in the areas of training “Applied Mathematics and Informatics” and “Software Engineering” is impossible today without inculcating the skills and abilities of students to solve following problems [4–8]: the ability

Ontological Approach for the Organization of Intelligent Tutoring

161

to construct on the basis of the “self-expert” models of the simplest situations of the problem domain based on frames and semantic networks, modeling the strategies of direct/reverse reasoning in the expert system, construction the components of the linguistic model of the sublanguage of business prose, and others. The training tasks listed above are based on non-formalized expert methods, the experience of which has been accumulated in the technologies of traditional expert system and IES, in particular, in knowledge engineering [3,6]. 2.1

Integration Between Ontology Elements of Courses/Disciplines with Individual Models of Students and Adaptive Tutoring Models

In the context of constructing tutoring IES on the basis of a problem-oriented methodology, special software tools of AT-TECHNOLOGY workbench were implemented and tested in practice at the NRNU MEPhI and other higher educational institutions implementing “manual” methods for solving various non-formalized tasks, in particular, presented in [5,6,8]. It is necessary to point out that all these software tools, in accordance with the concept of “intelligent software environment” [4] are designed as reusable components (RUC), used to implement the standard design procedure “Construction of tutoring IES” (described in [4,5,9]). For several years of experimental software research on several generations of students and continuous improvement of the methodical, algorithmic and software of all the above RUC, it was possible to create quite unique methods and software to elicitation and evaluate the skills of students to solve informal practical problems within the ontology of a specific subject area. Since all RUCs were developed and operated autonomously without connection with the corresponding ontology of courses/disciplines, special algorithms and tools were developed to integrate the ontology elements of courses/disciplines with a variety of RUCs to elicitation the student’s ability to solve tutoring tasks. Typically, in the context of the ontological approach, a conceptually close problem arose in the construction of an adaptive tutoring model that, in accordance with [4,5], contains knowledge of the planning and organization of the tutoring process, depending on the individual tutoring models. An important feature is that each strategy (plan) of education consists of a certain sequence of learning impacts of different types, the application of which is completely determined by the state of the current model of the student (in particular, by “problem zones” and other parameters). At the present time it were developed, decorated as a RUC [4,5,7,8] and have undergone experimental testing such classes of training impact such as the solution of ETT of several types, reading sections of the hypertext textbook (HT-textbook) and “training with ES/IES”. The greatest expansion of applied ontologies of courses/disciplines was associated with the implementation of the integration of ontology elements of courses/disciplines with a set of the following ETTs, designed as operational RUCs: “Arrangement of correspondences between blocks”, “Filling in blanks in the text”, “Marking or correction of the text”, “Choosing answer options”, “Arranging graphic images”.

162

3

G.V. Rybina and E.S. Sergienko

Conclusion

In this paper intelligent tutoring problems with use of onthological approach was discussed. Basic ontology model of course/discipline was defined. The properties of this model were analysed, in particular connection between ontology elements, student models and adaptive tutoring models. Some implementation details of the ontology model was given, and involved software components like different educational training tasks and skills evaluation components were listed. Acknowledgements. The work was supported by the Russian Foundation for Basic Research support (project No. 15-01-04696) and the MEPhI Academic Excellence Project (Contract No. 02.a03.21.0005).

References 1. Surez-Figueroa, M.C., Gmez-Prez, A., Motta, E., Gangemi, A.: Ontology Engineering in a Networked World. Springer Science & Business Media, Berlin (2012) 2. Sosnovsky, S., Mitrovic, A., Lee, D., Brusilovsky, P., Yudelson, M.: Ontology-based integration of adaptive educational systems. In: 16th International Conference on Computers in Education (ICCE 2008) (2008) 3. Gavrilova, T.A., Kudryavtsev, D.V., Muromtsev, D.I.: Knowledge Engineering. Models and Methods: A Textbook. Lan, Saint Petersburg (2016) 4. Rybina, G.V.: Theory and technology of construction of integrated expert systems. Monography. Nauchtehlitizdat, Moscow (2008) 5. Rybina, G.V.: Intelligent systems: from A to Z. Monography Series in 3 Books, vol. 1. Knowledge-based systems. Integrated expert systems. Nauchtehlitizdat, Moscow (2014) 6. Rybina, G.V.: Fundamentals of building intelligent systems. Tutorial. Finance and Statistics, Moscow (2014) 7. Rybina, G.V.: Modern approaches to the implementation of intelligent computer learning based on the development and use of tutoring integrated expert systems. Instrum. Syst. Monit. Control Diagn. 5, 10–15 (2010) 8. Rybina, G.V., Sergienko, E.S., Sorokin, I.A.: Some aspects of intellectual tutoring based on the integrated tutoring expert systems usage. In: Biologically Inspired Cognitive Architectures (BICA) for Young Scientists. Proceedings of the First International Early Research Career Enhancement School (FIERCES 2016) (2016) 9. Rybina, G.V., Blokhin, Y.M.: Use of intelligent planning for integrated expert systems development. In: 8th IEEE International Conference on Intelligent Systems, IS 2016, Sofia, Bulgaria, 4–6 September 2016, pp. 295–300 (2016)

A Continuous-Attractor Model of Flip Cell Phenomena Alexei V. Samsonovich1,2 ✉ (

)

1

Department of Cybernetics and BICA Lab, Institute for Cyber Intelligence Systems, National Research Nuclear University “Moscow Engineering Physics Institute”, Kashirskoe Shosse 31, Moscow 115409, Russian Federation 2 George Mason University, Fairfax, VA 22030, USA [email protected]

Abstract. This paper is devoted to the problem of understanding mechanisms underlying behavioral correlates of head direction (HD) cells in the mammalian retrosplenial cortex. HD cells become active when an animal, such as rat, is facing a particular direction in its environment. The robustness of this phenomenon is usually attributed to attractor dynamics of the HD cell system. According to the standard view, a ring attractor exists in some abstract space, with HD cells symbolically allocated on the ring, so that any natural state of the system corre‐ sponds to a bump of activity on the ring. In apparent contradiction with this standard model are recent discoveries of so-called “flip cells”, that constitute a minority of HD cells and can either rotate their directional tuning by 180° when an animal transitions between two environments, or interpolate between discordant cues, or demonstrate a bimodal tuning curve. Here a continuous attractor network model is described that is capable of a qualitative reproduction of these phenomena, while being consistent with the ring attractor hypothesis. The model assumes that there is more than one attractor ring in the HD system. Results of the concept-proof simulation suggest a correction to the standard view of how the internal sense of direction is formed in the rat brain. Keywords: Attractor neural networks · Continuous attractor · Navigation · Head direction cells · Animal cognition

1

Introduction: Head Direction and Flip Cells

Many neurons in the mammalian brain are behaviorally modulated, in the sense that their dynamic state correlates with certain semantic characteristics of the current behav‐ ioral state of the animal and its environment. Understanding behavioral correlates of neuronal activity and mechanisms responsible for them is one of the main goals in neuroscience. One of the seemingly simplest and yet mysterious examples that still evades understanding is presented by the phenomena of head direction (HD) cells [1]. HD cells are neurons that become active when the animal is facing in a particular direction, and are silent otherwise. Typically, each HD cell has a sharp tuning curve with a width of approximately 60°, and is fixed in allocentric coordinates: e.g., North could

© Springer International Publishing AG 2018 A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6_23

164

A.V. Samsonovich

be always a preferred direction of a given HD cell. The ensemble of HD cells uniformly covers all directions, complementing other navigational systems [2]. Remarkably, not only tuning curves of individual HD cells, but also relations among tuning curves of different HD cells typically remain the same for all environments [3]. The entire set of tuning curves of HD cells rotates as a whole, when the rat moves from one environment to another or gets disoriented due to various factors, such as displace‐ ment of landmarks or the lack of any directional sensory cues. This property stands at odds with the various remapping phenomena [4] observed in the place cell system, which is tightly connected to the HD system [2]. The above and other observations inspired the widely accepted attractor [5] hypoth‐ esis about HD cells, expressed in, and supported by numerical study of mathematical models based on a ring attractor [6, 7]. This attractor hypothesis postulates that HD cells form a continuous-attractor neural network [8], that behaves as one coherent system. According to the standard model [6], HD cells together form a ring in some abstract space, such that cells that are close to each other on the ring have excitatory connections to each other, and more distant cells inhibit each other. This assumption is supported by recent observations of millisecond-scale correlations of HD cell activity during sleep by the group of Buzsaki [9]. The threshold angle appears to be around 60°, which roughly corresponds to the width of an HD cell tuning curve. As a result, the total distribution of activity of HD cells forms a bump on the ring. This bump can be shifted by both selfmotion cues and rearrangement of landmarks, but cannot dissociate under normal condi‐ tions. In apparent contradiction with this standard attractor ring model is an observation of what we call here flip cells (also called “bidirectional cells”), recently discovered in the Jeffery lab [10], in the dysgranular retrosplenial cortex of the rat (Fig. 1C: b, c, d). These flip cells represent a minority of HD cells, and yet they may also have a critical role in linking landmarks to the direction sense. Unlike HD cells, which are sensitive to the absolute orientation of the animal’s head regardless of its location, flip cells change their directional preferences by 180°, when triggered by a cue or on entry into a new compartment. Moreover, some of these cells can exhibit a mixture of the two represen‐ tations: rotation and non-rotation, thus exhibiting a bimodal tuning curve (Fig. 1C: c, d). The directional firing of flip cells thus appears disconnected from classic HD neurons that presumably are controlled by an attractor network. Another observation that further challenges the standard model was made earlier by the same team [11]. In this case, discordant representations form in the HD system following cue rotation. When a prominent distant cue such as an external light is rotated around the apparatus, the activity bump representing the internal sense of direction in the HD system is rotated as well; however, instead of following the light, it “under‐ shoots”, interpolating between the rotated light and the remaining background (Fig. 1A, B). It should be noted that local multimodal cues were carefully eliminated in this experiment: e.g., the floor was washed with alcohol between sessions (Kate Jeffery, priv. comm.). The animal was not present during the light rotation [10]. The idea used here to reconcile these challenging observations with the ring attractor model is to assume that there is not one, but possibly two, or rather a hierarchy of weakly connected to each other attractor rings formed by HD cells, each of which is consistent

A Continuous-Attractor Model of Flip Cell Phenomena

165

Fig. 1. Summary of the flip cell phenomena (based on [10, 11]). A: The light cue rotation paradigm [11]. The external light is shown in its original (solid arrow) and rotated (open arrows) positions. B: Under conditions shown in A, the internal sense of direction represented by the bump under-rotates, following the light up to a critical angle (120°), at which point it dissociates, and the distribution of activity on the ring of HD cells becomes bimodal [11]. C: Types of HD-like cells observed in retrosplenial cortex [10]. The two parts of the environment look identical, when rotated 180° with respect to each other. Transition point is in the middle. The polar plots show angular tuning curves of different cell types; a: normal HD cell, b: classical flip cell (or “betweencompartment bidirectional cell”), c: “within-compartment bidirectional cell”, d: “mixed betweencompartment bidirectional cell”, a mixture of b and c.

with the standard model. Some rings are strongly bound to particular landmarks, while others are mostly driven by background cues. To validate the concept, simulations of a simple two-ring continuous-attractor neural network model were performed, using continuous firing rate units with a Gaussian distribution of excitatory connections and a global inhibition that stabilizes the total activity level in each ring. Under a reasonable choice of parameters consistent with the observed range of excitatory connectivity on the ring (Peyrache et al. data [9]), a model was built that easily reproduced all qualitative aspects of observed dynamics, including discordant HD representations and the various flip cell phenomena, as described in the following sections.

166

2

A.V. Samsonovich

A Model-Based Interpretation of Flip Cell Phenomena

To understand the proposed mechanisms leading to observations described above, consider a continuous-attractor neural network (Fig. 2), consisting of two attractor rings (2, 3) and two sensory rings (1, 4) of neuronal units.

Fig. 2. Conceptual scheme of the model with two attractor rings (see text).

Figure 2 can be explained as follows. Each filled circle represents a neuronal unit. Active units are red, inactive are blue. Units are organized into rings. Three panels A, B, C show different states of the same system of rings. In each panel, the top ring (1) represents units that provide sensory input to the system given by distant landmarks, such as an external light or a cue card. The bottom ring represents a sense of direction that comes from background or local sensory cues, including other than visual modali‐ ties. Normally, the two sources are consistent with each other; however, in all three examples shown in Fig. 2 they become discordant. The sense of direction induced by each sensory input is shown by a solid black arrow. Dotted blue arrows show excitatory interactions of units. The two internal rings (2, 3) are attractor networks. A natural state of activity of any of these two networks is a bump of activity located somewhere in the ring. The bump can be moved around the ring with an arbitrarily weak external stimulation, and stays indefinitely at the same location without an input to the ring. In this model, however, the two rings are coupled to each other and to sensory rings. Ring 2 is coupled to rings 1 and 3, while ring 3 is coupled to rings 2 and 4. The relative strengths of coupling are adjustable model parameters. Figure 2, A: When the angle between bumps in two sensory rings 1, 4 (representing perceived directions of the head based on different cues) is less than 120°, the system of attractor rings (2, 3) interpolates between two stimuli, producing a unimodal distri‐ bution of activity in HD cells. Figure 2, B: When the angle between bumps in two sensory rings is more than 120°, the system still tries to interpolate, while the distribution becomes bimodal. In this case, the relative weight of each mode depends on the number of units in each ring and on the strength of connections between attractor and sensory rings.

A Continuous-Attractor Model of Flip Cell Phenomena

167

Figure 2, C: When the angle between bumps in two sensory rings is 180°, the system does not interpolate: mutual excitations of the rings are symmetrical and compensate each other; as a result, the two rings form bumps at 180° with respect to each other. This case corresponds to classical flip cells (Fig. 1C b). Assuming that some cells (not shown in Fig. 2) can receive input from both rings 2 and 3, one can see that those cells in the case of Fig. 2C will become “bidirectional” (Fig. 1C c) or “between-compartment bidirectional” (Fig. 1C d), while in the case of Fig. 2B they will interpolate between the two modes. This intuitive consideration is supported by numerical simulations of the model described below. A remark should be added here that mechanisms of updating of the HD representation based on the angular velocity, which are a part of the standard model [6], are not included in the present model, in order to keep it simple. Including them would only help: indeed, frequent synchronized across all rings rotations of the bumps should eliminate effects of inhomogeneities in distributions of neurons on the rings, that could in principle deteriorate or even destroy the continuous attractor (to be precise, in a real system with a finite number of neurons, one can only be dealing with quasicontinuous attractors [4]).

3

Model Implementation and Simulation Details

The model of an attractor neural network selected here serves the purpose of a qualitative validation of the hypothesis that flip cell phenomena are possible in a continuousattractor network with ring attractors. Therefore, model neurons are simple and are not selected to be realistic in these simulations. The network is composed of continuous firing rate units with symmetric excitatory connections and a global inhibition that keeps the total number of active units at a constant level. The entire network of N = 620 HD units with synchronous updating is divided into three populations: two ring attractor networks of 300 units each and one ring of 20 units intended as flip cells (Fig. 3). Dynamic equations for the firing rates v and postsynaptic potentials h are:

ht+1 = Iit + i

∑ j∈Ring(i)

( ) Wij − 𝜇i vtj ,

( ( )) vti = 1∕ 1 + exp 3 - hti ,

(1) (2)

where i is the neuron number, t is the discrete time, I originates from sensory, vestibular and proprioceptive inputs, W’s are excitatory connection weights, μ’s are inhibitory connection weights, and Ring(i) is the population of neurons (ring) to which the neuron i belongs. The weights W are computed via the Gaussian function of the Euclidean distance r between two neuronal units in the abstract space shown in Fig. 3, calculated in the units of the plot of Fig. 3; and similarly is calculated the input I:

( ) ( ) Wij = exp −2rij2 , Ii = Aexp −ri2 ∕2𝜎i2 ,

(3)

168

A.V. Samsonovich

where r is the distance from the unit to the center of input (a star in Fig. 3). Parameters are: A = 30, σ2 = 0.15 for the outer ring and 0.04 for the inner ring and the ring of flip cells; μ = 0.5, 0.4 and 0.2 for the inner, outer, and flip cell rings respectively.

Fig. 3. Implementation of the model with two attractor rings. Circles represent neuronal units in the rings 2 and 3 of Fig. 2: here they correspond to the inner and outer rings, respectively. Sensory rings are not shown. Each ring contains 300 units; plus, a ring of evenly spaced 20 “flip cells” (to become “within-compartment bidirectional cells”) is allocated in between the inner and the outer rings. The strength of excitatory connection between any two units is a Gaussian function of the Euclidean distance between them on this plot, measured in the abstract units of X and Y shown on the axes. There are no other connections or constraints, except uniform global inhibition. Stars represent centers of external stimulation, that also has a Gaussian distribution with the same width. The red star (representing background cues) is fixed, while the blue star (representing the rotating light or a cue card) moves along the dotted line. Red and blue arrows point to the centers of bumps on the two rings; the black arrow points to the center of the combined distribution. Shades of gray show the level of activity of each unit. The configuration shown in the figure is stable. Other stable configurations are shown in Fig. 4.

4

Simulation Results and Discussion

The model was implemented in Matlab R2017a on an iMac (Retina 5 K, 27-inch, Late 2015, running OS X 10.11.6) and simulated with various input conditions. Typically, running 200 iterations was more than sufficient for complete relaxation to an attractor state. Simulation results are represented in Figs. 3 and 4. It can be seen from Figs. 3 and 4B, C that intuitions of Fig. 2 are confirmed by simulations (cf. Fig. 2).

A Continuous-Attractor Model of Flip Cell Phenomena

169

Fig. 4. Examples of stable configurations of the system observed in simulations (notations are the same as in Fig. 3.). A: Concordant inputs. B: Discordant, with the angle of rotation more than 120°. C: Exactly opposite inputs: a 180° rotation. A typical time of convergence in all cases is less than 200 iterations. Animation of these and other examples is available at https:// player.vimeo.com/video/179615595

Fig. 5. Results of simulations of the two-ring model. A-C: Polar diagrams of the tuning curve of a “flip cell” (one of the intermediate ring of 20 units in Fig. 3) as a function of the cue rotation angle α. When α < 120°, the tuning curve is unimodal (A); otherwise it is bimodal (B) and eventually becomes that of a “bidirectional cell” (C). D: Internal sense of direction, represented by the bump center in each ring, as a function of the distal cue rotation angle α (cf. Fig. 1B). 1: the inner ring (Figs. 3 and 4), 2: the outer ring, 3 is the center of mass of the overall circular distribution of HD cell activity. Since the width of each bump is approximately 60°, the combined distribution becomes bimodal at α ~ 120°.

170

A.V. Samsonovich

Indeed, when the two sensory inputs – cue and background – applied each to its ring become discordant, i.e., differ from each other by an angle α, the rings interpolate between the two senses of direction (Figs. 3 and 4B). In this case, the overall circular distribution of HD cell activity can be unimodal (α < 120°) or bimodal (α > 120°). When the disagreement between sensory inputs is zero or 180°, the rings behave as if they were independent. Accordingly, the intermediate ring of “flip cells” shows a unimodal distribution of activity when α < 120° and a bimodal distribution when α > 120° (Fig. 5A, B). At α = 180° (Fig. 5C), the distribution becomes similar to that of Fig. 1C, c or d. The simulated rotation of centers of activity in each ring and in the entire HD system as a function of the cue rotation (similar to the experiment of [11], see also [12]) is shown in Fig. 5D. It should be pointed that the critical value of α = 120° that agrees with experimental observations [11] was achieved by adjustment of the model parameters. Other choices of parameters yielded smaller values of the critical angle: e.g., α = 80°.

5

Discussion

Thus, the entire set of simulation results successfully reproduces the observed phenomena [10, 11] at a qualitative level. The question of a quantitative agreement would require a much more thorough modeling study based on realistic units. There are many aspects of the model that would need to be addressed in future studies. For example, plasticity was not taken into account in this model. It is more important, however, to understand and verify model predictions with and without plasticity, and then see whether a corrected model can be consistent with all documented observations. 5.1 Related Work and Predictions Other modeling studies have also addressed these phenomena. For example, an earlier study by the same group who discovered them offered an alternative interpretation in terms of a computational neural network model (Page et al. [12]) that is based on assumptions about plasticity. The Page model successfully reproduces observed phenomena quantitatively and deserves further verification. One of its prediction is that if a cue rotation were to be performed in the opposite direction, then the system would overshoot, instead of undershooting. In other words, a curve in Fig. 5D would go above the dashed line. In contrast, the model of the present work predicts undershooting, regardless of the direction of rotation (Fig. 4B). Recently an experimental test of this sort was made in the Jeffery lab, with prelimi‐ nary results indicating consistency with Fig. 4B, i.e., undershooting, therefore, supporting the proposed two-ring model (Kate Jeffery, private communication). On the other hand, the present two-ring model might have its own shortcomings. E.g., one of its predictions is that whenever the cue conflict is eliminated, all units should become again normal HD cells. This, however, may not be necessarily the case in the model, if plasticity will be introduced into it. This is one example of many questions that cannot be discussed here due to the limited volume.

A Continuous-Attractor Model of Flip Cell Phenomena

6

171

Conclusions

Mammalian HD cells work together to form a representation of the sense of direction in a local environment, so that an animal knows what direction it is facing, what it can see there and how to navigate. Understanding how this representation comes about will not only illuminate how the brain extracts knowledge from sensory inputs, but may contribute to development of navigation aids for humans, as well as navigation systems for robots. The concept of continuous attractor dynamics, popular in the recent past, seems to be central to understanding of the mechanisms underlying HD and place cell phenomena [4], as well as many other cognitive abilities of the brain. In this context, the attractor concept deserves serious attention and further validation in biology. New experimental paradigms need to be proposed to fully address this topic [13]. This work presented a novel mechanism of attractor dynamics, which may be responsible for the variety of recently observed mysterious HD cell phenomena, among which are flip cells (bidirectional cells) and asymmetric flip cells (within- and betweencompartment bidirectional cells), which seem hard to reconcile with the original model of a ring attractor in the HD system. The explanation proposed here has been supported by numerical simulations of a neural network model, which keeps the attractor hypoth‐ esis [6]. Further investigation is necessary to make a quantitative connection of this model to the brain in order to support, correct or reject the model. Many alternative theoretical possibilities remain (e.g., [12]), which need to be judged and reconciled with the experimental data. The study of flip cells needs to be connected to studies of related phenomena, such as behavioral correlates of HD cells in 3D space, in other species [14], and the interaction of HD representations with other spatial representations, primarily including place cells, grid cells, border cells and so on. Unfortunately, it is impossible to address those impor‐ tant topics in this short note. Acknowledgments. The author is grateful to Drs. Kate J. Jeffery and Hector Page from the Institute of Behavioural Neuroscience, University College London, London, United Kingdom, for fruitful discussions of the ideas of this work and its outcome. This work was supported by the RSF Grant # 15-11-30014.

References 1. Taube, J.S., Muller, R.U., Ranck, J.B.: Head direction cells recorded from the postsubiculum in freely moving rats. I. Description and quantitative analysis. J. Neurosci. 10, 420–435 (1990) 2. Dumont, J.R., Taube, J.S.: The neural correlates of navigation beyond the hippocampus. Prog. Brain Res. 219, 83–102 (2015) 3. Taube, J.S.: Head direction cells and the neurophysiological basis for a sense of direction. Prog. Neurobiol. 55(3), 225–256 (1998) 4. Samsonovich, A., McNaughton, B.L.: Path integration and cognitive mapping in a continuous attractor neural network model. J. Neurosci. 17(15), 5900–5920 (1997) 5. Strogatz, S.H.: Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry, and Engineering, p. 324. Addison-Wesley, Reading, MA (1994)

172

A.V. Samsonovich

6. Skaggs, W.E., Knierim, J.J., Kudrimoti, H.S., McNaughton, B.L.: A model of the neural basis of the rat’s sense of direction. In: Tesauro, G., Touretzky, D., Leen, T. (eds.) Advances in Neural Information Processing Systems, pp. 130–180. MIT, Cambridge (1995) 7. Zhang, K.: Representation of spatial orientation by the intrinsic dynamics of the head-direction cell ensemble: a theory. J. Neurosci. 16(6), 2112–2126 (1996) 8. Samsonovich, A.V.: Continuous attractor network. In: Izhikevich, E.M. (ed.) Scholarpedia: The Free Peer-Reviewed Encyclopedia (2010). http://www.scholarpedia.org/article/ Continuous_Attractor_Network 9. Peyrache, A., Lacroix, M.M., Petersen, P.C., Buzsaki, G.: Internally organized mechanisms of the head direction sense. Nat. Neurosci. 18(4), 569–575 (2015). doi:10.1038/nn.3968 10. Jacob, P.Y., Casali, G., Spieser, L., Page, H., Overington, D., Jeffery, K.: Nat. Neurosci. 20(2), 173–175 (2017). doi:10.1038/nn.4465 11. Knight, R., Piette, C.E., Page, H., Walters, D., Marozzi, E., Nardini, M., Stringer, S., Jeffery, K.J.: Weighted cue integration in the rodent head direction system. Philos. Trans. R. Soc. Lond. B 369(1635), 20120512 (2013) 12. Page, H.J.I., Walters, D.M., Knight, R., Piette, C.E., Jeffery, K.J., Stringer, S.M.: A theoretical account of cue averaging in the rodent head direction system. Philos. Trans. R. Soc. Lond. B 369(1635), 20130283 (2013) 13. Samsonovich, A.V.: Bringing consciousness to cognitive neuroscience: a computational perspective. J. Integr. Des. Process Sci. 11(3), 19–30 (2007) 14. Finkelstein, A., Derdikman, D., Rubin, A., Foerster, J.N., Las, L., Ulanovsky, N.: Threedimensional head-direction coding in the bat brain. Nature 517(7533), 159-U65 (2015). doi: 10.1038/nature14031

Neural Network Classification Method for Solution of the Problem of Monitoring Theremoval of the Theranostics Nanocomposites from an Organism Olga Sarmanova1 ✉ , Sergey Burikov1,2, Sergey Dolenko2, Eva von Haartman3, Didem Sen Karaman3, Igor Isaev2, Kirill Laptinskiy1,2, Jessica M. Rosenholm3, and Tatiana Dolenko1,2 (

)

1

2

Physical Department, M.V. Lomonosov Moscow State University, Moscow, Russia [email protected] D.V. Skobeltsyn Institute of Nuclear Physics, M.V. Lomonosov Moscow State University, Moscow, Russia 3 Faculty of Science and Engineering, Abo Akademi University, Turku, Finland

Abstract. In this study artificial neural networks were used for elaboration of the new method of monitoring of excreted nanocomposites-drug carriers and their components in human urine by their fluorescence spectra. The problem of clas‐ sification of nanocomposites consisting of fluorescence carbon dots covered by copolymers and ligands of folic acid in urine was solved. A set of different archi‐ tectures of neural networks and 4 alternative procedures of the selection of signif‐ icant input features: by cross-correlation, cross-entropy, standard deviation and by analysis of weights of a neural network were used. The best solution of the problem of classification of nanocomposites and their components in urine provides the perceptron with 8 neurons in a single hidden layer, trained on a set of significant input features selected using cross-correlation. The percentage of correct recognition averaged over all five classes, is 72.3%. Keywords: Artificial neural network · Inverse problem · Fluorescent spectroscopy · Carbon nanocomposite · Drug carrier

1

Introduction

Nowadays it is extremely important to create novel theranostic nanoagents that can simultaneously be used for the diagnostics and treatment of diseases [1]. Such agents can simultaneously perform the following functions in the body: (1) to identify the diseased tissues by change of their fluorescence properties; (2) after loading drugs on their surface to carry out targeted delivery of these drugs, preventing their invasion in the healthy tissues; (3) to control the localization of the drug in the body at the cellular level by fluorescence spectra agents. Because of an ability to stable fluorescence, possi‐ bility of targeted surface functionalization and immobilization of drugs on it, nontoxicity and high biocompatibility, carbon nanoparticles are more suitable than many other nanoparticles for such applications in nanomedicine [1, 2]. © Springer International Publishing AG 2018 A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6_24

174

O. Sarmanova et al.

During development of such theranostics nanoagents one needs to pay attention to control of removal of them and their components from the body. In this paper we propose a new method of monitoring extracted nanocomposites and their components in human urine by fluorescence spectra using neural network classification method. One of the ways of such control is an optical visualization of carbon nanoparticles with the use of their fluorescent properties. But the fluorescence spectra of carbon nanocomposites and their components overlap significantly with the spectra of fluorescence of natural fluo‐ rophores of biological tissues, i.e. autofluorescence. Therefore, the problem of optical visualization of carbon nanocomposites in biological tissue lies in the development of effective methods of the separation of their fluorescence from the background of auto‐ fluorescence of urine. Previously, such problems were successfully solved using artifi‐ cial neural networks (ANN) for nanodiamonds and carbon dots in the egg white [3] and in human urine [4]. The authors have demonstrated that ANN allow to detect the fluo‐ rescence of detonation nanodiamonds and carbon dots on the background of natural fluorescence of egg white and urine and to determine the concentration of nanodiamonds in these objects with sufficiently high accuracy - not worse than 0.002 mg/ml [4] and carbon dots in the egg white – with the accuracy of 0.004 mg/ml [3]. In this study, the method of optical imaging in biotissue of novel theranostic nano‐ agents, which simultaneously can be used as fluorescent markers and drug carriers, was elaborated.

2

Experiment

In this study nanocomposites CD@cop@FA – carbon dots coated with poly(ethylene‐ glycol)–poly(ethylene imine) copolymers (cop) tagged with folic acid (FA) [5] were used. As it is known, folic acid is necessary for the organism for development and growth of new cells, including cancer cells [6]. That is why, in tumors an expression of the folate receptors occurs. As a result, tumors actively “take” from the body free folic acid, which is used for their growth [6]. In this regard, as therapy such ligands of folic acid are used, that block the expression of folate receptors and stop the outflow of folic acid from the body and thus stop the tumor growth. Exactly such ligands were attached to the surface of nanocomposites [5]. During injection of such nanocomposites in the body the following situations are possible: (1) nanocomposite has not delivered the drug and excreted unchanged; (2) nanocomposite has delivered from its surface ligands of folic acid, and only the CD@cop component has been excreted; (3) drug and copolymer were separated from the nanocomposite, CD was excreted; (4) copolymer and excess folic acid are excreted separately. Thus, the following 5 classes of substances can be present in urine: CD@cop@FA, CD@cop, CD, cop, and FA. In the study the samples with all possible combinations of these classes of nano‐ composites and their components in urine from two different donors aged 18 to 25 years were simulated. The suspensions with 32 combinations of components in urine were prepared in the concentration range from 2.1 to 2.7 mg/l for each component. Fluorescence spectra of all the prepared suspensions of the nanocomposites and their components in urine were obtained experimentally. For excitation of fluorescence the

Neural Network Classification Method for Solution of the Problem

175

diode laser with a wavelength 405 nm was used. The registration system consisted of monochromator (Acton, grade 1800 grooves/mm, focal length 500 mm) and PMT (Hamamatsu, H-8259-01). Fluorescence spectra were recorded in the range 410– 750 nm. Every spectrum had 341 channels. Processing of the spectra consisted of subtraction of the pedestal caused by elastic light scattering, and normalization of spectra to the area of the Raman band of stretching vibrations of water. In Fig. 1 one can see the experimental fluorescence spectra of urine and all of the considered classes of components. In total 248 fluorescence spectra were obtained – 2 series with 124 spectra for the same combinations of components with the same concentrations in urine from two donors.

Fig. 1. Raman and fluorescence spectra of urine and suspensions of nanodiamonds and their components in urine. The concentration is 2.7 mg/l.

3

Results

The data array was randomly divided into training (trn), validation (valid) (used to determine the moment to stop training), and examination (exam) (out-of-sample) sets in the ratio of 70:20:10, respectively. Thus, the training set consisted of 175 patterns (spectra), the validation set – 49 patterns, the examination set – 24 patterns. For solution of the classification problem the following architectures of multi-layer perceptrons (MLP) were used: with one hidden layer – N01 (with 8, 16, 32 and 64 neurons in the hidden layer), and with two hidden layers – N02 (with (8 + 2), (8 + 4), (12 + 3), (16 + 8), (32 + 16) neurons in the hidden layers). 3 equal neural networks (MLP) were trained with different initial weight values, and the results of their appli‐ cation were averaged, to eliminate the influence of the initial MLP weights choice. All ANN had 5 outputs corresponding to each of five classes: CD@cop@FA, CD@cop, CD, cop, and FA. Decision about presence of the substance from the particular class in urine was accepted when the amplitude on the corresponding output exceeded the defined threshold.

176

O. Sarmanova et al.

3.1 ANN Training on the Complete Set of Input Features In Fig. 2 one can see the dependence of percentage of the correct answers of network on the values of threshold for each of 5 classes separately and for all classes together.

Fig. 2. The dependence of percentage of the correct answers of ANN on the value of threshold.

In Fig. 3 the best results of classification of all used architectures of MLP trained on the complete set of input features are presented. The percentage of the correct answers was calculated for all classes together.

Fig. 3. Percentage of correct recognition for different architectures of MLP trained on the complete set of input features (the best results).

As it can be seen from the obtained results, the best classification was demonstrated by the perceptron with 8 neurons in the single hidden layer – 67.9%.

Neural Network Classification Method for Solution of the Problem

177

3.2 ANN Training After Significant Input Features Selection To reduce the input dimensionality of the problem and possible retraining of the neural network 4 alternative procedures for selection of significant input features - by crosscorrelation, by cross-entropy, by standard deviation and by analysis of weights of a neural network [7] - were carried out. For comparison of the efficiency of the methods of selection, at the use of each of them, the sets with about 50, about 150 and about 250 significant input features were formed. In all cases the perceptron with 8 neurons in the single layer (which have demonstrated the best results in p. 3.1) was trained on the selected sets of input features. 3.2.1 Cross-Correlation The values of cross-correlation between the values in each spectral channel with the values of each of output were calculated. For every output the significant input features were determined separately, then all features, which are significant for at least one output, were used for further perceptron training. The best result was demonstrated by network with 252 significant features - 72.3% of correct recognition on the examination set. 3.2.2 Cross-Entropy The values of cross-entropy between the values in each spectral channel with the values of each of output were calculated. For every output the significant input features were determined separately, then all features, which are significant for at least one output, were used for further perceptron training. In this case we could not find parameters, providing selection of 50, 150 and 250 significant features due to the structure of the data. That is why the networks with 52, 85 and 318 input features were trained. The best result was provided by the network with 318 input features– 69.7% of correct recognition on the examination set. 3.2.3 Standard Deviation We calculated the value of standard deviation of the values in each spectral channel for all patterns. It is proportional to the entropy, i.e. amount of information in the given channel. For every output the significant input features were determined separately, then all features, which are significant for at least one output, were used for further perceptron training. The best result was provided by the network with 248 input features– 71.6% of correct recognition on the examination set. 3.2.4 Analysis of Weights of a Neural Network This method of the selection of significant features [8] is based on the weights of neural networks, trained on the complete dataset. The idea of the method based on the fact that a significant input feature usually has a large weighting coefficients at least for some of the links that connect it to the output layer; on this basis, the index of significance is

178

O. Sarmanova et al.

determined for each input feature (spectral channel). For each class of substances the determination of significant channels was made separately as follows: 5 identical neural networks with 32 neurons in a single hidden layer were trained on the full dataset, varying the initial value of the weights. Further the analysis of weights of each of obtained neural networks was carried out. For each network the mean value for all channels and its standard deviation were calcu‐ lated. If the value of index of significance in the given channel exceeded the “mean value + k standard deviations”, this channel was considered as significant. Further, if the channel inside the class was significant at least for three networks from five, this channel was considered as significant for the class. If the channel was significant for at least one class, it was considered as significant and used for further ANN training. In dependence on the parameter k it was possible to vary the number of significant channels. The best result was demonstrated by the network with 112 significant features - 66% of correct recognition on the examination set.

4

Conclusions

Thus, best results of solution of the problem of classification of nanocomposites and their components in urine were provided by the perceptron with 8 neurons in the single hidden layer trained on the set of significant features selected by cross-correlation. Percentage of correct recognition averaged over all five classes was 72.3%. Acknowledgments. The following parts of this study were supported by the following foundations: (i) elaboration of optical visualization of nanocomposites using ANN (O.E.S.,I.V.I.,T.A.D.) have been performed at the expense of the grant of Russian Science Foundation (project no. 17-12-01481); (ii) the test of nanocomposites properties (S.A.B., K.A.L.) were supported by the grant of the Russian Foundation for Basic Research no. 15-29-01290 ofi_m.

References 1. Doane, T.L., Burda, C.: The unique role of nanoparticles in nanomedicine: imaging, drug delivery and therapy. Chem. Soc. Rev. 41(7), 2885–2911 (2012) 2. Hong, G., Diao, S., Antaris, A.L., Dai, H.: Carbon nanomaterials for biological imaging and nanomedicinal therapy. Chem. Rev. 115(19), 10816–10906 (2015) 3. Dolenko, T., Burikov, S., Vervald, A., Vlasov, I., Dolenko, S., Laptinskiy, K., Rosenholm, J.M., Shenderova, O.: Optical imaging of fluorescent carbon biomarkers using artificial neural networks. J. Biomed. Opt. 19(11), 117007-1–117007-9 (2014) 4. Laptinskiy, K., Burikov, S., Dolenko, S., Efitorov, A., Sarmanova, O., Shenderova, O., Vlasov, I., Dolenko, T.: Monitoring of nanodiamonds in human urine using artificial neural networks. Phys. Status Solidi A 213(10), 2614–2622 (2016) 5. Prabhakar, N., Näreoja, T., von Haartman, E., Karaman, D., Burikov, S., Dolenko, T., Deguchi, T., Mamaeva, V., Hänninen, P., Vlasov, I., Shenderova, O., Rosenholm, J.: Functionalization of graphene oxide nanostructures improves photoluminescence and facilitates their use as optical probes in preclinical imaging. Nanoscale 7, 10410–10420 (2015)

Neural Network Classification Method for Solution of the Problem

179

6. Kim, Y.: Role of folate in colon cancer development and progression. J. Nutr. 133, 3731S– 3739S (2003) 7. Efitorov, A., Burikov, S., Dolenko, T., Laptinskiy, K., Dolenko, S.: Significant feature selection in neural network solution of an inverse problem in spectroscopy. Procedia Comput. Sci. 66, 93–102 (2015) 8. Gevrey, M., Dimopoulos, I., Lek, S.: Review and comparison of methods to study the contribution of variables in artificial neural network models. Ecol. Model. 160, 249–264 (2003)

Realization of the Gesture Interface by Multifingered Robot Hand Pavlovsky Vladimir and Stepanova Elizaveta(B) Keldysh Institute of Applied Mathematics, Moscow, Russia [email protected], [email protected]

Abstract. The paper considers theoretical mechanical model of a multifingered arm with 21 degrees of freedom. The main objective of the work is the creation of gesture interface. Gesture interface includes the set of gestures, the synthesis of finger control schemes for 26 gestures, as well as gesture recognition task with the help of convolutional neural network training. As the demonstration we propose to observe the results of 26 gestures recognition with the help of constructed convolutional network. For 26 classes 15600 images at different distance and at different angles were created. As a result of convolutional neural network training the accuracy of a test set classification is 76%.

1

Introduction

The collaborative robotics is a human-computer interaction. One of the methods of collaborative robotics is an interaction based on human gestures. Gesture recognition task can be used in different spheres of our life: on the factory in the bustling workshop; in space, where sound waves can not propagate because of vacuum; in communication with deaf people; in household appliances control and many other tasks.

2

Problem Formulation

A hand model in the software package Universal Mechanism is considered. The model consists of a palm with a single rotational degree of freedom and five fingers connected by rotational joints. Also, in this model the thumb metacarpal bone is considered similarly with other fingers’ bases, i.e. a spherical type of joint connection. First phalanges are connected with a palm by a spherical joint with two degrees of freedom each. Thus, the model has 21 degrees of freedom, which is close to an actual number of degrees of freedom of the human hand. The size of a model hand is similar to my own hand’s size. The main objective of the work is the synthesis of finger control schemes, as well as the model hand’s gesture recognition with the help of neural network training. On the image below one can see a schematic representation of a model hand with degrees of freedom (Fig. 1). c Springer International Publishing AG 2018  A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6 25

Realization of the Gesture Interface by Multifingered Robot Hand

181

Fig. 1. Model hand, degrees of freedom

3

Experiment

Artificial neural network - mathematical model, as well as its software implementation, built on the principle of the organization and functioning of biological neural networks. The idea of convolutional neural networks is similar to the idea of our brains structure. We used classical convolutional neural network in this task because according to our research and study of other similar works, it shows good results in tasks of image recognition and classification. Description of the system: To solve the tasks of gesture recognition we will work with a convolutional neural network. The convolutional neural network was created in the Neural network toolbox in a software package Matlab. This program allows us to classify gesture images. A vocabulary of sign language for one hand was created, where one static gesture corresponds to one letter of the English alphabet. We teached the program to perceive static gestures of hands at different distances, at different angles of shooting (Fig. 2). This convolutional neural network for this task consists of 7 layers: 1. ImageInputLayer - receives the data set of images of 60 × 60 × 3 dimension, where 60 × 60 (pixels) is the size of the training images, 3 is the color channels (RGB). The task of this system is to distribute a set of images by classes (class - one gesture corresponding to the letter of the English alphabet). For each class we prepared 600 images. The program receives images randomly from folders with labels corresponding to classes’ names (A, B, C, D, etc.). This system uses 90% of the images for training, and 10% for testing. The result of testing is accuracy of classification. 2. Convolution Layer is the main block of a convolutional neural network. Input data for this layer is a 60 × 60 × 3 matrix. One of the features of the convolution layer - the kernel that runs through the entire data set, starting from the upper left corner and moving to the right step by step, multiplies with each section. The result of this matrix multiplication is summed and put to the appropriate place of new matrix. That is, the result of one multiplication

182

P. Vladimir and S. Elizaveta

Fig. 2. The set of gestures A–Z

must be one number. The dimension of the convolution kernel is a variable parameter and depends on the task, to solve this problem, we used a 6 × 6 × 3 kernel. With these values the training was most successful. Unlike the size of the kernel, we can’t choose the kernel’s values, since they must be revealed during the learning process. After passing through the entire data set, the output of the kernel is a matrix (55553) - a map of features. Where a feature is an image property (lines at a certain angle, color, curves, etc.), the number of features is an adjustable parameter. For this task, the number of features required is 30. Let’s consider this process in a simplified example. The program received an image. Let the first kernel be a detector of curves. To simplify the understanding, we ignore the fact that the depth of the core is 3, we consider only the upper layer. The kernel has a pixel structure in which numerical values are higher along the region that determines the shape of the curve (Fig. 3). In the initial position, the kernel is in the upper left corner, it multiplies the kernel values by the pixel values of this region. Let’s look at the example

Fig. 3. Curve detecting kernel

Realization of the Gesture Interface by Multifingered Robot Hand

Fig. 4. Input image

3.

4.

5.

6. 7.

183

Fig. 5. Target curve on the image

of the image that we want to classify, and put the kernel in the upper left corner (Figs. 4 and 5). Multiply the value of the kernel by the image area values. As a result, we get the value 6600 - a large number. Let’s try to multiply the kernel to another area of the image. As a result, the product is zero, which means that the kernel has not found the desired feature in this area. The filter we described is simplified. In fact, the feature card would look differently (Figs. 6 and 7). ReLULayer is the activation function after the convolutional layer, however, the maximum function f (x) = max(0, x) is selected for activation. This function cuts out unnecessary or bad signs. A high value of 6600 from the previous argument shows that, perhaps, there is a curve on the image, and such a probability activated the filter. In the right upper corner, the value for the feature map will be 0, because there was nothing in the picture that could activate the kernel (in other words, there was no curve in this area). MaxPooling - this layer takes small individual image fragments (2 × 2 in the case of this system) and combines each fragment into one value. The operation of the pooling reduces the spatial volume of the image (it becomes 27 × 27 × 3). FullyConnectedLayer, The fully-connected layer refers to the output of the previous layer, and the features that are more associated with the individual class are determined. The output of this layer is an 26 -spatial vector. SoftMaxLayer is an activation layer, that maps information at the input of a set of elements to classes. ClassificationOutputLayer - displays and classifies information. For training the backpropagation method is used.

184

P. Vladimir and S. Elizaveta

Fig. 6. successful curve detection

Fig. 7. Unsuccessful curve detection

4

Conclusion

As a result, the gesture interface has been created, including the control scheme the image recognition program with the help of convolutional neural network. The network was tested on the task where we tried to classify 26 gestures and received the 76% accuracy of test images classification. One can see on the image below the result of test images classification accuracy (Fig. 8).

Fig. 8. Accuracy

Realization of the Gesture Interface by Multifingered Robot Hand

185

In the nearest future we are planning to increase the accuracy of the classification task. Moreover we will consider gestures in motion (at the moment we use only images of static gestures).

References 1. Nagapetyan, V.G.: Gesture recognition methods on the base of long-range images analysis, Moscow (2013) 2. Craig, J.J.: Introduction to robotics. Mechanics and Control. Monograph. SRC “Regular and Chaotic Dynamics”, Institute for Computer Research, Izhevsk (2013) 3. Yurevich, E.I.: Fundamentals of Robotics, 2nd edn. BHV-Petersburg, St. Petersburg (2005) 4. Formalski, A.M.: Anthropomorphic mechanisms movement (1982) 5. Haykin, S.O.: Neural Networks. A Comprehensive Foundation. McMaster University, Ontario, Canada 6. Nielsen, M.: Neural network and deep learning. http://neuralnetworksand deeplearning.com/

A Conscious Robot that Can Venture into an Unknown Environment in Search of Pleasure Yuichi Takayama and Junichi Takeno ✉ (

)

Robot Science Laboratory, Computer Science, Meiji University, 1-1-1 Higashi-Mita, Tama-ku, Kawasaki-shi, Kanagawa 214-8571, Japan [email protected], [email protected]

Abstract. A “conscious system that can venture into an unknown environment” has been proposed. This study models the process of consciousness of a person who is going into an unknown environment. First, we assumed that to go into an unknown environment, the person needs to be curious about that environment and assured of its safety. Curiosity is a tendency to become interested in unknown phenomena and draw information from them. We consider that to acquire the behavior of going into an unknown environment (curiosity behavior), firstly the person needs in some way to go through many experiences of pleasure in unknown environments and increase curiosity and interest in such environments. To enter an unknown environment the person must also be assured that the environment is safe. We have developed a conscious system that can venture into an unknown environment and tested whether a robot can voluntarily enter an unknown envi‐ ronment. Keywords: Conscious robot · Venture · Unknown environment · Curiosity · Pleasure

1

Mechanism of Searching the Unknown

The authors considered the process of consciousness that occurs when humans “venture into an unknown environment in search of something pleasant” and modeled that process. Basically, something “unknown” is thought to cause an unpleasant sensation in humans [2]. As such, unknown environments should also cause unpleasantness to humans. Humans often feel uncomfortable because of emotions, such as pain or solitude, resulting from physical changes and external stimuli. However, the authors consider an “unknown environment” to be a source of particular unpleasantness, and this is because the unknown environment is brought about by the cognition of the external environment. Reason is what determines the behavior taken upon the cognition of this external envi‐ ronment and the logical judgment performed from the information obtained from the environment. The authors think that there is one source of unpleasantness in this “logical judgment” here. And therefore, we think that the unpleasantness caused by an unknown © Springer International Publishing AG 2018 A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6_26

A Conscious Robot that Can Venture into an Unknown Environment

187

environment is brought about by reason. In order for humans to venture into an unknown environment, it is necessary to change the behavior that is determined by reason. In addition, while the unknown produces a feeling that is initially uncomfortable to humans, if they are given the opportunity to know the unknown after that, the authors think that this is an opportunity for humans to change toward a feeling of pleasantness regarding the unknown. We speculate that by repeating experiences in which unknowns can become known, humans gradually become able to represent unknown subjects. This would seem to be the development of a sort of curiosity concept. Curiosity is commonly said to be an interest in unknown events, and that there is a tendency to derive information from them [3]. We may presume that the interest in an unknown environment arises due to human curiosity, and that it serves to aid the desire to move forward in an unknown environment. However, although interest in an unknown environment may arise due to human curiosity, there is a doubt as to whether that alone will lead the human to the act of actually entering the unknown environment. Here we consider the relationship between “death,” which is an example of an unknown event, and curiosity. Although curiosity may lead to an interest in death, it seems that no one dies from an interest in death. From this example, in order for humans to challenge the unknown, we think that, in addition to the interest generated by curiosity, something different is necessary. The authors think that that is the “growth of curiosity in the unknown.” As humans receive pleasure related to unknown events, their curiosity about the unknown events grows and their interest in them grows. The authors thought that curiosity will grow to challenge unknown events when crossing certain lines. In this study, we don’t consider what subjects humans are interested in because of curiosity. The curiosity of the system described later is interested in all unknown subjects. The authors also believe that there is another factor that enables humans to move into an unknown environment. That is securing a safe environment. If unpleasantness should occur in an unknown environment, as long as one has secured safety beforehand, that will be a place that you can escape to. In this way, we thought that securing a safe environment — a place where one could escape to — would become a spiritual support and make it easy to move forward into an unknown environment. Based on the above points, the hypothesis of the mechanism by which humans venture into an unknown environment as proposed by the authors is as follows. “When humans encounter an unknown environment, they are interested in the environment due to their curiosity, but reason will stop human moving forward and prevent it from moving forward into the unknown environment. To be able to proceed into an unknown envi‐ ronment it is necessary to have a growing curiosity caused by pleasure associated with the unknown environment. It is also necessary to secure a safe environment that can be a place to escape to when unpleasantness occurs in an unknown environment. Only after this growth in curiosity and securing a safe environment can humans proceed into an unknown environment.”

188

2

Y. Takayama and J. Takeno

Conscious System that Ventures into Unknown Environments

In this study we use a consciousness module called a “MoNAD,” which stands for Module of Nerves for Advanced Dynamics(Takeno, 2011). The MoNAD is a model that is composed of neural networks and achieves consciousness functions. We have also constructed a conscious system by using multiple MoNADs. Our conscious system consists of three subsystems, the “Reason subsystem,” “Emotion & Feeling subsystem,” and “Association subsystem.” In the Reason subsystem, the state of the conscious system itself in the external environment is determined based on information input from the environment, and emotion and feelings are represented in the Emotion & Feeling subsystem. The role of the Association system is to arbitrate (i.e., settle) the information of the Reason subsystem and the information of the Emotion & Feeling subsystem. Our conscious system that can venture into unknown environments roughly consists of six MoNADs. The Reason subsystem consists of the reason MoNAD (Re), safety MoNAD (Sa), curiosity MoNAD (Cu), and search MoNAD (Se). The Emotion & Feeling subsystem consists of the pleasant MoNAD (P) and unpleasant MoNAD (UP). We do not use the MoNADs in the emotion subsystem in this study. Incidentally, Re is the part that would intelligently make judgments in humans. Sa confirms safety. Cu mimics human curiosity. Se searches for unknown environments and makes them known. P represents pleasure. UP represents unpleasantness (Fig. 1).

Fig. 1. Conscious system that ventures into unknown environments and Cu’s switching.

In our experiment, the robot judges whether to move forward or stop (rationally) depending on the external situation. Also, information input from Cu and Sa becomes judgment information for the behavior decision. Sa confirms the safety. In this study, we assume that safety is secured when the current position of the robot is a known place. If the next location to move to from the current position of the robot is unknown, Cu issues a go forward signal or a stop signal. Cu uses two MoNADs, Cu1 and Cu2, to represent the growth of curiosity. Since in the initial state Cu1 is weakly interested in the unknown environment, Cu1 outputs a signal stopping the forward movement of the robot with respect to Re. As Cu1 grows with the experience of receiving pleasure in the unknown environment, Cu1 is set to switch to Cu2. Cu2 outputs a go forward signal when recognizing an unknown environment.

A Conscious Robot that Can Venture into an Unknown Environment

189

Se is the MoNAD that enables the robot itself to be conscious of the known envi‐ ronment and the unknown environment. Se recognizes the present location of the robot itself in the experimental environment using a one-dimensional array M[k] (where k is an integer) that is set in the conscious system. That is, M is the environment map of the robot. At the start of the experiment, M[0] = 1 is set as the initial value, and the other array elements are set to 0. Element 1 means that the position k of M is a known envi‐ ronment. Element 0 corresponds to an unknown environment. P represents a “pleasant” state. In this study, the robot represents a current pleasant state (p0) when its present location is a known location and when the unknown envi‐ ronment is made known by a forced forward movement. There are cases in which the environment in front of the robot may be a known place or an unknown place, but if the robot judges that the environment in front of it is a place that will bring it pleasure, even if that is an unknown place, a future pleasant state (p1) is represented. UP represents an “unpleasant” state. In this study, the robot represents a current unpleasant state (up0) when its current location is unknown. And when the environment in front of the robot is unknown, a future unpleasant state (up1) is represented. All of the MoNADs perform in advance the initial learning that is necessary for the experiments, and the learning method uses back propagation.

3

Robot Experiments

The conscious system as described in the previous chapter was mounted on a robot and experiments were conducted. In these experiments, we used the “e-puck,” a commercial small robot. The robot has red LEDs on its exterior. In our experiments, the LEDs corresponding to the emotions represented by the robot light up. In our experiments, the robot uses the environment map M to move forward or stop. The actual distance between M[k] and M[k+1] in the experimental environment is approximately 10 cm. We begin the experiment on a sufficiently large and flat desk. First, we set the area (M[0] = 1) where the robot is placed (this is a 10 cm × 10 cm wide region, and the position is M[0]), and we set the area (M[1]) in front of the robot, which is where the robot will move to, to an unknown area (M[1] = 0). Since the area in front of the robot is an unknown area, the feeling MoNAD UP represents an unpleasant state (up1) and the robot does not move forward. As mentioned above, in order for the robot to move forward into the unknown area by itself, it is necessary for it to know the unknown environment first, to experience pleasure, and for its curiosity Cu to grow. However, the robot can not experience pleasure as it is. Therefore, the experimenter forcibly pushes the robot by hand and moves it 10 cm forward from its current position M[0] into the unknown area M[1]. Through this process, the system activates the search MoNAD Se to correspond to an unknown environment and sets the information (M[1] = 0) of the unknown area to be known information (M[1] = 1). The robot can feel pleasure in a known environment, so when it knows an unknown area, the robot has more opportu‐ nities to feel pleasure.

190

Y. Takayama and J. Takeno

Through the process of experiencing a pleasant state for about n times (n = 4 in an experiments), the robot’s curiosity can grow adequately and the robot can move forward into an unknown environment voluntarily. In our experiment, the distance the robot can travel in one movement is 10 cm, and for every 10 cm of movement forward, if the place is an unknown environment, the robot changes it to a known environment. Then, the robot judges whether the next area that it will move to 10 cm forward is unknown or already known (Fig. 2).

Fig. 2. State of the experiment a: represent p0, up1 for an unknown area, b: represent up0, up1 when forced to move forward to an unknown area, c: represent p0, up1 when successfully making an unknown area known, d: represent p0, p1 with the unknown area in front.

Figure 3 shows the log of experiment results. The horizontal axis of the figure repre‐ sents the time series of the action steps of the robot. The vertical axis represents four important data.

Fig. 3. Log of experiment results

A Conscious Robot that Can Venture into an Unknown Environment

191

Line “a” rises up when robot represents pleasant because of successfully making an unknown area known. Line “b” rises up when robot secures safety. Line “c” rises up when Cu outputs go forward signal. Line “d” rises up when robot is forced to move forward to an unknown area. Figure 3 shows that robot experiences pleasure after it is forced forward movement iteratively. Since Cu is switched to Cu2 due to the fourth pleasant experience at time 12 and safety is secured, at time 14, go forward signal for the unknown region by Cu2 outputs.

4

Conclusions and Considerations

This experiment was conducted 10 times, and in all of the experiments the robot was able to voluntarily venture into unknown areas. This experiment was the first demon‐ stration of “a robot equipped with a conscious system that voluntarily ventured into unknown areas.” In this paper, the authors first consider the mechanism of searching the unknown by humans, and in order for humans to be able to voluntarily venture into an unknown environment, the authors have confirmed and determined that the growth of curiosity and the securing of a safe environment are important factors. Based on this consideration, the authors developed a conscious system using MoNAD consciousness modules. This is a “conscious system that voluntarily ventures into unknown environments.” Then, we performed experiments by installing the conscious system on a robot and demonstrated that the robot voluntarily ventured into unknown environments. The authors pointed out that this demonstration shows that the growth of curiosity and the securing of a safe environment are important factors when humans voluntarily venture into unknown environments.

References 1. Takeno, J.: MoNAD structure and self-awareness. In: Biologically Inspired Cognitive Architectures (2011) 2. Leon, F.: A Theory of Cognitive Dissonance. Stanford University, California (1957) 3. Della, S.: Longman Dictionary of Contemporary English (6E) Paperback. Pearson PLC (2014)

Algorithms for Intelligent Automated Evaluation of Relevance of Search Queries Results Anna Tikhomirova ✉ and Elena Matrosova ✉ (

)

(

)

National Research Nuclear University MEPhI (Moscow Engineering Physics Institute), Moscow, Russia [email protected], [email protected]

Abstract. This paper is devoted to the problem of automated evaluation of rele‐ vance of search queries results. High relevance of search algorithm output is the base of effective large quantities of data processing, which is worked at by users of modern informational systems. Automated and reliable estimate of relevance of search queries results will give the opportunity to lower time expenditures for the best algorithm choice. The usage of improved from this perspective algorithms will allow to raise effectiveness and user satisfaction when dealing with automatic search systems in any activities. Keywords: Neural network · Semantic analysis · Algorithm · Search query · Teaching model · Machine learning

1

Introduction

The successful solution of current scientific and practical problems cannot be achieved without relevant and complete information about the state of the problem, the latest methods in the subject of research and trends in science development. Nowadays one cannot imagine a complete scientific work that didn’t use materials publicly available on the Internet or materials from private information systems. A quick and precise response to a search query is a crucial part of effective user operation and a competitive advantage of search engines. The basis of the successful operation of a search engine is an effective algorithm of providing a response to a search query according to user expectations. Companies producing search engines and services are constantly carrying out research to improve algorithms in use [1]. The term ‘neural network’ was coined in the middle of 20th century. Nowadays neural networks are the main instrument of developers of intelligent search engines. Despite many capabilities of this instrument and its ability to self-learn in the process, however, all new or updated algorithms need to be tested before their practical application. It is done to ensure that users get the engine with an algorithm that works correctly and efficiently. This makes the problem of algorithm preliminary quality evaluation imperative for developers. Such quality evaluation is done by analyzing rele‐ vance of search query results.

© Springer International Publishing AG 2018 A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6_27

Algorithms for Intelligent Automated Evaluation

2

193

Algorithm of Forming and Evaluating Results of a Search Output

The workings of search engines that give access to information based on queries become more and more complicated. An enormous amount of information that can be found with one query demands intelligent ranking based not only on the presence of keywords but also on semantic correspondence of the material. Users grow more and more demanding to search results and if they feel that a search engine is not effective they will likely choose another search engine that gives them more relevant results. Technology is constantly improving and researchers aim to use artificial intelligence to solve complicated multivariate problems [2]. For this problem it means using machine learning of search algorithms [3]. In the context of this article machine learning is a system self-learning to find more and more relevant output results based on positive and negative examples. Self-learning means that a machine improves its work quality without any human involvement. Result relevance is a characteristic of a degree of material (ranked and outputted by a search engine) correspondence with user expectations. In the context of the given problem it is semantic correspondence of a search query to an outputted document [4] Since the problem is that results should correspond with human expectations, its solution can be successfully achieved by artificial intelligence systems that are based on selflearning machines optimizing search algorithms. Considering this, it is suggested to look into a possibility of using neural networks to solve this problem. According to the suggested approach, to rate the quality of a search algorithm work one can apply the following diagram [5] (Fig. 1). Data processing

Feature generation

Model quality analysis

Evaluation quality analysis aceccopoB

Fig. 1. Diagram of the quality evaluation of a search algorithm.

2.1 Data Processing This stage is highly important. User input can contain various mistakes and the system should transform any input into easy to analyze format and give relevant results. So, an effective search algorithm must be able to work with inputs of various qualities and successfully execute procedures such as spell-check, correction of lexical mistakes, unit conversion, lemmatization and stemming [6]. Development of such an algorithm requires not only machine learning specialists but also competent linguists [7]. 2.2 Feature Generation Feature generation is a process of formalizing input data and identifying certain signs that can become input for a teaching model. In this stage should be realized the following procedures: identification and removal of noise words, generation of a ‘bag’ of words,

194

A. Tikhomirova and E. Matrosova

use of TF-IDF. Work with the ‘bag’ of words means encoding all the words from a sampling and forming of a unified vocabulary. At the same time a space of dichotomic or serial variables is created, with one dimension for every time a word with a given index is present in the processed text (1).

D = D1 ∪ D2 ∪ … ∪ Dn ,

(1)

Where Di is a set of words in an object i, n is a number of objects. The dimensionality of a feature space becomes equal to the number of unique notnoise words in the whole sampling, and the matrix of features becomes weak. One needs to take into account the word weight. Static measure TF-IDF can be used for that purpose. After processing the data one will get a huge feature matrix. To work with it further one can use latent semantic analysis. It allows picking key features based on the revealed correlation between materials and words (Fig. 2). The set of words

The set of cross points with weight coefficients

The set of documents

Fig. 2. The representation of the latent semantic analysis.

With machine learning, the space of features is expanded in the process of working with a natural language to get various heuristic statistics since they can contain hidden important information [8]. These features include the length of the text and the ration of the query length to the headline length. It is necessary to make the procedure of normalization and centering of the features that prevents the network difficulty and computational effort from going up. 2.3 Model Quality Analysis Algorithm quality evaluations, or quality metrics, can be calculated in different ways. One of the more frequently used methods is a loss function ( ) 1 ∑n (a, xi ),  a, X l = i=1 n

(2)

that can determine the quality of an algorithm a(x) on the plurality of objects X l, and RMSE evaluation (3) [9].

Algorithms for Intelligent Automated Evaluation

√ RMSE =

∑n i=1

(yi − t)2 n

,

195

(3)

where yi is a predicted value, t is a target value. Sample X L is divided into two disjoint sub-samples: teaching X l of length l and control X k of length (4). k =L−l

(4)

For every decomposition n = 1, …, N an algorithm is created and the work quality ( ) is rated based on the control sample n =  a, X k . Then the arithmetic average n for all the decompositions is calculated. It is the evaluation of a sliding control (5). ( ) 1 ∑N (𝜇(X l ), X k ) CV 𝜇, X L = n=1 N

(5)

It is advisable to also trace base lines, i.e. to rate the quality by constant algorithms. These algorithms are not very time-consuming and one can judge the quality of their work by comparing their results with the results of a complicated model. 2.4 Assessors’ Evaluation Quality Analysis The quality of an evaluation model can be determined by comparing it with assessor evaluation quality. Assessor is a person who is tasked by developers with evaluating the search engine based on how well the query and user expectations correspond with found materials. Generally, assessors’ evaluations are discrete data. The procedure of assessors evaluating the quality of work of a search engine is, at its core, a task of group multi‐ criteria evaluation of the given object, and after its completion one must find one unified consolidated evaluation [10]. Such evaluation can be found by different ways. The simplest one is calculating an arithmetical mean, but this evaluation can be incorrect, since assessors’ qualification can differ from person to person. Assessors’ different levels of computer literacy, different professional fields and education all directly influence their evaluation. But it must be said that the true authenticity cannot be achieved without their inherent differences. In that case it necessary to take this peculiarity into account, but at the same time to get an average evaluation, that is close to the true value. To solve this problem one can take into account the weight of each assessor’s eval‐ uation and average them based on the analysis of deviation of an individual evaluation from the expectation value of the sample. With this method the evaluation that is the closest to the average mean in the group will get the highest weight. For n number of evaluation criteria of the output relevance of m assessors there will be m × n evaluations that can be presented as a matrix (6) [11]: ⎡ z11 … z1n ⎤ Z = ⎢… … …⎥ ⎢ m m ⎥ ⎣ z1 … zn ⎦

(6)

196

A. Tikhomirova and E. Matrosova

where zij is an evaluation of the criterion j by assessor i. Then one must calculate the arithmetical mean value for evaluations of each criterion (7):

∑m ∑m 1 ∑m ⃖⃖⃖(i)⃗ ⃗= 1 (A) z⃖⃖⃖⃖ z1 , … , z⃖⃖⃖(i)⃗) = (z1(A) , … , z(A) z⃖⃖⃖(i)⃗ = ( n ) i=1 i=1 i=1 n m m

(7)

(A) ⃗ is an average evaluation of criterion j. where z⃖⃖⃖⃖ j It is suggested to assume the arithmetical mean of the evaluations plurality as the true value. For each assessor we have the evaluation of dispersion of the introduced random variate (8).

𝜎 (i)2 =

1 ∑n (z(i) − z(A) )2 j j=1 j n−1

(8)

Then we calculate the sum of reciprocal values of deviation dispersions of all asses‐ sors (9): ∑m

1 𝜎 (i)2

i=1

(9)

and the weight coefficient for each (10): w(i) =

1 𝜎

∕ (i)2

∑m i=1

1 𝜎 (i)2

(10)

After calculation the weights we calculate the refined criteria evaluations as average weighted evaluation with the account of various levels of assessor evaluations’ error (11):

z(B) = j

∑m i=1

w(i) z(i) j

(11)

The suggested method of assessor evaluations analysis can increase the evaluation relevance authenticity. Since assessors’ opinions are taken as a comparison standard (accounting for errors), the standard variant must also be correctly mathematically processed.

3

Conclusions

The problem of replacing manual labour with automated system is always difficult and complicated. Human evaluation of any object (in this article it’s the relevance of a search engine output) must be transformed into an algorithm realizable by artificial intelligence. Assessor’s evaluation cannot be taken as is, since there is always a factor of varying competencies that makes the evaluation result not uniform enough and demanding corrections.

Algorithms for Intelligent Automated Evaluation

197

The successful solution of this problem is in the interests of not only developers of various automated search engines that are nowadays used everywhere from small busi‐ nesses and network organizations to science information libraries, but also of average users of various services who are interested in reducing the time and effort consumed in processing the search information that doesn’t meet their expectations. Acknowledgments. This work was supported by Competitiveness Growth Program of the Federal Autonomous Educational Institution of Higher Professional Education National Research Nuclear University MEPhI (Moscow Engineering Physics Institute).

References 1. Gabrilovich, E., Markovitch, S.: Feature generation for text categorization using world knowledge. IJCAI 5, 1048–1053 (2005) 2. Samsonovich, A.V.: Functional possible biologically inspired by cognitive architectures. In: XVII All-Russian Scientific-Technical Conference “Neuroinformatics-2015”. National Research Nuclear University MEPhI (Moscow Engineering Physics Institute), Moscow (2015) 3. Samsonovich, A.V., Klimov, V.V., Rybina, G.V.: Biologically inspired cognitive architectures (BICA) for young scientists. In: Proceedings of the First International Early Research Career Enhancement School (FIERCES 2016) (2016). ISBN: 978-3-319-32553-8 (Print) 978-3-319-32554-5 (Online) 4. Hjorland, B.: The foundation of the concept of relevance. J. Am. Soc. Inf. Sci. Technol. (2010) 5. Skorokhodov, I.S., Tikhomirova, A.N.: Key stages of text processing and feature generation in text classification. Probl. Mod. Sci. Educ. 15(57), 18–22 (2016) 6. Balakrishnan, V., Lloyd-Yemoh, E.: Stemming and lemmatization: acomparison of retrieval performances. IACSIT (2014) 7. Balandina, A., Chernyshov, A., Klimov, V., Kostkina, A.: Usage of language particularities for semantic map construction: affixes in Russian language. In: International Symposium on Neural Networks, ISNN 2016, Advances in Neural Networks – ISNN, pp. 731–738 (2016) 8. Klimov, V.V., Chernyshov, A.A., Shchukin, B.A.: Composition of web-services based on semantic description. In: WEBIST 2015 – Proceedings 11th International Conference on Web Information Systems and Technologies (2015) 9. Altman, J.M., Bland, D.G. Statistics notes: measurement error. BMJ (1996) 10. Tikhomirova, A.N., Sidorenko, E.V.: Optimization of the process of scientific and technical expertise projects in nanobiomedical technologies. Nanotechnics 1(29), 26–28 (2012) 11. Kryanev, A.V., Tikhomirova, A.N., Sidorenko, E.V.: Group expertise of innovative projects using the Bayesian approach. Econ. Mathe. Methods 49(2), 134–139 (2013)

“Re:ROS”: Prototyping of Reinforcement Learning Environment for Asynchronous Cognitive Architecture Sei Ueno1,2(&), Masahiko Osawa3,4, Michita Imai4, Tsuneo Kato1, and Hiroshi Yamakawa5,6 1

4

Faculty of Science and Engineering, Doshisha University, Kyoto, Japan 2 Kyoto University of Informatics, Kyoto, Japan [email protected] 3 Japan Society for Promotion of Science, Tokyo, Japan Graduate School of Science and Technology, Keio University, Tokyo, Japan 5 Dwango Artificial Intelligence Laboratory, Dwango Ltd., Tokyo, Japan 6 The Whole Brain Architecture Initiative (A Specified Non-Profit Organization), Tokyo, Japan

Abstract. Reinforcement learning (RL), which is a field of machine learning, is effective for behavior acquisition in robots. Asynchronous cognitive architecture, which is a method to model human intelligence, is also effective for behavior acquisition. Accordingly, the combination of RL and asynchronous cognitive architecture is expected to be effective. However, early work on the RL toolkit cannot apply asynchronous cognitive architecture because it cannot solve the difference between the asynchrony, which the asynchronous cognitive architecture has, and the synchrony, which RL modules have. In this study, we propose an RL environment for robots that can apply the asynchronous cognitive architecture by applying asynchronous systems to RL modules. We prototyped the RL environment named “Re:ROS.” Keywords: Reinforcement learning learning environment
Robotics


Cognitive architecture
Reinforcement

1 Introduction Reinforcement learning (RL), which is a field of machine learning, is a method of learning behavior on a basis of a criterion of maximizing a numerical reward. Unlike supervised learning, RL works without a set of training data prepared. The RL algorithm needs to find the proper actions based on the rewards given by the environment. An agent must not only choose the known action to maximize a reward at the moment, but must also choose the unknown action to find better actions to obtain more rewards. RL has a trial-and-error characteristic. Therefore, it can achieve its goal even though state or action spaces are very large. The state or action spaces for the control of robots are very large, and RL is effective for behavior acquisition in robots. An early work by [5] reported that a four-legged robot successfully learned to walk in practical learning steps. [12] also reported that arm robots can acquire a cooperative behavior through RL. © Springer International Publishing AG 2018 A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6_28

“Re:ROS”: Prototyping of Reinforcement Learning Environment

199

Cognitive architecture is a method used to model human intelligence. Many types of cognitive architecture have already been reported. The asynchronous cognitive architecture, in particular, is expected to be applicable and effective for robots because the real world has asynchrony. In other words, changes in the environment, state, and reward signal do not occur strictly at the same time. Subsumption architecture [2], adaptive control of thought—rational (ACT-R) [1], and cortical capacity-constrained concurrent activation-based production system (4CAPS) [7], which are asynchronous cognitive architectures, also exist. Genghis and Roomba, which were developed by iRobot, are robots that use the subsumption architecture. Therefore, researchers aim to easily apply the asynchronous cognitive architecture to solve RL problems. The toolkit, called Gym-Gazebo [4], is used to develop RL algorithms. It extends Gym [3] using a robot operation system (ROS) [8] and Gazebo [6] and can use robots. Gym provides environments in video games, such as Pong and Go, and a toolkit for developing and comparing RL algorithms. However, Gym-Gazebo has a problem in applying the asynchronous cognitive architecture because of Gym. Gym assumes synchrony, and all actions or policies are executed in synchrony. Gym-Gazebo is scalable to a 3D simulator and robots in the real world, but does not solve the difference between synchrony and asynchrony. Moreover, it cannot apply the asynchronous cognitive architecture. This study proposes an RL environment for robots that can apply the asynchronous cognitive architecture. In this proposed environment named “Re:ROS”, the RL modules are applied with asynchronous distributed systems, such that the proposed RL environment can adapt to the asynchronous cognitive architecture. Re:ROS uses ROS to create asynchronous systems. ROS has nodes that are functions for robots or publishing/subscribing orders. These nodes are distributed and asynchronous, thereby allowing a distributed operation over multiple cores and processors, GPUs, and clusters. Re:ROS applies nodes to the RL state, reward signal, changes of the environment, and action of the agent. Each node publishes and subscribes the necessary information. By doing this, Re:ROS is scalable to the asynchronous cognitive architecture. The remainder of this paper is organized as follows: Sect. 2 describes the RL elements that should be asynchronous; Sect. 3 describes the Re:ROS elements; Sect. 4 presents an example applying the subsumption architecture to an agent, which is a robot called Turtlebot, and a problem of dribbling a soccer ball and shooting it to a goal; and Sect. 5 draws the conclusions.

2 Elements of Reinforcement Learning The most important elements of RL are the agent and the environment. An RL agent interacts with its environment. The following four main sub-elements of RL exist beyond the agent and the environment: a policy, a reward signal, a value function, and, optionally, a model of the environment [10]. A policy determines the learning agent’s behavior at a given time and is the core of an agent in the sense that it alone is sufficient to determine a behavior.

200

S. Ueno et al.

A reward signal defines the goal in an RL problem. At each step, the environment sends a reward signal to the RL agent. A value function specifies what is desirable in a long term. While rewards immediately determine the desirability of the environmental states, values indicate the long-term desirability of states after considering the states that are likely to follow and the rewards available in those states. A model of the environment entails information of behavior of the environment that allows inferences to be made on how the environment will behave.

2.1

RL with Asynchronous Distributed Systems

The functions that the agent and environment should have are listed. Agent should have actions (policies) and value functions. Environment should have changes of the environment, reward signals, states, and optional models of the environment. Each function receives information to operate itself and returns the calculation result. We define the agent herein as robots. The agent has, for example, a high-level action, rule-based action, and the action for exploration. These actions are not simultaneous, but asynchronous operations. The action functions only determine options and are different from the function that actually moves the agent.

2.2

Asynchronous Cognitive Architecture

The functions of the asynchronous cognitive architecture cannot be uniquely determined. For example, Re:ROS using the subsumption architecture has a function for decision making. This function needs information on the lowest-layer action, second-layer action, and so on until the highest-layer action. Figure 1 shows that action1, action2, and action3 in the “Agent” asynchronously send information to the “Subsumption Architecture”. The “Subsumption Architecture”

Fig. 1. Example of Re:ROS elements. Each module has functions that are asynchronously executed and are related with each Re:ROS package. The others are processes needed for the environment, except for the RL modules.

“Re:ROS”: Prototyping of Reinforcement Learning Environment

201

chooses which actions the agent performs. The state, reward signal, and others asynchronously receive information on the action from the subsumption architecture. Each function then sends information to the functions that need it. Table 1. Re:ROS packages and descriptions Package name re_ros

re_rule re_agent re_environment

Description To start three additional packages and determine the training environment’s settings: which environments to use, which agents to learn, and which cognitive architecture to select To define cognitive architecture. In Sect. 4, its main role is to select the action to send to the environment To define an agent for RL. It includes robot management and action (policy) of RL To define the environment and the RL problem. This package includes reward signals, state, and models of the environment

3 Elements of Re:ROS The previous section introduced our idea of RL modules to apply the asynchronous cognitive architecture. We describe herein the implementation of Re: ROS. Re:ROS uses ROS and Gazebo to create asynchronous distributed systems. Gazebo is a physical simulator developed by the Open Source Robotics Foundation (OSRF).

3.1

Re:ROS Packages

Re:ROS performs RL with four packages created by ROS. Figure 1 shows that each package is associated with the previous section. “re_ros” is needed for the integration of all packages. Table 1 lists each package name and description.

4 Example In this example [9], we used Turtlebot as the agent to dribble a soccer ball and shoot it to a goal. The previous researchers used Re:ROS to apply the subsumption architecture as the asynchronous cognitive architecture. The agent has five actions: Forward/Backward (v = 1 m/s), Left/Right (w = ± 2 rad/s), and stop. The state is detected by using Microsoft Kinect (60  60 depth image), as shown in Fig. 3. The policies are Deep Q-Network (DQN) [11], random walk, and suppressor. The reward signals are three types. (1) ball near the goal: 0–1.0, (2) goal: 1.0, and (3) no goal in 10 s: −1.0 (Fig. 2).

202

S. Ueno et al.

Fig. 2. Environment: a soccer field.

Fig. 3. State: 60  60 depth image.

Figure 4 shows the nodes in Re:ROS.

Fig. 4. Nodes in Re:ROS using “rqt_graph” in ROS, which is a ROS package.

The “/accumulator” node subscribes messages from “/agent1/random_walk,” “/ agent1/suppressor,” and “/agent1/dqn_walk.” The “/accumulator” publishes “/gazebo” as the environment. “/agent1/dqn_walk” needs the state and a reward signal, and subscribes to information from “/reward_soccer” and “/gazebo.”

5 Conclusion In this study, we proposed and showed the implementation of Re:ROS, an RL environment applying the cognitive architecture with asynchronous systems. We then developed a simple application with Re:ROS to robots in a simulated world for the first step, as we have not verified operation of robots in the real world. We plan to extend our work on the RL environment by increasing the number of robots, environments, and cognitive architecture as templates.

References 1. Anderson, J.R.: ACT: a simple theory of complex cognition. Am. Psychol. 51(4), 355 (1996) 2. Brooks, R.: A robust layered control system for a mobile robot. IEEE J. Robot. Autom. 2(1), 14–23 (1986)

“Re:ROS”: Prototyping of Reinforcement Learning Environment

203

3. Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., Zaremba, W.: OpenAI Gym. arXiv preprint arXiv:1606.01540 (2016) 4. Zamora, I., Lopez, N.G., Vilches, V.M., Cordero, A.H.: Extending the OpenAI Gym for robotics: a toolkit for reinforcement learning using ROS and Gazebo. arXiv preprint arXiv: 1608.05742 (2016) 5. Kimura, H., Yamashita, T., Kobayashi, S.: Reinforcement learning of walking behavior for a four-legged robot. IEEJ Trans. Electron. Inform. Syst. 122(3), 330–337 (2002) 6. Koenig, N., Howard, A.: Gazebo-3D multiple robot simulator with dynamics (2006) 7. Just, M.A., Varma, S.: The organization of thinking: what functional brain imaging reveals about the neuroarchitecture of complex cognition. Cogn. Affect. Behav. Neurosci. 7(3), 153– 191 (2007) 8. Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Berger, E., Wheeler, R., Ng, A.Y.: ROS: an open-source robot operating system. In: ICRA Workshop on Open Source Software, vol. 3(3.2), p. 5 (2009) 9. Osawa, M., Ashihara, Y., Shimada, D., Kurihara, S., Imai, M.: Arbitration of multiple learner and application of cognitive architecture using accumulator utilizing prefrontal area. In: 4th SIG-AGI (2016) 10. Richard, S., Sutton, G., Barto, A.: Reinforcement Learning: An Introduction, 1(1). MIT Press, Cambridge (1998) 11. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M.: Playing Atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013) 12. Yamada, K., Ohkura, K., Ueda, K.: Cooperative behavior acquisition of autonomous arm robots through reinforcement learning. Trans. Soc. Instr. Control Eng. 39(3), 266–275 (2003)

Adaptive Control of Modular Robots Alexander V. Demin1 and Evgenii E. Vityaev2(&) 1

Institute of Informatics Systems SB RAS, Lavrentieva 6, Novosibirsk, Russia [email protected] 2 Sobolev Institute of Mathematics, pr. Koptyuga 4, Novosibirsk, Russia [email protected]

Abstract. This paper proposes a learning control system of modular systems with a large number of degrees of freedom based on joint learning of modules, starting with finding the common control rules for all modules and finishing with their subsequent specification in accordance with the ideas of the semantic probabilistic inference. With an interactive 3D simulator, a number of successful experiments were carried out to train three robot models: snake-like robot, multiped robot and trunk-like robot. Pilot studies have shown that the approach proposed is quite effective and can be used to control the complex modular systems with many degrees of freedom. Keywords: Control system


Patterns detection
Knowledge elicitation

1 Introduction. Modular Robots The task of developing control systems for modular robots faces serious difficulties resulting from the hyper-redundancy of such systems. The existing approaches based on the reinforcement learning techniques are not effective for systems with a big number of degrees of freedom. The purpose of the paper is to develop the control systems for hyper-redundant modular systems which enable learning and adaptation of such systems in real time. To meet this challenge, it is proposed to use logical-and-probabilistic methods of knowledge elicitation adapted for control purposes. As result, this work proposes a new approach to creating the learning control systems for modular robots, based on the use of the modules functional similarity and the logical-and-probabilistic algorithm of the guided search of rules. The approach is based on the joint learning of the control modules, starting with finding the common control rules for all modules and finishing with their subsequent specification in accordance with the ideas of the probabilistic inference. The main advantage of the proposed approach is the high learning rate and the teach-and-learn capability in real-time mode based only on the experience of the system’s interaction with the

The first author financially supported by the RFBR Grants #14-07-00386, 15-07-03410. The second author financially supported by the Russian Science Foundation grant # 17-11-01176 in part concerning the learning procedure. © Springer International Publishing AG 2018 A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6_29

Adaptive Control of Modular Robots

205

environment. The paper gives the examples of creation and learning of control systems for three virtual robot models: snake-like robot, multiped robot and elephant’s trunk robot. The experiments carried out have confirmed the learning high rate and quality of control. Practically speaking, the results of the experiments show that the proposed approach to adaptive control can be used in the tasks of control systems development by intelligent agents - software or robotic systems, including hyper-redundant, which require learning capability and adaptation to changing circumstances. Currently, the new area in robotics under the name “modular robots” [1, 2] is developing actively. The basic idea of this approach is the robotic engineering from a variety of the simple, single-type modules that themselves have limited locomotion, but by connecting with each other can form complex mechanical systems with a big number of degrees of freedom. Such robots have a number of interesting possibilities exceeding the abilities of traditional robots. Firstly, it is the ability to create different designs from the same modules, that enables solving different tasks using the same set of modules. It is much more cheaper and more convenient than constructing a lot of specialized robots for each specific task. Moreover, it is possible to create robots-transformers that independently change their design, depending on the environmental objectives and conditions. Second, the modular structure and availability of a large number of degrees of freedom (hyper-redundancy) allows one to create fault-tolerant robot models. Such robot’s individual modules’ failure is not critical to the operation of the entire system, and causes minimal performance degradation. Thirdly, the production and use of such robots is economically advantageous since the modules of the same type are simpler and cheaper to manufacture and repair. However, the broad capabilities of modular robots associated with hyper-redundancy also have the downside – significant complexity of control. In particular, the relevant task is creating a locomotion control system for a predefined robot configuration. Whereas for usual robots the traditional approach to creating control systems is the manual programming by a man-developer, for modular robots this approach is inefficient. Because of the large number of degrees of freedom it is extremely difficult for a developer to foresee and to program all the possible forms of movement and the situations where they need to be applied, and particularly to take into account the ability of adaptation in the event of individual modules’ failure or a sudden environment change. Therefore, it becomes relevant to develop the ways of control system automatic generation based on different learning models. Such popular methods as reinforcement learning, for generation of control systems of hyper-redundant robots is difficult to use due to the large number of degrees of freedom of such robots. In this paper, we propose a learning control system using the logic-probabilistic approach to knowledge elicitation for the control rules generation from the system’s environment interaction experience [3–8, 10, 11]. The specificity of the suggested approach is that the system is primarily trying to locate the control rules, common to all modules, and only then – the rules that are specific to each individual module. The effectiveness of the approach is suggested to evaluate by the example of teaching the typical representatives of simple hyper-redundant modular robots: snake-like robot, multiped robot and trunk-like robot.

206

A.V. Demin and E.E. Vityaev

2 Simulator For conducting the experiments with the proposed control model an interactive 3D-simulator with graphical user interface was developed. The main purpose of the program is to conduct experiments on robot control in the environment pushed closer to the real world. The program has virtual environment visualization capabilities as well as the capability of recording experiments in a video file. As a physics engine, the simulator uses the Open Dynamic Library (ODE) [9], which allows you to simulate the dynamics of solid bodies with different types of joints. The advantage of this library is the speed, high stability of integration, as well as built-in collision detection. Using this simulator, three robot models were built: snake-like, multiped and trunk-like robot. Snake-like robot model is presented in the simulator as a set of six identical rectangular blocks (“vertebrae”) combined by universal joints (Fig. 1). All joints are identical and have two angular engines (“muscles”) that ensure the rotation of the joints in the vertical and horizontal planes. The proposed design, despite its simplicity, provides sufficient flexibility of the model and provides body’s position typical for the biological snakes.

Fig. 1. Snake-like robot model, multiped robot model, trunk-like robot model.

The second model, a multiped robot, is presented as a structure of six identical modules connected to each other with rigid joints (Fig. 1). Each module has a pair of L-shaped legs on the right and left side. So, the robot has totally twelve legs-limbs. Each leg is connected to the module by a universal joint with two angular motors allowing a joint to rotate the leg in the horizontal and vertical planes. In general, the robot’s design resembles its biological type of millipede, and allows you to implement specific ways of movement. Model of trunk-like robot is represented in the form of a multi-segmental “trunk”, connected by a versatile joint with a massive fixed platform in the form of cube. The trunk itself is represented as a sequence of five identical rectangular blocks associated with versatile joints provided with angular motor. The dimensions of the blocks and the position of the joints have been chosen in such a way as to ensure the system’s sufficient flexibility and attainability to carry out the experiments.

3 Control System To create the modular robots control system, it is proposed to use neural networks consisting of trainable logic neurons, each of which controls a separate robot module.

Adaptive Control of Modular Robots

207

Logical neurons operate in a discrete time t ¼ 0; 1; 2; . . .. Each neuron contains a set of inputs input1 ; . . .; inputk , assuming valid values and one output, assuming the values from a predefined set fy1 ; . . .; ym g. At any time point t, the neuron inputs are supplied with information by assigning real values of the inputs input1 ¼ x1 ; . . .; inputk ¼ xk ,x1 ; . . .; xk 2 R. The result of the neuron’s work is the output signal output ¼ y, y 2 fy1 ; . . .; ym g assuming one of the possible values fy1 ; . . .; ym g. After all the neurons of the network have completed the work, the reward comes from the external environment. The award function is set depending on the ultimate goal and is used to evaluate the control quality. Control system task is detecting such patterns of neuron functioning that would ensure getting the maximal reward. Variety of the patterns that define the work of neurons are suggested to search in the form of logical patterns with estimates 8i ðPðiÞ; X1 ðiÞ; . . .; Xm ðiÞ; YðiÞ ! rÞ, i ¼ 1; ::; n – the variable on the neurons, where: Xj ðiÞ 2 X – predicates from the specified set of input predicates X that describe the inputs j of the neuron Ni ði ¼ 1; ::; nÞ; Yj ðiÞ 2 Y – predicates from a given set of output predicates Y describing the output of neurons Ni ði ¼ 1; ::; nÞ and looking as Yj ðiÞ ¼ ðoutputðiÞ ¼ yr Þ, yr – some constants from the range of output signals; PðiÞ 2 P – predicates from a set of predicates P look as ði ¼ jÞ, j ¼ 1; ::; n that narrow the scope of rules to specific neurons; r – reward whose maximization is the task of a neuron. These patterns predict that if neuron gets input signals, Ni ; i ¼ 1; ::; n that satisfy the input predicates X1 ðiÞ; . . .; Xm ðiÞ of the rule premises, and the neuron sends output signal specified in the output predicate YðiÞ, then reward mathematical expectation will be equal to a certain value r. Additionally, we will note that if a neuron Nj has an input specific only to that neuron, it is assumed that the predicate XðiÞ describing this input will take the value “0” for all i 6¼ j, i.e. for all other neurons. Similarly, if the output of any neuron Nj can take any value y specific only to that neuron, the corresponding output predicate ðoutputðiÞ ¼ yÞ will also take the value of “0” for all i 6¼ j. We will now explain the need for introducing a set of predicates P. Should a rule does not contain predicates from P, it will look like 8iðX1 ðiÞ; . . .; Xm ðiÞ; YðiÞ ! rÞ and will describe patterns common to all neurons Ni , i ¼ 1; ::; n. Adding a predicate to premises of the rule from P automatically narrows the scope of the rule application to a specific neuron. Thus, the rules containing predicates from P, describe patterns specific to particular neurons. Learning the rules based on the algorithm of semantic probabilistic inference described in the papers [6, 12]. The behavior of the neural network as a whole is as follows: In each operating cycle of network, the incoming signals are received on the neuron inputs. After that, the decision-making procedure is started successively for each neuron, in course of which from the set of rules describing the neurons behavior, those ones shall be selected that can be applied to the current neuron on the current input signals. A single rule is then selected among the selected rules that forecasts the maximum value of the mathematical expectation of the reward r. Further, the neuron output receives the output signal output ¼ y specified in the rule. At the initial stage of the network functioning, when the set of rules describing the neurons behavior is still empty, or when there are

208

A.V. Demin and E.E. Vityaev

no rules applicable to the current set of incoming signals, the neuron’s output is determined randomly. After all the neurons generate their output signals, the external environment provides a reward and learning in the process of which the new rules are searched and current rules of operation are adjusted in accordance with the suggested patterns of algorithms search.

4 Snake-like Robot Locomotion Control System The purpose of this experiment was the teaching the simplest model of the snake-like robot locomotion forward (Fig. 1). In previous studies [10, 11] we proposed model of the neural control circuit of the robot locomotion of nematode C. Elegans, which proved to be highly effective in experiments on undulating way of locomotion learning process. The diagram of this circuit assumed that the head of nematode acted as a source of oscillations, based only on feedback from stretch receptor. Then the signal is distributed over the nematode’s body with a certain time delay, thereby enabling the distinctive undulation. Since the design of the snake-like robot has many commonalities with the nematode model, it was decided in this paper to use a similar neural circuit scheme to control the robot locomotion. Finally a neural circuit consisting of five neurons (Fig. 2) was selected. Each neuron Ni , i ¼ 1; . . .; 5 controls one joint of the robot body, sending activation signals

Fig. 2. Diagram of the neural control circuit of the robot locomotion. Sequence of the snake-like robot movements when moving forward.

Adaptive Control of Modular Robots

209

to the angular motors located in that joint. The head neuron N1 receives input information about the bending angles between the head and the subsequent segment. In addition to this, the neuron receives a signal from its own output with a time delay Dt via feedbacks. The remaining neurons Ni , i ¼ 2; . . .; 5 receive only the signal from the previous neuron’s Ni1 output with a time delay of Dt. The set of input and output predicates for neurons is specified by quantizing the range of possible values of the neuron corresponding inputs and outputs. The reward for the entire neural circuit of locomotion control is determined depends on the speed that the robot will develop on the time span of Dt: the higher speed - the greater reward. A number of successful experiments were conducted using the 3D-simulator on learning of the proposed model for ways of locomotion. As the results of the experiments have shown, the control system has been able to consistently learn an effective way of locomotion forward based on the undulating body movement in the horizontal plane. This way of movement is the most common among the biological snakes and is also typical for some other animals, for example, nematodes. Figure 2 shows the best sequence of motions found by the system during learning when moving forward.

5 Multiped Robot Locomotion Control System The purpose of this experiment was to teach the multiped robot model (Fig. 1) the way of movement forward. For multiped robot control system, the neural circuit consisting of six neurons was selected - one neuron for the robot each module (Fig. 3) Each neuron Ni , i ¼ 1; . . .; 6 controls the movement of its module’s left and right leg by sending out signals to the appropriate angular motors rotating limbs in the joint. To make the task a little easier, the right and left legs of the robot were synchronized so that the movement of one leg always occurs in counterphase to the other, for example, the forward movement of the left foot is always accompanied by the backward movement of the right foot. So the neuron is essentially enough to control the movement of only one leg, because the second leg will repeat the same movements only in counterphase. Neuron of the first module N1 gets the incoming information about the first module’s leg position. The remaining neurons Ni , i ¼ 2; . . .; 6 receive the incoming information about the legs position of the previous module. Information about the position of the legs is set by a pair of angles of the limb bending in joint in the vertical and horizontal planes. The set of input and output predicates for neurons is specified by quantizing the range of possible values of the neuron corresponding inputs and outputs. The reward for the entire locomotion control system is determined depends on the speed that the robot will develop on the time span Dt of: the higher speed – the greater reward. A number of successful experiments were conducted using the 3D-simulator on learning locomotion of the multiped robot model. The results of the experiments showed that the control system successfully discovers cooperative movements of limbs ensuring effective movement forward. Figure 3 shows an example of the best sequence of motions found during the learning.

210

A.V. Demin and E.E. Vityaev

Fig. 3. Diagram of the neural control circuit of the multiped robot locomotion. Sequence of the multiped robot movements when moving forward.

6 Trunk-like Robot Locomotion Control System In this experiment, the robot task was to grasp a target that appears in a random position within the trunk’s range (Fig. 4). A certain sphere with a radius equal to the length of the trunk’s one segment acts as a target. The target is considered to be grasped if the end of the trunk is found to be inside the sphere. After grasping, the sphere-target disappears and appears in a new random position. Thus, the experiment may continue uninterrupted for unlimited period of time. The purpose of the control system is the detection of such control rules of trunk movement that would ensure the target grasping in any position that is accessible for the manipulator. To accomplish this task, we have selected the control scheme of five neurons Ni , i ¼ 1; . . .; 5; each of which controls a single segment of the trunk (Fig. 4). Neurons serve as triggers for angular motors, thereby causing the trunk bend in the appropriate joints. Since in this task the target can only be at one level, all motions are limited to a

Adaptive Control of Modular Robots

211

Fig. 4. Diagram of trunk-like robot neuron circuit. Example of the manipulator path when grasping a target.

horizontal plane simplifying the task. The first neuron N1 receives incoming binary information on which side of the trunk the target is: on the right or left. The rest neurons Ni , i ¼ 2; . . .; 5 receive the following incoming signals: (1) information about bend angle between the controlled and previous segment (2) signal from the previous neuron’s output Ni1 at the previous timepoint; (3) binary information about the position of the target with respect to the end of the trunk and its attachment point – the result of comparing the distances from attachment to the target and from attachment to the manipulator end. Thus, the manipulator is actually “blind” – it does not see the exact position of the target, but only “feels” it: right-left and closer-farther. The reward for control system is calculated based on the fact of target grasp by the manipulator, as follows: Suppose the target appeared in a new position at the moment of time t0 , and the manipulator grasped it at the moment of time t1 . Then all actions from the moment of time t0 to the moment of time t1 will receive the reward in the amount of r ¼ 1=ð1 þ ðt1  tÞÞ where t the moment of time for which the reward is

212

A.V. Demin and E.E. Vityaev

calculated. With a 3D simulator, the experiments were carried out on learning the manipulator to grasp the targets at any position within the manipulator’s grasp. The results of the experiments showed, the control system is able to detect effective rules ensuring 100% grasp of targets in the specified zone. Figure 4 shows examples of the paths for already taught trunk when grasping the target. The examples from second to fourth show that the control system has learned to make special preparatory movements to be ready for grasp.

References 1. Yim, M., Duff D., Roufas, K.: Modular reconfigurable robots, an approach to urban search and rescue. In: 1st International Workshop on Human Welfare Robotics Systems (HWRS 2000), pp. 19–20 (2000) 2. Stoy, K., Brandt, D., Christensen, D.J.: Self-Reconfigurable robots: an introduction. In: Intelligent Robotics and Autonomous Agents Series. MIT Press (2010) 3. Demin, A.: Logical model of the adaptive control system based on functional systems theory. In: Young Scientist USA. Applied Science, pp. 113–118. Auburn, Washington (2014) 4. Demin, A.: The model of the adaptive control system and its application for a virtual robot locomotion control. Molodoy Uchyony (Young scientist) 11(46), 114–119 (2012) 5. Demin, A.: Adaptive locomotion control system for modular robots. Int. J. Autom. Control Intell. Syst. 1(4), 92–96 (2015) 6. Demin, A., Vityaev, E.: Logical model of the adaptive control system. Neuroinformatics 3 (1), 79–107 (2008) 7. Demin, A.V.: Learning locomotion control system for 3D multiped robot model. Molodoy Uchyony (Young scientist) 19(99), 74–78 (2015) 8. Demin, A.: Teaching of locomotion ways of the snake-like robot virtual model. Molodoy Uchyony (Young scientist) 19(78), 147–150 (2014) 9. Smith, R.: Open Dynamics engine. http://ode.org/ 10. Demin, A., Vityaev, E.: Learning in a virtual model of the C. elegans nematode for locomotion and chemotaxis. Biologically Inspired Cogn. Architectures 7, 9–14 (2014) 11. Demin, A.: Learning control model of chemotaxis for C. Elegans nematode. Neuroinformatics 7(1), 29–41 (2013) 12. Evgenii, V.: The logic of prediction. In: Proceedings of the 9th Asian Logic Conference, pp. 263–276. World Scientific Publishers (2006)

Model of Heterogeneous Interactions Between Complex Agents. From a Neural to a Social Network Liudmila Zhilyakova ✉ (

)

V.A. Trapeznikov Institute of Control Sciences of Russian Academy of Sciences, 65 Profsoyuznaya street, Moscow 117997, Russia [email protected] Abstract. We describe a heterogeneous neural network where neurons interact by means of various neurotransmitters using the common extracellular space. Every neuron is sensitive to a subset of neurotransmitters and, when excited, secretes its specific neurotransmitter. This feature enables establishing the selec‐ tive connections between neurons according to sets of their receptors and to their outputs. We use a simplification of this formalism as a basis for modeling inter‐ actions between agents in a social network, where the two opposite types of activity are spreading. Agents have beliefs of different strength and activation thresholds of different heights (which correspond to neuronal excitation thresh‐ olds) and can be more or less sensitive to an external influence (which corresponds to weights of neuron receptors). The main properties of the agents and the prin‐ ciples of activity spreading are defined. The classification of agents according to their parameters is provided. Keywords: Discrete dynamics · Heterogeneous neural network · Social network · Activity in networks

1

Introduction

We introduce a threshold network model with several kinds of activity. The nodes of the network are heterogeneous artificial neurons, i.e. threshold elements (some of them have endogenous activity) communicating by means of a set of neurotransmitters. The network operates in discrete time. At each time step, active neurons secret a certain amount of their specific transmitter. At the next step, the released transmitters become available to all other neurons with appropriate receptors. Every neuron has its own set of receptors. The activity of neurons at time step t is determined by input signals and its internal state. This state is characterized by a single generalized parameter, which in some sense corresponds to the membrane potential and can be considered as the read‐ iness for activation. Numerous studies are devoted to different dynamic network models. However, despite the diversity of approaches and the wide range of simulated tasks, as a rule, these models consider the propagation of one type of activity. A wide class of such models is described by random walks or diffusion on graphs (see, for example, surveys [1, 2]). A completely different kind of dynamics is described in the integer threshold model called © Springer International Publishing AG 2018 A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6_30

214

L. Zhilyakova

chip-firing game. This model on both directed and undirected graphs is well studied and described analytically [2–4]. Such threshold models, in particular, describe phenomena of self-organized criticality “avalanche” or “sand- (rice-) pile” [5, 6]. Basically, these models consider the series of consecutive firing, when the vertices fire in turn in random order. It was proved that if the final configuration exists, it does not depend on the firing order. Therefore, such models are often called “abelian sandpiles” [6]. Social networks research is a rapidly developing area. Numerous researchers have obtained many interesting results on the general structural regularities of social networks (small-world property, power-law degree distribution, modularity, rich club phenom‐ enon, assortativity, etc.), and on various dynamic processes occurring in these networks [7]. Various mathematical models of activity spreading in social networks are developed: threshold and cascade models [8–10], models of epidemic spreading [11], Markovian models [12], and a number of others. Based on these models, optimization problems are solved, in particular, the determination of the initial set of active agents that ensure the maximum propagation of activity over the network [8, 13]. Models of control in social networks were proposed and developed in a number of papers (see e.g. [14]). In this article, we use a model of multitransmitter neuronal interaction [15–17] as a basis for simulating the spreading of two types of opposite activities in a social network. Some approaches to this problem were described in [18] and then developed in [19].

2

Model Description

We consider a heterogeneous neural network S = < N, E, C > , where N = {1, …, n} is the set of neurons, E is the set of edges (i.e. connections between the neurons), and C = {c1, …, cm} is the set of transmitters. The network operates in discrete time T. The rules of neuron interactions are as follows. If the neuron i emits the transmitter ck, and the neuron j has a receptor to this transmitter, there exists an oriented edge between the neurons. We assume that each transmitter has its own color. Then the set of edges E is separated to colored subsets: E = Ec1 ∪ … ∪ Ec-m. Weights of edges rijc-m correspond to the sensitivity of receptors of the receiving neurons. We will make no distinction between synaptic and extrasynaptic connections. When activated, every neuron transmits a specific neurotransmitter over all its outgoing connections. We assume that a neuron can always produce a sufficient amount of transmitter, and each outgoing edge receives an amount of transmitter equal to its weight rijc-m. Another characteristic of a neuron is the membrane potential (MP) Ui(t). In the model, it is believed that the higher the membrane potential of the neuron, the easier it becomes excited, and vice versa. Neuron i activates if its membrane potential Ui(t) is above a certain threshold value Pi. The membrane potential of neurons at time t is calculated by the formula

Ui (t) = 𝛼 ⋅ Ui (t − 1) + 𝛽 ⋅

m n ∑ ∑ j=1 k=1

rjick yj (t − 1),

(1)

Model of Heterogeneous Interactions Between Complex Agents

215

where α ∈ (0, 1] is a discount factor and β is a scale factor. The activity function yi(t) of neuron i at time t is determined by the formula { yi (t) =

3

1, if Ui (t) ≥ P; 0 otherwise.

(2)

Model of a Social Network with Five Types of Agents

Using a specified formalism we construct a model of social network S = , where N = {1, …, n} is the set of agents, C = {c1, …, cm} is the set of types of their activities, and E = Ec1 ∪ … ∪ Ec-m is the set of colored weighted edges. There may be several edges of different colors and weights between a pair of nodes i, j; their weights indicate the degree of influence. The only membrane potential, in this case, is replaced by m parameters Uick(t) corresponding to the agent’s readiness to be activated by one of the m types. Accordingly, each agent i has m thresholds Pick. Here, we consider a reduced case with m = 2 and assume that there are two activities in a network (called 1 and 2). In this case, each agent will have two parameters Ui1,2(t) – one for each activity type, and two threshold values Pi1,2. Types 1 and 2 denote antagonistic activities (revolutionary/reactionary, constructive/ destructive, sharp/blunt end of the egg should be cracked etc.). Agents have beliefs about the types of activity. These beliefs are characterized by values si1,2∈{–1, + 1}. If si1,2 = +1 (or si1,2 = –1), then the agent i considers corresponding activity as positive (or nega‐ tive, accordingly). At each time step t, agents receive from their neighborhood the activity of one or two types. These signals effect the potentials of agents Ui1,2(t) according to a formula similar to (1):

Ui1,2 (t) = 𝛼 ⋅ Ui1,2 (t − 1) + 𝛽

n ∑ j=1

rji1,2 yj1,2 (t − 1).

The agent activates if the value of either Ui1(t) or Ui2(t) exceeds the threshold. However, the activation rule is slightly different from (2) due to the beliefs of agents. This difference will be described below. 3.1 States of Agents and Their Mutual Influence The agent’s beliefs about the two kinds of activity can be combined in different ways. We consider three cases: 1. si1 = + 1, si1 = –1; 2. si1 = –1, si1 = + 1; 3. si1 = + 1, si1 = + 1. In cases 1 and 2, agents have moral certainty about types of activity, one of which is regarded as positive, the other as negative. The activity with the “+” sign will be called

216

L. Zhilyakova

own for the agent. Whichever threshold (Pi1 or Pi2) is exceeded, the agents activate only in their own type. Case 3 describes the situation where agents do not have a clear position for these types of activity. However, the agent i can activate if one of thresholds Pi1 or Pi2 is exceeded. In this case, agent i activates by the type of the threshold exceeded. This means that the agent succumbs to the charm of the crowd and acts with the majority. If two thresholds were exceeded at the same time, the agent selects the type of activity with stronger influence as own; in the case when Pi1 = Pi2 and Ui1(t) = Ui2(t), agent i chooses an activity type with equal probability. At the next time step after activation both potentials are reset: Ui1,2(t + 1) = 0. The values of thresholds together with the values of si1,2 have an obvious interpre‐ tation. Let some agent i has the following set of parameters: si1 = + 1, si2 = –1, Pi1 = 0.2, and Pi2 = 0.99. This means that the agent has some beliefs, and the first activity is his own. Absolute threshold values show the agent’s tolerance to the activity of his envi‐ ronment. The higher the values of Pi1 and Pi2, the more stable the agent is to an external activity. Thus, agent i easily activates in a friendly environment and almost never acti‐ vates in an environment with beliefs alien to him. On the contrary, the set of parameters si1 = + 1, si2 = –1, Pi1 = 0.99, and Pi2 = 0.2 characterizes the agent with a strong sense of contradiction. If the “own” type of activity prevails, he is silent, because everything is good without his participation. But if the activity is alien, he activates to resist the majority. If both thresholds of the agent are low, this means that he is easily activated by any external activity. High thresholds characterize cautious agents. 3.2 Types of Agents Each agent belongs to one of the five disjoint classes depending on the values of the parameters Pi1 and Pi2: V = VR1 ∪ VR2 ∪ VA∪ VC ∪ VP, where VR1 – revolutionary agents; VR2 – reactionary agents; VA – precautionary agents; VC – conformist agents; VP – passive agents. Revolutionaries and reactionaries. The numbers of agents in the classes VR1 and VR2 are relatively small. These agents have almost zero thresholds. For revolutionary agents we have si1 = + 1, si2 = –1, Pi1 = 0, and Pi2 = ε, while for reactionary agents si1 = –1, si2 = + 1, Pi1 = ε, and Pi2 = ε. These agents have no memory and become active as soon as an activity of any type comes to them. Revolutionaries differ from reaction‐ aries only by their ability to start the new activity. Precautionary agents, as well as agents of the first two types, have their own beliefs, that is, si1 ·si2 < 0. However, they tend to act cautiously, and their values of Pi1 and Pi2 vary in the intervals (0, 1), without taking extreme values. Conformist agents do not give preference to the kind of activity: si1 = si2 = + 1. At small values of Pi1 and Pi2 we have “hooligan agents” supporting any activity for the sake of the activity itself. At Pi1 and Pi2, close to 1, the agent activates, yielding to the influence of a large crowd.

Model of Heterogeneous Interactions Between Complex Agents

217

Passive agents always have Pi1, Pi2 > 1, and no activity of neighbors is able to draw them into activity.

4

Conclusion

The paper presents a model of a heterogeneous neural network where the neurons of different types interact by means of different neurotransmitters. The basic principles of this model are applied to describe the interactions of agents in social networks. In our opinion, the basic concepts can be applied to many other areas with common properties: multiproduct traffic, an internal structure of nodes, the threshold switching of activity, and variability in behavior. Acknowledgments. This work was supported by the Russian Science Foundation, project no. 15-07-02488.

References 1. Blanchard, P., Volchenkov, D.: Random Walks and Diffusions on Graphs and Databases: An Introduction. Springer Series in Synergetics. Springer, Heidelberg (2011) 2. Lovasz, L., Winkler, P.: Mixing of random walks and other diffusions on a graph. In: Rowlinson, P. (ed.) Surveys in Combinatorics. London Mathematical Society Lecture Note Series, vol. 218. pp. 119–154. Cambridge University Press (1995) 3. Biggs, N.L.: Chip-firing and the critical group of a graph. J. Algebr. Comb. 9, 25–45 (1999). Kluwer Academic Publishers. Netherlands 1999 4. Bjorner, A., Lovasz, L.: Chip-firing games on directed graphs. J. Algebr. Comb. 1, 305–328 (1992) 5. Bak, P.: How Nature Works: The Science of Self-Organized Criticality. Copernicus, New York (1996) 6. Dhar, D.: The abelian sandpile and related models. Physica A Stat. Mech. Appl. 263(1–4), 4–25 (1999) 7. Newman, M.E.J.: The structure and function of complex networks. SIAM Rev. 45(2), 167– 256 (2003) 8. Kempe, D., Kleinberg, J., Tardos, E.: Maximizing the spread of influence through a social network. In: Proceedings of the 9-th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 137–146 (2003) 9. Watts, D.J.: A simple model of global cascade on random networks. Proc. Natl. Acad. Sci. U.S.A. 99(9), 5766–5771 (2002) 10. Goldenberg, J., Libai, B., Muller, E.: Talk of the network: a complex systems look at the underlying process of word-of-mouth. Market. Lett. 12(2), 11–34 (2001) 11. Pastor-Satorras, R., Vespignani, A.: Epidemic spreading in scale-free networks. Phys. Rev. Lett. 86(14), 3200–3203 (2001) 12. DeGroot, M.H.: Reaching a consensus. J. Amer. Stat. Assoc. 69(345), 118–121 (1974) 13. Goyal, A., Bonchi, F., Lakshmanan, L.V.S., Venkatasubramanian, S.: On minimizing budget and time in influence propagation over social networks. Soc. Netw. Anal. Min. 2(1), 179–192 (2012)

218

L. Zhilyakova

14. Gubanov, D.A., Chkhartishvili, A.G.: Models of information opinion and trust control of social network members. In: Proceedings of the 18th IFAC World Congress, 2011 World Congress, pp. 1991–1996. International Federation of Automatic Control (IFAC), Milano (2011) 15. Bargmann, C.I.: Beyond the connectome: how neuromodulators shape neural circuits. BioEssays 34(6), 458–465 (2012) 16. Dyakonova, V.Ye.: Neurotransmitter mechanisms of context-dependent behavior. Zhurn. vyssh. nerv. deyat. 62(6), 1–17 (2012) 17. Sakharov, D.A.: Biological substrate for the generation of behavioral acts. Zhurn. obshch. biologii. 73(5), 334–348 (2012). (in Russian) 18. Zhilyakova, L.Y.: Network model of spreading of several activity types among complex agents and its applications. Ontol. Des. 5(3(17)), 278–296 (2015). (in Russian) 19. Gubanov, D.A., Zhilyakova, L.Y.: On a threshold model of the activity spreading in a social network. In: Proceedings of 8-th National Multi-Conference on Control Problems, vol. 1, pp. 51–53. SFedU publishing, Rostov-na-Donu (2015). (in Russian)

Methods of Artificial Intelligence in Cybersecurity

Stochastic Data Transformation Boxes for Information Security Applications Ahmad Albatsha and Michael A. Ivanov(&) National Research Nuclear University “MEPhI” (Moscow Engineering Physics Institute), Kashirskoe highway 31, 115409 Moscow, Russian Federation [email protected]

Abstract. Stochastic methods are commonly referred to as methods which are directly or indirectly based on using a pseudo-random number generator (PRNG). In some cases, stochastic methods are the only possible mechanism of protecting information from an active adversary. In this paper we examine a construction of R-boxes, which are a generalization of S-boxes, classical structural elements of cryptographic primitives of hashing, block and stream encryption. R-boxes are in fact stochastic adders, i.e. adders with an unpredictable operating result, which depends on the key table H. A distinguishing feature of R-boxes is their efficient software and hardware implementation. Keywords: Stochastic transformation
R-box Register (RFSR)
Non-linear M-sequence




Random Feedback Shift

1 Introduction An analysis of information security threats and development of computer technologies allows to arrive at a firm conclusion that the role of stochastic methods of information security is constantly increasing. Stochastic methods are commonly referred to as methods which are directly or indirectly based on using pseudo-random number generators (PRNG). As an example of a universal stochastic method of information security we can mention the method of introducing unpredictability in the operation of means and objects of security. By using PRNG all tasks of information security can be solved successfully. Thus, in some cases stochastic methods are the only possible mechanism of protecting information from an active adversary. A particular case of stochastic methods are cryptographic methods of information security. The term “stochastic” in relation to information security applications was, apparently, first used by S.A. Osmolovsky in constructing codes which detect and correct mistakes arising when transferring data through communication channels (Osmolovsky 1991, 2003). The stochastic codes suggested by him offer unique properties, two of which are worth mentioning. They are: the ability to provide a predefined probability of correct information reception and the possibility to solve, beside the task of error detection and correction during data transmission, two other important tasks of information security – providing confidentiality and integrity of the information transferred.

© Springer International Publishing AG 2018 A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6_31

222

A. Albatsha and M.A. Ivanov

2 Stochastic Transformation Blocks. R-Boxes The reference (Asoskov et al. 2003) suggests a block of stochastic transformation (Rbox), which can be effectively applied when solving various tasks related to information security. The construction of one of the possibly simplest variants of stochastic transformation block R, which was first proposed for solving the task of error correcting coding in operation (Osmolovsky 1991), and its graphical representation are shown in Fig. 1. The key information of n-bit R-box is filling the table H ¼ fH ðmÞg; m ¼ 0; . . .; ð2n  1Þ; of dimension n  2n , which contains elements GF(2n), mixed in a random fashion, i.e. H ðmÞ 2 GF ð2n Þ. In other words, the table H contains consecutive states of n-bit PRNG. The result RH ðA; BÞ of the transformation of the input n-bit binary set A depends on how the table H is filled as well as on the transformation parameter B, specifying displacement in the table with respect to the cell holding the value A, in the following way RH ðA; BÞ ¼ H ððmA þ BÞ mod 2n Þ; where mA is the address of the cell in the table H, containing the code A, i.e. H ðmA Þ ¼ A. Otherwise speaking, the result of the operation of R-box consists in reading the cell content in the table H, repeatedly displaced at B positions toward major addresses with respect to the cell containing the code A. To ensure the independence of transformation time from the source data we introduce into R-box the table H−1 = {H−1(j)} of dimension n  2n , where 8j = 0, 1, …, (2n – 1) H−1(j) = mj. To put it differently, the cell having the address j in the array H−1 holds the address of the cell of the array H containing the code j. Below are the facts which deserve attention: H -1 A B

n

H n

n

n

n

R(A, B)

A

R(A, B)

R

B а

б

Fig. 1. The behavior of R- box (a) and its graphical representation (b). ⊞ – modulo 2n adder.

– when H−1 = {0, 1, …, (2n – 1)} and B = 0, we get a classical S-box (substitution box) with the substitution table H; – when writing its own address in each cell of the arrays H and H−1 we get a classical 2n adder, which means that the R-box can be rightfully called a stochastic adder, i.e. adder with an unpredictable operating result, which depends on how the key table H is filled. R-box has an easy software implementation. Below follows an example of implementing an 8-bit box of stochastic transformation in Assembler (a system of commands x86, standard Intel notation) (Fig. 2). R-boxes can be applied for implementing stream encryption. In this case, a Plaintext is fed to input A, a Keystream is fed to input B, and a Ciphertext is removed from output RH ðA,BÞ. We should bear in mind that it is essential to apply the transformation R1 (reverse R) at the receiver.

Stochastic Data Transformation Boxes

223

;=================================================================== ;==== RBox – the procedure of stochastic transformation.============== ;=================================================================== ;==== When called: AL – input byte, AH – transformation parameter, === ;==== DS – segment address of array H–1&H, ========================== ;==== BX – relative address of array H–1&H(fig.2), ================== ;==== CX – dimension of arrays Addr and H (HSize). ================= ;==== Upon return: AL – output byte. =============================== ;=================================================================== RBox PROC push bx xlat ; Reading from list H–1 add al, ah ; AL – output byte address in array add bx, cx ; BX – relative address of array xlat ; Reading from list pop bx ret RBox ENDP ;=================================================================== DS: BX

DS: [BX + HSize]

H–1&H H–1[0] H–1[1] H–1[2] ... H[0] H[1] H[2] ...

H–1

H

Fig. 2. Array H−1&H

The second area of application of R-boxes is substitution of modulo 2n adders in modifying known algorithms of stochastic data transformation, for example, stream algorithms PIKE (Fig. 3), RC4, and several others (Asoskov et al. 2003; Stallings 2016; Hammood et al. 2016; McKague 2016; Rivest and Schuldt 2016). In addition, with the help of an R-box it is possible to substitute modulo 2n adder in two ways in 2 Fig. 1 and thus get two kinds of R -boxes. Let us consider an example of a nonlinear transformation on the basis of RFSR (Random Feedback Shift Register), which is obtained after substituting the modulo 2n adder with an R-box in the architecture of an additive generator (Asoskov et al. 2003). Let the number of bits in Q state (the number of storage elements) RFSR be 128: jM j ¼ jQj ¼ 128; Q ¼ ðQ16 . . . Q1 Þ; Qi 2 GF ð28 Þ; i ¼ 1; . . .; 16: The nonlinear stochastic transformation on the basis of RFSR, constructed in accordance with Galois module (Fig. 4), is defined by the following expressions: F ðQÞ ¼ f 16 ðQÞ ¼ f 16 ðQ16 k. . .kQ1 Þ:

224

A. Albatsha and M.A. Ivanov

cri A 8

8

RSm

8 RSm

8 R(A, B)

cro

A

Q55

Q1

Q7

Q57

cro2

cri 8

M2

H 8

Sm 8

8

RSm

8

8 γ

8

RSm

a H-1

Q24

cro1

B

8

Q1

Q1

Q58

Q19

cro3

B

cro Control Unit

c

b Clock

Fig. 3. Modified PRNG of stream cipher PIKE: a – the graphical representation of the stochastic adder; b – the scheme of the stochastic adder; c – the scheme of PRNG. Sm – modulo 256 adder, RSm – stochastic 8-bit adder, M2–8-bit modulo 2 adder, c – PRNG output, cri – Carry In, cro – Carry Out.

Q1

R1

Qi

Ri

QN

Q1

Qi

RN Ci

a R

QN

b

Fig. 4. RFSR: a – general scheme; b – RFSR with a single R-box

The equation of the base transformation f takes the following form: 


Qj ¼ Rj Q1 ; Qj þ 1 ; j ¼ 1; . . .; 15; Q16 ¼ R16 ðQ1 ; Ci Þ:

where C ¼ C1 . . .Ci . . .C16 is the control sequence (which probably depends on the key). Finally, RFSR has a remarkable property: when choosing the appropriate table H of stochastic transformation it is possible to obtain a nonlinear maximum length sequence (M-sequence) generator, whose properties differ fundamentally from those of linear M-sequences, formed by LFSR. Figure 5 shows an example of the M-sequence generator of the length 63 (N = 3, n = 2, │Qi│ = 2).

Stochastic Data Transformation Boxes

A

225

R

B

Q1

Q3

Q2 a

1 0 0

3 1 2

2 0 3

3 3 1

0 1 0

0 3 1

2 2 0

1 3 3

1 0 1

1 0 3

2 2 2

0 1 3

0

3 1 0

2 1 0

3 2 2

3 0 1

3

1 3 1

1 2 1

3 3 2

3 3 0

1 1 3

0 1 2

2 3 3

3 3 3

1

3 1 1

0 0 1

0 2 3

0 3 3

2 3 1

3 0 0

1 0 2

0 0 3

1 2 3

0 3 0

1 1 0

2 0 0

1 1 2

3 0 3

1 1 1

0 2 0

0 1 1

2 3 0

2 1 1

2 0 2

2 0 1

3 2 3

2 2 1

1 2 0

3 2 0

1 3 2

0 2 2

2 1 2

2 3 2

2 1 3

3 0 2

0 2 1

2 2 3

3 2 1

1 3 0

0 0 2

1 2 2

0 3 2

3 1 3

2 b

0 0 0

c

Fig. 5. Example of RFSR: a – the scheme of the nonlinear M-sequence generator; b – the table of stochastic transformation; c – the switching diagram of the generator.

3 Conclusion We have analyzed the construction of R-boxes, which are a generalization of S-boxes, classical structural elements of cryptographic primitives of hashing, block and stream encryption. R-boxes are in fact stochastic adders, i.e. adders with an unpredictable operating result, which depends on the filling of the key table H, which can be formed by using a method similar specified in the stream cipher RC4. A distinctive feature of R-boxes is their effective software and hardware implementation. R-boxes can be applied in the following areas: – Constructing blocks of direct-to-reverse stochastic transformation in the implementation of stochastic methods of data transfer (Osmolovsky 1991); – Implementing stream encryption; in this case a Plaintext is fed to input A, a Keystream is fed to input B, and a Ciphertext is removed from output RH ðA; BÞ; in this case it is essential to use the transformation R1 (reverse R) at the receiver;

226

A. Albatsha and M.A. Ivanov

– Increasing the cryptographic security of known algorithms by substituting modulo 2n adders with n-bit R- boxes. Implementing nonlinear transformations through mixing cipher state (MixState), including those depending on the key information, in the construction of 2D and 3D iterative cryptographic algorithms (Ivanov et al. 2012a; Ivanov and Chugunkov 2012b; Ivanov et al. 2014; GOST R 34.12-2015 2015). – Constructing shift registers with stochastic feedback (RFSR), which are a generalization of classical linear feedback registers (LFSR), i.e. PRNG, functioning in finite fields, which work well in practice as structural elements in primitives of symmetric cryptography. When choosing the appropriate table H RFSR forms nonlinear M-sequences, which have different properties from those characteristic of linear M-sequences. Innovative solutions of forming sequences of the length 2m, where m stands for the number of storage elements in PRNG, forming sequences with a tail, forming universally programmed PRNG, forming sequences of any length, less than or equal to 2m, working in the case of LFSR, also work in a more common case, that of RFSR (Ivanov et al. 2009). Acknowledgement. The publication is prepared in accordance with the scientific research under the Agreement between the Federal State Autonomous Educational Institution of Higher Education “National Research Nuclear University MEPhI” and the Ministry of Education and Science № 14.578.21.0117 on 27.10.2015. The unique identifier for the applied scientific research (project) is RFMEFI57815X0117.

References Osmolovsky, S.A.: Stochastic Methods of Data Transmission. Radio i Svyaz, Moscow (1991) Osmolovsky, S.A.: Stochastic Methods of Information Defense. Radio i Svyaz, Moscow (2003) Asoskov, A.V., Ivanov, M.A., Mirsky, A.A., et al.: Stream Ciphers. Kudits-Obraz, Moscow (2003) Stallings, W.: The RC4 stream encryption algorithm, 5 July 2016. people.cs.clemson.edu/ *jmarty/courses/Spring-2016/CPSC424/papers/RC4ALGORITHM-Stallings.pdf Hammood, M.M., Yoshigoe, K., Sagheer, A.M.: RC4-2S: RC4 stream cipher with two state tables, 5 July 2016. ualr.edu/computerscience/files/2014/01/Paper-12.pdf McKague, M.E.: Design and analysis of RC4-like stream ciphers, 5 July 2016. etd.uwaterloo.ca/ etd/memckagu2005.pdf Rivest, R.L., Schuldt, J.C.N.: Spritz—a spongy RC4-like stream cipher and hash function, 5 July 2016. people.csail.mit.edu/rivest/pubs/RS14.pdf Ivanov, M.A., Vasilyev, N.P., Chugunkov, I.V., et al.: Three-dimensional pseudo-random number generator for implementing in hybrid computer systems. Vestnik NRNU MEPhI 1(2), 232–235 (2012a) Ivanov, M.A., Chugunkov, I.V.: Cryptographic methods of information defense in the computer systems and networks: teaching guide. National Research Nuclear University MEPhI, Moscow (2012b)

Stochastic Data Transformation Boxes

227

Ivanov, M.A., Spiridonov, A.A., Chugunkov, I.V., et al.: Three-dimensional data stochastic transformation algorithms for hybrid supercomputer implementation. In: Proceedings of 17th IEEE Mediterranean Electrotechnical Conference, Beirut, Lebanon, pp. 451–457 (2014) GOST R 34.12-2015. Information Technology. Cryptographic Information Defense. Block Ciphers. Standartinform, Moscow (2015) Ivanov, M.A., Chugunkov, I.V., Matsuk, N.A., et al.: Stochastic Methods of Information Defense in Computer Systems and Networks. Kudits-Press, Moscow (2009)

An Innovative Algorithm for Privacy Protection in a Voice Disorder Detection System Zulfiqar Ali1(B) , Muhammad Imran2 , Wadood Abdul3 , and Muhammad Shoaib2 1

Digital Speech Processing Group, Department of Computer Engineering, College of Computer and Information Science, King Saud University, Riyadh 11543, Saudi Arabia [email protected] 2 College of Computer and Information Science, King Saud University, Riyadh, Saudi Arabia [email protected], [email protected] 3 Department of Computer Engineering, College of Computer and Information Science, King Saud University, Riyadh 11543, Saudi Arabia [email protected]

Abstract. Health information is critical for the patient and its unauthorized access may have server impact. With the advancement in the healthcare systems especially through the Internet of Things give rises to patient privacy. We developed a healthcare system that protects identity of patients using innovative zero-watermarking algorithm along with vocal fold disorders detection. To avoid audio signal distortion, proposed system embeds watermark in a secret key of identity by visual cryptography rather than audio signal. The secret shares generated through visual cryptography are inserted in the secret watermark key by computing the features of audio signals. The proposed technique is evaluated using audio samples taken from voice disorder database of the Massachusetts Eye and Ear Infirmary (MEEI). Experimental results prove that the proposed technique achieves imperceptibility with reliability to extract identity, unaffected disorder detection result with high robustness. The results are provided in form of Normalized Cross-Correlation (NCR), Bit Error Rate (BER), and Energy Ratio (ENR).

1

Introduction

The privacy of patient’s health related information is always a serious concern [1,2]. Health information can include patients’ demographics with administrative and legal clinical massages which are stored, managed and transmitted electronically. The main objective of the research is to design and implement a protected health diagnostic system and to detect vocal fold disorders. The proposed protected healthcare system has two main modules. The first module ensures the privacy of an audio sample implemented with zero-watermarking, c Springer International Publishing AG 2018  A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6 32

An Innovative Algorithm for Privacy Protection

229

whereas the second module is responsible for the detection of voice disorder in an audio sample implemented with Mel-frequency Cepstral Coefficients (MFCC) extraction method with Support Vector Machine (SVM). To the best of our knowledge, existing research lacks with using zero-watermarking to audio medical signals and their after attack diagnostic accuracy. The rest of the paper is organized as follows: Sect. 2 describes implementation of two modules. Section 3 details the proposed zero-watermarking algorithm with embedding and extracting processes. Section 4 provides the results. Finally, Sect. 5 presents some conclusions.

2

Proposed Implementation in Healthcare System

The proposed healthcare system consists of two modules: the privacy protection and disorder detection. The main components used to secure privacy in Module 1 are the image generation for subject identity, feature extraction for zero-watermarking, and creation of secret shares for subjects’ identities through visual cryptography. Whereas, Module 2 consists of the speech extraction features from audio samples of normal and dysphonic subjects by applying the 1D-LBP operator [3,4], and pattern matching for automatic diagnosis of a voice disorder by implementation of SVM. The audio samples are taken from a voice disorder database recorded at the Massachusetts Eye and Ear Infirmary (MEEI) voice and speech laboratory [5] which has been used in a number of previous studies [6–11].

3

Proposed Zero-Watermarking Algorithm

The proposed zero-watermarking algorithm used to protect the subject’s privacy is implemented in Module 1. The features selected into audio samples are determined by analyzing the histograms of the computed 1D-LBP codes. Then embedding is implemented to insert the identity of a subject is shown (Fig. 1). To recover the identity, two healthcare staff has the keys and transmitted audios as shown (Figs. 2 and 3)

Fig. 1. Embedding process to insert identity of the proposed algorithm.

230

Z. Ali et al.

Fig. 2. Watermark extraction process.

Fig. 3. Evaluation of patient’s voice signal.

4

Experimental Results and Discussion

In the proposed disorder detection system, Module 2 is implemented with MFCC and SVM which consist of two phases: the training phase and the testing phase. The training phase takes labeled audio samples and extracts MFCC features. Then, SVM generates the model for each type of subject by using the computed features. The testing phase takes unlabeled/unknown audio samples and calculates the MFCC features. Then, SVM uses these features to predict the class of unknown audio samples through pattern matching. The frame size for MFCC is 512 samples, a hamming window with 512 points, and 29 band-pass filters are used in a Mel-spaced filter bank. The performance is measured by: sensitivity (SEN), specificity (SPE), and accuracy (ACC) as shown in Table 1. Table 1. Performance result for disorder detection Kernel % SEN ± STD % SPE ± STD %ACC ± STD AUC Linear 96.32 ± 4.1

81.96 ± 6.7

90.64 ± 5.8

0.89

98.72 ± 2.2

83.22 ± 8.3

92.39± 4.4

0.95

RBF

The parameters NCR (Normalized Cross-Correlation), BER (Bit Error Rate), and ENR (Energy Ratio), given by Eqs. (1), (2) and (3), for the audio sample of patient identity are 1,0, and 1, respectively. These parameters are

An Innovative Algorithm for Privacy Protection

231

computed for different audio samples of the MEEI subset database as shown in Table 2. The experimental results validated the performance of the proposed algorithm. a

b

SID (i, j)SID (i, j)  12   12 b b 2 a 2 i=1 j=1 SID (i, j) i=1 j=1 SID (i, j)

N CR(SID , SID ) =  a

i=1

j=1

t × 100 a×b a b i=1 j=1 SID (i, j) EN R = a b SID (i, j) BER(%) =

i=1

(1)

(2) (3)

j=1

where SID and SID are original and retrieved patient’s identities, respectively. The parameter t represents the number of erroneously extracted bits and a × b is the dimension of SID image. Table 2. Performance of the proposed algorithm for the M EEIsubset Modules

Performance Parameters

Module 1 NCR: 1 (proposed algorithm)

BER: 0

ENR: 1

Module 2 SEN: 98.72% ± 2.2 SPE: 83.22% ± 8.3 ACC:92.39% ± 4.4 AUC:0.95 (disorder detection)

In the proposed zero-watermarking algorithm the identity is inserted into the secret key instead of the audio sample to avoid inaccurate diagnosis. Table 3 shows imperceptibility analysis with SNR (signal-to-noise ratio) of host audio and watermarked audio implemented in module 1along with and SEN, SPE, ACC, and AUC implemented in module 2. Table 3. Performance analysis of the proposed algorithm for imperceptibility Modules

Performance parameters

Module 1 SNR:inf (proposed algorithm) Module 2 SEN: 98.72%±2.2 SPE: 83.22%±8.3 ACC:92.39%±4.4 AUC:0.95 (disorder detection)

232

Z. Ali et al.

Detection reliability examines whether the proposed zero-watermarking algorithm has the undesired property of watermark extraction by using secret keys of a different subject. The proposed algorithm detects the identity of a subject reliably, and a key of a different subject cannot be used to disclose the identity of some other subject. The proposed algorithm is robust against malicious attacks. The results of the proposed algorithm after adding the white-Gaussian noise in the watermarked audio sample with identity extracted from the attacked audio sample SID and original identity SID is shown in Table 4. Table 4. The performance of the proposed zero-watermarking algorithm for noise attack dB NCR BER ENR Diagnosis

5

No 1

0

1

True

60 0.99

1.11

0.98

True

50 0.98

3.01

0.95

True

40 0.92

9.64

0.85

True

30 0.81

22.61 0.65

false

Conclusion

In this paper, a secure healthcare system is developed implemented with the proposed zero-watermarking algorithm, which generates two secret shares of a subject’s identity using visual cryptography. Our proposed zero-watermarking algorithm distinguish with traditional approach is that it will not intervene with audio samples as the secret shares of the identity are embedded into the secret keys instead of the host audio to avoid probable audio degradation with and diagnostic accuracy. The proposed algorithm is evaluated using the MEEI voice disorder database. The experimental results validated the reliability to detect of a subject’s identity and robustness against noise attacks.

References 1. Gong, T., Huang, H., Li, P., Zhang, K., Jiang, H.: A medical healthcare system for privacy protection based on IoT. In: Paper Presented at the 2015 Seventh International Symposium on Parallel Architectures, Algorithms and Programming (2015) 2. Hsu, C.-L., Lee, M.-R., Su, C.-H.: The role of privacy protection in healthcare information systems adoption. J. Med. Syst. 37(5), 9966 (2013) 3. Chatlani, N., Soraghan, J.J.: Local binary patterns for 1-D signal processing. In: Paper Presented at the 2010 18th European Signal Processing Conference (2010) 4. Houam, L., Hafiane, A., Boukrouche, A., Lespessailles, E., Jennane, R.: One dimensional local binary pattern for bone texture characterization. Pattern Anal. Appl. 17(1), 179–193 (2014)

An Innovative Algorithm for Privacy Protection

233

5. Massachusette Eye & Ear Infirmry Voice & Speech LAB. Disordered Voice Database Model 4337 (Ver. 1.03) (1994) 6. Ali, Z., Elamvazuthi, I., Alsulaiman, M., Muhammad, G.: Detection of voice pathology using fractal dimension in a multiresolution analysis of normal and disordered speech signals. J. Med. Syst. 40(1), 20 (2015) ˜ sn, N., Osma-Ruiz, V., ˜ so, J.D., Godino-Llorente, J.I., SA ˜ aenz-LechA¸ 7. Arias-LondoA´  ˜ Castellanos-DomAnguez, G.: An improved method for voice pathology detection by means of a HMM-based feature space transformation. Pattern Recogn. 43(9), 3100–3112 (2010) 8. Godino-Llorente, J.I., Gomez-Vilda, P., Blanco-Velasco, M.: Dimensionality reduction of a pathological voice quality assessment system based on Gaussian mixture models and short-term cepstral parameters. IEEE Trans. Biomed. Eng. 53(10), 1943–1953 (2006) 9. Markaki, M., Stylianou, Y.: Voice pathology detection and discrimination based on modulation spectral features. IEEE Trans. Audio Speech Lang. Process. 19(7), 1938–1948 (2011) 10. Muhammad, G., Melhem, M.: Pathological voice detection and binary classification using MPEG-7 audio features. Biomed. Signal Process. Control 11, 1–9 (2014) 11. Villa-Canas, T., Belalcazar-Bolamos, E., Bedoya-Jaramillo, S., Garces, J.F., Orozco-Arroyave, J.R., Arias-Londono, J.D., Vargas-Bonilla, J.F.: Automatic detection of laryngeal pathologies using cepstral analysis in Mel and Bark scales. In: Paper Presented at the XVII Symposium of Image, Signal Processing, and Artificial Vision (STSIVA), 12–14 September 2012

Handwritten Signature Verification: The State of the Art Anastasia Beresneva, Anna Epishkina ✉ , Sergey Babkin, Alexey Kurnev, and Vladimir Lermontov (

)

National Research Nuclear University MEPhI (Moscow Engineering Physics Institute), Moscow, Russia [email protected], [email protected], [email protected], [email protected], [email protected] Abstract. Nowadays handwritten signature and its verification is utilized in a lot of applications including e-commerce. An analysis of verification algorithms and areas of their practical usage is provided. The focus of the investigation is on verification method based on neural network. This type of verification algorithm is realized as a mobile application and its main characteristics are obtained. The directions of further work are concluded including a modification of an algorithm and its realization in order to remove its disadvantages. Keywords: Handwritten signature · Verification · Neural network · Mobile application

1

Introduction

Currently, the task of verification or identification of the user is one of the priority tasks in the field of information security. The most promising algorithms of verification based on the use of personal biometric data: fingerprints, iris, retina, DNA, handwriting signa‐ ture. Verification a handwritten signature is a biometric technology that uses the signed for identification purposes with the aim of establishing the authority for making auto‐ mated transactions, obtaining access to computer terminals, or physical access to the controlled area. Signatures are particularly useful for identification because the signature of each person is unique, especially if along with a static form are considered her dynamic performance. These features include time stamping the signature, size, speed, number of segments and the pressing force of the pen. Verification of a handwritten signature can be used to ensure the security of financial transactions. If the verification algorithm is guaranteed to be able to determine the identity of the signatory, a hand‐ written signature will replace e-due to more simple and human-readable mechanism for placing signatures. In addition, such verification will be used in mobile application security. Modern mobile devices have the necessary hardware for online verification of a handwritten signature. Such identification does not require memorizing passwords or PIN codes, however, can provide the necessary level of security. The rest of the paper is organized as follows. In Sect. 2 we consider related works. Section 3 is devoted to analysis of the main verification algorithms. In addition, the © Springer International Publishing AG 2018 A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6_33

Handwritten Signature Verification: The State of the Art

235

section considers the implementation of algorithms and test results. We give the conclu‐ sion and future research directions in Sect. 4.

2

Related Works

Nowadays there are few approaches to handwritten signature verification. Algorithms of verification a handwritten signature can be divided into two groups: online verification algorithms and offline verification algorithms. The system uses online verification algorithms can be implemented using graphical tablets and mobile devices to retrieve dynamic characteristics in the process of entering the signature. System based on algorithms offline verification, use only the static char‐ acteristics of signatures that are extracted from the image. Nowadays developed several different approaches to the problem of verification a handwritten signature. Autonomous system of signature verification presented in [1], built on the basis of several statistical methods, in particular, uses hidden Markov models (HMM) in building reference model for each local object. A hidden Markov model consists of a sequence of states S1 , … , Sn which are associated by probabilistic transitions with probability pij, i.e. the probability of transition from i -th state to j-th. Possible transitions only to the next state or looping. Each time the model performs a probabilistic transition from one state to another or in the same condition. Thus, there is a vector of observations yk with the output probability distribution of bn (yk ) corresponding to the specific condition. This approach defines two concurrent random process, one of which is the main and unobservable (i.e., the sequence of HMM-States). Judge it is possible only with the help of another random process, i.e. a sequence of observations. In the verification algorithm of a probabilistic comparison of the sample and the signature is based on the HMM. The signing process is modelled with several States that constitute a Markov chain. Each of these States corresponds to a separate part of the signature described by a set of characteristics that are not observed directly (i.e. hidden). There are only local features of a signature (such as tangents of angles). The observed data are associated statistically with state models and conditionally inde‐ pendent in each state. When training the model parameters are estimated for the set that contains the authentic signature. During verification, we compute the probability that the signature is genuine. If this probability exceeds the threshold, the signature is accepted, otherwise it is considered tampered with. This approach can be viewed as a statistical conformity check of the signature and the signature based on the HMM. Another system proposed in [2] was based on machine learning. For the application of machine learning for verification of signatures also required the training sample. In the process study examined the possibility of applying such algorithms as KNN classi‐ fication algorithm [2], support vector machines, and logistic regression analysis. However, machine learning is on example of these algorithms has a major drawback for accurate recognition, the training required for a much larger sample than for the algo‐ rithm based on neural network.

236

A. Beresneva et al.

In [3] described a technique for verification of signatures, which was based on the use of neural networks. For each object, it sets a special two stage perceptron and intro‐ duced structural classification. The application technology of neural networks is widely accepted to solve such kind of problems. It is possible to allocate following advantages of using neural networks to verify the signature: • Class of multilayer networks as a whole can represent any desired function in the form of a set of attributes, and signatures can easily be modelled as a function from the set of characteristics. • Neural networks are an excellent tool to summarize and help to cope with the diversity and variations inherent in handwritten signatures. • Neural networks are very tolerant to noise in the input data. • The performance of a neural network gradually decreases with a sharp deterioration in conditions of recognition.

3

Analysis of the Main Verification Algorithms

Common online verification algorithm of a handwritten signature consists of the following steps: • • • • • •

Obtaining source data using the hardware. Preprocessing of the handwritten signature. Extraction of the characteristics of a handwritten signature. Build a model of a signature on its characteristics. Computation of the similarity measure of the test signature to the sample. The decision on the authenticity of the signature.

The advantages of using dynamic features in that they are much more difficult to forge because they are not visible when reviewing a paper copy of the signature. The test results of the algorithms verify the signature is represented in a ratio of type I error and type II error. Type I error associated with denial of access to legitimate user, type II error – of a false identification. Used for further processing signature: • Graphic display (in graphic or vector form). • Number of touches. • Temporal characteristics (minimum, maximum, average, total time without lifting the pen from the screen). • The characteristics of the pen movement speed (minimum and maximum values of the projections of the velocities on the axis and module speed). Analyzed the following approaches, allow for the verification of handwritten signatures: • • • •

KNN algorithm. Range Classifier algorithm. Algorithm based on hidden Markov models. The simplest perceptron neural network.

Handwritten Signature Verification: The State of the Art

237

The KNN classification algorithm [2] (k-nearest neighbor), the input accepts a vector containing the values of the characteristics of the signature, and the output issues a decision, a genuine signature or a forgery. To classify each of characteristics from the training samples, you must execute the following operations: • Calculate the distance to each of objects of training set. • To select k objects in the training set, the distance to which is minimal. Next, a decision is made, if the characteristics are within acceptable tolerances. This algorithm has the following disadvantages: • Low accuracy. • Type I error and type II error at turns, scaling, shifts the signature. Range Classifier algorithm verify the signature consists of the following steps: • For each signature is calculated the centroid. • For each signature vector is formed from the angles and lengths of the radius vectors from centroid to each point. • Overlap of a sequence of test vectors of the training sample feature with an error. • Determined the range of values of each vector according to the training sample. If the threshold number of parameters within the specifications and applying vectors match, the signature is determined genuine. Algorithm based on hidden Markov models input also accepts a vector of charac‐ teristics of the signature, they are as follows: • The signing process is modelled with several States that constitute a Markov chain. • Each of these states corresponds to a separate part of the signature that are not observed directly (i.e. hidden). • The observed data are associated statistically with state models and conditionally independent in each state. • When training the model parameters are estimated for the set that contains the authentic signature. At the time of verification to calculate the probability that the signature is genuine. If this probability reaches a preset threshold, the signature is accepted, otherwise, it is rejected. This approach can be viewed as a statistical conformity check signatures and signatures based on hidden Markov model. The following algorithm is neural network, it takes the input vector containing the values of the characteristics of the signature. It has 12 inputs, 2 hidden layers of 6 neurons each and 1 output. The network operates on the principle of “learning with a teacher”. In the first stage to the inputs of the neural network serves signature: • The proportion of matching points in the vector sequence. • The temporal characteristics (total time, maximum, minimum and medium time without interruption). • Number of breaks pen. • Projections of the minimum and maximum speeds on the coordinate axes. • Modules maximum and minimum speed.

238

A. Beresneva et al.

Then, the logistic activation function sets the weight of the synapses of the neural network. After that, the signature is deemed to be correct if the output of the neural network value is greater than threshold. This verification algorithm is recognition accuracy and insensitivity to changes in scale and offset of the signature. However, this algorithm showed the most accurate results for the task of verification. The algorithms were implemented in the form of a Java mobile application for the Android platform. While testing a sample of 100 signatures for the considered algo‐ rithms the obtained results are shown in Table 1. All implemented algorithms are based on prior learning. The user makes several signatures from which to retrieve the necessary characteristics. Each handwritten signature of the user is different from the previous one. As a result, some characteristics, such as changing speed, different dimensions or the collars of the pen must be removed from several samples of the signature to account possible error. The extracted features from digitized signatures are stored in a matrix, and then can be used for verification. Table 1. The results of testing the developed algorithms Algorithm KNN algorithm Range Classifier Hidden Markov model based algorithm Neural network

4

Type I error 0.13 0.04 0.10 0.08

Type II error 0.20 0.20 0.17 0.12

Computation time, ms 3.37 5.17 4.80 2.16

Conclusions

The study revealed that the most promising methods for further work is the algorithm based on hidden Markov model and neural network, since the proportion type I and type II errors for these algorithms is minimal relative to the others. In addition, they are less sensitive to noise and can be scaled. In the future studies on this topic assume to develop an improved algorithm for verification of a handwritten signature, taking into account the pressing force of the pen, reduce the number of errors and time of learning. Acknowledgments. Authors acknowledge support from the MEPhI Academic Excellence Project (Contract No. 02.a03.21.0005).

References 1. Kashi, R.S., Hu, J., Nelson, W.L., Turin, W.: On-line handwritten signature verification using hidden markov model features. In: IEEE Proceedings 4th International Conference Document Analysis and Recognition, pp. 253–257 (1997) 2. McCabe, A., Trevathan, J., Read, W.: Neural network-based handwritten signature verification. J. Comput. 3(8), 9–22 (2008) 3. Beatrice, D., Thomas, H.: On-line handwritten signature verification using machine learning techniques with a deep learning approach. Master’s Theses in Mathematical Sciences (2015)

The Port-in-Use Covert Channel Attack Dmitry Efanov(B) and Pavel Roschin National Research Nuclear University MEPhI (Moscow Engineering Physics Institute), Kashirskoe shosse, 31, 115409 Moscow, Russian Federation {dvefanov,pgroschin}@mephi.ru

Abstract. We propose a port-is-in-use attack, which is intended for leaking sensitive information in multilevel secure operating systems. Our approach is based on TCP socket mechanism widely used in Linux for interprocess communication. Despite the strong limitations inherent in operating systems with mandatory access control, sockets may not be restricted by the security policy, which makes it possible theoretically to transfer information from one process to another from a high security level to a low one. The proposed attack belongs to the operating system storage transition-based class attack. The main idea is to use the availability of TCP port, which is shared among processes at more than one security level, as the communication medium. The possibility or impossibility of binding a socket to a predefined port is used to transmit a bit of 0 or 1 respectively. We implement proof-of-concept exploit, which was used to check the idea and to evaluate covert channel capacity. Experimental results show that the proposed technique provides high rate covert channel, that means a significant threat of confidentiality in multilevel secure operating systems. Keywords: Covert channel · Information flow · TCP socket · Proofof-concept exploit · Multilevel security · Mandatory access control · Interprocess communication

1

Introduction

The development of the concept of covert channels in operating systems includes several important steps. The term of covert channels was introduced as one kind of communication channel that a service program should be confined to use [7]. Later, the shared resource matrix methodology was proposed [6], which focused primarily on the discovery of covert channels in a formal top-level design specification of operating system kernels and trusted processes rather than in source code. Covert channels are of great interest in the context of multilevel secure (MLS) operating systems researching from the security point of view. It is thus important that there be no covert channels that allow a malicious user (a rogue c Springer International Publishing AG 2018  A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6 34

240

D. Efanov and P. Roschin

employee, who abuses his trusted privileges) to transfer information from a high security level to a low one. We propose a get-port-in-use attack, which is based on TCP socket mechanism widely used in Linux for interprocess communication. Despite the strong limitations inherent in MLS operating systems, sockets may not be restricted by the security policy and may be used to transfer information from one process to another regardless of their security levels. Our attack corresponds to the classical three communication steps in covert channel attack [10]. First, a send mechanism emits bits of data by manipulating a shared resource. Then, a receive mechanism infers the bits by monitoring a shared resource. Finally, an optional feedback mechanism back to the sender provides direct synchronization and reduces noise.

2

Covert Channel Classification

For the last twenty years different approaches have been taken to classify the covert channels [1–3,9,12,17]. The problem is that there are a variety of mechanisms in operating systems to transmit timing and storage information between processes, including many data structures that can potentially be manipulated (directly or indirectly). New techniques of attack are constantly emerging [4,5,11,13,14,16,19]. Traditionally the covert channels are divided into two main types: storage and timing channels [8]. Although fundamentally the same, they differ in the way that information is encoded. In a storage channel, there is a shared global resource in the system that acts as the medium for information transfer, where a user can potentially change its value, and another user can potentially view the change directly or indirectly. Timing channels send information in a way that involves manipulating the timing properties of a component of the system. There are basically two classes of covert storage channels: resource exhaustion and event count [15]. Resource-exhaustion channels exist wherever system resources are shared among users at more than one security level. In event-count channels, the sending process encodes multiple bits by requesting or not requesting a shared system resource. Furthermore, information can be transmitted by value-based and transition-based channels [18]. The key distinction between these methods is that value-based channels transmit information based on the actual value present somewhere in the system, whereas transition-based channels use the change of a value to transmit information. In this paper we use the classification scheme with three orthogonal criteria: storage/timing, network/OS/hardware, and value/transition-based [10].

3

Attack Scenario

Consider the operating system with MLS security policy. Malicious user has access to some sensitive files with high security level. He wants to violate MLS security policy and compromise sensitive information. He has advanced technical

The Port-in-Use Covert Channel Attack

241

knowledge. He has a range of security clearances and has the rights to run processes at different security levels. The idea of possible covert channel was noted in [20]. We assume two processes are running at different security levels. The sending process has a high security clearance and the receiving processes has a low security clearance. The security policy prohibits processes from communicating directly in read/write manner. The low process can send a signal (by kill() function) to the high process. Both processes have permissions to create a socket and to bind a socket to the predefined port. Obviously they cannot simultaneously bind the socket to the same port. Suppose that there is some mechanism that allows the processes to agree in advance on the port number. The proposed attack is based on using of the TCP socket mechanism. The main idea is to use a state of network port, which is shared among processes at more than one security level, as the communication medium. The possibility (port is free) or impossibility (port is in use) of binding a socket to a predefined port is used to transmit a bit of 0 or 1 respectively from high security level to low security level. An TCP socket address is defined as a combination of an IP interface address and a 16-bit port number. Only one TCP socket may be bound to any given local (address, port) pair. If a process try to bind to an address already in use, it will get an EADDRINUSE error. The sending process encodes a bit of 1 or 0 by binding or not binding a socket to a local address respectively. The receiving process detects the bit by trying to bind a socket to the same port number. By observing the return result of bind() system call that allocates the port number, the receiving process can determine the value of the bit from the sending process. The proposed attack belongs to operating system storage transition-based class attack according to [10]. The transition-based channels appear to be the more difficult to prevent, since all that is required is the ability to change a value to anything, not necessarily a specific value [18]. It also belongs to event-count class channel according to [15].

4

Implementation Details

We present a proof-of-concept exploit, which implements suggested the get-portin-use attack. The sequence diagram shows processes interactions arranged in time sequence (Fig. 1). The source code of exploit is also provided for study and evaluation here github.com/scriptum/port-in-use-covert-channel. As shown earlier, the attack is based on the AF INET protocol family with SOCK STREAM socket type, which provides sequenced, reliable, two-way, connection-based byte streams. Since the specific address does not matter, INADDR ANY (0.0.0.0) is specified in the bind call, so the socket will be bound to all local interfaces.

242

D. Efanov and P. Roschin

The high process generates a message and encodes it in the form of 0 and 1. In our implementation, the message is an array of bits. After that the high process sets the disposition of the signal SIGUSR1 to a signal handler. When the signal SIGUSR1 is delivered to the high process, handler is called and performs the following actions: 1. Closes the server socket, if it was opened previously. 2. Depending on the value of the current bit of the message it performs: a. If the bit is 1, it creates a server socket, binds it to the predefined port, and starts listening to it. b. If the bit is 0, then it does not create a server socket. 3. Increments the counter by one. 4. Frees the processor (call sched yield()). After setting the signal handler, the high process enters the infinite loop in which it sleeps. The low process starts and enters a loop in which the message is sequentially received bit by bit. To do this, it forms a buffer for receiving the message and sets the bit counter to 0. Then the low process sends a signal (in our implementation – SIGUSR1) to the high process, creates a client socket and try to bind to the predefined port. If the server listens to this port, that means that this port is in use. So, the client will not be able to bind to and this will mean that the client received a bit with a value of 1. If the server does not create a socket and the predefined port is free, then the client will be able to bind to the port and this will mean that the client received a bit with a value of 0. After that, the low process increments the counter by one and closes the socket. To increase the capacity and reduce the noise of covert channel significantly, we propose using an operating system scheduler to synchronize sender and receiver processes. Modern hardware come with CPU with more than one core. The scheduler attempts to use all available cores, so synchronization using the scheduler is difficult, since each core dedicated to a particular process. However, Linux allows to assign a process to a specific CPU core by calling sched setaffinity() function. Thus, by assigning the sender and receiver to the same CPU core, scheduler will have to split the time between those two processes. We use the call sched yield() as synchronization tool, which interrupts the current process and forces it to the waiting state and causes the kernel to switch to the next process that is waiting in the queue. Since there are only two CPU consumers (sender and receiver), they are switching one by one and synchronize without system timer. This technique works well on modern Linux kernels and allows to transfer the data between processes on relatively high speed without the noise. Our attack is implemented on a computer with an Intel(R) Core(TM) i73770 CPU at 3.40 GHz. We were running CentOS 7.2 with kernel version 3.10.0327.22.2. We obtained high rate more than 73000 bps. The results are obtained without any attempt to optimize the implementation. The channel capacity is measured, in each case, by sending a large bit stream through the channel, timing the transfer, and counting the error bits.

The Port-in-Use Covert Channel Attack

243

Fig. 1. The diagram of the port-is-in-use attack

5

Conclusion

In this paper we proposed a new get-port-in-use attack, which is based on TCP socket mechanism and belongs to the class of operating system storage transitionbased attack. The proof-of-concept exploit was presented as well. Experimental results showed high rate covert channel, that means a significant threat of confidentiality in multilevel secure operating systems. Acknowledgements. This work was supported by the MEPhI Academic Excellence Project (agreement with the Ministry of Education and Science of the Russian Federation of August 27, 2013, project no. 02.a03.21.0005).

References 1. Gallagher Jr., P.R.: A guide to understanding covert channel analysis of trusted systems provides a set of good (1993) 2. Girling, C.G.: Covert channels in LAN’s. IEEE Trans. Softw. Eng. SE–13(2), 292– 296 (1987) 3. Handel, T.G., Sandford, M.T.: Hiding data in the OSI network model, pp. 23–38. Springer, Heidelberg (1996) 4. Harnik, D., Pinkas, B., Shulman-Peleg, A.: Side channels in cloud services: Deduplication in cloud storage. IEEE Secur. Priv. 8(6), 40–47 (2010) 5. Hovhannisyan, H., Qi, W., Lu, K., Yang, R., Wang, J.: Whispers in the cloud storage: A novel cross-user deduplication-based covert channel design. Peer-to-Peer Netw. Appl. 1–10 (2016)

244

D. Efanov and P. Roschin

6. Kemmerer, R.A.: Shared resource matrix methodology: An approach to identifying storage and timing channels. ACM Trans. Comput. Syst. 1(3), 256–277 (1983) 7. Lampson, B.W.: A note on the confinement problem. Commun. ACM 16(10), 613– 615 (1973) 8. Lipner, S.B.: A comment on the confinement problem. SIGOPS Oper. Syst. Rev. 9(5), 192–196 (1975) 9. Mileva, A., Panajotov, B.: Covert channels in TCP/IP protocol stack - extended version-. Cent. Eur. J. Comput. Sci. 4(2), 45–66 (2014) 10. Okhravi, H., Bak, S., King, S.T.: Design, implementation and evaluation of covert channel attacks. In: 2010 IEEE International Conference on Technologies for Homeland Security (HST), pp. 481–487, November 2010 11. Pulls, T.: (More) side channels in cloud storage, pp. 102–115. Springer, Heidelberg (2012) 12. Rowland, C.H.: Covert channels in the TCP/IP protocol suite. First Monday 2(5) (1997) 13. Sala¨ un, M.: Practical overview of a xen covert channel. J. Comput. Virol. 6(4), 317–328 (2010) 14. Salih, A., Ma, X., Peytchev, E.: Implementation of hybrid artificial intelligence technique to detect covert channels attack in new generation internet protocol IPv6, pp. 173–190. Springer, Cham (2017) 15. Shieh, S.-P.: Estimating and measuring covert channel bandwidth in multilevel secure operating systems. J. Inf. Sci. Eng. 15(1), 91–106 (1999) 16. Wang, S., Qiang, W., Jin, H., Yuan, J.: Covertinspector: Identification of shared memory covert timing channel in multi-tenanted cloud. Int. J. Parallel Prog. 45(1), 142–156 (2017) 17. Wang, Z., Lee, R.B.: Covert and side channels due to processor architecture. In: 2006 22nd Annual Computer Security Applications Conference (ACSAC 2006), pp. 473–482, December 2006 18. Wang, Z., Lee, R.B.: New constructive approach to covert channel modeling and channel capacity estimation. In: Proceedings of the 8th International Conference on Information Security, ISC 2005, pp. 498–505. Springer, Heidelberg (2005) 19. Wang, Z., Yang, R., Fu, X., Du, X., Luo, B.: A shared memory based cross-VM side channel attacks in IaaS cloud. In: 2016 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), pp. 181–186, April 2016 20. Wilson, G., Weidner, K., Salem, L.: Extending Linux for Multi-Level Security. DEStech Publications Inc., Lancaster (2007)

Discovering and Clustering Hidden Time Patterns in Blockchain Ledger Anna Epishkina(&) and Sergey Zapechnikov National Research Nuclear University MEPhI (Moscow Engineering Physics Institute), Moscow, Russia {AVEpishkina,SVZapechnikov}@mephi.ru

Abstract. Currently, immutable blockchain-based ledgers become important tools for cryptocurrency transactions, auditing, smart contracts, copyright registration and many other applications. In this regard, there is a need to analyze the typical, repetitive actions written to the ledger, for example, to identify suspicious cryptocurrency transactions, a chain of events that led to information security incident, or to predict recurrence of some situation in the future. We propose to use for these purposes the algorithms for T-patterns discovering and to cluster the identified behavioral patterns subsequently. In case of having labeled patterns, clustering might be replaced by classification. Keywords: Audit trails Clustering




Blockchain




Data mining




Classification




1 Introduction The advent of decentralized cryptocurrencies started from Bitcoin [1] has brought a lot of blockchain-based systems and databases. They are created and maintained through network consensus and can be used as a public ledger. The most widely known blockchain-based decentralized databases are BigchainDB [2], BlockCypher, Openchain etc. Almost all of them provide some form of API which can be used for developing distributed application. Besides blockchain-based databases, applications may also use decentralized file systems such as IPFS (InterPlanetary File System) or BitTorrent-like storage systems. The further important adventure in the area of decentralized computations is generalized decentralized virtual machines like Ethereum [3]. It provides the possibility to support two types of accounts. They are manually managed Externally Owned Accounts (EOAs) and automatically executable Contact Accounts (CAs). The latter is able to execute special programs in byte codes of Ethereum Virtual Machine (EVM) called smart contracts. It is very important that EVM language is Turing-complete, so in principle any possible functionality can be realized through smart contracts. Ethereum platform supports a lot of programming languages such as Solidity, Serpent, Mutan etc. There are some ideas for more advanced privacy-preserving smart contract platforms, i.e. Hawk [4] and Enigma [5] but no one of them has implementation yet. © Springer International Publishing AG 2018 A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6_35

246

A. Epishkina and S. Zapechnikov

The novelty of such platforms like Ethereum is their ability to provide immutable storage for any transactions among the accounts. These transactions may be not only financial but any other involving state changes of EOAs or CAs. Besides that, blockchain-based application ensures perfect integrity for all data or references stored in blockchain and high availability of system services. In particular, such platforms have a lot of applications in cybersecurity including audit trails management and many others. The rest of the paper is organized as follows. In Sect. 2 we consider related works. In Sect. 3 we suggest a technique to apply T-pattern analysis for discovering hidden behavior patterns in audit logs. In Sect. 4 we discuss how to evaluate distance among such patterns and to cluster them. Section 5 is about the clustering-based anomaly detection in behavior patterns. We give the conclusion and future research directions in Sect. 6.

2 Related Works There are a lot of applications of blockchain-based techniques in the area of cybersecurity. Some projects such as ShoCard and ChainAnchor [6] provide blockchain-based identity management and anonymous permissions. Others like DECENT [7] are decentralized highly-available content distribution systems. One more area for blockchain is distributed Certification Authorities and certificate validation systems [8]. There are some other applications such as software authentication and version control, IoT devices authentication, data provenance, secure messaging and so on. However, the most evident cybersecurity application of blockchain is events’ tracing and audit [9]. Traditional logs can be replaced by immutable ledger storing system’s events history. Every audit log should be useful for event incident management. For instance, if log contains security events it should be possible to investigate a chain of events leading to the incident. That is why blockchain transactions should also be auditable. We suggest a technique for blockchain ledger auditing based on computation of hidden behavior patterns discovery, clustering them and outlier detection. We assume blockchain-based audit log because currently blockchain is the most prominent tool for auditing. However, such or similar technique may quite evidently be used for analysis of traditional read-only service or network events log. In our work, we use Magnusson’s T-pattern discovery technique [10] followed by a standard technique for agglomerative clustering and cluster-based outlier detection [11].

3 Discovery of Hidden Patterns in Audit Logs As it is well known, blockchain ledger growths by new blocks in regular time intervals. The length of interval may be different from 10 min in Bitcoin to near 1 s in BigchainDB. The number of transactions into one block may also be different, in the limit decreasing to 1 transaction per block, reducing from blockchain, to transaction-chain.

Discovering and Clustering Hidden Time Patterns

247

So, ever blockchain-based audit trail can be represented as a sequence of events, each placed into one of time intervals. One interval can contain strictly one event or a lot of events. Such log may contain huge amount of different events. Thus, it may be very difficult to trace a sequence of events preceding some other event. The technique of T-pattern discovery introduced by M. Magnusson [10] can be used to solve the task. Let N be the length of time interval (measured in time slots) and A1 ; A2 be two security events. Let NA1 be the number of blocks where the A1 event was written to blockchain ledger (among the total amount of N blocks). Then PðA1 Þ ¼ NA1 =N is the frequency of
 A1 event occurring (this is the evaluation of A1 probability). So, P A1 ¼ 1  PðA1 Þ is
t the frequency of the A1 event non-occurring. Thus P A1 is the estimated probability of the event non-occurring during t segmental slots, where t ¼ t2  t1 þ 1 and 1 
t P A1 is the estimated probability of at least one occurring of A1 during this interval. Let f ðk; p; nÞ ¼ Cnk pk ð1  pÞnk be polynomial distribution of occurring k-of-n events each of which has probability p. Thus, the apriory probability that A1 event will occur at least NAB times followed by A2 event in the next t blocks is P¼1

NX AB 1


 f NA ; i; PðA2 Þt ;

ð1Þ

i0

where
 f NA ; i; 1  PðA2 Þt ¼ CNi A ð1  PðA2 Þt Þi PðA2 ÞtðNA iÞ :

ð2Þ

This probability should be compared with the actual frequency of event occurring to decide if it is above or below the significance level. Let’s show, how it can be evaluated. According to [10] there are four cases: • event A2 occurs after a series of events A1 only once, so no pattern exists; • if there are no NAB cases when event A2 occurs by some blocks after event A1 , we can take minimum and maximum distance between A1 and A2 (in blocks) and evaluate P using (1); • if this probability is quite significant, this is most likely random coincidence; • if it is not significant (i.e. less than 0.05), this is likely pattern. What can be seen as different type of events? It strongly depends on the application. For instance, if we trace financial transactions, it may be a transfer from one certain account to another. After mining a pair of certain events A1 ; A2 the every discovered pattern B ¼ A1 ! A2 (where ! is a sign denoting that A2 is following A1 ) can be thought as one event. The second-order patterns can be discovered similarly. In such pattern every compound event B became the first or the second part of wider pattern where the remaining part is also elementary or compound event. Thus the tree of events grows. It can be visualized by means of dendrogram. After all, each pattern can be written as S ¼ Ai1 ! Ai2 ! . . . ! Aik .

248

A. Epishkina and S. Zapechnikov

4 Clustering Behavior Patterns For auditing, discovered behavior patterns should be analyzed further. We suggest that some different pattern types may be discovered. Moreover, repeating patterns may be not strictly equal one to other, but rather similar. For this purpose Levenshtein distance may be used as a measure of similarity between the patterns [12]. Let S1 and S2 be two event strings with lengths M and N accordingly. The Levenshtein distance is defined by the following recursive formula: 8 0; if i ¼ 0; j ¼ 0 > > > > > > < i; if i [ 0; j ¼ 0 Dði; jÞ ¼ j; if i ¼ 0; j [ 0 > > > minfDði; j  1Þ; Dði  1; jÞ þ 1; Dði  1; j  1Þg þ > > > : þ mðS1 ½i; S2 ½jÞ; if i [ 0; j [ 0

ð3Þ

where mða; bÞ ¼ 0; if a ¼ b and mða; bÞ ¼ 1; if a 6¼ b; Dði; jÞ – the distance between the first i symbols of S1 and the first j symbols of S2. Using Wagner – Fischer’s algorithm [13] optimal distances between event strings can be evaluated. Thus, after that we have a matrix of distances between the discovered event strings. After that, any type of agglomerative hierarchical clustering may be used. We recommend using Ward’s method [14] because of its monotonicity and tension property. As it is well known, the best visual technique for hierarchical clustering is dendrogram, so the optimal number of clusters may be found easily as maximum inter-cluster distance. For example, Lance – Williams algorithm [15] may be used. It starts from n one-element clusters, where n is the number of different patterns. On each step two clusters U; V with minimal distance are united into one cluster W: Let Z be any other cluster that is not merged on this step. The Ward’s distance is defined as ! jZ jjW j 2 X w X z RðW; ZÞ ¼ D ; ; jZ j þ jW j jW j z2Z jZ j w2W

ð4Þ

where Dðu; vÞ is Levenstein distance between patterns, jZ j; jW j are cardinalities of Z and W clusters.

5 Anomaly Detection in Behavior Patterns As a rule, the main purpose of audit is discovering a sort of non-typical behavior. Therefore, outlier detection is the main technique for solving such task. Clustering-based outlier detection using distance to the closest cluster is more convenient choice for our case. For each pattern S we can assign it an outlier score according to the distance between the pattern and the center of a cluster that is closest to the

Discovering and Clustering Hidden Time Patterns

249

pattern [16]. For this purpose, the centers of all discovered clusters should be evaluated. In our case, the center of each cluster is the mean value of Levenshtein distance of all cluster’s patterns from the null pattern. Suppose that closest center to S is CS ; the distance between them is DðS; Cs Þ and the average distance between CS and the patsÞ will be a measure evaluating how far terns assigned to cluster is LCS : The ratio DðLS;C C S

the pattern S stands out from the average distance. The larger this ratio, the relatively farther away pattern S is from the cluster’s center, so it is more likely that S is an outlier.

6 Conclusions Finally, we have the following algorithm for discovering and clustering hidden time patterns in blockchain ledger. 1. Repeating event sequences S are discovered in blockchain ledger using Magnusson’s technique of T-pattern discovery. Pattern can be discovered if it repeats no less twice. The database D of event sequences is created. 2. For each pair of sequences Levenstein distance is evaluated using Wagner-Fischer’s algorithm. In particular, Levenstein distance between each sequence and null sequence should be defined. The distance between all sequences is append to the database D: 3. Event sequences are combined in clusters using Lance – Williams agglomerate hierarchical algorithm and Ward’s distance between clusters. The optimal number of clusters is evaluated using maximal inter-cluster distance criterion. The membership of sequences in clusters is appended to the database D: 4. For each cluster, its center and the average distance between the center and the sequences assigned to cluster is evaluated and appended to the database D: Outliers (anomaly sequences) are discovered as objects with high ratio of the distance between the object and the nearest center of a cluster and the average distance between the sequences and the center of a cluster. These outliers are most likely abnormal event sequences that may be traces of attacks, user’s errors and so on. The main advantage of the suggested technique is that it does not require any additional parameters and may be fully automated. The main drawback is that any sequence to be clustered or estimated as anomaly must repeat at least twice. So, absolutely new anomalous sequences of events could not be discovered by the technique. Thus, further research may be associated with prediction of completely new sequences of anomalous events. Acknowledgments. Authors acknowledge support from the MEPhI Academic Excellence Project (Contract No. 02.a03.21.0005).

250

A. Epishkina and S. Zapechnikov

References 1. Nakamoto, S.B.: A peer-to-peer electronic cash system (2008). https://bitcoin.org/bitcoin.pdf 2. McConaghy, T., et al.: BigchainDB: a scalable blockchain database (2016). https://www. bigchaindb.com/whitepaper/bigchaindb-whitepaper.pdf 3. Wood, G.E.: A secure decentralized generalized transaction ledger (2017). http://gavwood. com/paper.pdf 4. Cosba, A., et al.: Hawk: the blockchain model of cryptography and privacy-preserving smart contracts (2015). http://eprint.iacr.org/2015/675 5. Zyskind, G., Nathan, O., Pentland, A.: Enigma: decentralized computation platform with guaranteed privacy (2016). http://www.enigma.co/enigma_full.pdf 6. Hardjono, T., Smith, N., Pentland, A.: Anonymous identities for permissioned blockchains (2016). http://connection.mit.edu/wp-content/uploads/sites/29/2014/12/Anonymous-Iden tities-for-Permissioned-Blockchains2.pdf 7. Michalko, M., Sevcik, J.: DECENT whitepaper (2015). http://www.the-blockchain.com/ docs/Decentralized%20Open-Source%20Content%20Distribution%20(DECENT)% 20whitepaper.pdf 8. Matsumoto, S., Reischuk, R.: IKP: turning a PKI around with blockchains (2016). http:// eprint.iacr.org/2016/1018 9. Cucurull, J., Puiggali, J.: Distributed immutabilization of secure logs (2017). https://www. scytl.com/wp-content/uploads/2017/01/Distributed-Immutabilization-of-Secure-Logs_Scytl. pdf 10. Magnusson, M.: Discovering hidden time patterns in behavior: T-patterns and their detection. Behav. Res. Methods Instrum. Comput. 32(1), 93–110 (2000) 11. Han, J., Kamber, M., Pei, J.: Data Mining: Concepts and Techniques. Morgan Kaufmann Pubs, San Francisco (2012) 12. Levenshtein, V.I.: Binary codes capable of correcting deletions, insertions, and reversals. Sov. Phys. Dokl. 10(8), 707–710 (1966). (In Russian) 13. Wagner, R., Fischer, M.: The string-to-string correction problem. J. Assoc. Comput. Mach. 21(I), 168–173 (1974) 14. Ward, J.H.: Hierarchical grouping to optimize an objective function. J. Am. Stat. Assoc. 58, 236–244 (1963) 15. Lance, G., Williams, W.: A general theory of classificatory sorting strategies I hierarchical systems. Comput. J. 9(4), 373–380 (1967) 16. Tao, Y., Xiao, X., Zhou, S.: Mining distance-based outliers from large databases in any metric space. In: Proceedings of 2006 ACM SIGKDD International Conference on Knowledge Discovery in Databases (KDD 2006), Philadelphia, PA, August 2006, pp. 394– 403 (2006)

On Attribute-Based Encryption for Access Control to Multidimensional Data Structures Anna Epishkina ✉ and Sergey Zapechnikov (

)

National Research Nuclear University MEPhI (Moscow Engineering Physics Institute), Moscow, Russia {AVEpishkina,SVZapechnikov}@mephi.ru

Abstract. Multidimensional data structures are widely used in modern infor‐ mation technologies. They sometimes contain private or other sensitive informa‐ tion. We argue that attribute-based encryption is a handy tool for access control to multidimensional data structures, discussing the advantages and disadvantages of ciphertext-policy and key-policy attribute-based encryption for this task. We propose a scheme of attribute management for multidimensional data structures. Keywords: Multidimensional data structures · OLAP · Access control · Attribute-based encryption

1

Introduction

Currently, it is hard to overestimate role of information in the life of modern society. Information technologies became essential part in every area of human life. Special attention is paid to data storage and processing technologies. Millions of databases across the world provide storage for large amount of data. OLAP (On-Line Analytical Processing) systems are extremely popular in the field of data processing. Such systems are used in different areas such as sales, financial transactions, telecommunications, healthcare etc. Over the last few years, this approach was widely used in vital areas of human life. Because of that, security had a profound impact on OLAP systems devel‐ opment. Privacy of information, stored in data warehouses, have an important role to play in these technologies. Confidentiality of data stored in cloud services is particular acute nowadays. Recent trends show a transition from companies’ proprietary data storage to cloud storage. Besides economic benefits, the main motive power for outsourcing data storage is the versatility of such decisions. However, at the same the security aspect become more and more important, which implies the necessity of using encryption. In this scenario Cloud Storage Service provider always has access to the information, stored in its system. The progress of cloud technologies makes possible efficient and secure data storage. However, access control is very important to make cloud storage secure. Existing solutions provide a versatility access control system, but they are not fully secure because in most cases cloud provider can access to plaintext data. So, in most cases users prefer not to send confidential information to the cloud at all. As an example, © Springer International Publishing AG 2018 A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6_36

252

A. Epishkina and S. Zapechnikov

such information may concern organization’s partners or clients, and in accordance with the agreement on the inadmissibility it must not be disclosed to a third party. Currently, one of the most prominent techniques for access control to databases and data ware‐ houses is attribute-based access control (ABAC). We consider the problem of access control to multidimensional datasets and databases. It is shown in our paper, that cipher‐ text-policy attribute-based encryption schemes can be effective tools for access control to multidimensional datasets. We offer attribute and key management schemes to support such technique. All data in such system is encrypted by means of special encryption schemes before being sent into the cloud. The rest of the paper is organized as follows. In Sect. 2 we consider related works. In Sect. 3 we discuss the possibilities of using attribute-based encryption for access control to multidimensional datasets. An attribute management scheme is proposed in Sect. 4. We give the conclusion and future research directions in Sect. 5.

2

Preliminaries and Related Works

Formal model of multidimensional databases was introduced in [1]. This technology can be used to effectively store, interactively process and analyze multidimensional data from multiple perspectives. In particular, it became a basis of OLAP [2]. For example, one can use this technology to store and analyze healthcare data, predict diagnoses and create specific therapy for specific patients. Main advantage of this approach against simple relational databases is speed of large queries processing. Complex large multi-table queries are often used during data analytic processing. In case of relational databases, this leads to problems with performance of query processing. OLAP technology was developed with these ideas in mind to negate performance issues for complex queries. Main object of this technology is multidimensional cube also called hypercube that represents multidimensional coordinate system. Dimensions are used as axes of this system. Dimension is the hierarchical system, which consists of attributes of the process being analyzed. For example, sales business process can have such dimensions as Data, Goods, and City and so on. At the same time, Data dimension can have such members as Quarter-Month-Week-Day, organized in hierarchy. Measure is one of the quantitative properties of the process being analyzed in multi‐ dimensional system. Measures are stored in the cell of the coordinate system represented by cube. In the above example, sale amounts can be used as measures for quantitative description of sales process. Each cell store one value of any type, but all values of one measure have the same type. If a cube contains some measures, then the set of all meas‐ ures is considered to be one more dimension. Another essential part of multidimensional dataset technology is aggregates. Aggre‐ gate is the pre-calculated analytical query, stored in data warehouse and based on the source data, aggregated accordingly to the received request. Thus, combination of every possible aggregate and source data can be used to process any query. However, only small amount of determined aggregates are calculated during initialization of the system, while rest of the aggregates are being calculated on demand. The aggregation of many

On Attribute-Based Encryption for Access Control

253

cells at some level of hierarchy gives the value of one cell at the next, higher level of hierarchy at the same dimension. There are three main approaches to store data and aggregates in OLAP systems [3]: MOLAP (Multidimensional OLAP) – data and aggregates are fully stored in multidi‐ mensional database; ROLAP (Relational OLAP) – data and aggregates are stored in relational database of special form and are virtually interpreted as multidimensional structure and HOLAP (Hybrid OLAP) – data stored in relational database, but aggre‐ gates are stored in multidimensional database. In this work, we will keep in mind MOLAP as main approach. Even though this approach has large data overhead, it can achieve good performance.

3

Using of Attribute-Based Encryption for Multidimensional Datasets

Other crucial technique is attribute-based encryption (ABE) proposed by Sahai and Waters [4]. It is intended for “one to many” encryption, so that ciphertexts are created for users who meet certain requirements. There are two types of such schemes: cipher‐ text-policy attribute-based encryption (CP-ABE) and key-policy attribute-based encryp‐ tion (KP-ABE). CP-ABE implies that ciphertexts are associated with the access policy, and the corresponding attributes are included in the private key, and KP-ABE vice versa. In both cases, ciphertext can be decrypted if and only if the attributes correspond to specified access policy [5]. Thus, CP-ABE is suitable for implementation of attribute-based access control for data storage into untrusted environment (Fig. 1). According to [6], the following main components should be identified: ABAC Servers, including server resource attributes, server entity attributes, server environment attributes, and server actions attributes; CA; server working as a gateway for interaction with cloud storage; access control server and client’s application.

Fig. 1. Using ABE scheme to access control in cloud storage.

254

A. Epishkina and S. Zapechnikov

Access control policies are directly connected with attributes. Data encryption and access are determined by a variety of attributes, some of which are required for CP-ABE (principal attributes). These principal attributes are responsible for encryption and decryption of all stored files. Other attributes, which we called non-principal, are checked only by access control server and are not associated with the encrypted data by means of CP-ABE scheme. Dividing attributes into two categories is made manually by administrator and/or data owner. To decide which attributes will be principal and which is not we should take into account existing access control system and company’s business model. The client application consists of a key generation algorithm, a symmetric block cipher algorithm, an, of course, an encryption (EnCP-ABE) and a decryption (DeCPABE) algorithms for CP-ABE. It also provides information about the possible access policies and the values of user attributes. Attributes are divided into those that are appointed by system’s security administrator and those that are generated automatically (time, IP-address, etc.). To encrypt a file, at first user selects an access policy. Then it starts the key generation algorithm, which generates a key for the symmetric algorithm (symmetric key, for short). User’s data may be encrypted using, for example, GOST R 34.12-2015 symmetric cipher [7] or some other block cypher. Then encrypted file is delivered to a cloud gateway server. It is worth emphasizing, that the encrypted data must be associated with a partic‐ ular access policy. After that, the application receives a revoked user list from Certifi‐ cation Authority (CA), which is required for the EnCP-ABE algorithm as well as public key attributes, access policy and symmetric key. The algorithm produces symmetric key encrypted by means of CP-ABE scheme, which is sent to the cloud gateway with the encrypted file. To decrypt data, the application sends the user attributes to the access control server. After checking for compliance with the policy of the user attributes are sent to secret keys and private key. Having the keys of CA and application ABAC server it is possible to decrypt symmetric key. By means of symmetric key algorithm, the application decrypts the file. The computational complexity of decryption strongly depends on the number of attributes, so to improve performance it is undesirable to associate all of the attributes with encryption policy, some of them can be simply checked by the access control server without ABE. If the user meets all the rules of the access policy for the data it receives unencrypted file. ABAC Servers manage key attributes for ABE. Each of these keys holds implicitly a set of user’s attributes. CA provides support for current users of the system and generates a list of application user’s secret keys. Gateway for interaction with the cloud provides a lot of functions for search and delivery of encrypted files; also it indexes the encrypted data and keys before sending to the cloud.

4

The Scheme of Attribute Management

Now let us consider what kind of attribute-based encryption and how can be used to implement the above-mentioned scheme. We will show CP-ABE schemes allow

On Attribute-Based Encryption for Access Control

255

assigning a set of attributes for each cell of dataset for fine-grained access control. More precisely, each cell must be associated with a set of attributes as follows: • “Cube_C”; • “Dimension_i1”; “Dimension_i1.Level_j1”; “Dimension_i1.Level_j1.Cell_k1”; … • “Dimension_id”; “Dimension_id.Level_jd”; “Dimension_id.Level_jd.Cell_kd”, where “Cube_C” is the attribute that indicates that the cell belongs to a particular cube C and its content may be read by any user who has no restrictions on the access to the hypercube C, “Dimension_i” is the attribute that indicates that the cell may be read by any user who has no restrictions on reading hierarchies on i-th hypercube’s dimension, “Dimension_i.Level_j” is the attribute that indicates that the cell may be read by any user who has access for reading cell having j-th level of hierarchy at i-th dimension, and finally “Dimension_i.Level_j.Cell_k” is the attribute that indicates that the cell may be read by any user who has access for reading k-th cell at j-th level of hierarchy at i-th dimension. Let hypercube C has d dimensions. Then every cell must have 3d + 1 attributes, because each cell has exactly d coordinates on each of hypercube’s dimensions, and each coordinate is a set of three values: a dimension name, a level of hierarchy name for the given dimension, and an index at the given level of hierarchy. However, our ABE is ciphertext-based, so we finally must construct an access policy, such that only users having a set of necessary and sufficient attributes but nobody else will have access to the specified cell. Such access policy can be simply constructed as the following predicate: (( Dimension_i1 ∨ Dimension_i1 .Level_ji ∨ ) ( ∨ Dimension_i1 .Level_ji .Cell_k1 ∧ … ∧ Dimension_id ∨ )) ∨ Dimension_id .Level_jd ∨ Dimension_id .Level_jd .Cell_kd .

Policy: = Cube_C ∨

(1)

This predicate means that the access to the cell is granted either to the users who are entitled to access to hypercube without restrictions, or those who have a right of access to such subsets of cells for each dimension which include this cell. The right of access to subsets of cells for each dimension is given for such users who can access all cells along the dimension without restrictions or such users who can access all the hierarchical level to which the cell belongs or such users who can access this particular cell. It is important that every cell must have the above-mentioned set of attributes and access policy in the specified form associated with it. To implement the access policy, each user must have a set of attributes assigned by ABAC servers according to the following set of rules: 1. If user should have right to access all cells of hypercube C, it is assigned the attribute “Cube_C”. 2. If user should have right to access all cells along the dimension i, it is assigned the attribute “Dimension_i” (however, she can have no rights to access every cell along

256

A. Epishkina and S. Zapechnikov

one or more other dimensions). Simultaneously it is must not be assigned the attribute “Cube_C” because it makes no sense to assign attribute “Dimension_i”. 3. If user should have right to access all cells at the certain hierarchical level j of the specified dimension i, it is assigned the attribute “Dimension_i.Level_j” (however, she can have no rights to access one or more other hierarchical level). Simultaneously it is must not be assigned the attribute “Dimension_i”, because it makes no sense to assign attribute “Dimension_i.Level_j”. 4. If user should have right to access certain cell with index k at certain hierarchical level j of the specified dimension i, it is assigned the attribute “Dimen‐ sion_i.Level_j.Cell_k” (however, he can have no rights to access one or more other cells at the given hierarchical level). Simultaneously it is must not be assigned the attribute “Dimension_i.Level_j”, because it makes no sense to assign attribute “Dimension_i.Level_j.Cell_k”. Access control is very essential for privacy-preserving OLAP over encrypted data [8].

5

Conclusions

We have analyzed the problem of using attribute-based encryption to implement access control to multidimensional datasets. The main result of the research is a model of attribute-based access control to multidimensional datasets. It is based on a set of rules for assigning access rules to users and a predicate for access control policy which must be applied to each cell of multidimensional dataset. We will continue to develop our solution to implement attribute-based data authentication in multidimensional datasets. We suppose to use attribute-based signature scheme. Acknowledgments. Authors acknowledge support from the MEPhI Academic Excellence Project (Contract No. 02.a03.21.0005).

References 1. Pedersen, T.: Multidimensional database technology. J. Comput. 34(12), 40–46 (2001) 2. Jensen, C., Pedersen, T., Thomson, C.: Multidimensional Databases and Data Warehousing. Morgan and Claypool, San Rafael (2010) 3. Thomsen, E.: OLAP Solutions: Building Multidimensional Information Systems. Wiley, New York (2002) 4. Sahai, A., Waters, B.: Fuzzy identity-based encryption. In: Cramer, R. (ed.) Advances in Cryptology – EUROCRYPT 2005, vol. 3494, pp. 457–473. Springer, Heidelberg (2005) 5. Zhenfu, C.: New Directions of Modern Cryptography. CRC Press, Boca Raton (2012) 6. Sukhodolskiy, I., Zapechnikov, S.: An access control model for cloud storage using attributebased encryption. In: Proceedings of the 2017 ElConRus, pp. 578–581 (2017) 7. GOST R 34.12-2015 Information technology. Cryptographic protection of information. Block cyphers. Russian state standard. Moscow, Standartinform (in Russian) (2015) 8. Gorlatykh, A., Zapechnikov, S.: Challenges of privacy-preserving OLAP techniques. In: Proceedings of the 2017 ElConRus, pp. 404–408 (2017)

Gamma-Probe for Locating the Source of Ionizing Radiation Jake Hecla1, Timur Khabibullin2, Andrey Starikovskiy2, and Anastasia Tolstaya2(&) 1

National Research Nuclear University MEPhI (Moscow Engineering Physics Institute), Moscow, Russia 2 Massachusetts Institute of Technology, Cambridge, USA [email protected]

Abstract. The radionuclide diagnostics unit, described in the article, detects pathological changes of organs and systems of a person. The device is a portable detector of gamma rays that allows to diagnose superficial malignancies using radiopharmaceuticals injected into the body. The gamma probe uses crystal LaBr3:Ce as a scintillator and silicon photomultiplier SiPM as a photodetector. The focus of this paper is the improvement of the amplifier, which originally produced misshapen pulses unsuitable for energy discrimination. Using LTSPICE, a free circuit-modelling program, we performed extensive simulation of both the SiPM and the amplifier. From this work, we determined that high input impedance and unnecessarily high gain were the source of the distortion. Another amplifier better suited to the SiPM parameters was simulated and then prototyped. Keywords: Gamma-probe
Cancer detection resolution
SiPM
Amplifier
Collimator




Lymph nodes




Detector

1 Introduction Cancer is a generic term for a large group of diseases that can affect any part of the body. Other terms used for that group are malignant tumours and neoplasms. A characteristic feature of cancer is rapid creation of abnormal cells that grow beyond their usual boundaries, and which can invade adjoining parts of the body and spread to other organs. This process is called metastasis. Metastases are the major cause of death from cancer [1, 2]. Cancer is a major cause of morbidity and mortality all over the world: in 2012, 14 million new cases were detected and 8.2 million deaths associated with cancer occurred. According to forecasts, the number of cases of cancer will continue to grow from 14 million to 22 million over the next two decades [3, 4]. According to the World Health Organization, more than 30% of cancer deaths can be prevented by avoiding or changing the main risk factors, which include: tobacco use, overweight or obesity, eating insufficient amounts of fruit and vegetables, physical inactivity, alcohol consumption, ionizing and non-ionizing radiation [1]. Nuclear medicine is an independent branch of beam diagnostics and radiology aimed at, in particular, recognizing pathological changes of organs and systems of a © Springer International Publishing AG 2018 A.V. Samsonovich and V.V. Klimov (eds.), Biologically Inspired Cognitive Architectures (BICA) for Young Scientists, Advances in Intelligent Systems and Computing 636, DOI 10.1007/978-3-319-63940-6_37

258

J. Hecla et al.

person using radiopharmaceuticals (RPh). RPh is a chemical compound, which contains a specific radionuclide in its molecule. Usually short-lived radionuclides of technetium are used. In clinical practice, the following types of radionuclide investigations are implemented: visualization of organs, i.e. getting their radionuclide imaging; measuring the accumulation of radiopharmaceuticals in the body and their elimination; measuring the radioactivity of samples of biological fluids and tissues of the human body and in vitro tests. RPh are able to selectively accumulate in organs and tissues affected by malignant neoplasms, which is what happens after a certain period of time characteristic of this RPh. The radiation emitted by a radioactive isotope, which is part of the RPh can be registered by means of a special device: the count rate of gamma rays will be maximum at the location of the tumour. Thus, it is possible to diagnose the tumour and its location in the body, which is defined as an area of increased concentration of RPh [5, 6]. Diagnostic instruments used in nuclear medicine, usually include a detector, a scanner, a converter unit and a memory block. A special type of such devices are compact gamma-probes designed to identify areas of local accumulation of the radiopharmaceuticals in the body [7–9]. Main applications of gamma-probes are intraoperative search for sentinel lymph nodes and non-invasive body scan of patients to detect superficial malignancies [10–13]. The first method is implemented in the following way: the patient is preoperatively administered with a radiopharmaceutical that accumulates in the hearth of a malignant neoplasm (tumour), and in the network of nearby lymph nodes affected by metastases. The surgeon removes the tumour and then extracts the lymph nodes, which are scanned for the presence of metastases one by one with the gamma probe. Since the network of lymph nodes in the body is a diverging network, their consequent check for metastasis is a valid criterion for the spread of metastasis in the body. This procedure can reduce the invasiveness of tumour removal procedures and save the greatest number of healthy tissue of the patient without the risk of relapse. The second method is completely non-invasive and is an addition to the traditional radionuclide procedures. In some cases (surface location of the tumour or its small diameter) using single-photon or positron emission tomography of the whole body is irrational, since the cost of a procedure is high, but due to the limited number of scanners in the medical centres of the Russian Federation and their bandwidth is small [11]. In such cases, the rational solution is to conduct local radio diagnostic procedures in the vicinity of the area of the tumour, which comprises administering of RPh to a patient and subsequent scanning of the surface of malignancy location zones using a portable gamma-ray detector. The gamma-probe is a device aimed to locate gamma emitters intraoperatively. Several devices currently exist on the market for these purposes [14, 15]. For example, a device called a surgical gamma-probe with TlBr semiconductor for identification of sentinel lymph node is quite known, as described in [16]. This surgical gamma-probe uses a crystal TlBr as a scintillator. The disadvantages of this device are the need for surgical intervention, lack of precision, sensitivity to magnetic fields, lack of immunity to noise of a photomultiplier,

Gamma-Probe for Locating the Source of Ionizing Radiation

259

small light output of the scintillation crystal and dependence of counting rate from the temperature. Another device is described in the article by [17]. This gamma-probe is based on R8900U-00-C12 position-sensitive photomultipliers coupled to the scintillator, which uses CsI (T1) crystal, and a general-purpose parallel collimator. The disadvantages of the device are small light output of the scintillation crystal, sensitivity to magnetic fields, and the counting rate dependence on the temperature. In most cases, a gamma-probe is the main station with an indicator of the radiation intensity and a probe recording gamma rays, which is connected with the station through a wire. The absence of an indicator of radiation intensity on the probe itself is forcing surgeons to often switch their attention from the sensing zone to the display at the main station. This practice not only slows down and complicates the process of tumour search, but also can lead to a loss of the zone of an already detected tumour in the event of external distractions and it can also lead to repeating of the search procedure. Given the shortcomings of existing devices, it was decided to develop a gamma-probe. The gamma-probe has features that significantly improve on the state of the art through the use of SiPM technology developed at the National Research Nuclear University “MEPhI” (NRNU MEPhI), Russia. The article discusses two versions of the gamma-probe: Mk. I and Mk. II.

2 Gamma-Probe Prototype In general, the developed gamma-probe consists of the following components: Collimator. It serves to reduce the angle of gamma rays fixing. It is a metal construction – usually a cylinder with a hole. The main area of improvement – the construction itself, its shape and the shape of the hole. Developments in this area are little, and there is no obvious way to achieve a sharp qualitative leap. Scintillator. It is the main component of the detector. The material, of which it is composed, emits light when capturing gamma rays; the intensity of flashes is directly proportional to the intensity of gamma radiation. The gamma-probes commonly use scintillators LYSO, Lu1.8Y0.2SiO5:Ce), Cadmium Zinc Telluride (CdZnTe), Cadmium Telluride (CdTe), which have the described qualities, but the search for new materials is time consuming and does not give fast efficiency growth. Silicon photomultiplier. It is a matrix of avalanche photodiodes. It allows detecting and amplifying a weak flash of light intensity (at the level of single photons) and having the duration of the order of ones-hundreds of nanoseconds. The ability to recognize the weakest flash defines the accuracy of the boundaries determination of the cancerous tumour, and specialists keep working on the improvement of these qualities. The development of photomultipliers goes towards increasing the number of diodes and reducing the size of the entire matrix. It depends on the size of technology process, and progress in this area is related to the general miniaturization of electronics. Thus,

260

J. Hecla et al.

there are new photomultipliers, more sensitive than those used in common gamma-probes. The new photomultipliers are supposed to improve overall efficiency. Microcontroller. It is a chip responsible for the processing and interpretation of the signals. The element base used in the current generation of gamma-probes is not only outdated, but also requires a lot of production costs, since the period of support of microcontrollers used in the production already ended or will end soon. In most cases, three generations of microcontrollers have already changed since the last solution development. Using the latest developments in this area will not only add functionality and improve usability, but it will also reduce production costs. An example of a specific implementation of the proposed device in a general form is shown in Fig. 1 and comprises: a detecting element (1) placed in the collimator (2), an amplifier (3), a comparator (4), a logic analyser FPGA (field-programmable gate array – circuit logic elements in the programmable operating conditions) (5), data interface (6), a digital-to-analogue converter (DAC) (7), the SPI (Serial Peripheral Interface) bus (8), the power supply circuit (9) and the connection circuit (10). The connection circuit is connected to the switching power supply circuit, which in its turn is connected to each element of the system via the SPI bus.

Fig. 1. The general scheme of the gamma locator.

Fig. 2. LTSpice implementation of the 3  3 MPPC model with values taken from Siefert et al. to reflect a pulse consisting of 2500/14400 cells firing.

The detection element is placed in the collimator and consists of a LaBr3:Ce crystal and a silicon photomultiplier SiPM which is used as a photomultiplier tube. Protection from scattered radiation and background activity surrounds the detecting part at sides and forms a narrow field of view of the detector to improve the ratio “signal – noise”. LaBr3:Ce crystal was selected as a scintillator due to its following features: • high detection efficiency (efficiency level of NaI detection); • non-hygroscopicity of the scintillator, i.e. the possibility to work with it in the open air without additional protection from moisture for the crystal; • high value of effective atomic number;

Gamma-Probe for Locating the Source of Ionizing Radiation

261

• short decay time. SiPM was selected as the photodetector of the gamma-probe selected due to its following features: • the operating voltage is approximately 2 V higher than Ubreak (breakdown voltage) and Ubreak is only tens of Volts; • an excellent signal/noise ratio (SNR) compared to conventional avalanche photodiode; • no damage from excessive light; • durable (rugged) and stable; • minimum requirements for electronics; • small spread gain (less than 10%); • low sensitivity gain to changes in temperature and voltage supply (to temperature change *3% 10 °C, to change of the voltage bias of *1% per 30 mV); • the possibility of registering nanosecond flashes of light without distortion of detected pulse form; • the ability to work both in the pulse counting mode, and in spectrometric mode; • good time resolution (tens of picoseconds); • compact size (the size of the sensitive SiPM area – 1 mm2, 9 mm2, 25 mm2). In the particular case, it is proposed to produce the collimator of lead or tungsten, including the connection circuit for gamma-probe for locating the source of ionizing radiation is: reed sensor; mechanical switch; sensor surface; infrared distance sensor. The scheme of inclusion, represented by one of the above-mentioned methods, is necessary to activate the power supply circuit. The current prototype gains these advantages using a 3  3 Hamamatsu silicon photomultipliers (SiPM) (Microsoft Point-to-Point Compression, MPPC) coupled to a 2  2  15 mm LYSO scintillator. This enables energy resolution across a wide range of emitters, as well as high pulse rates (faster than a gas-tube device or avalanche photodiode (APD)). The SiPM device itself is an array of 25–100 micron self-quenching APDs on a common substrate. Light from a scintillating medium triggers the avalanche of one (or multiple) cells, which creates a mV-scale electrical pulse lasting several hundred nanoseconds. The amplitude of such a pulse is proportional to the number of triggered APDs, and therefore proportional to the number of photons in the pulse of light incident on the SiPM plate. This allows direct correlation between the energy of the gamma rays hitting the scintillator and the amplitude of the pulses from the SiPM. Unlike the competition, this allows the device only to record the impact of gamma rays in a selected energy range if the user wishes. Though extensive documentation exists regarding the modelling of various SiPMs, no manufacturer has made a SPICE (a free program for circuit simulations available from Linear Technologies) model available for researchers. However, it is not a difficult device to simulate. Since the SiPM is a massive array of avalanche photodiodes in parallel, it can be approximately modelled using passive components and a fast rise-time switch to simulate the momentary conduction of a triggered APD. As the physics of the avalanche process are outside of the scope of even the most advanced

262

J. Hecla et al.

circuit simulation programs, this model is only approximate, and does not represent the physics at work. As models of our type of MPPC have been constructed previously, we relied heavily on the work of [18, 19]. The model itself was implemented in LTSPICE. The model in Fig. 2, with limited modifications to adjust pulse height and shape, was used to simulate the amplifier input from the SiPM. To better reflect the behaviour of the real SiPM in the gamma-probe model, a small inductance was added in series with the SiPM output. Even with this addition, the raw SiPM output pulse-height is *20% higher than the simulated value. Qualitatively, the pulse shape is nearly identical. This could be ascribed to a number of factors, but it is not significant enough to warrant further investigation. Excluding the energy-resolution feature, the success of a device like the gamma-probe is determined by its angular selectivity, sensitivity and dynamic range. According to tests performed by Yagnyukova [20] the SiPM and crystal arrangement can achieve angular selectivity better than 26°, which is far better than existing devices. Unfortunately, the device suffers from a high background count rate from the 1.19 MeV decay of Lu-176 in the LYSO scintillator. Further limiting the dynamic range, scatter-background produced by the physical arrangement of the radioisotope within the patient can contribute significantly to noise. Gammas that traverse such a path, however, have necessarily lost energy through Compton scattering [21]. The wavelength shift is given below in Eq. 1. k  k0 ¼

h ð1  cosðhÞÞ; me c

ð1Þ

where k is the initial wavelength; k0 is the wavelength after scattering; h is the Planck constant; me is the electron rest mass; c is the speed of light; h is the scattering angle (Figs. 3 and 4).

Fig. 3. Am-241 (59.5 keV) gamma peak resolved by the gamma-probe Mk. I detector with the Mk. II amplifier. Red line indicates the setting of the backscatter filter.

Fig. 4. A plot of Eq. 1 in the energy domain for .5 MeV photon [22].

These Compton-scattered photons can be removed from the number of counts displayed using energy discrimination. By filtering large-angle or multiple scatters (with consummately large energy loss), the level of spurious counts can be reduced greatly.

Gamma-Probe for Locating the Source of Ionizing Radiation

263

The plot shows the energy of the scattered photon and the ejected electron versus the scattering angle. The other advantages of the SiPM/LYSO system are achieved through the geometry of the scintillator/housing, as well as the use of a self-quenching device (as opposed to an active APD), which gives a higher maximum pulse-rate. Further, by using a RPP crystal with its longest dimension along the axis of the SiPM, proper collimating can be achieved without significant impact on the total mass or footprint of the scintillator/SiPM package.

3 MK. I Device Description Specifications (see Fig. 5):

Fig. 5. Mk. I board with SiPM/LYSO scintillator module installed (black canister on the right).

• • • • • • • • •

Fig. 6. The trace is 1 V/div (amp out).

Mass: 12 h). Below are photos showing pulses from the SiPM in the presence of a radioisotope captured on a Teledyne LeCroy WaveAce 2024 oscilloscope (Fig. 6). Note the time interval is 250 ns/div horizontally. This pulse is longer than ideal (*750 ns), but the amplitude and shape are proper. Amplifier impedance as a function of frequency. Generated with LTSPICE.

264

J. Hecla et al.

Fig. 7. The amplifier design from the Mk. Fig. 8. Amplifier gain as a function of frequency I prototype. when DC gain is set to 2 (top).

The Mk. I device is able to properly register pulses from the SiPM, but lacks pulse-height discrimination ability. As a result of higher-than-optimal gain the amplifier saturates, washing out all pulses larger than a fixed value. This causes obvious problems with energy resolution. Further, the SiPM is capable of pulses with s\100 ns on the tail. However, in the recorded pulses above, s [ 500 ns. This limits the maximum pulse-rate to around 1.3 MHz to avoid pile-up. Note that R1 was later changed to 7.5 k to reduce output-offset voltage (Figs. 7, 8, 9, 10 and 11).

Fig. 9. Amplifier output with simulated SiPM Fig. 10. Scope shot of the amplifier (actually Mk. II) chopping pulses when the gain is set input pulse (see above). excessively high.

Gamma-Probe for Locating the Source of Ionizing Radiation

265

Note that the time and amplitude settings are different from the first image, though the input pulses are near identical. This was the situation with the Mk. I design, but the extreme rarity (1/100 pulses) of saturation events made them hard to capture for an image.

4 MK. II Amplifier Development The development of the next version of the amplifier was informed by two ideas: to prevent signal chopping and to improve the recovery time of the amplifier. This was accomplished by building an amplifier circuit with a much lower input impedance, and with a full 0–3.3 V range (Fig. 12). As in the previous section, all simulations were performed with LTSPICE.

Fig. 11. Image of LTSPICE simulated amplifier saturation in Mk. I amplifier (achieved by boosting the input signal amplitude).

Fig. 12. Mk. II amplifier design.

The same IC (LMH6624) is used, though the LMH6629 has been suggested as a replacement (Figs. 13, 14, 15 and 16).

Fig. 13. Amplifier Mk. II input impedance vs. signal frequency

Fig. 14. Amplifier gain as a function of signal frequency.

Note that this is roughly two orders of magnitude smaller than the Mk. I device. This was conducted with an arbitrarily selected direct current gain of 18 dB (the final value selected was 15 dB).

266

J. Hecla et al.

Fig. 15. Sample output pulse from the amplifier.

Fig. 16. Input (blue) and output (yellow) pulses from the Mk. I SiPM and the Mk. II amplifier.

However, this example pulse was significantly shorter than that observed in the test device, the Mk. II amplifier nonetheless made considerably shorter pulses (*200 ns) than the Mk. I device (500–750 ns). This discrepancy may be due to parasitic capacitance that went unaccounted for in the model. Both are 500 mV/div. The fall-time is *200 ns, and there is no saturation of the signal.

5 Testing and Results The next step after verifying that the amplifier was stable and not chopping pulse amplitudes was to gather a pulse-height spectrum from a known, minimally shielded gamma-emitter. Two sources were available, a Cs-137 source (*1uCi, 662 keV) and an Am-241 source (*.1uCi, 59.5 keV). Mk. II testing consisted of placing the samples as close to the end of the LYSO crystal as possible, then histogramming the pulse heights for 600 s using a Teledyne LeCroy WaveRunner 620Zi. Background spectra were gathered in the same manner (Fig. 17).

Fig. 17. Cs-137 spectrum superimposed on a background count.

The raw spectra showed significant noise, especially at low energies. This was reduced using simple background subtraction.

Gamma-Probe for Locating the Source of Ionizing Radiation

267

Fig. 18. Americium−241 peak. 11.9% full Fig. 19. Cesium-137 peak, 7% FWHM from width at half maximum (FWHM) at 59.5 keV. a 1uCi source (662 keV). 600 s integration time from a 1uCi source.

The background peak at −0.045 was eliminated in the Cs spectrum because the trigger level was bumped up slightly. After the measurements, background was subtracted out, but no smoothing was applied to the data. As usual, Gaussian peaks were fitted to the emitter peaks to measure resolution. The channel spacing was worked out using the known energies of the two emitters and the number of channels between them. For this, the SiPM response and the LYSO energy-photon yield curve was assumed to be linear. The results are shown below (Figs. 18 and 19). Direct comparison of pulse-height histograms gathered from the amplifier input and output shows no significant distortion, excluding that related to the finite maximum pulse height. Lowering the gain of the amplifier could alleviate this problem, but at the cost of poorer channel spacing. This problem is a low-priority issue, however, because the targeted gamma-emitters are below 200 keV characteristic energy, whereas the max pulse height exceeds 1 MeV. The resolution of the detectors is also within the range predicted by prior literature. This indicates proper function, but does not bode well for attempts to improve angular selectivity using scattering-angle. The energy loss for a Tc-99 m photon (140 keV) in a low angle scattering event (