Internet of Things, Infrastructures and Mobile Applications: Proceedings of the 13th IMCL Conference [1st ed.] 9783030499310, 9783030499327

Table of contents :
Front Matter ....Pages i-xxiii
Front Matter ....Pages 1-1
Autism Serious Game Framework (ASGF) for Developing Games for Children with Autism (Geoffrey Gaudi, Bill Kapralos, Alvaro Uribe-Quevedo, Geoffrey Hall, Diana Parvinchi)....Pages 3-12
LinkLearn: Blockchain Technology as a Learning Tool (Dawid Benjamin Jordaan)....Pages 13-24
New Era of the Nano-Electronic Devices – One of the Most Adaptive Learning Areas for the Next Period (Cristian Ravariu, Doru Ursutiu, Dan Mihaiescu, Alina Morosan, Mihai Tanase, Thrasyvoulos Tsiatsos)....Pages 25-35
Open Source Online Conference System for Industry Experts Participation in Education (Dan Robu, Radu Curpen, Daniel Ilie, Titus Balan)....Pages 36-44
Interactive TV and Music Education (Rafail Tzimas, Dimitrios Margounakis, Dionysios Politis, Nektarios-Kyriakos Paris)....Pages 45-53
Front Matter ....Pages 55-55
An Analysis for the Identification of Use and Development of Game Design Strategies as Problem Posing Activities for Early Childhood Learners (George Kalmpourtzis, Margarida Romero, Cindy De Smet, Andreas Veglis)....Pages 57-68
Measuring Knowledge Gains in an SMS m-Learning Intervention: The Case of ChildConnect South Africa (Nicky Roberts, Ingrid Mostert, Lydia-Ann Plaatjies)....Pages 69-80
Measuring Uptake and Engagement in an m-Learning Intervention: The Case of ChildConnect South Africa (Ingrid Mostert, Nicky Roberts, Lydia-Anne Plaatjies)....Pages 81-90
Development of a Classroom Response System: A Web-Based Approach Used in SEPT (Dan Centea, Konstantinos Apostolou, Moein Mehrtash)....Pages 91-101
Analysis of the Perception of Students of the Autonomous University of Baja California Sur for the Use of m-Learning (Jesús Andrés Sandoval Bringas, Mónica Adriana Carreño León, Francisco Javier Alvarez Rodriguez)....Pages 102-108
Work-in-Progress: Development of a Framework for Incorporating Usability Aspects with Digital Didactical Design for Mobile/Tablet Based Learning in Pre-primary Education (Uthpala Samarakoon, Hakim Usoof)....Pages 109-116
Promoting Authentic Student Assessment for STEM Project-Based Learning Activities (Andri Vrioni, Anna Mavroudi, Ioannis Ioannou)....Pages 117-126
Predictive Modeling Concerning Mobile Learning Advance (Malinka Ivanova)....Pages 127-135
Assessing Early Grade Mathematics Learner Outcomes Using m-Learning (Nicky Roberts)....Pages 136-146
Users’ and Experts’ Evaluation of TARGET: A Serious Game for Mitigating Performance Enhancement Culture in Youth (Panagiotis Stylianidis, Agisilaos Chaldogeridis, Nikolaos Politopoulos, Vassilis Barkoukis, Thrasyvoulos Tsiatsos)....Pages 147-157
Poster: Exploring the Educational Affordances of an Academic ePortfolio for Engineer Students Through a Self-regulated Learning Framework (Foteini Paraskeva, Eleni Neofotistou, Angeliki Alafouzou, Aikaterini Alexiou)....Pages 158-166
Front Matter ....Pages 167-167
M-Health as a Tool in the Cognitive Flexibility of the Elderly (Cristina Páez-Quinde, Sonia Armas-Arias, Dorys Cumbe-Coraizaca, Santiago Velastegui-Hernández)....Pages 169-177
Critical Categorization of Android and IOS Applications Available for STEAM Education in Early Childhood (Tharrenos Bratitsis, Michalis Ioannou, George Palaigeorgiou)....Pages 178-188
Museum Exhibits that Interact with Pupils’ Mobile Devices. The Case of Hellenic Maritime Museum (Dimitris Rammos, Tharrenos Bratitsis)....Pages 189-201
Can Elementary Students Co-design the Learning Content of Educational Apps: The We!Design!Fractions Participatory Design Approach (George Palaigeorgiou, Vasiliki Sidiropoulou)....Pages 202-214
Pedagogical Considerations for Mobile-Based Augmented Reality Learning Environments (Betul Czerkawski, Margherita Berti)....Pages 215-222
Machine Learning and Deep Learning: Recent Overview in Medical Care (Nour Elhouda Chalabi, Abdelouahab Attia, Samir Akrouf)....Pages 223-231
Learn to Code, an Interactive Application to Promote Mobile Student-Centred Learning (Anne-Gaelle Colom, Wendy Purdy)....Pages 232-241
Emerging Technologies and Augmented Reality in the Development of Learning and Human Potential (Barba Téllez María Nela, Pullas Tapia Paúl Santiago, Mocha-Bonilla Julio Alfonso, Morales Jaramillo María Belén)....Pages 242-252
Front Matter ....Pages 253-253
Level of Digital Literacies Among Austrian College Students Assessed with an Online Survey (Anita Kloss-Brandstätter, Andreas Pester, Gila Kurtz)....Pages 255-265
A Gamified Educational Network for Collaborative Learning (Andrei B. B. Torres, Bill Kapralos, Alvaro Uribe-Quevedo, Enilda Zea Quero, Adam Dubrowski)....Pages 266-275
Dynamic Mobile Student Response System (Evangelos Sakkopoulos, Pantelis Krasadakis, Rozita Tsoni, Vassilios S. Verykios)....Pages 276-288
Poster: The Use of a Virtual Personal Assistant (FENNChat) as a Platform for Providing Automated Responses to ODL Students’ Queries at UNISA (Chaka Chaka, Tlatso Nkhobo)....Pages 289-296
Poster: Proposal of an Intelligent Remote Tutoring Model (Bounama Gueye, Amadou Dahirou Gueye, Assane Gueye, Omar Kasse, Claude Lishou)....Pages 297-306
A Comparative Study of Augmented Reality Platforms for Building Educational Mobile Applications (George Terzopoulos, Ioannis Kazanidis, Maya Satratzemi, Avgoustos Tsinakos)....Pages 307-316
M-Learning: Are We Ready to Go Mobile? (Ana M. B. Pavani, Guilherme P. Temporão)....Pages 317-326
Framework for Automatic VPN Access to Remotely Discovered Resources (Baboucar Diatta, Cherif Bachir Deme, Adrien Basse, Samuel Ouya)....Pages 327-336
Front Matter ....Pages 337-337
Poster: Learn to Love My Grandchild Design-with-the-User (Lydia-Anne Plaatjies)....Pages 339-346
Enhancing Second Language Listening Skills Through Smartphones: A Case Study (Mallikarjuna Sastry Mallampalli, V. Surya Seshagiri Anumula, Sherine Akkara)....Pages 347-356
Exposing Rural Indian Students to Mobile Assisted Language Learning: A Case Study (Sherine Akkara, Mallikarjuna Sastry Mallampalli, V. Surya Seshagiri Anumula)....Pages 357-366
How Can Facebook Use in Education Be Realized as Crowdsourcing of Learning? an Exploration of Junior, Senior and Graduates Working Together (Christiana Varda, Andri Ioannou)....Pages 367-378
Concept of Digital Competences in Service Training Systems (Sergiy Bronin, Alexander Kuchansky, Andrii Biloshchytskyi, Olga Zinyuk, Volodymyr Kyselov)....Pages 379-388
Means of Cyber Security Aspects Studying in Maritime Specialists Education (Vladlen Shapo, Maksym Levinskyi)....Pages 389-400
Evaluating a Coaching MOOC Course to Support Dual Career of Athletes (Thrasyvoulos Tsiatsos, Nikolaos Politopoulos, Panagiotis Stylianidis, Vasiliki Zilidou, Efthymios Ziagkas, Stella Douka)....Pages 401-409
Front Matter ....Pages 411-411
LoRa Technology Benefits in Educational Institutes (Apostolos Gkamas)....Pages 413-424
Three IoT Wearables in Six European Cities! Reality and Perception (Rasha Ibrahim, Holly Towndrow, Dorothy Monekosso)....Pages 425-436
Work-in-Progress: Designing an e-Coaching System for Chronic Heart Failure Patients (Evdokimos Konstantinidis, Niki Pandria, Antonis Billis, Sophia-Anastasia Mouratoglou, Panagiotis D. Bamidis)....Pages 437-444
A Comparative Examination of AR and Video in Delivering Assembly Instructions (Kaija Petrone, Richard Hanna, G. Shankaranarayanan)....Pages 445-456
Wearable E-Textile as a Narrative Mediator for Enhancing Empathy in Moral Development (George Palaigeorgiou, Grigoria Vroikou, Charoumenou Nikoleta, Tharrenos Bratitsis)....Pages 457-467
RSSI Fingerprinting Techniques for Indoor Localization Datasets (Angelos Chatzimichail, Athina Tsanousa, Georgios Meditskos, Stefanos Vrochidis, Ioannis Kompatsiaris)....Pages 468-479
Mulsemedia Data Representation Based on Multi-image Concept (Yevgeniya Sulema, Abhishek Bhattacharya, Niall Murray)....Pages 480-491
Cryptographic Systems and Threats in e-Commerce (Javier Sánchez Guerrero, Sandra Carrillo Ríos, Darwin García Herrera, Julio Mocha-Bonilla)....Pages 492-506
Work in Progress. SportSWARES, Towards an Intelligent Way of Physical Training (Thrasyvoulos Tsiatsos, Ippokratis Apostolidis, Nikolaos Politopoulos, Agisilaos Chaldogeridis, Ioannis Stamelos)....Pages 507-513
Greek Traditional Dances Capturing and a Kinematic Analysis Approach of the Greek Traditional Dance “Syrtos” (Terpsichore Project) (Efthymios Ziagkas, Vasiliki Zilidou, Andreas Loukovitis, Styliani Douka, Thrasyvoulos Tsiatsos)....Pages 514-523
Front Matter ....Pages 525-525
Educational Mobile Applications on Computational Thinking and Programming for Children Under 8 Years Old (George Terzopoulos, Maya Satratzemi, Despina Tsompanoudi)....Pages 527-538
Work-in-Progress: GameLet: Readers’ Theater in Media-Based Gamification for Reading Skills (Chrystalla Neofytou, Thanasis Hadzilacos, Ute Massler)....Pages 539-546
Students’ Experiences of Learning Mathematics Through Games Design (Wayne Gallear, Petros Lameras, Craig Stewart)....Pages 547-558
A Serious Game for Amplifying Awareness on Multimodal Teaching: Game Design and Usability Study (Petros Lameras, Stephanie Philippe, Lars Oertel)....Pages 559-570
Smart Citizens for Smart Cities – (Benjamin Stelzle, Anja Jannack, Torsten Holmer, Fabrice Naumann, Andreas Wilde, Jörg Rainer Noennig)....Pages 571-581
ADDventurous Rhythmical Planet: A 3D Rhythm-Based Serious Game for Social Skills Development of Children with ADHD (Marina Giannaraki, Nektarios Moumoutzis, Elias Kourkoutas, Katerina Mania)....Pages 582-593
The Design and Development of a Game-Based Approach to Entrepreneurship Education (Ian Dunwell, Petros Lameras)....Pages 594-605
Interactive Serious Games for Cultural Heritage (Dimitrios Margounakis, Themistoklis Karalis, Theodoros Iliou)....Pages 606-617
Mobile Technologies Serious Games for the Development of Social Skills in Children with Autism Spectrum Disorders, in Enhanced with Socially Assistive Robots Interventions (Sofia Pliasa, Nikolaos Fachantidis)....Pages 618-628
Creating Magic-Matt, An Interface to Transform Video Games to a Sports Experience (Nikolaos Politopoulos, Panagiotis Stylianidis, Ippokratis Apostolidis, Agisilaos Chaldogeridis, Angeliki Mavropoulou, Thrasyvoulos Tsiatsos)....Pages 629-637
Front Matter ....Pages 639-639
“The Greek Steelbook (TGS)” The Home of Steelbook Presentations (Nikolaos Giannoulopoulos, Dimitrios Kotsifakos, Christos Douligeris)....Pages 641-651
Poster: Determining a Network and Pedagogical Efficient Approach to Learning in Disruptive Environments (Collins Nnalue Udanor, Agozie H. Eneh, Ogbonna U. Oparaku)....Pages 652-663
An Escape Room Game for Learning Digital Electronics in Vocational Education and Training (VET) (Romanos Dochtsis, Dimitrios Kotsifakos, Christos Douligeris)....Pages 664-674
The e-Facilitator as a Key Player for Interactive Dissemination of STEAM Resources for e-Learning via Webinar (Radoslav Yoshinov, Toni Chehlarova, Monka Kotseva)....Pages 675-686
Work-In-Progress: Interactive Lab Manuals and Videos for a Unit Operations Course (Konstantinos Apostolou, Dan Centea)....Pages 687-692
Towards a Learning Analytics Dashboard for Collaborative Conversational Agent Activities in MOOCs (Stergios Tegos, Thrasyvoulos Tsiatsos, Georgios Psathas, Stavros Demetriadis)....Pages 693-704
Interactive Educational Practices and Distance Learning: A Small Connection with Mobile Learning and the Challenges of Deregulation in Connectivity (Anastasios Nikiforos, Dimos Charidimou, Ioannis Inglezakis)....Pages 705-713
Front Matter ....Pages 715-715
Automatic Source Code Generation from Owl Pseudocode (Baboucar Diatta, Adrien Basse, Chérif Bachir Deme, Samuel Ouya)....Pages 717-725
Gamifying Early Foreign Language Learning (Eleni Korosidou, Tharrenos Bratitsis)....Pages 726-737
Ontology-Based System for Automatic SQL Exercises Generation (Adrien Basse, Baboucar Diatta, Samuel Ouya)....Pages 738-749
Augmented Reality Application Based on Information Barcoding (Ivan Dychka, Olga Sulema, Anton Salenko, Yevgeniya Sulema)....Pages 750-761
Front Matter ....Pages 763-763
Work in Progress: The Impact of the Project OnBoardMed on Development of Study Courses in Maritime Emergency Management (Inese Barbare)....Pages 765-772
An Approach for Supporting Space Orientation of the Blind Using Ontologically-Based Object Map (Dariusz Mikułowski, Marek Pilski)....Pages 773-784
Soupa and Integration of Ontologies Verl for Conceptualizing Contexts in Video Surveillance and Ubiquitous Computing (Susana Arias T, Xavier Arias, Claudia Cartuche, Lozada J. Francisco)....Pages 785-799
A Small Robotic Step for the Therapeutic Treatment of Mental Illnesses (Hector F. Gomez A, Diego Freire, Elena Malo, Francisco Naranjo Cobo)....Pages 800-805
The Use of Gamification in Evaluating Children’s Emotional Intelligence (Abbas Narimani, Ali Khaleghi, Hadi Haedar, Farzad Semnani)....Pages 806-813
Using Gamification Based on Mobile Platform in Therapeutic Interventions for Children with Dyslexia (Mahsa Behnamghader, Ali Khaleghi, Pegah Izadpanah, Farzaneh Rahmani)....Pages 814-824
Combined Approach to Diagnose ADHD: Gamifying Conners Rating Scale (Ali Khaleghi, Fatemeh Heydari, Maedeh Takhttavani, Hadi Haedar, Alireza Soltaninezhad)....Pages 825-835
Using Gamification Based on Virtual Reality Mobile Platform for Treatment of Adults with Amblyopia (Fateme Hosseinnia, Ali Khaleghi, Kamran Mahmoudi)....Pages 836-843
Linear Programming Model Applied to the Optimization of Nutritional Diets for Athletes (Julio A. Mocha-Bonilla, Victor Hugo Guachimbosa, Carolina Guachimbosa Santiago, Javier Sánchez Guerrero)....Pages 844-866
White Blood Cells Detection and Classification Using Convolutional Neural Network (Muaad Hammuda Siala, M. Samir Abou El-Seoud, Gerard McKee)....Pages 867-878
Work-in-Progress: The Use of Big Data and Data Analytics in the Prevention, the Diagnosis and the Monitoring of Long-Term Diseases (Oualid Mecili, Barkat Hadj, Farid Nouioua, Samir Akhrouf, Rachid Malek)....Pages 879-887
An Interactive Augmented Reality Volume Rendering Mobile Application (Amr S. Mady, Samir Abou El-Seoud)....Pages 888-896
Design of an Accessible Web Portal for the Labor Insertion of People with Blindness (Javier Sánchez Guerrero, Julio Alfonso Mocha-Bonilla, Esmeralda Zapata-Mocha, Sandra Carrillo Ríos)....Pages 897-908
Front Matter ....Pages 909-909
MassiveLearning: Online Masterclass Course (Ndeye Massata Ndiaye, Cheikh Ahmadou Lamine Yakhine Diop)....Pages 911-918
Touch Gesture Performance of Kindergarten Children in E-learning Applications: A Case Study in Sri Lanka (Uthpala Samarakoon, Hakim Usoof)....Pages 919-929
Learning Diaries—A Valuable Companion of Mobile Learning for Higher Education in Software Engineering (Sigrid Schefer-Wenzl, Igor Miladinovic)....Pages 930-937
A ‘Small and Thick’ Portrait of Kabelo’s Digital Play (Shafika Isaacs)....Pages 938-949
Use of the Fractal Analysis of Non-stationary Time Series in Mobile Foreign Exchange Trading for M-Learning (A. Kuchansky, A. Biloshchytskyi, S. Bronin, S. Biloshchytska, Yu. Andrashko)....Pages 950-961
Work-in-Progress: SMART-WATER, a Νovel Τelemetry and Remote Control System Infrastructure for the Management of Water Consumption in Thessaloniki (Christos Mourtzios, Dimitrios Kourtesis, Nikolaos Papadimitriou, Gerasimos Antzoulatos, Ioannis-Omiros Kouloglou, Stefanos Vrochidis et al.)....Pages 962-970
Educational Robotics for Creating “Tangible Simulations”: A Mixed Reality Space for Learning the Day/Night Cycle (Stefanos Xefteris, George Palaigeorgiou, Helen Zoumpourtikoudi)....Pages 971-982
TimeTracker App: Facilitating Migrants’ Engagement in Their Second Language Learning (Olga Viberg, Mohammad Khalil, Gustav Bergman)....Pages 983-994
“School – University – Industry” Cooperation (Doru Ursutiu, Cornel Samoila, Patrick Kane, Magdalena Ciurea, Mircea Stremtan, Cristian Ravariu)....Pages 995-1001
PerFECt: A Performative Framework to Establish and Sustain Onlife Communities and Its Use to Design a Mobile App to Extend a Digital Storytelling Platform with New Capabilities (Nektarios Moumoutzis, Alexandros Koukis, Marios Christoulakis, Ioannis Maragkoudakis, Stavros Christodoulakis, Desislava Paneva-Marinova)....Pages 1002-1014
Exploring Impact of Olfactory Stimuli on User Performance on Mobile Platforms (Sergio Caro-Alvaro, Anas Ali Alkasasbeh, Eva García-López, Antonio García-Cabot, Gregor Rozinaj, Gheorghita Ghinea)....Pages 1015-1023
Building a Virtualized Cybersecurity Lab (Titus Bălan, Dan Robu, Florin Sandu, Alexandra Bălan)....Pages 1024-1032
Work-in-Progress: Developing a Master Programme for Specialists in Industry 4.0 (Tom Savu, Andrei Dumitrescu)....Pages 1033-1038
Teachers’ Perceptions Towards the Use of Mobile Augmented Reality (Christina Pasalidou, Nikolaos Fachantidis)....Pages 1039-1050
NavMusApp: Exploring the Instrumental Continuum (Dionysios Politis, Veljko Aleksić, Gregory-Telemachos Stamkopoulos, Georgios Kyriafinis)....Pages 1051-1057
Developing Communities of Practice to Maximize the Usability and Impact of Clean Sport Education in Europe: IMPACT Project (Lambros Lazuras, Antonia Ypsilanti, Vassilis Barkoukis, Panagiotis Stylianidis, Nikolaos Politopoulos, Thrasyvoulos Tsiatsos)....Pages 1058-1064
Front Matter ....Pages 1065-1065
A Fiber Wireless A-RoF/IFoF Uplink Transmission of up to 0.6 Gb/s User Data Rate Over a 32-Element 60 GHz Beam-Steering Antenna for 5G Fronthaul Networks (Eugenio Ruggeri, Apostolos Tsakyridis, Christos Vagionas, George Kalfas, Amalia Miliou, Nikos Pleros et al.)....Pages 1067-1075
An eHealth-Care Driven Perspective on 5G Networks and Infrastructure (Dimitrios Konstantinou, Simon Rommel, Alvaro Morales, Thiago R. Raddo, Ulf Johannsen, Idelfonso Tafur Monroy)....Pages 1076-1088
25 Gb/s Colorless Transmitter Based on Reflective Electroabsorption Modulator for Ultra-Dense WDM-PON Application (Kebede Atra, Giancarlo Cerulo, Jean-Guy Provost, Karim Mekhazni, Cosimo Calo, Frederic Pommereau et al.)....Pages 1089-1100
Challenges of Using Phased Array Antennas in Commercial Backhaul Equipment at 26 GHz (Steven Caicedo, Matteo Oldoni, Stefano Moscato)....Pages 1101-1108
Towards Intelligent Multi-Access Edge Computing Using Machine Learning (Igor Miladinovic, Sigrid Schefer-Wenzl)....Pages 1109-1117
Performance Analysis of NB-IoT Random Access Channel (Subin Narayanan, Dimitris Tsolkas, Nikos Passas, Lazaros Merakos)....Pages 1118-1128
Back Matter ....Pages 1129-1132

Citation preview

Advances in Intelligent Systems and Computing 1192

Michael E. Auer Thrasyvoulos Tsiatsos   Editors

Internet of Things, Infrastructures and Mobile Applications Proceedings of the 13th IMCL Conference

Advances in Intelligent Systems and Computing Volume 1192

Series Editor Janusz Kacprzyk, Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland Advisory Editors Nikhil R. Pal, Indian Statistical Institute, Kolkata, India Rafael Bello Perez, Faculty of Mathematics, Physics and Computing, Universidad Central de Las Villas, Santa Clara, Cuba Emilio S. Corchado, University of Salamanca, Salamanca, Spain Hani Hagras, School of Computer Science and Electronic Engineering, University of Essex, Colchester, UK László T. Kóczy, Department of Automation, Széchenyi István University, Gyor, Hungary Vladik Kreinovich, Department of Computer Science, University of Texas at El Paso, El Paso, TX, USA Chin-Teng Lin, Department of Electrical Engineering, National Chiao Tung University, Hsinchu, Taiwan Jie Lu, Faculty of Engineering and Information Technology, University of Technology Sydney, Sydney, NSW, Australia Patricia Melin, Graduate Program of Computer Science, Tijuana Institute of Technology, Tijuana, Mexico Nadia Nedjah, Department of Electronics Engineering, University of Rio de Janeiro, Rio de Janeiro, Brazil Ngoc Thanh Nguyen , Faculty of Computer Science and Management, Wrocław University of Technology, Wrocław, Poland Jun Wang, Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong

The series “Advances in Intelligent Systems and Computing” contains publications on theory, applications, and design methods of Intelligent Systems and Intelligent Computing. Virtually all disciplines such as engineering, natural sciences, computer and information science, ICT, economics, business, e-commerce, environment, healthcare, life science are covered. The list of topics spans all the areas of modern intelligent systems and computing such as: computational intelligence, soft computing including neural networks, fuzzy systems, evolutionary computing and the fusion of these paradigms, social intelligence, ambient intelligence, computational neuroscience, artificial life, virtual worlds and society, cognitive science and systems, Perception and Vision, DNA and immune based systems, self-organizing and adaptive systems, e-Learning and teaching, human-centered and human-centric computing, recommender systems, intelligent control, robotics and mechatronics including human-machine teaming, knowledge-based paradigms, learning paradigms, machine ethics, intelligent data analysis, knowledge management, intelligent agents, intelligent decision making and support, intelligent network security, trust management, interactive entertainment, Web intelligence and multimedia. The publications within “Advances in Intelligent Systems and Computing” are primarily proceedings of important conferences, symposia and congresses. They cover significant recent developments in the field, both of a foundational and applicable character. An important characteristic feature of the series is the short publication time and world-wide distribution. This permits a rapid and broad dissemination of research results. ** Indexing: The books of this series are submitted to ISI Proceedings, EI-Compendex, DBLP, SCOPUS, Google Scholar and Springerlink **

More information about this series at http://www.springer.com/series/11156

Michael E. Auer Thrasyvoulos Tsiatsos •

Editors

Internet of Things, Infrastructures and Mobile Applications Proceedings of the 13th IMCL Conference

123

Editors Michael E. Auer Carinthia University of Applied Sciences Villach, Austria

Thrasyvoulos Tsiatsos Department of Informatics Aristotle University of Thessaloniki Thessaloniki, Greece

ISSN 2194-5357 ISSN 2194-5365 (electronic) Advances in Intelligent Systems and Computing ISBN 978-3-030-49931-0 ISBN 978-3-030-49932-7 (eBook) https://doi.org/10.1007/978-3-030-49932-7 © Springer Nature Switzerland AG 2021 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Preface

IMCL2019 was the 13th edition of the International Conference on Interactive Mobile Communication, Technologies and Learning. This interdisciplinary conference is part of an international initiative to promote technology-enhanced learning and online engineering worldwide. The IMCL2019 covered all aspects of mobile learning as well as the emergence of mobile communication technologies, infrastructures and services and their implications for education, business, governments and society. The IMCL conference series actually aims to promote the development of mobile learning to provide a forum for education and knowledge transfer, to expose students to latest ICT technologies and encourage the study and implementation of mobile applications in teaching and learning. The conference was also the platform for critical debates on theories, approaches, principles and applications of mobile learning among educators, developers, researchers, practitioners and policy-makers. IMCL2019 has been organized by Aristotle University of Thessaloniki, Greece, from 31 October to 01 November 2019. This year’s theme of the conference was “Internet of Things, Infrastructures and Mobile Applications”. Again, outstanding scientists from around the world accepted the invitation for keynote speeches: • Olga Viberg, KTH Royal Institute of Technology, Sweden: Supporting SelfRegulated Learning with Mobile Learning Analytics. • Ralf Klamma, RWTH Aachen University, Germany: The Future of Learning and Teaching Augmented Reality – A European Perspective. • In addition, two invited speeches have been given by • Ioannis Kompatsiaris, Centre of Research and Technology Hellas—Information Technologies Institute (CERTH-ITI), Greece: Integrating Sensors, Multimedia and Semantic Analysis for Health and Security IoT Applications. • Petros Nicopolitidis, Aristotle University of Thessaloniki, Thessaloniki, Greece: Security issues in Mobile Communications.

v

vi

Preface

Furthermore, one very interesting workshop and one tutorial have been organized: • Tutorial titled “A Gameful Approach Towards Tutors’ Professional Development on Mobile Learning and Interactive Blended Learning” by Anna Mavroudi (Norwegian University of Science and Technology, Norway) & Olga Viberg (KTH Royal Institute of Technology, Sweden). • Workshop titled “5G Networks: Technologies, Challenges, Deployments and Demo” by Thrasyvoulos Spyropoulos (EURECOM, France), Kostas Tsagkaris (Incelligent/Wings ICT Solutions, Greece), Markos Anastasopoulos (University of Bristol, UK) & Evangelos Pikasis (Eulambia Advanced Technologies Ltd, Greece). Since its the beginning, this conference is devoted to new approaches in learning with a focus to mobile learning, mobile communication, mobile technologies and engineering education. We are currently witnessing a significant transformation in the development of working and learning environments with a focus on mobile online communication. Therefore, the following main topics have been discussed during the conference in detail: • Mobile Learning Issues: – – – – – – – – – – – – – – – – – –

Dynamic learning experiences Large-scale adoption of mobile learning Performance support in the workplace Ethical and legal issues Assessment, evaluation and research methods in mobile learning Mobile learning models, theory and pedagogy Lifelong and informal learning using mobile devices Open and distance mobile learning Social implications of mobile learning Design of adaptive mobile learning environments Cost-effective management of mobile learning processes Quality in mobile learning Case studies in mobile learning Interactive Communication Technologies and Infrastructures: Wearables & Internet of Things (IoT) Tangible, embedded and embodied interaction Location-based integration Cloud computing and future Internet research and experimentation (fire) environments – Emerging mobile technologies and standards

Preface

vii

– Interactive and collaborative mobile learning environments – Crowd sensing – 5G network Infrastructure • Mobile Applications: – – – – – – – – –

Smart cities Online laboratories Game-based learning Mobile health care and training Learning analytics Mobile learning in cultural institutions and open spaces Mobile systems and services for opening up education Social networking applications Mobile Learning Management Systems (mLMS)

The following special sessions have been organized: • Designing and Developing Mobile Serious Games for Augmenting Arts and STEM Competencies, Capabilities and Skills (DG-STEAM) • University–Industry–Cooperation in Mobile Technologies (UIC-MT) • Mixed Reality Applications for Industry and Education (MIRINDE) • Digital Technology in Sports Program Committee (DiTeS) • 5G Wireless and Optical Technologies for Mobile Communication Systems (5G Fi-Wi for MC) • Social Networks and Mobile Applications for Health (SNMAH) • Interactive Learning Interfaces for Music Education (iLIME’2019) Also, the “3rd IMCL International Student Competition for Mobile Apps” has been organized in the context of IMCL2019. The winning team of the competition presented “Magic-Matt, An Interface To Transform Video Games To A Sports Experience” and were composed by Nikolaos Politopoulos, Agisilaos Chaldogeridis, Hippokratis Apostolidis, Panagiotis Stylianidis, Angeliki Mavropoulou by Aristotle University of Thessaloniki, Greece, presenting the As submission types have been accepted: • • • •

Full paper, short paper, distant/pre-recorded presentation Work in progress, poster Special sessions Round-table discussions, workshops, tutorials and students’ competition

All contributions were subject to a double-blind review. The review process was very competitive. We had to review about 250 submissions. A team of about 160 reviewers did this terrific job. Our special thanks go to all of them.

viii

Preface

Due to the time and conference schedule restrictions, we could finally accept only the best 105 submissions for presentation. Our conference had again more than 175 participants from 31 countries. IMCL2021 will be held again at Aristotle University of Thessaloniki, Greece. Michael E. Auer IMCL Steering Committee Chair Thrasyvoulos Tsiatsos IMCL General Chair

Organization

Committees Steering Committee Chair Michael E. Auer

CTI, Villach, Austria

General Conference Chair Thrasyvoulos Tsiatsos

Aristotle University of Thessaloniki, Greece

International Chairs Samir A. El-Seoud Neelakshi C. Premawardhena Alexander Kist Alaa Ashmawy David Guralnick

The British University in Egypt, Africa University of Kelaniya, Sri Lanka, Asia University of Southern Queensland, Australia, Australia/Oceania American University Dubai, Middle East Kaleidoscope Learning, New York, USA, North America

Technical Program Chairs Ioannis Stamelos Stavros Demetriadis Sebastian Schreiter

Aristotle University of Thessaloniki, Greece Aristotle University of Thessaloniki, Greece IAOE, France

IEEE Liaison Russ Meier

IEEE Education Society Meetings Chair

ix

x

Organization

Workshop, Tutorial and Special Sessions Chair Andreas Pester

Carinthia University of Applied Sciences, Villach, Austria

Publication Chair Sebastian Schreiter

IAOE, France

Local Organization Chair Stella Douka

Aristotle University of Thessaloniki, Greece

Local Organization Committee Members Christos Temertzoglou Vasiliki Peana

Aristotle University of Thessaloniki, Greece Aristotle University of Thessaloniki, Greece

Program Committee Members Abul Azad Achilles Kameas Agisilaos Konidaris Alexander Chatzigeorgiou Anastasios Economides Anastasios Karakostas Anastasios Mikropoulos Andreas Veglis Apostolos Gkamas Barbara Kerr Carlos Travieso-Gonzalez Charalampos Karagiannidis Christos Bouras Christos Katsanos Christos Douligeris Christos Georgiadis Christos Panagiotakopoulos Christos Pierrakeas

Cleo Sgouropoulou Costas Mourlas

Northern Illinois University, USA Hellenic Open University, Greece Technological Educational Institute of Ionian Islands, Greece University of Macedonia, Greece University of Macedonia, Greece Information Technologies Institute, Greece University of Ioannina, Greece Aristotle University of Thessaloniki, Greece University Ecclesiastical Academy of Vella of Ioannina, Greece Ottawa University, Canada Universidad de Las Palmas de Gran. Canaria, Spain University of Thessaly, Greece University of Patras, Greece Aristotle University of Thessaloniki, Greece University of Piraeus, Greece University of Macedonia, Greece University of Patras, Greece Technological Educational Institute (TEI) of Western Greece and Hellenic Open University, Greece Technological Educational Institute of Athens, Greece National and Kapodistrian University of Athens, Greece

Organization

Daphne Economou Demetrios Sampson Despo Ktoridou Dieter Wuttke Dimitrios Kalles Dimitris Alimisis Dimitris Gouscos Dionysios Politis Doru Ursutiu George Ioannidis George Magoulas George Palaigeorgiou Giasemi Vavoula Golberi S. Ferreira Helen Karatza Ioannis Vogiatzis Khitam Shraim Kostas Apostolou Maiga Chang Manuel Castro Maya Satratzemi Maria Meletiou-Mavrotheris Michail Giannakos Michalis Xenos Minjuan Wang

Monica Divitini Nektarios Moumoutzis Nikolaos Avouris Nikolaos Samaras Nikolaos Tselios Panagiotis Bamidis Panagiotis Petridis Panagiotis Politis Petros Lameras Petros Nicopolitidis Rhena Delport Santi Caballé Stelios Xinogalos

xi

University of Westminster, UK University of Pireaus, Greece University of Nicosia, Cyprus Technical University Ilmenau, Germany Hellenic Open University, Greece Edumotiva, Greece National and Kapodistrian University of Athens, Greece Aristotle University of Thessaloniki, Greece University Transylvania Brasov, Romania Patras University, Greece Birkbeck College, UK University of Western Macedonia, Greece University of Leicester, UK CEFET/SC, Brazil Aristotle University of Thessaloniki, Greece Technological Educational Institute of Athens, Greece Palestine Technical University, Palestine McMaster University, Canada Athabasca University, Canada Universidad Nacional de Educación a Distancia, Spain University of Macedonia, Greece European University Cyprus, Cyprus Norwegian University of Science and Technology, Norway University of Patras, Greece Shanghai International Studies University (Oriental Scholar); San Diego State University, USA Norwegian University of Science and Technology, Norway Technical University of Crete, Greece University of Patras, Greece University of Macedonia, Greece University of Patras, Greece Aristotle University of Thessaloniki, Greece Aston University, UK University of Thessaly, Greece The Serious Games Institute, UK Aristotle University of Thessaloniki, Greece University of Pretoria, South Africa Open University of Catalonia, Spain University of Macedonia, Greece

xii

Stavros Demetriadis Stavros Nikou Symeon Retalis Tharenos Bratitsis Ting-Ting Wu Vassilis Komis

Organization

Aristotle University of Thessaloniki, Greece University of Strathclyde, UK University of Piraeus, Greece University of Western Macedonia, Greece National Yunlin University of Science and Technology, Taiwan University of Patras, Greece

3rd IMCL Student International Competition for Mobile Apps Chairs Andreas Pester Ioannis Stamelos

Carinthia University of Applied Sciences, Austria Aristotle University of Thessaloniki, Greece

Judges Petros Nikopolitidis Teresa Restivo Ilias Trohidis Athena Vakali George Palaigeorgiou

Aristotle University of Thessaloniki, Greece University of Porto, Portugal Tero Consulting, Greece Aristotle University of Thessaloniki, Greece University of Western Macedonia, Greece

Contents

Interactive and Collaborative Mobile Learning Environments Autism Serious Game Framework (ASGF) for Developing Games for Children with Autism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Geoffrey Gaudi, Bill Kapralos, Alvaro Uribe-Quevedo, Geoffrey Hall, and Diana Parvinchi LinkLearn: Blockchain Technology as a Learning Tool . . . . . . . . . . . . . Dawid Benjamin Jordaan

3

13

New Era of the Nano-Electronic Devices – One of the Most Adaptive Learning Areas for the Next Period . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cristian Ravariu, Doru Ursutiu, Dan Mihaiescu, Alina Morosan, Mihai Tanase, and Thrasyvoulos Tsiatsos

25

Open Source Online Conference System for Industry Experts Participation in Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dan Robu, Radu Curpen, Daniel Ilie, and Titus Balan

36

Interactive TV and Music Education . . . . . . . . . . . . . . . . . . . . . . . . . . . Rafail Tzimas, Dimitrios Margounakis, Dionysios Politis, and Nektarios-Kyriakos Paris

45

Assessment, Evaluation and Research Methods in Mobile Learning An Analysis for the Identification of Use and Development of Game Design Strategies as Problem Posing Activities for Early Childhood Learners . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . George Kalmpourtzis, Margarida Romero, Cindy De Smet, and Andreas Veglis

57

xiii

xiv

Contents

Measuring Knowledge Gains in an SMS m-Learning Intervention: The Case of ChildConnect South Africa . . . . . . . . . . . . . . . . . . . . . . . . . Nicky Roberts, Ingrid Mostert, and Lydia-Ann Plaatjies

69

Measuring Uptake and Engagement in an m-Learning Intervention: The Case of ChildConnect South Africa . . . . . . . . . . . . . . . . . . . . . . . . . Ingrid Mostert, Nicky Roberts, and Lydia-Anne Plaatjies

81

Development of a Classroom Response System: A Web-Based Approach Used in SEPT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dan Centea, Konstantinos Apostolou, and Moein Mehrtash

91

Analysis of the Perception of Students of the Autonomous University of Baja California Sur for the Use of m-Learning . . . . . . . . . . . . . . . . . 102 Jesús Andrés Sandoval Bringas, Mónica Adriana Carreño León, and Francisco Javier Alvarez Rodriguez Work-in-Progress: Development of a Framework for Incorporating Usability Aspects with Digital Didactical Design for Mobile/Tablet Based Learning in Pre-primary Education . . . . . . . . . . . . . . . . . . . . . . . 109 Uthpala Samarakoon and Hakim Usoof Promoting Authentic Student Assessment for STEM Project-Based Learning Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 Andri Vrioni, Anna Mavroudi, and Ioannis Ioannou Predictive Modeling Concerning Mobile Learning Advance . . . . . . . . . . 127 Malinka Ivanova Assessing Early Grade Mathematics Learner Outcomes Using m-Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 Nicky Roberts Users’ and Experts’ Evaluation of TARGET: A Serious Game for Mitigating Performance Enhancement Culture in Youth . . . . . . . . . 147 Panagiotis Stylianidis, Agisilaos Chaldogeridis, Nikolaos Politopoulos, Vassilis Barkoukis, and Thrasyvoulos Tsiatsos Poster: Exploring the Educational Affordances of an Academic ePortfolio for Engineer Students Through a Self-regulated Learning Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 Foteini Paraskeva, Eleni Neofotistou, Angeliki Alafouzou, and Aikaterini Alexiou Mobile Learning Models, Theory and Pedagogy M-Health as a Tool in the Cognitive Flexibility of the Elderly . . . . . . . . 169 Cristina Páez-Quinde, Sonia Armas-Arias, Dorys Cumbe-Coraizaca, and Santiago Velastegui-Hernández

Contents

xv

Critical Categorization of Android and IOS Applications Available for STEAM Education in Early Childhood . . . . . . . . . . . . . . . . . . . . . . 178 Tharrenos Bratitsis, Michalis Ioannou, and George Palaigeorgiou Museum Exhibits that Interact with Pupils’ Mobile Devices. The Case of Hellenic Maritime Museum . . . . . . . . . . . . . . . . . . . . . . . . 189 Dimitris Rammos and Tharrenos Bratitsis Can Elementary Students Co-design the Learning Content of Educational Apps: The We!Design!Fractions Participatory Design Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202 George Palaigeorgiou and Vasiliki Sidiropoulou Pedagogical Considerations for Mobile-Based Augmented Reality Learning Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215 Betul Czerkawski and Margherita Berti Machine Learning and Deep Learning: Recent Overview in Medical Care . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 Nour Elhouda Chalabi, Abdelouahab Attia, and Samir Akrouf Learn to Code, an Interactive Application to Promote Mobile Student-Centred Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232 Anne-Gaelle Colom and Wendy Purdy Emerging Technologies and Augmented Reality in the Development of Learning and Human Potential . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242 Barba Téllez María Nela, Pullas Tapia Paúl Santiago, Mocha-Bonilla Julio Alfonso, and Morales Jaramillo María Belén Open and Distance Mobile Learning Level of Digital Literacies Among Austrian College Students Assessed with an Online Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255 Anita Kloss-Brandstätter, Andreas Pester, and Gila Kurtz A Gamified Educational Network for Collaborative Learning . . . . . . . . 266 Andrei B. B. Torres, Bill Kapralos, Alvaro Uribe-Quevedo, Enilda Zea Quero, and Adam Dubrowski Dynamic Mobile Student Response System . . . . . . . . . . . . . . . . . . . . . . 276 Evangelos Sakkopoulos, Pantelis Krasadakis, Rozita Tsoni, and Vassilios S. Verykios Poster: The Use of a Virtual Personal Assistant (FENNChat) as a Platform for Providing Automated Responses to ODL Students’ Queries at UNISA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289 Chaka Chaka and Tlatso Nkhobo

xvi

Contents

Poster: Proposal of an Intelligent Remote Tutoring Model . . . . . . . . . . 297 Bounama Gueye, Amadou Dahirou Gueye, Assane Gueye, Omar Kasse, and Claude Lishou A Comparative Study of Augmented Reality Platforms for Building Educational Mobile Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307 George Terzopoulos, Ioannis Kazanidis, Maya Satratzemi, and Avgoustos Tsinakos M-Learning: Are We Ready to Go Mobile? . . . . . . . . . . . . . . . . . . . . . . 317 Ana M. B. Pavani and Guilherme P. Temporão Framework for Automatic VPN Access to Remotely Discovered Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327 Baboucar Diatta, Cherif Bachir Deme, Adrien Basse, and Samuel Ouya Life-Long and Informal Learning Using Mobile Devices Poster: Learn to Love My Grandchild Design-with-the-User . . . . . . . . . 339 Lydia-Anne Plaatjies Enhancing Second Language Listening Skills Through Smartphones: A Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347 Mallikarjuna Sastry Mallampalli, V. Surya Seshagiri Anumula, and Sherine Akkara Exposing Rural Indian Students to Mobile Assisted Language Learning: A Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357 Sherine Akkara, Mallikarjuna Sastry Mallampalli, and V. Surya Seshagiri Anumula How Can Facebook Use in Education Be Realized as Crowdsourcing of Learning? an Exploration of Junior, Senior and Graduates Working Together . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367 Christiana Varda and Andri Ioannou Concept of Digital Competences in Service Training Systems . . . . . . . . 379 Sergiy Bronin, Alexander Kuchansky, Andrii Biloshchytskyi, Olga Zinyuk, and Volodymyr Kyselov Means of Cyber Security Aspects Studying in Maritime Specialists Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389 Vladlen Shapo and Maksym Levinskyi Evaluating a Coaching MOOC Course to Support Dual Career of Athletes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401 Thrasyvoulos Tsiatsos, Nikolaos Politopoulos, Panagiotis Stylianidis, Vasiliki Zilidou, Efthymios Ziagkas, and Stella Douka

Contents

xvii

Wearables and Internet of Things (IoT) LoRa Technology Benefits in Educational Institutes . . . . . . . . . . . . . . . . 413 Apostolos Gkamas Three IoT Wearables in Six European Cities! Reality and Perception . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425 Rasha Ibrahim, Holly Towndrow, and Dorothy Monekosso Work-in-Progress: Designing an e-Coaching System for Chronic Heart Failure Patients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437 Evdokimos Konstantinidis, Niki Pandria, Antonis Billis, Sophia-Anastasia Mouratoglou, and Panagiotis D. Bamidis A Comparative Examination of AR and Video in Delivering Assembly Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445 Kaija Petrone, Richard Hanna, and G. Shankaranarayanan Wearable E-Textile as a Narrative Mediator for Enhancing Empathy in Moral Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457 George Palaigeorgiou, Grigoria Vroikou, Charoumenou Nikoleta, and Tharrenos Bratitsis RSSI Fingerprinting Techniques for Indoor Localization Datasets . . . . . 468 Angelos Chatzimichail, Athina Tsanousa, Georgios Meditskos, Stefanos Vrochidis, and Ioannis Kompatsiaris Mulsemedia Data Representation Based on Multi-image Concept . . . . . 480 Yevgeniya Sulema, Abhishek Bhattacharya, and Niall Murray Cryptographic Systems and Threats in e-Commerce . . . . . . . . . . . . . . . 492 Javier Sánchez Guerrero, Sandra Carrillo Ríos, Darwin García Herrera, and Julio Mocha-Bonilla Work in Progress. SportSWARES, Towards an Intelligent Way of Physical Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 507 Thrasyvoulos Tsiatsos, Ippokratis Apostolidis, Nikolaos Politopoulos, Agisilaos Chaldogeridis, and Ioannis Stamelos Greek Traditional Dances Capturing and a Kinematic Analysis Approach of the Greek Traditional Dance “Syrtos” (Terpsichore Project) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514 Efthymios Ziagkas, Vasiliki Zilidou, Andreas Loukovitis, Styliani Douka, and Thrasyvoulos Tsiatsos Game Based Learning Educational Mobile Applications on Computational Thinking and Programming for Children Under 8 Years Old . . . . . . . . . . . . . . . . 527 George Terzopoulos, Maya Satratzemi, and Despina Tsompanoudi

xviii

Contents

Work-in-Progress: GameLet: Readers’ Theater in Media-Based Gamification for Reading Skills . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 539 Chrystalla Neofytou, Thanasis Hadzilacos, and Ute Massler Students’ Experiences of Learning Mathematics Through Games Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 547 Wayne Gallear, Petros Lameras, and Craig Stewart A Serious Game for Amplifying Awareness on Multimodal Teaching: Game Design and Usability Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 559 Petros Lameras, Stephanie Philippe, and Lars Oertel Smart Citizens for Smart Cities – . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 571 Benjamin Stelzle, Anja Jannack, Torsten Holmer, Fabrice Naumann, Andreas Wilde, and Jörg Rainer Noennig ADDventurous Rhythmical Planet: A 3D Rhythm-Based Serious Game for Social Skills Development of Children with ADHD . . . . . . . . 582 Marina Giannaraki, Nektarios Moumoutzis, Elias Kourkoutas, and Katerina Mania The Design and Development of a Game-Based Approach to Entrepreneurship Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 594 Ian Dunwell and Petros Lameras Interactive Serious Games for Cultural Heritage . . . . . . . . . . . . . . . . . . 606 Dimitrios Margounakis, Themistoklis Karalis, and Theodoros Iliou Mobile Technologies Serious Games for the Development of Social Skills in Children with Autism Spectrum Disorders, in Enhanced with Socially Assistive Robots Interventions . . . . . . . . . . . . . . . . . . . . . . 618 Sofia Pliasa and Nikolaos Fachantidis Creating Magic-Matt, An Interface to Transform Video Games to a Sports Experience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 629 Nikolaos Politopoulos, Panagiotis Stylianidis, Ippokratis Apostolidis, Agisilaos Chaldogeridis, Angeliki Mavropoulou, and Thrasyvoulos Tsiatsos Dynamic Learning Experiences “The Greek Steelbook (TGS)” The Home of Steelbook Presentations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 641 Nikolaos Giannoulopoulos, Dimitrios Kotsifakos, and Christos Douligeris Poster: Determining a Network and Pedagogical Efficient Approach to Learning in Disruptive Environments . . . . . . . . . . . . . . . . . . . . . . . . 652 Collins Nnalue Udanor, Agozie H. Eneh, and Ogbonna U. Oparaku

Contents

xix

An Escape Room Game for Learning Digital Electronics in Vocational Education and Training (VET) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 664 Romanos Dochtsis, Dimitrios Kotsifakos, and Christos Douligeris The e-Facilitator as a Key Player for Interactive Dissemination of STEAM Resources for e-Learning via Webinar . . . . . . . . . . . . . . . . . 675 Radoslav Yoshinov, Toni Chehlarova, and Monka Kotseva Work-In-Progress: Interactive Lab Manuals and Videos for a Unit Operations Course . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 687 Konstantinos Apostolou and Dan Centea Towards a Learning Analytics Dashboard for Collaborative Conversational Agent Activities in MOOCs . . . . . . . . . . . . . . . . . . . . . . 693 Stergios Tegos, Thrasyvoulos Tsiatsos, Georgios Psathas, and Stavros Demetriadis Interactive Educational Practices and Distance Learning: A Small Connection with Mobile Learning and the Challenges of Deregulation in Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 705 Anastasios Nikiforos, Dimos Charidimou, and Ioannis Inglezakis Mobile Systems and Services for Opening Up Education Automatic Source Code Generation from Owl Pseudocode . . . . . . . . . . 717 Baboucar Diatta, Adrien Basse, Chérif Bachir Deme, and Samuel Ouya Gamifying Early Foreign Language Learning . . . . . . . . . . . . . . . . . . . . 726 Eleni Korosidou and Tharrenos Bratitsis Ontology-Based System for Automatic SQL Exercises Generation . . . . . 738 Adrien Basse, Baboucar Diatta, and Samuel Ouya Augmented Reality Application Based on Information Barcoding . . . . . 750 Ivan Dychka, Olga Sulema, Anton Salenko, and Yevgeniya Sulema Mobile Health Care and Training Work in Progress: The Impact of the Project OnBoardMed on Development of Study Courses in Maritime Emergency Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 765 Inese Barbare An Approach for Supporting Space Orientation of the Blind Using Ontologically-Based Object Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 773 Dariusz Mikułowski and Marek Pilski Soupa and Integration of Ontologies Verl for Conceptualizing Contexts in Video Surveillance and Ubiquitous Computing . . . . . . . . . . 785 Susana Arias T, Xavier Arias, Claudia Cartuche, and Lozada J. Francisco

xx

Contents

A Small Robotic Step for the Therapeutic Treatment of Mental Illnesses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 800 Hector F. Gomez A, Diego Freire, Elena Malo, and Francisco Naranjo Cobo The Use of Gamification in Evaluating Children’s Emotional Intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 806 Abbas Narimani, Ali Khaleghi, Hadi Haedar, and Farzad Semnani Using Gamification Based on Mobile Platform in Therapeutic Interventions for Children with Dyslexia . . . . . . . . . . . . . . . . . . . . . . . . 814 Mahsa Behnamghader, Ali Khaleghi, Pegah Izadpanah, and Farzaneh Rahmani Combined Approach to Diagnose ADHD: Gamifying Conners Rating Scale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 825 Ali Khaleghi, Fatemeh Heydari, Maedeh Takhttavani, Hadi Haedar, and Alireza Soltaninezhad Using Gamification Based on Virtual Reality Mobile Platform for Treatment of Adults with Amblyopia . . . . . . . . . . . . . . . . . . . . . . . . 836 Fateme Hosseinnia, Ali Khaleghi, and Kamran Mahmoudi Linear Programming Model Applied to the Optimization of Nutritional Diets for Athletes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 844 Julio A. Mocha-Bonilla, Victor Hugo Guachimbosa, Carolina Guachimbosa Santiago, and Javier Sánchez Guerrero White Blood Cells Detection and Classification Using Convolutional Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 867 Muaad Hammuda Siala, M. Samir Abou El-Seoud, and Gerard McKee Work-in-Progress: The Use of Big Data and Data Analytics in the Prevention, the Diagnosis and the Monitoring of Long-Term Diseases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 879 Oualid Mecili, Barkat Hadj, Farid Nouioua, Samir Akhrouf, and Rachid Malek An Interactive Augmented Reality Volume Rendering Mobile Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 888 Amr S. Mady and Samir Abou El-Seoud Design of an Accessible Web Portal for the Labor Insertion of People with Blindness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 897 Javier Sánchez Guerrero, Julio Alfonso Mocha-Bonilla, Esmeralda Zapata-Mocha, and Sandra Carrillo Ríos

Contents

xxi

Case Studies in Mobile Learning MassiveLearning: Online Masterclass Course . . . . . . . . . . . . . . . . . . . . 911 Ndeye Massata Ndiaye and Cheikh Ahmadou Lamine Yakhine Diop Touch Gesture Performance of Kindergarten Children in E-learning Applications: A Case Study in Sri Lanka . . . . . . . . . . . . . . . . . . . . . . . . 919 Uthpala Samarakoon and Hakim Usoof Learning Diaries—A Valuable Companion of Mobile Learning for Higher Education in Software Engineering . . . . . . . . . . . . . . . . . . . 930 Sigrid Schefer-Wenzl and Igor Miladinovic A ‘Small and Thick’ Portrait of Kabelo’s Digital Play . . . . . . . . . . . . . . 938 Shafika Isaacs Use of the Fractal Analysis of Non-stationary Time Series in Mobile Foreign Exchange Trading for M-Learning . . . . . . . . . . . . . . . . . . . . . . 950 A. Kuchansky, A. Biloshchytskyi, S. Bronin, S. Biloshchytska, and Yu. Andrashko Work-in-Progress: SMART-WATER, a Novel Telemetry and Remote Control System Infrastructure for the Management of Water Consumption in Thessaloniki . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 962 Christos Mourtzios, Dimitrios Kourtesis, Nikolaos Papadimitriou, Gerasimos Antzoulatos, Ioannis-Omiros Kouloglou, Stefanos Vrochidis, and Kompatsiaris Ioannis Educational Robotics for Creating “Tangible Simulations”: A Mixed Reality Space for Learning the Day/Night Cycle . . . . . . . . . . . 971 Stefanos Xefteris, George Palaigeorgiou, and Helen Zoumpourtikoudi TimeTracker App: Facilitating Migrants’ Engagement in Their Second Language Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 983 Olga Viberg, Mohammad Khalil, and Gustav Bergman “School – University – Industry” Cooperation . . . . . . . . . . . . . . . . . . . . 995 Doru Ursutiu, Cornel Samoila, Patrick Kane, Magdalena Ciurea, Mircea Stremtan, and Cristian Ravariu PerFECt: A Performative Framework to Establish and Sustain Onlife Communities and Its Use to Design a Mobile App to Extend a Digital Storytelling Platform with New Capabilities . . . . . . . . . . . . . . . . . . . . . . 1002 Nektarios Moumoutzis, Alexandros Koukis, Marios Christoulakis, Ioannis Maragkoudakis, Stavros Christodoulakis, and Desislava Paneva-Marinova

xxii

Contents

Exploring Impact of Olfactory Stimuli on User Performance on Mobile Platforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1015 Sergio Caro-Alvaro, Anas Ali Alkasasbeh, Eva García-López, Antonio García-Cabot, Gregor Rozinaj, and Gheorghita Ghinea Building a Virtualized Cybersecurity Lab . . . . . . . . . . . . . . . . . . . . . . . 1024 Titus Bălan, Dan Robu, Florin Sandu, and Alexandra Bălan Work-in-Progress: Developing a Master Programme for Specialists in Industry 4.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1033 Tom Savu and Andrei Dumitrescu Teachers’ Perceptions Towards the Use of Mobile Augmented Reality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1039 Christina Pasalidou and Nikolaos Fachantidis NavMusApp: Exploring the Instrumental Continuum . . . . . . . . . . . . . . 1051 Dionysios Politis, Veljko Aleksić, Gregory-Telemachos Stamkopoulos, and Georgios Kyriafinis Developing Communities of Practice to Maximize the Usability and Impact of Clean Sport Education in Europe: IMPACT Project . . . 1058 Lambros Lazuras, Antonia Ypsilanti, Vassilis Barkoukis, Panagiotis Stylianidis, Nikolaos Politopoulos, and Thrasyvoulos Tsiatsos 5G Network Infrastructure A Fiber Wireless A-RoF/IFoF Uplink Transmission of up to 0.6 Gb/s User Data Rate Over a 32-Element 60 GHz Beam-Steering Antenna for 5G Fronthaul Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . 1067 Eugenio Ruggeri, Apostolos Tsakyridis, Christos Vagionas, George Kalfas, Amalia Miliou, Nikos Pleros, and Yigal Leiba An eHealth-Care Driven Perspective on 5G Networks and Infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1076 Dimitrios Konstantinou, Simon Rommel, Alvaro Morales, Thiago R. Raddo, Ulf Johannsen, and Idelfonso Tafur Monroy 25 Gb/s Colorless Transmitter Based on Reflective Electroabsorption Modulator for Ultra-Dense WDM-PON Application . . . . . . . . . . . . . . . 1089 Kebede Atra, Giancarlo Cerulo, Jean-Guy Provost, Karim Mekhazni, Cosimo Calo, Frederic Pommereau, Carmen Gomez, Catherine Fortin, Jean Decobert, Florence Martin, Estelle Derouin, Christophe Caillaud, Didier Erasme, Cédric Ware, Franck Mallecot, and Mohand Achouche Challenges of Using Phased Array Antennas in Commercial Backhaul Equipment at 26 GHz . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1101 Steven Caicedo, Matteo Oldoni, and Stefano Moscato

Contents

xxiii

Towards Intelligent Multi-Access Edge Computing Using Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1109 Igor Miladinovic and Sigrid Schefer-Wenzl Performance Analysis of NB-IoT Random Access Channel . . . . . . . . . . 1118 Subin Narayanan, Dimitris Tsolkas, Nikos Passas, and Lazaros Merakos Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1129

Interactive and Collaborative Mobile Learning Environments

Autism Serious Game Framework (ASGF) for Developing Games for Children with Autism Geoffrey Gaudi1 , Bill Kapralos1(B) , Alvaro Uribe-Quevedo1 , Geoffrey Hall2 , and Diana Parvinchi2 1

Ontario Tech University, Oshawa, ON, Canada [email protected], {bill.kapralos,alvaro.quevedo}@uoit.ca 2 McMaster University, Hamilton, ON, Canada {hallg,parvind}@mcmaster.ca

Abstract. Given their ability to motivate and engage users, digital media, and games in particular, are being adopted on a mass scale in a wide area of applications including education, training, health care, and therapy, amongst others. Traditional therapy for children diagnosed with autism comprises a set of activities aimed at improving behavioral traits often associated with social and communication skills. However, the therapy process is challenging as every child responds differently to the intervention. There are usually long wait lists for treatment and there are high demands for the time clinician’s can provide for treatment. Games, and serious games in particular, that can be easily tailored to the needs of specific individuals, and that can span a wide range of scenarios, are proving to be effective alternatives to traditional therapy. Here we present an autism serious game framework (ASGF) whose goal is to provide therapists with a simple interface that will allow them to develop various serious game interventions. Using the ASGF, we have developed two games, one whose goal is to help users identify emotions by matching facial expressions such as happiness, anger, fear, or sadness, and another, which focuses on response inhibition. The ASGF employs a graphical user interface-based, What You See Is What You Get interface, that simplifies the development of autism serious games. Keywords: Autism

1

· Serious games · Framework

Introduction

Serious games (SGs) are “games that do not have entertainment, enjoyment, or fun as their primary purpose” [9]. SGs take advantage of game elements to create engaging, fun, and challenging scenarios that motivate players to achieve the The support of the Social Sciences and Humanities Research Council of Canada, and the Natural Sciences and Engineering Research Council of Canada is gratefully acknowledged. c Springer Nature Switzerland AG 2021  M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 3–12, 2021. https://doi.org/10.1007/978-3-030-49932-7_1

4

G. Gaudi et al.

intended goals of the game. Although serious games are gaining momentum in a wide variety of health-based applications, there are various problems that must be overcome before their use can become more widespread. Developing SGs is not trivial, requiring the collaboration of an interdisciplinary team with expertise in game development/computer science, education, and content (health professionals when considering health-based SGs) [5]. Furthermore, most SGs include fixed scenarios that cannot easily be modified [7]. Such a fixed-scenario approach can produce predictable, boring, and repetitive game experiences, which after several sessions, can negatively impact their effectiveness. In medical applications, SGs have been employed to help patients increase physical health, or improve cognitive functioning. For example, SGs have been applied in treating dementia, and used to increase social and emotional health in depression. [9,20]. Serious games have also been used to treat autism spectrum disorder (ASD) [12,21]. As will be discussed in the following section, currently, therapists are using several methods (including the use of technology and serious games in particular), to help children with ASD. Children with ASD have difficulty imitating behaviour and around their first birthday are less interested to look at people [13]. According to the social motivation theory [8], since children with ASD do not find social interactions rewarding they attend less and less to faces and conversations and lose interest in social stimuli. Over time they become less adept in recognizing peoples’ intentions, goals and social cues. For this reason many existing interventions are aimed at motivating children to attend to faces and speech to reduce the cascading risk of ASD children’ s indifference to faces and emotional cues. Given the increase in autism diagnosis and the wide range of symptom severities, therapeutic efforts may benefit from the inclusion of SGs to increase their reach. The goal of this work was to develop an autism-specific serious game framework (ASGF) that therapists could use to create new SGs (or modify existing ones) in a simple manner (without having an extensive background and experience in software development), to supplement their therapeutic efforts. Although formal testing is required to gauge the effectiveness of the games developed using the ASGF, the aim of the ASGF, therefore, is to assist therapists to independently develop interactive and engaging scenarios that may have the potential to improve functioning in children with autism. We provide a tool to help therapists create games, by using the scenarios, to target repetitive behaviour and facial emotion processing. Following a review of previous work in Sect. 2, details regarding the ASGF are provided in Sect. 3. In addition, two games developed with the ASGF are also outlined. Concluding remarks and plans for future work are provided in Sect. 4. It should be noted that formal testing has yet to be conducted and thus although the ASGF was developed working closely with experts in autism treatment, formal evaluation of the ASGF and the two developed games is required to determine their effectiveness.

Autism Serious Game Framework (ASGF) for Developing Games

2

5

Literature Review

As previously described, currently, therapists are using several methods to treat ASD, including Applied Behavioural Analysis (ABA), the Treatment and Education of Autistic and Related Communication Handicapped Children (TEACCH) approach [19], and the Developmental Individual-Differences and RelationshipBased Model (DIR) [16]. The ABA intervention uses careful behavioural observation and positive reinforcement or prompting to teach each step of a behaviour. When a child performs the desired steps correctly, a reward, something (s)he likes very much, is provided, increasing the probability of positive behaviour in the future. By analyzing and modifying antecedent behaviour, and reactions or consequences, tutors can implement strategies for extinction of behaviours. Discrete trial training (DTT) breaks tasks into basic steps of increasing difficulty. Each trial is repeated until the child, under guidance, can accomplish the task successfully and autonomously. Successful ABA training involves early intervention and intensive treatment for as much time as possible (25–30 h per week), well-trained practitioners and consistent ABA applications, at home and at school [1]. Although this approach is generally recognized to be very successful, gains can be slow, and such gains may not be generalized. Children are often not motivated, and may demonstrate disruptive behaviours in order to interrupt or quit the sessions [11]. A variation of ABA, the Pivotal Response Treatment (PRT), which offers a more naturalistic approach, focuses on engaging with preferred toys and activities rather than artificial stimuli such as picture cards. In PRT, teaching is conducted within the context of natural play [11]. Recent work by Mohammadzaheri et al. [11], showed that the PRT intervention was more effective at improving social communication skills for children with autism than structured ABA treatment. The use of preferred toys in natural play settings appears to have created more interest, improved communication skills, reduced off-task behaviours [11]. Park et al. [14] examined the effects of TEACCH structured teaching on independent work skills in individuals with severe disabilities in a job setting. They demonstrated that TEACCH was effective, and performance gains were generalized into a different job setting. Carpente [3], combined the DIR/Floortime with Improvisational Music Therapy (IMT) while relying on the therapist’s ability to be creative, flexible, spontaneous, and emotionally attuned with the child. Four children with ASD, aged between 4–8 years old, participated in twenty-four 30 min classes for 13 weeks. Results using the Functional Emotional Assessment Scale (FEAS) indicated improvements in the areas of self-regulation, engagement, behavioural organization, and two-way purposeful communication. The DIR/Floortime Model enables clinicians, parents, and educators to construct tailored experiences matching the child’s unique skills. Malinverni et al. [6], proposed an inclusive method for developing therapeutic gaming interventions for children with ASD, and defined a model, which integrated the expertise of clinicians, the interests of children, and the experience of designers. Following this method allowed for the creation of an engaging therapeutic gaming experience using a Microsoft Kinect-based game called Pico’s

6

G. Gaudi et al.

adventures, where an alien requires assistance to get back to his own planet by repairing his space ship. To assess the effects on participants, 10 children with ASD between the ages of 4–6 years were studied. They each played the game independently in the first session, and for the second session, they required the help of their parent to retrieve items. The third session involved collaborative play with the parent to land the ship, and in the fourth session, the children earned rewards by collaborating with another child participating in the study. The authors concluded that social interaction could be promoted by designing cooperative games where the resources are distributed between the players to achieve a common goal. Conversely, it was suggested that game mechanics that use physical contact should be avoided since they may hinder social communication [6]. Van Zijl and Chamberlain [18], proposed a generic platform for the development of computerized ASD therapy and research tools with the specific aim of providing a culture-free independent software on a generic platform. The platform was built with OGRE, an open source game engine in conjunction with the Enginuity engine, which was modified to allow for multiple threads, thus improving better synchronization between video, sound, and reaction times in the therapy tools, among others. They used a number of third-party libraries to manage tasks involving input from peripheral devices, playing audio, executing scripts and game logic, and rendering graphics. These included the OIS (Open Input System) for managing input, and Nvidia PhysX for physics and collision detection. The platform was tested using two therapy tools, and one educational game. The game was designed in partnership with a local school, and targeted 1st graders. The situations were based on encounters children may experience in the classroom, and if the child successfully completed the exercise, the game advanced. All text was available in multiple languages with audio for those learners who were not capable of reading at a satisfactory level [18]. The game was tested at a local school for a period of 20 days. Teachers’ feedback was positive, and they found the data capturing aspect useful, which captures the learner’s attempts and answers to exercises [18]. The second therapy tool, which tested social skills in a restaurant scenario, required the player to order food, pay for it, and find a place to sit. The authors’ intended goal to develop a generic open-source 3D virtual environment platform, was reached. Caro et al. [2] proposed that exergames, that is, the coupling of video games with exercise, can offer children with severe autism a natural interaction using multisensory stimuli to keep them focused during motor therapeutic interventions. Traditionally, physical therapy relies on t repetitive limb movement, which requires the child with severe autism to aim for a visual target. Often, the visual stimuli are confusing, and this leads to aimless movements. FroggyBobby, is a game to improve hand-eye coordination in children with autism [2]. The goal of the game is to encourage participants to use their upper limbs to perform different eye-body coordination exercises. Children must help the frog avatar eat as many flies as it can. Their limb movements, which involve moving their right or left arm to reach a visual target, control the frog’s tongue. FroggyBobby

Autism Serious Game Framework (ASGF) for Developing Games

7

employs the Microsoft Kinect sensor to monitor and track which arm a child is using at a particular moment. Psychotherapists found FroggyBobby easy to use during eye-body coordination sessions, and experimental results indicate that children with severe autism maintained their attention for the total duration of the therapy, reduced their aimless limb movements, and developed aimed limb movements after several weeks of use [2]. The authors also contributes qualitative and quantitative empirical evidence to show how exergames help in the development of eye-body coordination. The authors concluded that further research is needed to fade out the guidance that children receive when doing the eye-body exercises [2].

3

The Autism Serious Game Framework (ASGF)

The ASGF was developed in conjunction with neuro-scientists who served as content experts and specified the requirements of the facial expression identification and the response inhibition games. The ASGF graphical user interface (GUI) follows a What You See is What You Get (WYSIWYG) [15] approach whereby users (therapists) are able to drag and drop (and assemble) components associated with each game. The ASGF allows users to import 2D, and 3D assets (with or without animations), and sound assets to configure the different types of games. For example, a card matching game can be re-purposed to create a facial expression identification game, where emotions can be displayed on pictures, illustrations, or rendered images and 3D models on the cards. Additionally, a 3D computer generated space can be tailored to focus on inhibitory control, whereby the therapist can replace the 3D models of the characters in the game and have the child interact with characters or the scene employing different audiovisual cues. The goal of inhibitory control is to teach the child to inhibit responses by not responding to a given character or event when required. Inhibition is the ability to suppress an action which is irrelevant, no longer needed, and/or inappropriate [4]. Set shifting “...is the ability to change from a learned rule to a new rule in response to changing behavioral contingencies [10]”. In addition to changing the look and feel of the games, the therapist can also change the difficulty, scoring, and obtain metrics to assess the player’s performance, which can be modified in a simple manner using the ASGF, providing flexibility to tailor the experience to the children instead of a one-size-fits-all solution. The ASGF has two end users, the therapists who can modify, create, and assess the outcomes from the games, and the child playing the resulting games. Feedback is provided from ASGF to both end users, for the therapist a report of the child interactions, and for the children, audiovisual cues associated to their performance. The system architecture is presented in Fig. 1. The ASGF enables the therapist to create a variety of games, which could raise the child’s confidence, motivate the child to follow through and succeed, and to potentially target different learning issues. Model driven engineering (MDE) provides the environment for domain experts to produce a serious game via modelling (either using language or visual tools) without worrying about the

8

G. Gaudi et al.

Fig. 1. The ASGF system architecture.

intricacy of game development [17]. It involves modelling the flow of information, the user interface (UI), designing a room layout (e.g., a 3D model of a room) or importing assets, and then using this information to generate code in the target language [17]. “MDE offers an increase in productivity, promotion of interoperability and portability among different technology platforms, support for generation of documentation, and easier software maintenance [17]”. The therapist is provided the option to focus on the problem and use the tool to create the solution, for example, a game that would teach autistic children to recognize faces. By using a planned event model, the therapist only has to model small sections of the code, and the objects connecting the assets, codes, and functions will know how to behave or run when they are built. The GUI for developing the games is presented in Fig. 2. The GUI allows for tailoring of the materials by importing pictures, videos, audio files, 3D models, along with difficulty settings, scores and timers within the design screen.

Fig. 2. Graphical user interface (GUI) components for creating the games.

Autism Serious Game Framework (ASGF) for Developing Games

3.1

9

The Games

The ASGF allows for two types of games to be created: i) card-style (for face expression recognition games), and ii) non-card style (for inhibitory games). The two games we developed include customizations such as add a score label, a level label, graphic and text labels and the inclusion of timers. These allow for the inclusion of the necessary game mechanics such as displaying the current score on the screen. Descriptive tool tips and a help display give feedback to the therapist in build mode. The ASGF includes a splash screen with terminology and tutorial sections. 3.2

Card Style Game

This is a face matching game (with one main sample card and two or more choice cards), that targets reading emotions from the face. The user must click on the card that matches the main card. Optionally, a timer can be added that counts up or down. When counting up, it records the game completion time. When counting down, if the counter reaches zero, the child loses that match and depending on the winning and difficulty conditions set by the therapist, the child may repeat or continue playing with at the same difficulty level. There are 20 matches (trials) per block and the therapist can adjust the number of trials per block. The difficulty level sets how many correct responses are required to advance to the next level or end the playback level. Face Emotion Matching Game: The child is asked to select the emotional expression that matches the main sample card (top card) For example, if the main card shows a happy face, the child is asked to identify the card that also shows a happy face (see Fig. 3).

Fig. 3. Face matching game screenshot.

10

3.3

G. Gaudi et al.

Non-card Style Game

The non-card style games aim to improve the ability to inhibit responses. In this game, a main character is moving on the screen in three different environments with customized assets and difficulty level provided by the game framework. It is worth noting that in these games the character moves and its trajectory can be altered to increase or decrease the difficulty when interacting with the objects in the scene. Rabbit Adventures Game: In this response inhibition game, the goal is for the player to catch as many rabbits as they can. While the user is exploring their game environment, rabbits and wolves may cross their path. When a rabbit crosses their path, they must press the space bar to catch the rabbit, and if a wolf crosses their path, they should not respond (i.e., they should not press any key). If they respond and press the space bar, they will catch the wolf who will steal the previously captured rabbits. The variable parts include: replacing the main character, setting the character that crosses the path, setting a different key other than the space bar (e.g., the “A” or “C” key), the number that must be correct to pass the level, and adding more characters that will cross the main character’s path. The difficulty decreases the number of consecutive correct responses to pass the level. Figure 4 shows a screenshot of a game created with the framework.

Fig. 4. Rabbit adventures game screenshot.

4

Conclusion

Previous works in the field of serious games for children with autism have focused on fixed scenario games that allow little customization and modification. Here

Autism Serious Game Framework (ASGF) for Developing Games

11

we present the development of an autism serious game framework (ASGF) that allows for the creation and customization of serious games that are aimed to help children with autism identify emotions and also inhibit responses. Using the ASGF, we developed two games. In the first game (Face Emotion Matching Game), the user selects a facial expression in the first game that matches the main card to develop facial recognition skills. In the second game (Rabbit Adventures), while exploring their game environment (world), the user rabbits and wolves may cross their path. They must press the space bar when they a rabbit crosses their path and avoid pressing the space bar (or any other key), when they a wolf crosses their path. The ASGF is GUI based and does not require programming skills, allowing it to be easily modified by a therapist or researcher. The ASGF has the potential to deploy the games on mobile devices so that the intervention can be extended while the child is at home, and the data recorded can help us to better understand the progression and customization of the therapy. Future work will involve conducting a series of user-based experiments to examine the usability of both the ASGF and the two serious games that we have developed and described here. Future work will also include user-based experiments to examine the effectiveness of the two games developed with the ASGF.

References 1. Artoni, S., Bastiani, L., Buzzi, M.C., Buzzi, M., Curzio, O., Pelagatti, S., Senette, C.: Technology-enhanced aba intervention in children with autism: a pilot study. Univ. Access Inf. Soc. 17(1), 191–210 (2018) 2. Caro, K., Tentori, M., Martinez-Garcia, A.I., Alvelais, M.: Using the froggybobby exergame to support eye-body coordination development of children with severe autism. Int. J. Hum. Comput. Stud. 105, 12–27 (2017) 3. Carpente, J.A.: Investigating the effectiveness of a developmental, individual difference, relationship-based (DIR) improvisational music therapy program on social communication for children with autism spectrum disorder. Music Ther. Perspect. 35(2), 160–174 (2016) 4. Geurts, H.M., van den Bergh, S.F., Ruzzano, L.: Prepotent response inhibition and interference control in autism spectrum disorders: two meta-analyses. Autism Res. 7(4), 407–420 (2014) 5. Kapralos, B., Haji, F., Dubrowski, A.: A crash course on serious games design and assessment: a case study. In: 2013 IEEE International Games Innovation Conference (IGIC), pp. 105–109. IEEE (2013) 6. Malinverni, L., Mora-Guiard, J., Padillo, V., Valero, L., Herv´ as, A., Pares, N.: An inclusive design approach for developing video games for children with autism spectrum disorder. Comput. Hum. Behav. 71, 535–549 (2017) 7. McCallum, S.: Gamification and serious games for personalized health. Stud. Health Technol. Inform. 177(2012), 85–96 (2012) 8. McPartland, J.C., Wu, J., Bailey, C.A., Mayes, L.C., Schultz, R.T., Klin, A.: Atypical neural specialization for social percepts in autism spectrum disorder. Dev. Psychopathol. 6(5–6), 436–451 (2005)

12

G. Gaudi et al.

9. Michael, D.R., Chen, S.L.: Serious Games: Games that Educate, Train, and Inform. Muska & Lipman/Premier-Trade, Florence (2005) 10. Miller, H.L., Ragozzino, M.E., Cook, E.H., Sweeney, J.A., Mosconi, M.W.: Cognitive set shifting deficits and their relationship to repetitive behaviors in autism spectrum disorder. J. Autism Dev. Disord. 45(3), 805–815 (2015) 11. Mohammadzaheri, F., Koegel, L.K., Rezaei, M., Bakhshi, E.: A randomized clinical trial comparison between pivotal response treatment (PRT) and adult-driven applied behavior analysis (ABA) intervention on disruptive behaviors in public school children with autism. J. Autism Dev. Disord. 45(9), 2899–2907 (2015) 12. Noor, H.A.M., Shahbodin, F., Pee, N.C.: Serious game for autism children: review of literature. World Acad. Sci. Eng. Technol. 64(124), 647–652 (2012) 13. Osterling, J.A., Dawson, G., Munson, J.A.: Early recognition of 1-year-old infants with autism spectrum disorder versus mental retardation. Dev. Psychopathol. 14(2), 239–251 (2002) 14. Park, I., Kim, Y.R.: Effects of teacch structured teaching on independent work skills among individuals with severe disabilities. Educ. Training Autism Dev. Disabil. 53(4), 343–352 (2018) 15. Petrovic, G., Fujita, H.: Springboard: game-agnostic tool for scenario editing with meta-programming support. Appl. Intell. 48(5), 1161–1175 (2018) 16. Ryan, J.B., Hughes, E.M., Katsiyannis, A., McDaniel, M., Sprinkle, C.: Research based educational practices for students with autism spectrum disorders. Teach. Except. Child. 43(3), 56–64 (2011) 17. Tang, S., Hanneghan, M.: A model-driven framework to support development of serious games for game-based learning. In: 2010 Developments in E-systems Engineering, pp. 95–100. IEEE (2010) 18. Van Zijl, L., Chamberlain, M.: A generic development platform for asd therapy tools. In: CSEDU (1), pp. 82–89 (2010) 19. Virues-Ortega, J., Julio, F.M., Pastor-Barriuso, R.: The teacch program for children and adults with autism: a meta-analysis of intervention studies. Clin. Psychol. Rev. 33(8), 940–953 (2013) 20. Wiemeyer, J., Kliem, A.: Serious games in prevention and rehabilitation-a new panacea for elderly people? Eur. Rev. Aging Phys. Act. 9(1), 41 (2012) 21. Zakari, H.M., Ma, M., Simmons, D.: A review of serious games for children with autism spectrum disorders (ASD). In: International Conference on Serious Games Development and Applications, pp. 93–106. Springer (2014)

LinkLearn: Blockchain Technology as a Learning Tool Dawid Benjamin Jordaan(&) School of Computer Science and Information Systems, Faculty of Natural and Agricultural Sciences, North-West University, Vanderbijlpark, South Africa [email protected]

Abstract. Higher education institutions will remain gatekeepers of opportunity if industry, governments, and students value their existing credentials. Keeping up with technological changes and new learning models may contribute to making learning environments more effective and desirable. With appropriate learning models like computer-based learning, contact time can be changed from absorbing information to situations where students inquire, think and challenge their lecturers and each other. A blockchain is a digital ledger of economic and financial transactions but is not limited to these kinds of transactions as it can record ‘virtually everything of value’. Blockchain is a disruptive technology that will influence our daily lives and organizations such as education, business, and governance. The focus of this study was on a model, called LinkLearn, that implements blockchain principals and used as a learning tool. The research question formulated for this research was whether LinkLearn can be used as a learning tool. A qualitative research process was followed, and lecturers were interviewed to obtain their views about LinkLearn in education. The outcomes of this study show that lecturers are positive about LinkLearn as a learning tool. The results of this study are important as it shows that the LinkLearn model has the potential to transform the way students are engaged, motivated and evaluated. Keywords: Blockchain


Mobile phones
Technology in education

1 Introduction The use of information technology (IT) is crucial at work, home and in education [17]. Students are attracted to digital technology and they like to share information. Mobile phones are essential tools for students as they use it for communicating by voice, text, digital photos and videos [27]. Students are also keen to discover new things and new ways of doing things, often at high speed [29]. Based on the “anytime, anywhere” paradigm students use mobile phones for wireless computing and posting to wikis and blogs [19]. According to Felder [5] students “only learn by thinking and doing, not watching and listening.” The mobile phone can play an important role in this regard. Felder and Brent [6] suggest to “do a think-pair-share, in which students work on something individually and then pair up to compare and improve their responses before you call on them.” Engagement plays an important role in active learning and with the © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 13–24, 2021. https://doi.org/10.1007/978-3-030-49932-7_2

14

D. B. Jordaan

right software students can be engaged online in ways that will appeal to them [4]. The availability and use of mobile phones provide an opportunity to integrate the learning process with ‘real-life’ activities. Mobile phones are so versatile and can allow students to carry their university in their hands [27]. Higher Education Institutions (HEI) will be the gateway to opportunities while their credentials are valued by industry, governments, and students. Students will attend and pay if HEIs are perceived as effective learning institutions [29]. According to Duan et al. [3], HEIs have a monopoly on learning certifications, learning processes and results. Incorporating technological changes and new learning models may contribute to making HEIs more effective and desirable. Implementing suitable learning models like computer-based learning, contact time can be changed from absorbing information to situations where students inquire, think and challenge their lecturers and each other. Universities are responsible for the continued advancement and communication of knowledge and students should be actively involved in this process [20]. Higher Education Institutions (HEI) need to be innovative especially concerning their pedagogy models. Their education models need to be constantly evaluated and redesigned and adapted according to technological changes and students’ demands. The broadcast model of learning where “the teacher is the broadcaster and the student is the supposedly willing recipient of the one-way message” is outdated [29]. This “traditional school-centered classroom learning” is slowly changing with the progression of technology [3]. Using interactive, self-paced computer learning programs students can master knowledge outside the formal classroom leaving contact time for conversation, debate, and teamwork around assignments and projects. Students should experience the joy of discovering new things for themselves and experience education as interesting and fun. The focus of this study was on a model, called LinkLearn, that implements blockchain principals and used as a learning tool. The research question formulated for this research was whether LinkLearn can be used as a learning tool.

2 Objective of the Study According to Haddon [9] wild or disruptive technology tends to be tamed into everyday objects very quickly. Lindh et al. [17] argue that IT generally develops into unexciting IT products or artifacts that we use for routine everyday tasks. These remarks together with the observation by Tapscott and Tapscott [28] that blockchain transactions can record “virtually everything of value” and a blockchain delivers fast, secure and transparent peer-to-peer transactions [14] prompted this study. Chen et al. [1] admit that with formative assessment it is not easy to track every detail of teaching and learning but they suggest that the implementation of a blockchain and smart contracts can address this challenge. Using the “collaborative learning” paradigm, for example, a student submits his/her work to the learning platform through his/her unique account. The smart contract running on it will review the student’s performance, and the results will be recorded into blocks. Chen et al. [1] believe that a student’s behavior and opinions during collaboration can also be saved into blocks and evaluated and in this context, the blockchain ensures fairness of evaluation. They admit

LinkLearn: Blockchain Technology as a Learning Tool

15

that the evaluation of formative learning activities is hard to do by pre-programmed smart contracts without human intervention. The focus of this study is on the question if LinkLearn, a model based on blockchain principals, can be used as a learning tool to advance and communicate knowledge. The main objective of this study is to investigate LinkLearn as a learning tool.

3 Background This section describes concepts that are used in this study. Section 3.1 describes blockchain technology while Sect. 3.2 highlights the use of blockchain in education. Section 3.3 deliberates on the LinkLearn model. 3.1

Blockchain Technology

There needs to be a closer relationship between HEIs, and they need to work together to design new educational models. A closer relationship between HEIs and society is also needed. The blockchain may help to realize these ideas. Chen et al. [1] indicate that blockchain technology is also known as distributed ledger technology. According to Tapscott and Tapscott [28], a blockchain is a digital ledger of economic transactions. Gupta [8] adds that the “blockchain is a shared, immutable ledger that facilitates the process of recording transactions and tracking assets in a business network.” Assets can be tangible for example land, a car, a house, cash, etc. or intangible for example patents, branding, copyrights, intellectual property, etc. Conducting and recording a financial transaction in an efficient, cost-effective, reliable and secure system determined the context in which blockchains was developed. Tapscott and Tapscott [28] advocate that the blockchain can be programmed to record financial transactions but is not limited to these kinds of transactions as it can record ‘virtually everything of value’. Figure 1 illustrates a simplified blockchain where each block shows different transactions and blocks are linked using unique signatures. Two blocks, containing some transaction data are linked or chained together. To accomplish this, every block gets a unique digital signature that corresponds to exactly the string of data in that block. If the contents of a block changes, the block will get a new signature. Signatures are calculated through hashing. The data in the first block is chained to the second block by adding the signature of the first block to the data of the second block. The signature of the second block is partially based on the signature of the first block because the signature of the first block is included in the string of data in the second block. These signatures link the blocks to each other, resulting in a blockchain.

16

D. B. Jordaan

Fig. 1. A simplified blockchain [12]

A blockchain network is characterized by the following features. Consensus (all participants must agree on its validity for a transaction to be valid), Provenance (the origin of assets and how its ownership has changed over time is known to participants), Immutability (the leger does not allow for participants to tamper with transactions after it has been recorded), and Finality (the completion of a transaction or the ownership of an asset is provided by only one single, shared ledger) [8]. Blockchain applications could be divided into three stages; Blockchain 1.0 is a peer-to-peer cash payment system for the deployment of cryptocurrencies; Blockchain 2.0 is more extensive including stocks, bonds, loans, smart property, and smart contracts; Blockchain 3.0 covers blockchain applications such as government, health, science, literacy, culture, and arts [26]. The benefits of blockchain technology might not be clear in the beginning, but there are real use cases and real issues that can be solved through the use of a blockchain database [22]. Blockchain has many potential applications as all transactions in the blockchain network are trustworthy, transparent and immutable [30]. King [14] states that the main purpose of a blockchain is to allow fast, secure and transparent peer-to-peer transactions. It is a trusted, decentralized network that allows for the transfer of digital values such as currency and data. Swan [26] predicts that the next gigantic trend in technology will be decentralized networks. A blockchain can be classified as permissioned or not permissioned. A permissioned blockchain network is crucial for blockchains meant for business, especially within regulated industries while not permissioned blockchain networks are open networks and anyone can access these networks. According to Gupta [8], a permissioned network offers enhanced privacy, improved audibility and increased operational efficiency. Chen et al. [1] state that blockchain is a disruptive technology that will influence our daily lives and organizations such as education, business, and national governance. Tapscott and Tapscott [29] agree that the blockchain holds the potential to disrupt higher education. Figure 2 shows the blockchain cycle. A transaction is requested and broadcasted to a peer-to-peer (P2P) network where the transaction gets validated by all nodes in the network. Once validated the transaction is combined with other transactions to create a new block of data that is linked or chained to an existing blockchain.

LinkLearn: Blockchain Technology as a Learning Tool

17

Fig. 2. Blockchain cycle [23]

3.2

Blockchain in Education

The demand for life-long learning is a priority today. Life-long learning, according to Harmse [10] and Tapscott and Tapscott [29] is the capacity to learn throughout life. This includes a) research, analyze, synthesize, contextualize, and critically evaluate information, b) apply knowledge in solving problems, and c) collaborate and communicate. As an example, Tapscott and Tapscott [29] refer to the experiences of Vitalik Buterin, founder of the Ethereum blockchain, in an interview they conducted with him. Some of Buterin’s remarks during the interview follows. “It’s actually surprisingly useful how much you can learn for yourself by debating ideas like politics with other people on forums. It’s a surprising educational experience all by itself” and “I had all these different interests, and somehow bitcoin seemed like a perfect convergence. It has its math. It has its computer science. It has its cryptography. It has its economics. It has its political and social philosophy. It was this community that I was immediately drawn into” and “I found it really empowering.” Blockchain technology has been implemented in many finance, business, and similar fields. Clark [2] reports that a blockchain database is an excellent way to verify and keep track of certificates that the educational sector issue. Chen et al. [1] list examples where blockchain technology has been used in educational environments. The University of Nicosia uses blockchain technology to manage students’ certificates received from MOOC platforms [24]. Sony Global Education used the blockchain technology to create a global assessment platform to provide services for storing and managing degree information [11]. Massachusetts Institute of Technology (MIT) and the Learning Machine company cooperated to design a digital badge for online learning based on blockchain technology [25].

18

D. B. Jordaan

Universities and other educational institutions can introduce or teach blockchain technology to students by linking blockchain applications and the Cloud as there is no need for dedicated or additional local hardware [2, 21]. The question that concerns Kursh and Gold [15] is the place in computer science and information technology curriculum where blockchain technology should be addressed. According to these authors advanced topics can be offered within existing courses, or through extracurricular activities such as symposiums or guest speakers, or new courses can be developed to cover the new topic. To introduce blockchain to students as an innovative example or subtopic in existing modules seems acceptable and achievable. Theoretical discussions and hands-on applications can add to the excitement of the course [15]. Purdon and Erturk [21] suggest possible computer science courses that may incorporate blockchain applications, for example database systems, cloud computing, computer networking, advanced business information systems, computer security, computer cryptography, web hosting, and web administration. How students approach learning can be affected by the way their learning outcomes are assessed. According to Duan et al. [3], the assessment of learning outcomes must be a continuously renegotiable working agreement between lecturers and students and captured in an Electronic Learning Contract (ELC). The authors explain that the ELC focuses on the use of electronic meetings to facilitate group discussions on learning outcomes. A Group Support System (GSS) can serve as a platform to assist students and lecturers during the discussions, negotiations, formulation, renegotiation, and revision of the ELC. The ELC indicates what to learn and when it will be assessed as well as the assessment criteria. To implement the ELC Duan et al. [3] propose that a blockchain and a smart contract could be used to implement the ELC. This idea is supported by Chen et al. [1] when they suggest a “Smart Contract running on the Ethereum blockchain network is essentially a computer protocol that simulates a real contract. It can facilitate contract negotiation, simplify contract terms, implement contract execution, and verify contract fulfillments state.” The discussions show examples of how blockchain technology is used in the education sector but not as a learning tool. The emphasis is mainly on implementing blockchain technology to “administrative applications” and not on how the blockchain principal can be used for learning purposes, which is the objective of this study. 3.3

The LinkLearn Model

LinkLearn is a model, based mainly on the first three steps of the blockchain logic flow as depicted in Fig. 3. The operational flow of the LinkLearn model is described in the next section. The LinkLearn model has several objectives. Tapscott and Tapscott [29] state that the traditional learning model of a one-way message where the lecturer is a presenter and the student a willing receiver is outdated. Students need to discover new things for themselves which will make education interesting, joyful and a fun experience. Therefore, lecturers need to think in creative ways to compile assignments, projects and learning material which is required in step 1 of the model. In line with Felder’s [5] recommendation that students learn by thinking and doing, step 2 of the model enforces student participation as individuals and in groups. This is

LinkLearn: Blockchain Technology as a Learning Tool

19

underlined by Felder and Brent [6] who suggest that students work on something individually and then pair up to compare and improve their responses. Students need to take responsibility for their learning and blockchain forces accountability by everybody in the system [13]. This happens in step 3. LinkLearn is a technology-driven model in which active learning plays a major role. Students are engaged online anytime, anywhere using their mobile phones which will appeal to them [4, 27]. This creates an opportunity to integrate the learning process with ‘real-life’ activities. Ultimately all assignments with their correct solutions will be contained in a blockchain. This will allow students to revise them at any stage if or when necessary. LinkLearn is not an evaluation system. If formal evaluation is built into the system, it becomes a ‘push’ system where students may think that they are forced to learn because they are going to be evaluated. LinkLearn is a ‘pull’ system where students are motivated to take responsibility and accountability of their learning [13].

Fig. 3. LinkLearn cycle

3.3.1 The Logical Flow of the LinkLearn Model The logical flow of the LinkLearn model is as follows. In step 1 the lecturer issues an assignment. In step 2 the requested assignment is broadcast to a permissioned peer-2peer (P2P) network accessible by all students who have access to the P2P network (a permissioned blockchain network is advisable as it can be regulated by the lecturer). In step 3, in the blockchain scenario, the transaction and the user’s status are validated by all nodes in the P2P network. In the LinkLearn model, step 3 differs somewhat from the blockchain scenario in that this step is repeated if necessary. All students solve the assignment individually and store their solutions. After that, all students select the correct or “best” solution and save it. The lecturer evaluates and rates the correct or “best” solution as determined by all students. If the correct or the “best” solution scores 100 percent, it is added to a blockchain else step 3 is repeated until the correct or the “best” solution scores 100%. The operational flow of the LinkLearn model is summarised in Table 1.

20

D. B. Jordaan Table 1. The logic flow of the LinkLearn model

Step 1 2 3a 3b 3c 3d 4

Action Lecturer issue assignment Broadcast assignment to a permissioned peer-2-peer network Students individually solve the assignment and save their solutions Students select the correct or “best” solution and save it Lecturer evaluate and rate the correct or “best” solution and save it If the correct or “best” solution do not score 100% then repeat step 3 Add the correct or “best” solution to a blockchain

Assignments pool P2P network Solutions pool Correct or “best” solutions pool Scores pool

Blockchain

3.3.2 Motivational System According to GrammerFlip [7], students love a leader-board system. Badges can be beneficial to students by giving them recognition for a diversity of acquired competencies [18]. To motivate students to participate and take responsibility for their learning, a badge system is introduced into the LinkLearn model. According to this system, a student receives badges for performing different activities as illustrated in Table 2. Each badge has a weight attached to it indicating the value of the activity that it presents. It is, for example more valuable to submit a correct assignment than to just submit one. Lecturers may alter these weights according to their specifications. The badges can be used to build a leader-board system which can also serve as a motivational tool. If traditional test and examinations are used to evaluate students’ learning, badges can be converted into marks that can contribute to students’ final test or examination scores.

Table 2. LinkLearn badge system Activity Submit assignment Report copied assignment Submit copied assignment Evaluate assignment Submit the correct or “best” assignment

Badge Weight Blue badge 0.1 Orange badge 0.5 Red badge −1.0 Yellow badge 0.2 Green badge 0.7

LinkLearn: Blockchain Technology as a Learning Tool

21

4 Objective of the Study According to Haddon [9] wild or disruptive technology tends to be tamed into everyday objects very quickly. Lindh et al. [17] argue that IT generally develops into unexciting IT products or artifacts that we use for routine everyday tasks. These remarks together with the observation by Tapscott and Tapscott [28] that blockchain transactions can record “virtually everything of value” and that a blockchain delivers fast, secure and transparent peer-to-peer transactions [14], prompted this study. Chen et al. [1] admit that with formative assessment it is not easy to track every detail of teaching and learning but they suggest that the implementation of a blockchain and smart contracts can address this challenge. Using the “collaborative learning” paradigm, for example, a student submits his/her work to the learning platform through his/her unique account. The smart contract running on it will review the student’s performance, and the results will be recorded into blocks. Chen et al. [1] believe that a student’s behavior and opinions during collaboration can also be saved into blocks and evaluated and in this context, the blockchain ensures fairness of evaluation. They admit that the evaluation of formative learning activities is hard to do by pre-programmed smart contracts without human intervention. The focus of this study is on the question if LinkLearn, a model based on blockchain principals, can be used as a learning tool to advance and communicate knowledge. The main objective of this study is to investigate LinkLearn as a learning tool.

5 Method To answer the objective formulated for this study the approach was to determine lecturers’ perceptions about LinkLearn as a learning tool. A mobile app demonstrating an example of a simplified blockchain was developed. The app has the functionality of a blockchain. The typical ‘transactional’ or ‘business’ data of a blockchain was replaced with ‘educational’ data. The logic flow of the app is based on the discussion by Rosic [23] and demonstrated in Fig. 2. The app was used to demonstrate the possible use of blockchain principals as a learning tool. A qualitative research approach was followed, and interviews were used as the data collection method. According to Kvale [16], interviews are a sensitive yet powerful method for collecting participants’ experiences and lived meanings and allow them to express their viewpoints of a situation in their own words. Interviews are generally classified as being unstructured, structured or semi-structured. Semi-structured interviews were conducted with lecturers from a higher education institute (HEI) and former students. The purpose was to discuss the use of blockchain principals as a “wild” idea to be domesticated and used as a mobile educational tool. Former students and lecturers from two schools, namely the School of Computer Science and Information Systems and the School of Mathematics and Statistical Sciences were selected to participate in this study. The empirical part of this study consists of semi-structured face-to-face interviews in places familiar to the participants. Data were gathered until data saturation occurred. The interviews were guided by the following themes: blockchain technology in general

22

D. B. Jordaan

and the LinkLearn model, based on blockchain principals, as a possible tool in education. Two pilot interviews were conducted before the actual data gathering process.

6 Results As mentioned before semi-structured interviews were conducted to gather data. Data saturation occurred after nine interviews. Responses were written down and immediately verified. Following is the analysis of the responses from participants. To establish participants’ knowledge of blockchains, the following two questions were asked at the beginning of the interviews: “Do you know what a blockchain is and how a blockchain works?” and “Do you think this principal can be used for educational or learning purposes?” All nine participants admitted that they had no knowledge of a blockchain, how it works or how blockchain principals can be used for educational or learning purposes. Referring to Fig. 1 and Fig. 2, the author explained blockchains using simple examples and the app (briefly described in Sect. 5) to demonstrate possible use of blockchain principals as a learning tool. The LinkLearn model and the logical flow, as illustrated in Fig. 3 and Table 1, were then explained to participants. Participants were again asked: “Do you think this principal can be used for educational or learning purposes?” This time the responses were very positive, but concerns were raised, like “How will you assure student participation?”, “What will motivate students to participate?”, “Why will students evaluate other students’ assignments?” and “How will you ensure that students will not copy another student’s assignment?” These concerns led to the introduction of the LinkLearn motivational system in the form of badges as discussed in Sect. 3.3.2 and Table 2. Another concern raised by participants concerns step 3b of Table 1 where students must select the correct or “best” solution. One participant mentioned “If there are for example 120 students in the class it means that every student must evaluate 120 solutions to select the correct or ‘best’ one. They will just not do it!” This might be an obstacle, but one possible solution is to divide the class into smaller groups of say thirty and give each group a similar but not identical assignment. This is where the lecturer needs to be very innovative and creative. The outcome of LinkLearn is a blockchain containing all the assignments with their correct solutions as agreed on by all students. Three participants remarked “But we provide the memorandum of an assignment to students in any case.” Participants agreed that providing a memorandum means that a student did not take any part in compiling the memorandum. Students can easily memorize the content of the memorandum in which case little learning took place. To finalize the solution in LinkLearn all students must participate and there must be consensus about the outcome before the solution goes to the blockchain. This observation excited all participants.

7 Discussion All participants in this study perceive the LinkLearn model as revolutionary but are positive about the idea of using LinkLearn as a learning tool. They all expressed their interest and willingness to take part in a follow-up study when a more sophisticated

LinkLearn: Blockchain Technology as a Learning Tool

23

working system or app is provided. A major challenge will be to formulate assignments in an appropriate manner that will strengthen the objectives of the LinkLearn model. The assignments should be creative and challenging to keep students interested, to have fun and ensure that learning takes place. The role of the lecturer changes from teacher to facilitator. When an assignment is referred for a third time, the lecturer needs to intervein and provide guidance. This can be in the form of a discussion or a debate during a contact session. This study shows how students can be actively involved and responsible for their own learning. The potential significance of this study is that it may help to transform the way we view the student’s role in the learning process. Research is needed to refine the motivational system and how badges can benefit students especially if a conventional evaluation system, where students must write tests and examinations, is still in place.

References 1. Chen, G., Xu, B., Lu, M., Chen, N.-S.: Exploring blockchain technology and its potential applications for education. Smart Learn. Environ. 5, 1 (2018) 2. Clark, D.: 10 ways blockchain could be used in education (2016). https://tinyurl.com/ ycd3uxtg 3. Duan, B., Zhong, Y., Liu, D.: Education application of blockchain technology: learning outcome and meta-diploma. In: 2017 IEEE 23rd International Conference on Parallel and Distributed Systems (ICPADS), pp. 814–817. IEEE (2017) 4. Felder, R.: You got questions, we got answers. 2. Active learning. Chem. Engr. Educ. 47(2) (2013) 5. Felder, R.M.: It goes without saying. Chem. Eng. Educ. 25, 132–133 (1991) 6. Felder, R.M., Brent, R.: Learning by doing. Chem. Eng. Educ. 37, 282–309 (2003) 7. GrammerFlip (2019). https://grammarflip.freshdesk.com/support/solutions/articles/1700003 8013-what-is-kahoot-and-how-does-it-work8. Gupta, M.: Blockchain for dummies. In: Burchfield, C.A. (ed.) IBM Limited Edition, US, Hoboken, NJ 07030-5774. John Wiley & Sons, Inc., Hoboken (2018) 9. Haddon, L.: The contribution of domestication research to in-home computing and media consumption. Inf. Soc. 22, 195–203 (2006) 10. Harmse, A.: Expectations of industry regarding the ICT competencies of HEI graduates. North-West University (2018) 11. Hoy, M.B.: An introduction to the blockchain and its implications for libraries and medicine. Med. Ref. Serv. Q. 36, 273–279 (2017) 12. Jimi, S.: How does blockchain work in 7 steps a clear and simple explanation (2018). https:// blog.goodaudience.com/blockchain-for-beginners-what-is-blockchain-519db8c6677a 13. Khan, I.: What is Blockchain Technology? A Step-by-Step Guide For Beginners. Blockgeeks (2019). https://blockgeeks.com/guides/what-is-blockchain-technology/ 14. King, R.: Blockchain Explained: The Ultimate Guide to Understanding How Blockchain Works. BitDegree Publishers (2019). https://www.bitdegree.org/tutorials/blockchainexplained/ 15. Kursh, S.R., Gold, N.A.: Adding fintech and blockchain to your curriculum. Bus. Educ. Innov. J. 8(2), 6–12 (2016) 16. Kvale, S.: Doing Interviews. SAGE, London (2008)

24

D. B. Jordaan

17. Lindh, M., Nolin, J., Hedvall, K.N.: Pupils in the clouds: implementation of Google apps for education. First Monday 21(4) (2016) 18. Mc Arthur, D.: Will blockchains revolutionize education? Educause (2018). https://er. educause.edu/articles/2018/5/will-blockchains-revolutionize-education 19. Norris, C., Soloway, E., Menchhofer, K., Bauman, B.D., Dickerson, M., Schad, L., Tomko, S.: Innovative leaders take the phone and run: profiles of four trailblazing programs. District Adm. 46, 35 (2010) 20. Oliveira, C.C., de Souza, R.C., Abe, É.H.S., Móz, L.E.S., de Carvalho, L.R., Domingues, M. A.: Undergraduate research in medical education: a descriptive study of students’ views. BMC Med. Educ. 14, 51 (2014) 21. Purdon, I., Erturk, E.: Perspectives of Blockchain technology, its relation to the cloud and its potential role in computer science education. Eng. Technol. Appl. Sci. Res. 7, 2340–2344 (2017) 22. Redka, M.: Top 10 problems that blockchain technology solves (2018). https://mlsdev.com/ blog/top-10-problems-that-blockchain-technology-solves 23. Rosic, A.: What is blockchain technology? A step-by-step guide for beginners. Blockgeeks, (2019). https://blockgeeks.com/guides/what-is-blockchain-technology/ 24. Sharples, M., Domingue, J.: The blockchain and kudos: a distributed system for educational record, reputation and reward. In: European Conference on Technology Enhanced Learning, pp. 490–496. Springer (2016) 25. Skiba, D.J.: The potential of blockchain in education and health care. Nurs. Educ. Perspect. 38, 220–221 (2017) 26. Swan, M.: Blockchain: Blueprint for a New Economy. O’Reilly Media Inc, Sebastopol (2015) 27. Taleb, Z., Sohrabi, A.: Learning on the move: the use of mobile technology to support learning for university students. Procedia-Soc. Behav. Sci. 69, 1102–1109 (2012) 28. Tapscott, D., Tapscott, A.: Blockchain Revolution: How the Technology Behind Bitcoin is Changing Money, Business, and the World. Penguin, New York (2016) 29. Tapscott, D., Tapscott, A.: The blockchain revolution and higher education. Educause Rev. 52, 11–24 (2017) 30. Underwood, S.: Blockchain beyond Bitcoin. Commun. ACM 59, 15–17 (2016)

New Era of the Nano-Electronic Devices – One of the Most Adaptive Learning Areas for the Next Period Cristian Ravariu1(&), Doru Ursutiu2, Dan Mihaiescu3, Alina Morosan3, Mihai Tanase1, and Thrasyvoulos Tsiatsos4 1

3

Faculty of Electronics, Department of Electronic Devices Circuits and Architectures, University “Politehnica” of Bucharest, Bdul Iuliu Maniu no 1-3, sect. 6, Bucharest, Romania [email protected] 2 Transilvania University of Brasov, Brasov, Romania [email protected] Faculty of Applied Chemistry, Department of Organic Chemistry Costin Nenitescu, University “Politehnica” of Bucharest, Str. Polizu 1-4, Bucharest, Romania [email protected] 4 Department of Informatics, Aristotle University of Thessaloniki, 54124 Thessaloniki, Greece [email protected]

Abstract. The Moore’s law rigorously fulfilled and in 2021 the CMOS technology seems to reach its ultimate limit. What’s next? All specialists anticipate that a circuit from 2031 will not keep the same performances offered by the last 2.5 nm CMOS from 2021. Proposals are multiple, alternative nano-devices are well-known, nanoscale technologies open amazing future facilities. Confluence of nano-electronics with organic semiconductors and biomaterials seems to be inherent. The paper intends to reply to this extraordinary challenging. Keywords: Beyond CMOS lessons Simulation in nanoelectronics


Mobile content
Adaptive learning


1 Introduction The CMOS roadmap reaches the physical limits till 2021. Despite to the prior alerts concerning the serious perturbation in the CMOS functioning sub-65 nm due to the quantum effects [1], this MOS technology successfully survives, breaking any pessimist barriers, reaching to 7 nm node [2] and continuing till 2021 by its ultimate stage of 2.5 nm node. Under 5 nm node, the source–drain tunneling becomes a main shortcut way. Additionally, the tunneling through an ultra-thin gate insulator injects gate currents [3], so that the transistor effect is lost. For a next decade, the industry predicts the co-integration of other nano-devices near CMOS circuitry [4] to increase some local performances and to keep the global costs as close as possible to the low costs of the CMOS technology. The effect in the © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 25–35, 2021. https://doi.org/10.1007/978-3-030-49932-7_3

26

C. Ravariu et al.

teaching area of the Electron Devices will require a maximum adaptive and mobile content. The pedagogy speaks about the maturity degree of a discipline, as an advantage. So we speak about hundreds year of discipline maturity for mathematics and physics, sixty year for bipolar and MOS devices and few year for nowadays nanoelectronics, [5]. But in the next decade, the content of an electron devices course will be extremely mobile. Both instructors and students must upgrade themselves so fast, so they are able to learn a year about tunneling transistors and in the next year about carbon nano-tube transistors, for instance. The pressure of the mobile content of the curricula will be obviously transferred to adaptive learning methods. A solution to face this dynamic learning experience with so fast slew rate comes from the simulations tools in this new era of the nano-electronic devices. Solutions to make them adaptive to novel technologies could be some open frames that are editable for the new models that depict the newest devices behavior.

2 Approach 2.1

Anticipated Directions for Alternative Nano-Devices

It is well-known that CMOS sub-22 nm means FDSOI-MOSFET (Fully Depleted Silicon On Insulator Metal Oxide Semiconductor Field Effect Transistor) and CMOS sub-15 nm means rather Fin-FET. A Fin-FET is a Field Effect Transistor that integrates in a “Fin” two elevated FDSOI-MOSFETs in a 3-dimensional manner, [6, 7]. Recent research proposes the 5-nm CMOS technological node as a NWT (NanoWire Transistor), [8]. However, the NWTs were produced many years ago, for other purposes than CMOS, using a multitude of materials, [9]. As the literature shows, the nano-wires technology stands for a practical solution to implement the concept of SET (Single Electron Transistor) [10] – that is by excellence an extreme low energy consumer. The SED (Single Electron Devices) family uses one by one electrons transfer from Source to Drain and can be a serious candidate for green electronics with quasinull power dissipation. On the other hand, the nano-wire technology was used to construct nanoscale organic transistors, [11] or CNT-FET (Carbon Nano-Tube Field Effect Transistors), [12]. The ballistic transport of the electrons through a middle axis of a CNT offers supra-conductor properties and fastest carriers transport, [13]. Despite of a non-mature CNT technology, since 2016 the CNT-FETs are already in use for the logical XOR circuits fabrication, [14]. From this stage, there was a little step to produce NCS (Nano-Core-Shell) CdS materials for optoelectronic devices, [15]. The fabricated NCS nano-wires presented enhanced surface photo-voltage properties, due to an efficient photo-induced charge transfer from the CdS shell to the carbon nanotube core, [15]. The TFT family has a maturity of ten years in industry, being applied especially to display manufacturing and touch mobile phones, [16]. A novel generation of Organic TFT (OTFT) allows integration of active electronic devices on flexible substrates like plastic, [17].

New Era of the Nano-Electronic Devices

27

Continuing this direction, very recent some TFT (Thin Film Transistors) with NCS nano-structured films were proposed in our group, [18, 19]. The organic transistors go forward to integrate on the same Si-chip some biomaterials like enzymes [20], DNA [21], antibodies, producing the Bio-FET (Transistors with Biomaterials) class. Not in the last, the tunnelling transistors are promising alternative nano-devices for future CMOS co-integration. Starting from the traditional pin-FET as TFET (TunnelFET) with lower sub-threshold slope than the ideal limit of 60 mV/dec for MOSFET [22] with length around 80 nm, they evolved to vacuum nano-transistors with 10 nm gap [23]. Our group proposed a NOI (Nothing On Insulator) transistor with 2 nm gap [24] or NOI nano-triode, able to work in the Tera-Hetz domain [25]. So, here is a list of alternative nano-devices that are not even exhausted, which are already in use or well known in the research branch and they are expected to enter into the industrial flow, as mass production. 2.2

What Tools Still Exists and What Is Missing

Fortunately, the industry of semiconductor devices offers haptic software tools. For instance the Athena tool can simulate any technological flow starting from a Si or glass substrate. This software precisely respects only the real technological facilities. This haptic feature helps students to find new nano-fabrication routes, in absence of en expensive white room, using dynamic and interactive design steps, which are fully restricted by the actual industry of micro-nano-electronics. Also, the Atlas tool from Silvaco allows the electrical characteristics extraction for any capricious 2-D or 3-D device shape. The last Atlas variants allow luminous or magnetic stimulus, beside to the classical electrical stimuli, but it is limited in organic materials and completely inefficient in bio-transistors. However, the Atlas tool allows defining any new material. Unfortunately, the new material must be initially defined as a semiconductor or insulator, following the template for Si and SiO2 respectively, adding the new parameters list. This feature rests ineffective in the next decade, because the nano-materials like CNT or NCS or any selfassembled core with external shell have individual mechanism for the current transport. Also, the main biomaterials do not find an appropriate correspondent in the Atlas or Athena templates. However, the tools must offer a maximum flexibility of the materials properties, in the next period. Not only the materials but also the devices principle is expected to be changed many times per year. Therefore, these software tools for future nanoelectronics must be re-built, so that the material model itself can be modified or renewed. A poor solution still exists: some C-routines can be released into an Atlas frame to define any mathematic profile of the interface states you want for thin film transistors, for instance. These features must be extended. Frequently the investigator initially does not know if his new material has a semiconductor behavior or rather an insulator one or what behavior has this material inside new device architecture. In the next decade, with the help of the software specialists, some complete new tools able to simulate biomaterials are compulsory necessary. The models missing represent a serious problem, due to the complexity and non-identical repeatability of the phenomena in the living matter. This subject was treated elsewhere, [26].

28

C. Ravariu et al.

3 Actual or Anticipated Outcomes 3.1

Actual Tools and Their Applications to E-Learning

In the next future, the computer scientists play an important role in this puzzle. The next international research projects need collaborative work and inter-disciplinary teams. For instance, institutions with white rooms facilities can final fabricate an ultimate nano-device, based on a solid design stage made by Athena-Atlas specialists, aided by new software able to include models of bio-, nano- and organic materials. The models come from living matter, material sciences, organic compounds, biochemistry. A recent practical example is provided as a case study. A master student from Faculty of Electronics from Bucharest, Romania, finished in June 2019 his dissertation thesis with the subject: Finding the applications of CMOS circuits co-integrated with a planar version of the Nothing On Insulator device. The student is employed to Infineon Company from Bucharest, to the department of circuits design. So, he developed good competences of circuits simulation by Spice-Cadence tools, outside of faculty, too. During the master of Nano-Micro-Electronics, the student received knowledge about Atlas-Athena device simulation tools. The dissertation subject is challenging because a pNOI (planar Nothing On Insulator) device has no model in Spice. Initially, the student was able to separately simulate the pNOI device [27] and to check its work principle by Athena-Atlas chain tools, Fig. 1. Fortunately, the pNOI device is suitable for the Sitechnology, so that the material model missing was not a problem for this kind of device. The Athena technological flow consists in sequential layer depositions, followed by removal of the non-useful parts by etchings, Fig. 1a. In Atlas, the pNOI device is simulated at VS = 5 V and VD = 0 V, usual voltages for logic CMOS applications, Fig. 1b. Figure 1b shows a crowding of the current vectors toward the upper part of the ntype Si-film. The tunneling current becomes significant for VS > 3 V, Fig. 2a, for an oxide thickness of 2 nm. The CMOS inverter can be simulated in Spice, either by its user-defined MOS models, or by their physical structure in Atlas. The simultaneous simulation of the pNOI device with the Source connected to the output of the CMOS inverter was the main novelty achieved by our student. This simulation can be done by mixed mode of the Atlas tool [28], letting run inside the input file a Spice sub-routine that depicts the components connections. The pNOI device was simulated by its physical structure from Atlas, while the CMOS inverter was simulated by Spice models of a NMOS and a PMOS transistor, [27].

New Era of the Nano-Electronic Devices

29

Fig. 1. (a) The pNOI device structure after Athena technology running; (b) the current vectors simulated by Atlas prove the main current transport trace.

The result of the mixed mode Atlas-Spice, allows to observe the CMOS inverter functioning with out = non-in (in = V(1) and out = V(3)), while the pNOI device measures the front rise/fall time, Fig. 2b. The pNOI drain peak current is positive for rising input signal and negative for falling input signal. From this mixed analysis, an application of this pNOI device with 2 terminals is suggested: a font detector. 3.2

Specific Research Aim and Objectives

Our specific research aims are related to: (i) NOI nanodevices with vacuum; (ii) organic thin film transistors with green technological routes; and (iii) biodevices, including integrated biosensors like BioFET.

30

C. Ravariu et al.

Fig. 2. (a) The ID − VD simulated curve of the pNOI device; (b) the mixed mode simulation reply.

Grace to the simulation tools, the following conclusion is revealed: the NOI device with a vacuum gap [29] keeps the transistor effect only if the source and drain islands presents semiconductor behavior [30, 31]. Rather the Carbon-related materials [32] are more efficient for the electrons emission in vacuum. From many years ago it is wellknown that diamond as mono-crystal of Carbon presents a suitable emission yield, for the vacuum nano-transistors, [33]. Grace to the Atlas facility that allow the artificial diamond describing, an improved version of the diamond-NOI transistor was recently simulated by our team, [31]. The advantage comes from the inherent roughnesses of the vertical walls, which increases the tunneling effect, due to the electrical field accentuation near any corner, Fig. 3.

Fig. 3. (a) The simulated electric field in a diamond-NOI transistor with roughnesses on the source and drain walls, when VDS = +5 V, VGS = −3 V.

New Era of the Nano-Electronic Devices

31

The critical electric field in vacuum is reached in the “Nothing” region, containing vacuum. When this value is exceeded, the tunneling current occurs between the source and drain diamond islands. The simulation accuracy of this NOI structure can be further improved in a next stage, if Athena tool allows the diamond deposition techniques. Exporting the structure with all imperfections toward the Atlas tool, we expect to offer a proper characterization, in agreement with the selected technology. The issue of multilevel layout NOI structures, in particular for the heat transfer is possible in Atlas, grace to the lattice temperature parameter activation. Fortunately, the NOI device doesn’t suffer from high power dissipation, Fig. 4.

Fig. 4. The simulated Joule heat power (W/cm3) inside the NOI transistor biased to a maximum drain voltage: VDS = +7 V, VGS = −1 V

Another recent research direction concerns some TFT with NCS nano-structured films, [18]. The internal ferrite core is self assembling by PABA (Para Amino Benzoic Acid) as external shell, offering semiconductor property for a PABA-NCS transistor, [19]. There are some simulators like Game or Hyperchem molecular modeling programs, able to extract the electrical charge inside a molecule, [18]. In opposition to PABA, the oleic acid molecule is an organic molecule that possesses poor electrical charge, Fig. 4, being inefficient for semiconductors. The polar heads of the oleic acid molecules, (Fig. 5), can be used for biomimetic layers, borrowed from the living matter structure. Alternatively, it can be applied in electronic devices, as liquid insulator. This analysis among different organic molecules as was achieved by another PhD student from our group, [34]. When the organic molecules evolve to bio-macro-molecules, like enzymes or antibody [35], frequently used in bio-recognition devices [36], the simulation tools are poorer than in in organic chemistry. For the moment, in Atlas, a simple simulation of a Bio-FET transistor can be done [20], defining the integrated bio-material as userdefined material with a reduced parameter list, Fig. 6. The potential deviations from (0, 1) Volts from Fig. 6 occurs in agreement to the metals work functions.

32

C. Ravariu et al.

Fig. 5. The simulation result of the electrical charge distribution over the oleic acid molecule; the values are normalized to the elementary electrical charge.

Figure 6 presents the current vectors and the potential contours in the Silicon part of an enzymatic Bio-FET, biased at VDS = 1 V and VGS = 0.4 V. The advantage of the simulation tool consists in main current trace emphasizing, which is impossible by experiment. In order to increase the BioFET sensitivity, that region of Silicon under the insulators must be carefully technological prepared. Details of these research objectives were presented elsewhere, [20]. The disadvantage of the simulation tool envisages the impossibility to simulate the bio-recognition effect of the glucose oxidase (GOX) enzyme entrapped onto nano-tubes of TiO2 layer to a glucose stimulus.

Fig. 6. The potential and current vectors through a simulated Bio-FET structure at VS = 0 V, VD = 1 V, VG = 0.4 V.

New Era of the Nano-Electronic Devices

3.3

33

Anticipated Mobile Character of the Future Learning Tools

Besides to the Athena-Atlas-Spice and C-interpreter simulation tools that are already presented, other new software tools are expected to be developed in the next period. They must to be able to face the few atomic layers or few molecules arrangement inside a nano-device. The molecular simulators like Game or Hyperchem modeling programs must be correlated with the Athena tool. As input files for Athena, they delivery all information for an accurate simulation of the devices with PABA materials in a final Atlas framework. Another mobile character has the learning tools for bioelectronics [37]. This mobile content result from the huge diversity of the bio-recognition mechanisms borrowed from the living matter to construct Bio-FETs, besides to an extreme large palate of the receptor immobilization techniques [38, 39]. In this area, the simulation tools are almost absent. In the didactical labs, we use the Atlas tool to simulate the transistor part. Separately, the students can estimate a biosensor calibration curve, applying some simple models, like Michaelis-Menten enzymatic kinetics [40]. In fact we have some experiences in the learning process in bioelectronics [26, 37]. When these investigations are learned by the students, one difficulty is related to the absence of experimental laboratories, to calibrate the simulations to measured curves. On the other side, the main participant students have electrical engineering background. Therefore we use as simple as possible models of the bio-recognition elements for a better assimilation. The target is to offer a demo method for students. So that, after years they can be able to develop more complicated and accurate models. We fructify their ability to elaborate short software tools that perform numerical analysis of BioFETs, during some projects per semester. From the autumn of 2019, a new master program is launched in our university – Nano-Bio-Engineering and the Environment Management – combining basic bio-nano-sciences elements with few engineering sections (electrical, devices and material science) at the beginning, continuing to adapt the content to the next year demands.

4 Conclusions Obviously, after 2021 new electron devices will be launched in industry. We still do not know if there is a successful sole candidate. Industry predicts an extreme large devices palette, as a new era in the nano-electronics devices: CNT-FETs, Tunnel-FETs, Bio-FETs, Org-TFT, SET, molecular devices. The fast adaptive content of a course of electronic devices will affect teachers and students alike. A convenient solution to face this dynamic content is adaptive Atlas-Athena software, able to simulate new device structures. Students successfully used these tools to design CMOS co-integrated with pNOI, by mixed mode. We faced some limitations of the software in the bio-materials facilities. Computer scientist contributions are necessary in this scope. Acknowledgments. This work was partially supported by a grant of Ministry of Research and Innovation, CNCS-UEFISCDI, project number PN-III-P4-ID-PCE-2016-0480, within PNCDI III, project number 4/2017 (TFTNANOEL).

34

C. Ravariu et al.

References 1. Taur, Y., Buchanan, D.A., Chen, W., Frank, D.J., et al.: CMOS scaling into the nanometer regime. Proc. IEEE 85, 486–504 (1997) 2. Xie, Q., Lin, X., Wang, Y., Chen, S., Dousti, M.J., Pedram, M.: Performance comparisons between 7 nm FinFET and conventional bulk CMOS standard cell libraries. IEEE Trans. Circuits Syst. II Express Briefs 62, 761–765 (2015) 3. Basak, R., Maiti, B., Mallik, A.: Analytical model of gate leakage current through bilayer oxide stack in advanced MOSFET. Superlattices Microstruct. 80, 20–31 (2015) 4. Han, J.W., Oh, J.S., Meyyappan, M.: Cofabrication of vacuum field emission transistor (VFET) and MOSFET. IEEE Trans. Nanotechnol. 13(3), 464–468 (2014) 5. Vacca, M., Turvani, G., Riente, F., Graziano, M., Demarchi, D., Piccinini, G.: TAMTAMS: an open tool to understand nanoelectronics. In: 12th IEEE International Conference on Nanotechnology (IEEE-NANO), Birmingham, UK, pp. 1–2 (2012) 6. Kaneko, A., Yagashita, A., Yahashi, K., Kubota, T., et al.: Sidewall transfer process and selective gate sidewall spacer formation technology for sub-15 nm FinFET with elevated source/drain extension. In: 10-th IEEE International Electron Devices Meeting (IEDM 2005), Washington, DC, USA, pp. 844–847, December 2005 7. Singh, J., Ciavatti, J., et al.: 14-nm FinFET technology for analog and RF applications. IEEE Trans. Electron Devices 65, 31–37 (2018) 8. Al-Ameri, T., Georgiev, V.P., Adamu-Lema, F., Asenov, A.: Simulation study of vertically stacked lateral Si nanowires transistors for 5-nm CMOS applications. IEEE J. Electron Devices Soc. 5, 466–472 (2017) 9. Colinge, J.P., Lee, C.W., Afzalian, A., Akhavan, N.D., et al.: Nanowire transistors without junctions. Nat. Nanotechnol. 5, 225–229 (2010) 10. Briseno, A.L., Mannsfeld, S.B., Lu, X., Xiong, Y., Jenekhe, S.A., Bao, Z., Xia, Y.: Fabrication of field-effect transistors from hexathiapentacene single-crystal nanowires. Nano Lett. 7, 668–675 (2007) 11. Thelander, C., Mårtensson, T., Björk, M.T., Ohlsson, B.J., Larsson, M.W., et al.: Singleelectron transistors in heterostructure nanowires. Appl. Phys. Lett. 83, 2052 (2003) 12. Wang, J., Dai, J., Yarlagadda-Langmuir, T., et al.: Carbon nanotube−conducting-polymer composite nanowires. Langmuir 21, 9–12 (2005) 13. Chin, H.C., Bhattacharyya, A., Arora, V.K.: Extraction of nanoelectronic parameters from quantum conductance in a carbon nanotube. Carbon 76, 451–454 (2014) 14. Kavitha, P., Musala, S., Vardhan, K.V., Vani, Y.S., Srinivasulu, A.: Carbon nano tube field effect transistors based ternary Ex-OR and Ex-NOR gates. J. Curr. Nanosci. 12, 520–526 (2016) 15. Cao, J., Sun, J.S., Hong, J., Li, H.Y., Chen, H.Z., et al.: Carbon nanotube/CdS core–shell nanowires prepared by a simple room-temperature chemical reduction method. Adv. Mater. 16, 84–87 (2004) 16. Lai, C.C., Tsai, C.C.: A modified stripe-RGBW TFT-LCD with image-processing engine for mobile phone displays. IEEE Trans. Consum. Electron. 53, 1628–1633 (2007) 17. Kim, S.J., Lee, J.S.: Flexible organic transistor memory devices. Nano Lett. 10, 2884–2890 (2010) 18. Ravariu, C., Mihaiescu, D., Istrati, D., Stanca, M.: From pentacene thin film transistor to nanostructured materials synthesis for green organic-TFT. In: IEEE Conference of Semiconductors, Sinaia, Romania, pp. 65–68 (2018) 19. Ravariu, C., Istrati, D., Mihaiescu, D., Morosan, A., Purcareanu, B., Cristescu, R.: Fe3O4 Nano-Core with PABA external shell applied for p-type Organic Thin Film Transistors, notyet published, in revision (2019)

New Era of the Nano-Electronic Devices

35

20. Ravariu, C., Manea, E., Babarada, F.: Masks and metallic electrodes compounds for silicon biosensor integration. J. Alloy. Compd. 697, 72–79 (2017). Elsevier 21. Liu, N., Hu, Y., Zhang, J., Cao, J., Liu, Y., Wang, J.: A label-free, organic transistor-based biosensor by introducing electric bias during DNA immobilization. Org. Electron. 13, 2781– 2785 (2012) 22. Koswatta, S.O., Lundstrom, M.S., Nikonov, D.E.: Performance comparison between p-i-n tunneling transistors and conventional MOSFETs. IEEE Trans. Electron Devices 56, 456– 465 (2009) 23. Han, J.W., Meyyappan, M.: Introducing the vacuum transistor: a device made of nothing. IEEE Spectr. 14, 25–29 (2014) 24. Ravariu, C.: Gate swing improving for the nothing on insulator transistor in weak tunneling. IEEE Trans. Nanotechnol. 16, 1115–1121 (2017) 25. Ravariu, C.: Vacuum nano-triode in nothing-on-insulator configuration working in terahertz domain. IEEE J. Electron Devices Soc. 6, 1115–1123 (2018) 26. Ravariu, C., Manea, E., Babarada, F., Ursutiu, D., Mihaiescu, D., Popescu, A.: Organic compounds integrated on nanostructured materials for biomedical applications. In: Auer, M., Langmann, R., (eds.) Chapter 2 in the book: Smart Industry & Smart Education, vol. 47, pp. 489–497. Springer, Cham (2019) 27. Tanase, M.: Mixed pNOI CMOS simulations, Thesis of Dissertation, under coordination of C. Ravariu, U.P.B, Faculty of Electronics, Bucharest, Romania (2019) 28. Atlas manual, pp. 345–475 (2019). https://www.silvaco.com/. Accessed 2019 29. Ravariu, C.: Compact NOI nano-device simulation. IEEE Trans. Very Large Scale Integr. (VLSI) Syst. 22(8), 1841–1844 (2014) 30. Ravariu, C.: Deeper insights of the conduction mechanisms in a vacuum SOI nanotransistor. IEEE Trans. Electron Devices 63(8), 3278–3283 (2016) 31. Ravariu, C.: Special features of the nothing on insulator transistor simulated with diamond lateral islands. Rom. Rep. Phys. 70, 4 (2018) 32. Majidi, R., Saadat, M., Davoudi, S.: Electronic properties of doped porous graphene and biphenylene carbon: a density functional theory study. Rom. Rep. Phys. 69, 509 (2017) 33. Subramanian, K., Kang, W.P., Davidson, J.L.: A monolithic nanodiamond lateral field emission vacuum transistor. IEEE Electron Device Lett. 29(11), 1259–1261 (2008) 34. Morosan, A., Mihaiescu, D.E., Istrati, D., Voicu, G., Fudulu, A., Stan, R.: Polar shell magnetic nanostructured systems for heterogeneous nanophase reactions. U.P.B. Sci. Bull. Series B 80, 53–64 (2018) 35. Ravariu, C., Manea, E., Parvulescu, C., Babarada, F., Popescu, A.: Titanium dioxide nanotubes on silicon wafer designated for GOX enzymes immobilization. Digest J. Nanomaterials Biostructures 6(2), 703–707 (2011) 36. Kaisti, M.: Detection principles of biological and chemical FET sensors. Biosens. Bioelectron. 98, 437–448 (2017) 37. Ravariu, C.: Learning in bioelectronics. In: Hijon-Neira, R., Lazinica, A. (eds.) Advanced Learning, pp. 381–396. InTech, Austria-Croatia (2009) 38. Sun, C., Liu, M., Sun, H., Lu, H., Zhao, G.: Immobilization free photoelectrochemical aptasensor for environmental pollutants: design, fabrication and mechanism. Biosens. Bioelectron. 140, 111352 (2019) 39. Hammami, A., Raouafi, N., Mirsky, V.M.: Electrically controlled Michael addition: addressing of covalent immobilization of biological receptors. Biosens. Bioelectron. 121, 72–79 (2018) 40. Chen, W.W., Neipel, M., Sorger, P.K.: Classic and contemporary approaches to modeling biochemical reactions. Genes Dev. 24(17), 1861–1875 (2010)

Open Source Online Conference System for Industry Experts Participation in Education Dan Robu(&), Radu Curpen, Daniel Ilie, and Titus Balan Transilvania University of Braşov, Brașov, Romania {dan.robu,radu.curpen,titus.balan}@unitbv.ro, [email protected]

Abstract. The latest technologies have enabled easier collaboration using commodity tools - mainly the Web Browsers. This facilitates online collaboration between universities and the experts in the industry. As an implementation of modern web services, our proposed solution is near to the real-life educational experience, aiming to replace or to complement assisted teaching and being a valuable add-on to the blended learning process. Keywords: Web RTC (Real Time Communications)
BigBlueButton
Open source conference system
e-Learning
Redis PubSub
FreeSWITCH communications platform

1 Introduction In the learning process, an important element is the accumulated hands-on experience. For example, in the field of telecommunications and computer networks there are some scenarios that are not obvious but are quite common in the daily work of experts. It would be a big plus to have their experience immediately available, on-demand, in the learning process. To achieve this, we propose a solution that fits both security requirements of the corporate and industry medium and the ease of deployment and cost required by the university educational platforms. 1.1

WebRTC as Base for eLearning

WebRTC was usually identified as a good platform for eLearning because it establishes a standard in the protocols and API-s. [1] A lot of proprietary solutions were developed, primarily targeted at the corporate customers, integrating all that was need for Online Collaboration. One such platform is Unify Circuit [2] that is bringing innovative features: one tool with one view (the conversation and the document editing happens in the same window), integration with Exchange Servers, online storage services like Box and Google Drive and storage of conferences and documents for later review and excellent mobility: not only there is a mobile client, but also the calls can be moved seamlessly from one device to the other. © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 36–44, 2021. https://doi.org/10.1007/978-3-030-49932-7_4

Open Source Online Conference System

37

Skype and Skype for Business are the Microsoft offers for online collaboration. The disadvantages of this solution are the requirements to install additional software and to add exceptions in the firewalls, while the mobility is reduced. On the other hand, integration with other Microsoft components is excellent (Microsoft Exchange and Microsoft Active Directory) which makes them a very good solution for the internal corporate use [3]. The solution that we are proposing has the big advantage that it can fill the gap between the academic and corporate networks, because it does not require installation of additional software on the users’ computers. The security is also enforced by the built-in mechanisms of the browsers (which are configured by the IT departments to fulfill the security requirements).

2 BigBlueButton BigBlueButton (BBB) [4] is an online open source conference system for eLearning. Its purpose is to provide a way to access an eLearning platform using only a web browser. To achieve this, it provides all the tools needed by a moderator to invite the students to a chatroom and all the tools that a presenter needs for his presentation. The presentation is real-time with audio, video and desktop sharing and moreover, it offers the possibility to record the session for later viewing. The usual configuration is with the teacher which is both the moderator and the presenter, but the configuration which fits best our scenario is the one with a moderator and another presenter (the guest expert - an industrial partner). It is worth mentioning that there can be more than two moderators and the moderators can assign the presenter role to the users. BigBlueButton also provides a virtual whiteboard and chatrooms for the students. It is compatible with JAWS (Job Access With Speech) so it is accessible to persons with disabilities and difficulties [5]. The viewer can participate in the audio/video conference, can use various emoji for actions like rising a hand for questions or making a remark. The typical PC interface, in a common browser - e.g. in Google Chrome - can be seen in Fig. 1, while the typical mobile phone interface, for a common OS - e.g. Android - can be seen in Fig. 2.

Fig. 1. The PC interface of BigBlueButton, with video and chat

38

D. Robu et al.

Fig. 2. The mobile interface of BigBlueButton

The moderator has all the facilities of a viewer, plus the ability to block the audio of participants, or even to remove the user from the conference. The BBB architecture can be seen in the Fig. 3. As it can be seen, the server is able to serve both older Adobe Flash Player clients and newer HTML5 clients. This has a big impact on the usage, as it will be detailed below, in the next subsection. Other important components are Redis PubSub, a service that provides a communication channel between different applications that run on BigBlueButton and the Redis DB where all the events are stored when a conference is recorded. When the conference ends, the Recording Processor saves the different files (PDF, Office) for processing. For conferencing it is used the FreeSWITCH communications platform, which also provides interconnection with VoIP providers. Red5 Apps are providing media streaming. Apps are based on Akka - a distributed, and message-driven set of methods for actor-based concurrency. They include mediators between FreeSWITCH and other voice conference systems via FreeSWITCH Event Socket Layer (FsESL).

Open Source Online Conference System

39

Fig. 3. The BigBlueButton architecture

2.1

Advantages of BigBlueButton – HTML5

Recently, starting with September 2018 and finalized in April 2019, the HTML5 client is production ready [6]. It enables the use of any modern web browser as client, without any other additional software. In the strict corporate environments that big companies are using, this is a great advantage, eliminating the need for additional software. The only software needed is the browser, which can be secured according to the security policy of the organization. Only the new client, based on HTML5 satisfies the security concerns of most organizations, since the Adobe Flash Player used in the older client, had a lot of vulnerabilities. Previously the only collaborative possibilities were either to use the corporate client software (hard to do because of the licensing and of the strict security), either to use the universities’ software (very improbable to be accepted in the corporate network, due to security concerns) [7]. The HTML5 client opened the possibility to use BigBlueButton also on mobile devices, while avoiding the security issues associated with the Adobe Flash Player [8]. The interface is intuitive and clean, as it can be seen in Fig. 4.

Fig. 4. Main conference window, without video

40

2.2

D. Robu et al.

HTML5 and WebRTC

One of the most important ideas in HTML5 is to simplify the audio and video implementation using WebRTC. In traditional web models, developed based on the client-server architecture, communication is unidirectional (from server to client, by the HTTP protocol). The WebRTC architecture extends the client-server model with the P2P (peer-to-peer) communication between web browsers [9]. The most used WebRTC model is the “triangular” one, in which the two web browsers access an application on a server (located at the top of the triangle) – via this application, the signaling messages are transmitted through the HTTP or WebSocket technologies [10]. The web server is used only to control the transfer of data – the large volume data streaming is made peer-topeer, directly between the browsers. The WebRTC stack can be seen in the Fig. 5. All the details of the media transport are offloaded to this WebRTC stack. This enables higher compatibility between applications and use cases (with or without external firewalls, different web browsers). Features like STUN (Session Traversal Utilities for NAT) that permits the NAT clients to use VoIP providers, TURN (Traversal Using Relays around NAT) that enables media sharing packets sending for NAT clients (which a STUN server can’t do) and ICE (Interactive Connectivity Establishment) that makes connectivity checks to collect the IP addresses of the conference participants.

Fig. 5. WebRTC Stack - the three dashed methods below can be overridden by the browser developers

Open Source Online Conference System

41

The WebRTC API has a central place in the stack of Fig. 5, being designed to create real-time connections using audio, video and/or data channels directly between users, via web browsers or servers that support WebRTC protocols. One of the common methods used by this API, the so-called getUserMedia(), enables the access to the microphone or webcam installed on the user’s terminal [11]. The other specific methods configure the P2P connections, create “data channels” and provide access to video and audio streams [12]. The codecs used are Opus, iSAC and iLBC for audio and VP8 and VP9 for video. As it can be seen, they are very efficient: a session with two users, using both microphone and camera used approximatively 70 kB/s as it can be seen in the Windows Resource Monitor (Fig. 6). The video stream doesn’t consume too much bandwidth since the images are mostly static (the faces of the participants and the shared desktop/application).

Fig. 6. Tests on bandwidth usage (only the BigBlueButton tab is opened).

2.3

Integration with Moodle

In the case that the university already has a Moodle LMS (Learning Management System) in place for eLearning, the Moodle integration plugin can be used. The plugin offers the following features [13]: • Create multiple activity links to online sessions within any course, depending on the scenario used • Restrict students from joining a session until a teacher (moderator) joins the session • Launch BigBlueButton conference in a separate window, to access at the same time also the course material

42

D. Robu et al.

• Create a custom welcome message that appears at the top of the chat window when joining the session, adapted to the course thematic • Specify join open/close dates for the session that appears in the Moodle’s calendar, to restrict the time frames when a BigBlueButton session is available • Record a session – add resources to the course that can be examined later • Access and manage recordings

3 Use Cases 3.1

Existing Laboratory that Can Be Improved

In the laboratories present at the Department of Electronics and Computers from Transilvania University of Brasov - Romania, there are Mobile Communications equipment either real or implemented with SDR (Software Defined Radio) technologies. Since there is usually only one of each necessary radio node, the participation via BigBlueButton can enhance the learning experience, each participant having the chance to observe the live work of the presenter in a console, for example, and each participant being able to take over the console to do a part of the practical work. After many years of difficult efforts to organize hands-on laboratories at the hightech sites of partner telecommunications companies, now their local experts (with the new possibility to invite international ones) “join the class” adding a lot of value to the learning experience, by enhancing the information presented with real life examples and manipulations. One such example is the laboratory of the Mobile Communication course where the mobile network nodes are located at the university, but the tests and configuration are performed by the experts from telecommunications companies. Some of the nodes of the mobile network are physical (the BTS, BSC, TRAU, NodeB, RNC), while some are emulated with protocol emulators (MSC, SMS-Center). In the case of the physical nodes the Operation and Maintenance terminal is shared by the students, teacher and experts, while for the emulated nodes the screen of the emulator is shared. 3.2

Courses with Experts’ Participation

Another use-case is the one where the course is presented mainly by the industry experts, remotely using the platform provided by the university. New fields like Cyber-Security require participation of highly-specialized experts from all around the world. The existing virtualization platform at the Transilvania University of Brasov - Romania enables the students to have their own environment for testing, but for the presentation, experts from all around the world are invited. Also, to solve complex issues, help from experts is needed so with BigBlueButton, this becomes possible.

Open Source Online Conference System

43

4 Conclusions As the Internet and Intranet bring not only instantly available information but also support - online expertise in valuable professional or volunteering schemes. Our solution, of direct, online, university-industry cooperation in modern technologies integrated solutions that extend WebRTC to educational platforms based on the “thin-client” paradigm: as much as possible, resources are available via normal browsers. With the new HTML5 implementation of the BigBlueButton client, external participation is now very easy. The experts can join a course or even entire courses can be presented remotely by experts, especially in newer fields where knowledge is scarce. Prior to BBB, due to security reasons and software restrictions, any external participation in the courses was very difficult. Now, no additional software besides the usual browser is needed to join an e-Learning/blended learning session. The fact that the BigBlueButton software is open-source means that the additional cost of implementation is very low, while allowing any customization needed. To assess the BBB solution, we have also performed tests on the used codecs efficiency. As expected, the bandwidth used was quite low, depending very little on the capabilities of the browsers and servers used.

References 1. Balan, T., Alexandra, S., Sandu, F., Stefania, S.: WebRTC based eLearning platform. In: The 13th International Scientific Conference eLearning and Software for Education, April 27–28, Bucharest (2017). https://doi.org/10.12753/2066-026x-17-000 2. Unify Circuit. https://unify.com/en/solutions/team-collaboration/circuit 3. Goetz, M., Igler, M., Meerkamm, S., Reithmeier, T., Jablonski, S., Ehmann, M.: Online communication and collaboration - a general approach. In: 2010 Fifth International Conference on Internet and Web Applications and Services, Barcelona, pp. 544–549 (2010) 4. Mavridis, A., Tsiatsos, T., Tegos, S.: Exploiting web conferencing to support collaborative learning. In: 15th Panhellenic Conference on Informatics (PCI), Kastoria, Greece, pp. 78–82 (2011) 5. Bohnsack, M., Puhl, S.: Accessibility of MOOCs (Massive Open Online Courses). In: Miesenberger, K., Fels, D., Archambault, D., Peňáz, P., Zagler, W. (eds.) Computers Helping People with Special Needs. (ICCHP 2014). LNCS, vol. 8547. Springer, Cham 6. HTML5 client is coming! https://blindsidenetworks.com/2019/03/15/html5-client-iscoming/ 7. Janitor, J., Fecilak, P., Jakab, F.: Enabling long distance education with realtime video. In: 2012 IEEE 10th International Conference on Emerging eLearning Technologies and Applications (ICETA), Stara Lesna, pp. 167–171 (2012) 8. Adobe Flash Player Security Vulnerabilities. https://www.cvedetails.com/vulnerability-list/ vendor_id-53/product_id-6761/Adobe-Flash-Player.html 9. Picek, R., Picek, S.: WebRTC Multipoint Web Real-Time Communication. Springer International Publishing, Heidelberg (2014). ISBN 978-3-319-07214-2

44

D. Robu et al.

10. Pearson, S., Yee, G.: Privacy and Security for Cloud Computing. Springer-Verlag, London (2013). ISBN 978-1-4471-5793-9 11. Zamfir, S., Bălan, T., Sandu, F.: Costache, C.: Mobile communication solutions for the services in the internet of things. In: Proceedings of the 7th International Conference on Exploring Service Science, IESS 1.6. Springer International, pp. 619–634, (2016). ISBN 978-3-319-32688-7 12. Pebriadi, P., Wuryandari, A.I., Setijadi, P.A.: Design and implementation learning methods HTML5 on iCode application. In: 2013 Joint International Conference on Rural Information Communication Technology and Electric-Vehicle Technology (rICT & ICeV-T), Bandung, pp. 1–6 (2013) 13. BBB plugin for Moodle. https://moodle.org/plugins/mod_bigbluebuttonbn

Interactive TV and Music Education Mobile Music Practices and Limitations: Learning Byzantine Music Online Rafail Tzimas1, Dimitrios Margounakis1, Dionysios Politis1(&), and Nektarios-Kyriakos Paris2 1

School of Informatics, Aristotle University of Thessaloniki, Thessaloniki, Greece [email protected], {dmargoun,dpolitis}@csd.auth.gr 2 Department of Music Science and Art, University of Macedonia, Thessaloniki, Greece [email protected]

Abstract. Music Learning online has been promoted the last decades as an Interactive methodology with considerable outcomes, in terms of efficiency, effectiveness and attainment of popularity. However, a situation where no noticeable progress would be inferred as accredited benefit of massive learning was reported a decade ago. The advent of mobile computing communication proved a forceful exertion to situated learning in music having the power to influence the acquisition of music knowledge and skills. The case of a novel Byzantine Music learning protocol is examined for its potential to achieve relatively a big “crop” from rather limited in extend lessons but with the capacity to have a significant effect on the context of musical instruction. Keywords: Massive communication


Online learning
Byzantine music
Mobile

1 Introduction Interactive Television (iTV) comprises a technology that dates back to 1990s as a concept. iTV is a form of media convergence, adding data services to traditional television technology. The core idea around iTV is the interactivity capabilities between the user and the TV set that it offers (e.g. change the camera angle during a sports event, vote for the show’s winner during a live broadcast, comment or ask a question during a talk show, etc.). Although the concept of interactive television provides many tempting features for various areas, it has been slow to develop over time, as there is a key factor to be met: fast Internet connectivity. But this factor has nowadays been resolved. It is a fact that new smart TVs are catching the market as the next step in home TV sets. A key feature of smart TV is the Internet connection and the use of Internet Protocol Television (IPTV). These TVs offer applications, media streaming, web surfing, online gaming, Video on Demand (VOD), and more.

© Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 45–53, 2021. https://doi.org/10.1007/978-3-030-49932-7_5

46

R. Tzimas et al.

The use of the new capabilities, coupled with the evolution of forms of interaction with a TV set, can undoubtedly provide a new potential in education, which requires the use of advanced neuro cognitive multimedia methodologies. iTV has a great deal of potential for facilitating interactive, collaborative, constructivist, situated, and authentic learning [1]. The next step of iTV was its migration to personalized viewing, i.e. its ability to form a sustainable, widespread mode of viewing, with strong attributes of pervasiveness [2].

2 Learning Models The education sector uses computer technology in countless ways. Courses are designed around course management software so that students can communicate outside the classroom, respond to online tests, and easily find their course material. Material is transformed into digital resources, often in interactive form, to increase the degree of interest and ease of assimilation by students. There are simulations and software programs, which are proven to be extremely useful learning tools. Virtual spaces can be used to enhance the learning experience and motivate students. Moreover, virtual museums and the use of 3D serious games for learning can be extended to valuable fields, e.g. cultural heritage. The digital classroom education can become the tool that will help teachers be more successful in their educational work, despite the increase in class sizes and tight economic policy. Moreover, numerous tools are designed to help educators create their own serious games or virtual educational materials. Open and distance learning is constantly spreading as a suitable and more flexible way of learning in today’s fast-paced lifestyle and enjoys the benefits of this development [3]. The new rising learning models can be seen in Fig. 1. In Fig. 1, two major factors mainly affect the models of learning: the classroom (whether it is a physical space with the presence of both teacher and student or a digital environment with no actual face-to-face meetings) and the knowledge to be transferred itself (learning content). On the axis of knowledge, the content is divided into the limited physical materials of a course (textbooks, notes, exercises on the blackboard and slides) and the unlimited digital resources that can be created for the course’s needs or already exist in the web (multimedia, digital libraries, resources on demand, synchronous and asynchronous communication tools etc.). In the case of a course delivering all or most of its content (over 80%) online, it is considered to belong to the pure online model. The intermediate ways of mixing elements from traditional and online learning are considered to belong to hybrid or blended models. Another model of learning (following the online advent of mobile devices) is mobile learning (mlearning), which has yet some drawbacks if used as a stand-alone model but will soon be an essential extension of e-learning [4]. Also, mobile devices can well be used for entertaining serious games to enhance the learning experience.

Interactive TV and Music Education

47

Fig. 1. A framework for learning models, portraying the subtle distinction between blended and hybrid models.

Although the terms blended learning and hybrid learning are often used interchangeably in literature, there is a slight difference between the two of them. Blended learning is an approach to education that combines traditional place-based classroom methods with ICT technologies and educational materials. The online component of the learning experience usually consists of exercises or additional content that complement the in-class lesson. In contrast with blended learning, in hybrid learning a significant portion of the course takes place online. As a result, in a hybrid learning scenario much of the student-teacher “face time” in a brick-and-mortar location is replaced by online interaction [5].

3 Online Learning and Music Online education systems serve mainly distance learning and can be proved as extremely helpful and successful in many disciplines, especially of theoretical nature. Another area with high rates of success in online learning is computer science, where programming languages and special purpose software can be mastered in their “natural environment”. In general, studies show that online learning is modestly more effective, on average, than the traditional face-to-face instruction. With the use of technology, Internet and multimedia dynamics, any course of theoretical nature can be presented in multiple desirable levels of complexity and accuracy, maybe even more effectively that its presentation in the traditional way of teaching. However, is this also true for music education? Music contains many different aspects and considerations (history of music, morphology, harmony, performance etc.), which cannot be integrated into the same context. For instance, history of music could well be taught with an on-line model with good results, as it is a theoretic course. There are already such online courses available. However, performing a musical instrument (or vocal/singing performance) does not belong to this category. In fact, there could be a parallelism with the science of medicine [6]. Would it be possible for somebody to be certified as a doctor (e.g. a

48

R. Tzimas et al.

surgeon) only from an online distance learning program? The answer is rather negative for obvious reasons. The main reason is: practice. The same stands for music too. Learning how to play a musical instrument or how to sing needs a lot of practice time (apart from understanding the theoretical rules). Consequently, this kind of learning requires cultivating specific skills [7]. If a student is left alone learning how to play the piano or sing in an autonomous and remote way, it is most likely not to perform well and eventually fail in the course. There are two reasons for that: technique and aesthetics. There is absolutely the need for guidance in person by the teacher possessing the specific skills and relevant experience. Even in the case of evaluating some kind of distant learning for such a discipline like music, there must be systems of special requirements to support it: high fidelity sound and a sense of space, fully synchronized supervision of the technique and bi-directional functionality. Mislearning of such a discipline may lead to repeated mistakes during practice by the learner. As a result, not early-corrected mistakes may lead to bad habits of practice, and, thus, improper performance. Of course, the audience that needs to learn a new skill (e.g. playing a musical instrument) should be taken into account. In case of beginners or people that want to learn something just for hobby, TV viewing or even YouTube videos with instructions prove to be adequate (amateur learning) [8]. This method, however, is not suggested for an academic course or for advanced studies at a professional level [9]. Although not much research is available in the literature concerning the effectiveness of a purely online model for teaching music performance, it seems that a blended approach to teaching music may be an effective solution that addresses the feedback component essential in performance-based courses, while retaining the benefits of an online-learning approach [5], as seen in Fig. 2.

Fig. 2. The musicological context (with some kinetic elements as well) for learning on-line the musical setting of an antiphon, historically sang in Eastern Mediterranean without parallax for more than 15 centuries. Composed by Prof. N.-K. Paris.

Interactive TV and Music Education

49

4 iTV for Educational Programs Interactive Television (iTV) in its current and upcoming forms has a great deal of potential for facilitating interactive, constructivist, situated and authentic learning. Several attempts have been made in the past to introduce iTV educational programs in schools worldwide. Some attempts were successful, while others showed indifference in their results comparing to traditional methods of learning. A range of educational content has been successfully delivered in many University environments worldwide with live interactive broadcasting and website interactivity features sometimes included. As a result, three different teaching modes (traditional face-to-face instruction, face-to-face instruction combined with originating an iTV class, remote classes receiving iTV transmissions) are ubiquitously compared in terms of the overall students’ ability to receive skillful instruction. Interestingly, as shown by their outcome, students of both faculties are most interested cumulatively in online quizzes as far as online teaching methods are concerned. Concerning foreign language learning, television as a medium has several advantages, especially for young learners, even in the form of passive viewing. The advent of Internet television has made content available to any internet-enabled devices and the viewer has yet access to programs in an anywhere-anytime manner. Extensive viewing of a TV program or TV series can improve both viewer’s listening comprehension and vocabulary, and the same situation is reported when music cognition is parallelized with language education [10]. The features that make television viewing a useful outof-class learning activity: accessibility, intensity of exposure, motivational, flexibility, multimodal and notice gaps [11].

5 iTV for Music Education: The Case of Byzantine Music Byzantine music, unlike the “usual” Western music, is characterized by features that make it harder for technical analysis and editing of musical phrases. These peculiarities concern both the rhythm and the tonic value of each note. Many times in its scores fuzziness is incorporated, leaving room to the performer to add his own interpretation and mentality, as opposed to the rather fixed stylistic explanation pertained in CMN scores. Besides, it is no coincidence that there is encouragement for improvisation, eagerly attested by a number of choirs and distinguished performers in general, mainly in traditional Byzantine music songs. Therefore, the number of performances of the same piece should not be considered static and always the same; on the contrary, it is commonplace to have diligent variants of historical compositions throughout the last decades, or better say, since the time we have recordings of such performances. However, the limits of improvisation and the boundaries of personal character stirred up for each performer are clearly defined by the music teachers and theorists. Byzantine and Western Music The following peculiarities are the basic elements that differentiate the two cultures. It is necessary to present them in order to understand the diversity involved in each performing paradigm, under the prism of instruction through iTV practices.

50

R. Tzimas et al.

Western music forms its scales according to fixed interval patterns. A typical example is the physical scale of the C major scale, and generally speaking the major diatonic mode, which permeates all contemporary musicality. When it comes to TV, for example, it is flooded with tons of such hearings, causing to its viewers a significant neurotropic submergence. On the other hand, the musical paradigm of Eastern countries relies heavily on the concept of Genre. Genres (from French, meaning “kind” or “sort”, originating from Latin GENUS and Greek CENOR) were consistently used for many centuries to classify in ordered sets of similar works literature pieces along with other forms of art and entertainment [12]. As new forms of music are invented, genres may be altered, discontinued or mixed together to produce new forms [13]. It is also possible for opuses of music to fit into two or more categories. As a result, the way that music is classified may be different when encountered as a commercial activity, strongly biased by the production tactics of the prolific music industry, and when academic criteria are applied. One of the greatest differences spotted between the so-called “Western” and “Eastern” Music is related to the musical scales and modes that are used in both cases. Eastern modes (e.g. oriental music, Byzantine music, etc.) have a very wide range of microtonal intervals and therefore strong use of music chromaticism, both in notation and during performance [14].

Fig. 3. The diatonic scale of Byzantine Music compared: (a) to the dominant Major scale, with the possible sharps and flats (b) to chromatic and enharmonic scales, without accentuations. Intervals denoted in echomoria.

This is pictorially explained in Fig. 3: the chromatic scale of Western Music, in its most decorative form, as far as semitones are concerned, is parallelized, in terms of similarity, with the most diatonic of the Byzantine Music scales, that of the 8th Mode. The strong lines denote the fixed notes and the intervals formed between them, in

Interactive TV and Music Education

51

echomoria, rather than in cents. The full octave thus extends to 72 echomoria, having as equivalent 1200 cents. Byzantine music does not strictly impose a specific frequency on every note, but only observes intervals during the course of the member. Thus, the possibility of identifying a note based on its specific frequency per se is rejected. However, in choral singing, it becomes obvious that common bases must be kept, between the choir members or the singing priests (Fig. 4).

Fig. 4. The choir of the Archon Protopsaltis in the Vlatadon Monastery. Keeping the tonal bases, rhythmic and melodic contours is basic priority of the choir leader.

All the above theoretical and practical should stances should exist in learning pretext content, in an iTV platform that is offering genuine training in Byzantine music. Diversity Problems It is necessary to refer to a typical example of different executions of the same musical text in order to understand the extent of the diversity presented by the singing music. The microtonal nature of Byzantine Music hymnology inhibits the danger of barely discernible states in executions to audiences with no prior experience in chromatic or otherwise Eastern hearings. As the dominant paradigm of musicality is clearly shifted to Western Music diatonicism, it is difficult for the non-expert to sing in authentic chromatic mode and understand in practice the particularities and features involved. Consequently, the novice members of the choir or the remote learners have an inherent difficulty in performing correctly. Therefore, iTV provides, perhaps, the only viable methodology to remedy the mishap. The learner is instructed to repeatedly indulge in the prototypal singing,

52

R. Tzimas et al.

hearing it by its mobile phone or TV again and again, until a clearly he has reached the faculty of memorizing the perceived sounds. This is critically true when reproducing “ancient” landmark melodies, dating from the first centuries AD. Only iTV or video lessons seem to be the proper Learning Objects to transfer to large audiences the teacher-student listening comprehension and synchronization with absolute accuracy. As a consequence, only this scheme, thus far, apart from the obvious, to be part of choir for at least quite a while, is able to provide the greater possibilities each student needs to render exactly the prototypal performance [15]. In Fig. 5 snapshots of the video lesson are depicted, in the self-training module of a Learning Object in Mode 2. As the student may become confused by the complexity of an hymn sang since the 5th century AD distinctively chromatically, the students should keep in their singing apart from the proper tonal height the appropriate rhythm and intonation (Fig. 5).

Fig. 5. Sequences of snapshots from the Learning Objects. The learner is instructed to modulate his singing according to the strong rhythmic elements of the hymn, as specifically directed by the hand moves of the conductor.

When the student has mastered the melody, he should be able to spot the perceptible differences between the Western Music rendition, and the original one.

6 Conclusion: The Influence of “Television” on Music The recent history of audiovisual pervasiveness shows that the strongest music influences affect primarily the style of execution, i.e. the tonality and rhythm of melody, while the less powerful, in terms of popularity, alter the qualitative characteristics of performance, like timbre, ensemble cohesion, polyphonic elements separation, etc. In 2002, with the release of the first Mp3 staff, the music industry is changing its form and way of cruising. Listening becomes cheap, easy and fast through a very small device. It is also productive in terms of cognition, reshuffling of sounds, apprehension of the accompanying semantics and interaction with a huge social network, amongst which some are willing to cooperate experts in he field.

Interactive TV and Music Education

53

In the years after 2010 with the huge explosion in the use of YouTube applications, Spotify, iTunes, and similar, listeners may enjoy the music they favor while in parallel exerting a significant learning potential, provided that they access within the Internet the appropriate music Learning Objects. The qualitative research finding of these survey have found that the world’s audience is shaped by new releases made daily. Music is used everywhere and is accessible from everywhere. It seems clear, that music iTV publicity does only promote gains for entrepreneurial ventures, but, most important, shapes the commonly attested features of a globally immersed community. National music, traditional music, Byzantine Music and the classics themselves are reshaped by a strong biasing mechanism readily promoted by smartphone viewing, interacting and neurophysiologically “learning” the features of a new brave world for singing.

References 1. Paulsen, M.F.: Online Education Systems: Discussion and Definition of Terms (2002). https://pdfs.semanticscholar.org/d8ce/57254f386171d0fbd8aeafbcd313b4fcd743.pdf 2. Liu, N.-H., Chiang, C.-Y., Chu, H.-C.: Recognizing the degree of human attention using EEG signals from mobile sensors. Sensors (Basel, Switz.) 13(8), 10273–10286 (2013). https://doi.org/10.3390/s130810273 3. Wegerif, R.: The social dimension of asynchronous learning networks. J. Asynchronous Learn. Networks 2(1), 34–49 (1998) 4. Conole, G.: The role of “mediating forms” of representation in learning design. In: Networked Learning Conference, Lancaster, UK (2006). https://www.lancaster.ac.uk/fss/ organisations/netlc/past/nlc2006/abstracts/pdfs/P32%20Conole.pdf 5. Lunce, L.M.: Simulations: bringing the benefits of situated learning to the traditional classroom. J. Appl. Educ. Technol. 3(1), 37–45 (2006) 6. Milner, B.: Laterality effects in audition. In: Mountcastle, V.B. (ed.) Interhemispheric relations and cerebral dominance, pp. 661–672. John Hopkins University Press, Baltimore, MD (1962) 7. Altenmüller, E., Gruhn, W., Parlitz, D., Liebert, G.L.: The impact of music education on brain networks: evidence from EEG-studies. Int. J. Music Educ. 35, 47–53 (2000) 8. Prout, E.: Harmony: Its Theory and Practice. University Press of the Pacific, Honolulu (2004) 9. Baeck, E.: The neural networks of music. Eur. J. Neurol. 9, 449–456 (2002) 10. The academy of neurologic music therapy. In: Memory of Robert F. Unkefer. https:// nmtacademy.co/ 11. Sulema, Y.: Mulsemedia vs. multimedia: state of the art and future trends. In: Proceedings of the 23rd IEEE International Conference on Systems, Signals and Image Processing IWSSIP 2016, Bratislava, Slovakia, pp. 19–23 (2016) 12. Mavroedis, M.: The Musical Modes in Eastern Mediterranean Sea – The Byzantine Echos – The Arabic Maqam – The Turkish Maqam (in Greek). Fagotto Editions, Athens (1999) 13. Hagel, S.: Reversing the abstraction of ancient music theory. the case of the genera. Studien zur Musik-archäologie Orient – Archäologie 22, 461–475 (2008) 14. Giannelos, D.: La Musique Byzantine. L’ Harmattan, Paris (1996) 15. Gomez-Pinilla, F., Hillman, C.: The influence of exercise on cognitive abilities. Compr. Physiol. 3(1), 403–428 (2013)

Assessment, Evaluation and Research Methods in Mobile Learning

An Analysis for the Identification of Use and Development of Game Design Strategies as Problem Posing Activities for Early Childhood Learners George Kalmpourtzis1(&), Margarida Romero2, Cindy De Smet2, and Andreas Veglis3 1

Infinitivity Design Labs, Laussonne, France [email protected] 2 Laboratoire d’Innovation et Numérique pour l’Education, Université Côte d’Azur, Nice, France {margarida.romero,cindy.de-smet}@univ-cotedazur.fr 3 Aristotle University of Thessaloniki, Thessaloniki, Greece [email protected]

Abstract. Examining and identifying the way that designers approach game design could support the improvement of game design curricula. This paper explores how kids in the early childhood approach develop and apply game design strategies during participatory game design sessions. During a period of three months, eighteen kindergarten learners participated in game design sessions, creating their own games. The data analysis suggests that those learners applied different game design strategies than the ones they initially started using. The frequency and intensity of the use of those strategies also changed, showing patterns that indicate learners’ development in their use of game design strategies and add empirical evidence to the field of game design education. Additionally, the Game Design Strategies Analysis (GDSA) is presented, which elaborates on the different types of strategies encountered when designing games. Keywords: Game design


Game based learning
Participatory design

1 Introduction and Theoretical Background Rapid and continuous technologic advances have strongly impacted modern education. An educational aspect with a continuously increasing research interest is that of game design, both as a tool for the support of curricular activities but also for their potential to create meaning making of children’s developing capacities in diverse contexts and practices [1]. Additionally, research interest on the development of game design skills has been developed in the last years [2]. Learning about game design is related to the general notion of learning by design, which requires the combination of problem-solving, critical thinking and creative thinking skills [3]. This process has been addressed through different approaches and © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 57–68, 2021. https://doi.org/10.1007/978-3-030-49932-7_6

58

G. Kalmpourtzis et al.

practices in order to establish and facilitate students’ involvement in the game design process [4]. According to Schell [5], games could also be viewed as problem situations, engaging the participant in a playful attitude. Hence, game design could be considered similar to problem posing [6]. The development of strategies to solve and design mathematical problems has been identified as a key aspect of students’ mathematical thinking [7]. So, the development of game design strategies by students could potentially help them better understand the process of game design. Game design education has also been addressed through the prism of participatory design. Designing games with the participation of students has been a topic of increasing interest in the field of human computer interaction. Previous studies have focused on students’ engagement in the design process [8], in the evaluation and communication of the process [9] and in organizing the design process through the creation of design tools [10]. The current paper aims at examining which types of game design strategies are developed and used by students in the early childhood while designing games during organized participatory game design interventions. The paper also presents the Game Design Strategies Analysis, a framework that stemmed from the qualitative analysis of this study, taking into account previous work in this field. 1.1

Game Design with Students

Engaging learners in the role of game designers is not a new concept. Previous research has focused on the impact of students’ involvement in game design on their narrative, cognitive, creative and design skills [11]. Design sessions with young partners have been studied through the prism of cooperative inquiry [12]. Those studies showed a positive impact on students’ interest in the field and the understanding of the concept of game design [13], the construction of knowledge and raised the question of the transfer of acquired knowledge and skills in other contexts [14]. Several frameworks have been proposed for the analysis and design of games in learning contexts [15]. Each of those frameworks approach game design through different perspectives, including game and learning mechanics, flow, interaction, decision making and problem solving and posing. Research has also focused on the identification of common patterns in order to address problems and issues that frequently appear in game design processes [16], while others have focused on the use of design strategies for greater player engagement [17]. Games have been examined as problemsolving environments. One approach considers problems presented in games and classifies them as either ill-structured or well-structured [18]. Contrary to wellstructured problems, which have fixed answers, ill-structured problems are more open and urge players to come up with diverse problem solving strategies [19]. 1.2

Problem Posing Strategies in Mathematics and Game Design Education

Problem posing has been the subject of research interest mainly for its connection to problem solving [20]. Problem posing education has proved to be a challenging endeavor, because of the subject’s difficulty [21]. Games, creating intrinsically

An Analysis for the Identification of Use and Development of Game

59

motivating learning experiences, are proposed as a tool for the facilitation of problem posing education [22]. In order to structure better learning environments, different approaches on identifying and analyzing problem posing strategies have been proposed [23]. Those approaches include the use of relevant questions, generating problem posing strategies and modifying existing problems in order to create new ones. To facilitate problem posing instruction, Stoyanova [24] classified problem posing situations as: Structured, Semi-Structured and Free of structure. Similar approaches have been proposed for the design of game spaces.

2 Methodology 2.1

Participants

The study involved a group of eighteen children in the kindergarten, eight girls and ten boys. The participants were 5–6 years old and were all children of the same kindergarten classroom, located at a suburb of Thessaloniki, Greece. 2.2

Setting

Game Design Situations For a period of three months, the study’s participants were introduced to two hourly game design sessions per week. The sessions would not occur during the same day. The structure of game design sessions was also based on participatory design techniques and the problem posing analysis by Stoyanova [24], as well as the Kalmpourtzis’ [25] structured, semi-structured and free situations of creating games. • Structured game design situations included already designed games that learners needed to reformulate or redesign at will, starting from a concrete and defined basis. • Semi-structured game design situations provided learners with half-finished games, requiring from learners to come up with their own fully working games. In semi-structured game design situations, players were asked to design their own games, taking into consideration those elements as a scaffolding mechanism. • Free game design situations did not offer external support to players, obliging them to design their own games in any way they deemed preferable. These different types of design situations were proposed to create a scaffolding mechanism where learners would be accustomed to the proposed game design process and familiarize themselves with the process of creating games. Game Design Sessions For the context of game design, an iterative process consisting of four steps was proposed. Those steps included: • Understanding the game: learners identify their resources and define their objectives, understand their technical restrictions and decide the direction they will take towards the design of their games.

60

G. Kalmpourtzis et al.

• Designing the game: learners brainstorm, exchange ideas, analyze the proposed game components, leading to their games. • Implementing the game: players construct their game prototypes. • Presenting and looking back: players present their games, observe their peers play them and receive their feedback. Game design sessions focused on the creation of low-tech prototypes for both physical and digital games. For this purpose, different kinds of materials were supplied to learners. Low-tech prototypes included wireframes, content representations and they were accompanied with instructions learners needed to present at the end of each design phase. During each session, learners worked in small groups [26]. Each group consisted of three to five participants and an adult observer. The role of the observer was to attend and take notes, as well as facilitate the team’s design process but only in cases where the team members were stuck [27]. The selection of the team’s members aimed to maintain the balance between the age of the children (teams are composed by 5 and 6 years) and the mix between boys and girls. During the period of the study, learners were also presented with a variety of different games (board games, outdoor games, card games, treasure-hunt games) and technologies in order to be aware of their various options and creative possibilities [12]. These activities included both physical and digital games. 2.3

The Game Design Strategies Analysis

The Game Design Strategies Analysis (GDSA) was proposed based on the work of Stoyanova and Ellerton [28]. The GDSA was proposed before the design sessions as a starting, considering that it will be modified, based on the findings of the current study. The initial GDSA consists of five main categories: 1. Reformulation game design strategies: describing cases where game elements are only rearranged or differently presented, with no apparent changes on gameplay. 2. Reconstruction game design strategies: describing cases where the content or materials of game components change with no other apparent change on game components and their interaction. 3. Imitation game design strategies: describing the modification of existing components, based on previously encountered experiences. 4. Expansion game design strategies: describing the expansion of existing games and components, based on previously encountered experiences and components. 5. Invention game design strategies: describing the proposal of novel, not previously encountered game components.

3 Design of the Research and Data Analysis For the present study, a qualitative research methodology was used. The selection of a qualitative methodology was also supported from the understanding that in order to better examine the process of designing games, several other factors needed to be taken

An Analysis for the Identification of Use and Development of Game

61

into account, such as practices, common patterns, interaction among peers, thought processes and emotions, which are difficult to extract through conventional research methodologies [29]. Our study focuses on the collection of data from the conversations among peers during design teams, their actions and decisions and their final creations. The collection of data was carried out in two ways. From one side, semi-structured interviews [30] with each participant were conducted at the beginning and the end of the interventions period. These interviews focused on the application, suggestion and incorporation of strategies and common practices of learners while they were designing their games. The interviews presented learners with three game design situations, one structured, one semi-structured and one free, during which leaners needed to create their own games, digital or physical, according to their preference. From the other side, learners’ work in design teams was audio recorded with the intention of later analysis. Additionally, the works of the design teams were collected and stored to provide additional information for the analysis of data. The findings deriving both from personal interviews and design teamwork were all transcribed. A line-by-line coding was used to identify emergent themes. The GDSA acted as a first point of reference. Initially codes would be based on the five strategy categories proposed by the initial GDSA. Through these codes and the analysis of teams’ deliverables and observers’ notes, categories were identified through a flexible category standard and an iterative process, where categories would emerge, change or be refined based on the constant examination of incoming information. A “thorough and interrogative” approach to data [31], where information would be cross-checked between cases, interviews, audio recordings and produced work did take place. This iterative process led to the refinement and re-structuring of a final GDSA because of the qualitative analysis that was conducted.

4 Results 4.1

Game Design Strategies

The final and revised GDSA (Fig. 3) describes the strategies incorporated in game designers’ decisions while preparing, implementing and presenting their games. Those game strategies are also related to learners’ perception of the structure and consistency of games, their structural elements, materials, representation and interaction with players. The analysis of data finally showed four principal game strategy categories: Reformulation, Reconstruction, Expansion and Invention, each of which consists of different subcategories. After the analysis of data, the initially proposed strategy category of Imitation was eventually merged with the category of Expansion. A detailed analysis is presented below.

62

G. Kalmpourtzis et al.

Reformulation Game Design Strategies Reformulation strategies describe a set of game design strategies during which designers’ resort to changing the presentation of a given game by altering its initial presentation description, and without impacting the initial game in any way. Learners used reformulation strategies in several occasions. Those cases were grouped in three subcategories. The first one is related to learners’ reformulation and different presentation of an existing game. When applying strategies of this category, learners were deliberately changing words related to the description of given games or were just repeating the initial game instructions with another word sequence. For instance, during a session where the base game was Hide and Seek, a team presented the same game by just rephrasing the initial instructions. The second subcategory describes players’ involuntary response to shifting away from an existing game implementation. When this strategy was encountered, learners wanted to recreate an exact replica of a previously encountered game. This is seen in the following discussion between learners, who were designing a matching game. One learner, proposed the use of color matching between cards, which was an addition to the game’s existing presentation where there were some animal images: Learner 1: This is not a butterfly that goes with a butterfly. The teacher said we should create the same combination: butterfly with butterfly. Learner 2: Yes, but this is red. They both have a red color. Learner 1: Yes, but it’s not a butterfly with a butterfly, a crown with a crown, and a princess with a princess. The third reformulation subcategory is related to providing playable examples of how a whole game or some parts of it, are being played. In such strategies, players do not change the game and do not repeat its instructions but provide a playable example of how the game could be played to present it to other learners who are potential codesigners or players. For example, when playing Snakes and Ladders, a team, instead of presenting the game’s instructions, gave an example of how it is played. The team had not changed the game at all during their design session and only focused on how to present it to their peers, which eventually ended up being in the form of an example. Reconstruction Game Design Strategies Reconstruction game design strategies describe the use of the same game mechanics of an existing game while changing the content, material and sizes of game components, but without affecting the nature of the game. In reconstruction game design strategies, learners addressed games as a defined system whose attributes are prone to change. Four subcategories for this category group were identified during this study. The first subcategory is related to the modification of symbolic content of game elements. When this type of strategy was applied, learners would change the content of cards, videos or board games. However, in this type of strategy, the final game would not change in nature. As presented in Fig. 1, changing the symbolic content led learners to redesign the video game Forest Maths, while they did not perform any other type of change on its game play.

An Analysis for the Identification of Use and Development of Game

63

Fig. 1. Forest Maths and learner prototype

The second subcategory is connected to the modification of game component sizes. This subcategory could be potentially merged with the first one, but its high frequency during the analysis led to the proposal of a separate subcategory. Strategies of this subcategory describe learners’ decision to change the size of the game objects, the repeatability of events, like the number of rounds required to complete a level, the number of game elements required for the completion of a game, the number of players, team sizes, point number or time duration. Figure 2 represents a prototype for a puzzle game, initially based on the concept of Forest Maths, where the number of holes and their position has been increased.

Fig. 2. The result of intentional modification reconstructive game design strategies

The third and fourth subcategories describe the rearrangement and modification of spatial elements and time sequences of events during the game. The third subcategory describes the repositioning of game components in different spatial arrangements. For example, for the game Snakes and Ladders, learners decided to arrange the tiles differently than the original square arrangement. The fourth subcategory describes the repositioning of game events in different chronological order. For example, for the game Forest Maths, a platform game where players need to complete patterns on a line to allow bugs to pass to the other side, learners decided to modify the intervals of bugs’ appearance so that they change the difficulty of the game.

64

G. Kalmpourtzis et al.

Expansion Game Design Strategies Expansion game design strategies describe a set of strategies used to expand the structure of a familiar game or game element with the incorporation and refinement of other known elements or whole games. Even if expansion strategies result to possible new game experiences, they describe cases where proposed game structures were based on previously known games or game components. Expansion strategies also include the total or partial omission or removal of game elements from a game. Expansion strategies consist of two subcategories. The first one describes the set of strategies that apply the use of familiar games or game structures to expand an existing game concept or base or the use of such structures in order to start a game from scratch. In one of the game sessions, learners that were influenced by the video game PiBot: Math & Action, decided to incorporate different familiar elements to expand the game. Consequently, they added new rules that corresponded to other games that were familiar to learners, such as collecting artefacts, defeating enemies by bumping on their heads and using super-powers. During a session the following discussion among learners took place: Observer: What type of game would you like to create? Learner 1: I propose French hide and seek! Observer: What is French hide and seek? Learner 1: One kid will hide and all the others are chasing. Observer: And why did you decide to name it like this? Learner 1: This is how I call it. It’s played outside in the school yard and this is how it’s played in France. This discussion shows that the learner modified the rules of the familiar game hide and seek, by changing its gameplay. The second subcategory describes strategies of generalizing or transferring a game structure and incorporating it as a component of another game or using the mechanics of existing game elements to their own. During one session, learners decided to create a game, based on their own narrative, inspired by the Turtle Mutant Ninja franchise. Wanting to keep the narrative element, they decided to create a game based on the mechanics of Snakes and Ladders. The designers generalized the use of one game to incorporate their narrative elements. Invention Game Design Strategies Invention game design strategies describe the use of strategies that led to the creation of new information or structures related to the produced games. Learners that use those strategies come up with new structures that are not relevant to previously familiar games or game elements. Invention strategies are related to whole games or game concepts. This study identified the use of invention strategies to expand familiar game structures with new and unique elements. Not one single complete game came up as the result of using this strategy.

An Analysis for the Identification of Use and Development of Game

Reformulation

Reconstruction

Change of presentation and description

Modification of content

Expansion

65

Invention

Expansion based on familiar logic Modification of intensity / repetition / magnitude

Repeating existing situations Modification of spatial components

Use of examples

Modification of chronological sequences

Expansion through generalizing

Fig. 3. The revised GDSA after the qualitative analysis

5 Discussion and Conclusions This study, which examines the use of strategies by learners during participatory game design sessions, offers a perspective on the nature and way of utilization of different types of game creation strategies. The use and development of the strategies presented by the GDSA was examined during the three months of interventions. The use and development of all encountered strategies was examined during three different phases, each of which corresponded to the three game design situations: structured, semistructured and free ones (Table 1). Each period corresponds to approximately a period of 4–5 weeks. The analysis shows three patterns that evolve during time. The first one is related to the decrease in the use of reformulation and reconstruction strategies. The use of those two strategy categories are very frequent during the first game design sessions and fall rapidly in the two last phases. The first one is the reversely proportional utilization of reformulation and reconstruction strategies during the duration of the sessions. Learners were observed to use such strategies at the early sessions and can be linked to the fact that learners were new to the design process, and were not accustomed to their functions as designers. To this direction points also the increased number of random uses of strategies with a trial and error approach on behalf of them, that tends to decrease later in the duration of the study. The great number of usages of different strategies during the first period, in comparison to the others, also points the experimental approach of learners, during which they resorted to a continuous and not always intentional or purposeful, proposition of strategies to create their games.

66

G. Kalmpourtzis et al. Table 1. Frequency of game design strategies in the course of the study

Reformulation strategies Reconstruction strategies Expansion strategies Invention strategies

Situation type Structured – phase 1 51

Semi-structured – phase 2 32

Free structured phase 3 35

71

38

33

5 0

17 2

33 5

The second one is connected to the increase of use of expansion and invention strategies during time, in the context of this study this period was three months. The second tendency is described by the proportional increase of use of expansion and invention strategies over time. This increase is also followed with a more cautious, intentional and targeted use of strategies during game design sessions. The increase of use of those types of strategies is also related to the nature of the tasks that design teams were asked to design. As part of the scaffolding mechanism that the study examined, the cases that learners would face would be structured, semi-structured or freestructured. As a result, initially learners would be asked to change existing games, then start designing games from half-made games and eventually come up with their ones. As presented in Table 1, the frequency of using different types of strategies is different in those phases. From one side, time and familiarity may play a role in this phenomenon. Additionally, the nature and different demands in terms the nature and structure of the activity in each of the three design situations may also play a role in the use of learners’ strategies. The third one is connected to the decrease in the overall use of strategies throughout the course of time. During phase one, the number of strategies used is almost twice as big as the number of strategies encountered during the next two phases. The strategies used during phase one are mainly reformulation and reconstruction and they consist of several trial and error efforts from learners’ side. This aspect is related both to the decrease in trial and error strategies, familiarity with the design process and resultant observation of the previous two observations. Consequently, this difference in the use of different strategies both as design situations change from structured to semi-structured and free-structured in the course of time indicating that experience and familiarity with game design processes impact the way that learners use different game design strategies that involve the recollection, identification and combination of other game elements or invent totally new game concepts. The shifting from reformulation and reconstruction strategies towards ones that fall into the categories of expansion and invention also seems to be followed by a decrease in the use of trial and error or arbitrary use of strategies without a purpose. Further studies around the topic, where the GDSA will be used and reflected upon will shed more light in the framework’s capacity to describe game design strategies during the process of game design.

An Analysis for the Identification of Use and Development of Game

67

References 1. Kafai, Y., Peppler, K.: Developing gaming fluencies with scratch. In: Steinkuehler, C., Squire, K., Barab, S. (eds.) Games, Learning, and Society: Learning and Meaning in the Digital Age, pp. 355–380. Cambridge University Press, Cambridge (2012). https://doi.org/ 10.1017/CBO9781139031127.026 2. Kalmpourtzis, G.: Developing kindergarten students’ game design skills by teaching game design through organized game design interventions. Multimed. Tools Appl. 78(14), 20485– 20510 (2019) 3. Kalmpourtzis, G.: Educational Game Design Fundamentals: A Journey to Creating Intrinsically Motivating Learning Experiences. A K Peters/CRC Press, New York (2018) 4. Fails, J.A., Guha, M.L., Druin, A.: Methods and techniques for involving children in the design of new technology for children. Found. Trends Hum.-Comput. Interact. 6, 85–166 (2013) 5. Schell, J.: The Art of Game Design: A Book of Lenses. CRC Press, Amsterdam (2014) 6. Chang, K.E., Wu, L.J., Weng, S.E., Sung, Y.T.: Embedding game-based problem-solving phase into problem-posing system for mathematics learning. Comput. Educ. 58, 775–786 (2012) 7. English, L.: Children’s problem posing within formal and informal contexts. J. Res. Math. Educ. 29, 83–106 (1998). https://doi.org/10.2307/749719 8. Djaouti, D., Alvarez, J.: The creation of newsgames as a teaching method - empirical observations. In: Kalmpourtzis, G. (ed.) Educational Game Design Fundamentals: A Journey to Creating Intrinsically Motivating Learning Experiences, pp. 72–77. CRC Press, Boca Raton, FL (2018) 9. Malinverni, L., Mora-Guiard, J., Pares, N.: Towards methods for evaluating and communicating participatory design: a multimodal approach. Int. J. Hum. Comput. Stud. 94, 53–63 (2015). https://doi.org/10.1016/j.ijhcs.2016.03.004 10. Triantafyllakos, G., Palaigeorgiou, G., Tsoukalas, I.A.: Designing educational software with students through collaborative design games: the we! design & play framework. Comput. Educ. 56, 227–242 (2011) 11. Könings, K.D., Brand-Gruwel, S., van Merriënboer, J.J.G.: Participatory instructional redesign by students and teachers in secondary education: effects on perceptions of instruction. Instr. Sci. 39, 737–762 (2011) 12. Druin, A.: Cooperative inquiry: developing new technologies for children with children. Hum. Factors Comput. Syst. 14, 592–599 (1999) 13. Bermingham, S., Charlier, N., Dagnino, F., Duggan, J., Earp, J., Kiili, K., Luts, E., Van Der, S.L., Whitton, N.: Approaches to collaborative game making for fostering 21st century skills. In: Proceedings 7th European Conference Games-Based Learning, pp. 45–52 (2013) 14. Habgood, M.P.J., Ainsworth, S., Benford, S.: Intrinsic fantasy: motivation and affect in educational games made by children. Learn. 36, 483–498 (2005) 15. Arnab, S., Lim, T., Carvalho, M.B., Bellotti, F., de Freitas, S., Louchart, S., Suttie, N., Berta, R., De Gloria, A.: Mapping learning and game mechanics for serious games analysis. Br. J. Educ. Technol. 46, 391–411 (2015) 16. Kelle, S., Klemke, R., Gruber, M., Specht, M.: Standardization of game based learning design. In: Murgante, B., Gervasi, O., Iglesias, A., Taniar, D., Apduhan, B.O. (eds.) ICCSA 2011. LNCS, vol. 6785, pp. 518–532. Springer, Heidelberg (2011) 17. Dickey, M.D.: Game design narrative for learning: appropriating adventure game design narrative devices and techniques for the design of interactive learning environments. Educ. Technol. Res. Dev. 54, 245–263 (2006)

68

G. Kalmpourtzis et al.

18. Hong, N.: The relationship between well-structured and ill-structured problem solving in multimedia simulation. Unpublished doctoral dissertation. The Pennsylvania State University, University Park (1998). www.cet.edu/research/papers.html 19. Papert, S.: The children’s machine. Technol. Rev. Nh-. 96, 28 (1993) 20. Crespo, S., Sinclair, N.: What makes a problem mathematically interesting? Inviting prospective teachers to pose better problems. J. Math. Teach. Educ. 11, 395–415 (2008). https://doi.org/10.1007/s10857-008-9081-0 21. Mestre, J.P.: Probing adults’ conceptual understanding and transfer of learning via problem posing. J. Appl. Dev. Psychol. 23, 9–50 (2002) 22. Chang, M.: Edutainment technologies. educational games and virtual reality augmented reality applications. In: 6th International Conference on E-learning and Games, Edutainment 2011, Taipei, Taiwan, September 2011 23. Kojima, K., Miwa, K.: A system that facilitates diverse thinking in problem posing. Int. J. Artif. Intell. Educ. 18, 209 (2008) 24. Stoyanova, E.N.: Extending and exploring students’ problem solving via problem posing (1997). https://ro.ecu.edu.au/theses/885 25. Kalmpourtzis, G.: Connecting game design with problem posing skills in early childhood. Br. J. Educ. Technol. 50, 846–860 (2019) 26. Patton, Q.M.: Qualitative Evaluation and Research Methods, 2nd ed. Sage Publications Inc., Newsbury Park, London, New Dehli (1990) 27. Kalmpourtzis, G., Vrysis, L., Ketsiakidis, G.: The role of adults in giving and receiving feedback for game design sessions with students of the early childhood. In: Auer, M.E., Tsiatsos, T. (eds.) IMCL 2017. AISC, vol. 725, pp. 266–275. Springer, Cham (2018). https:// doi.org/10.1007/978-3-319-75175-7_28 28. Stoyanova, E., Ellerton, N.: A framework for research into students’ problem posing in school mathematics. technology in mathematics education. In: Proceedings of the 19th Annual Conference Mathematics Education Research Group Australas. (MERGA), June 30– July 3 1996 Univ. Melbourne, 1996 (1993). ISBN 0959684468 29. Strauss, A., Corbin, J.: Basics of qualitative research: grounded theory procedure and techniques. Qual. Sociol. 13, 3–21 (1990) 30. Fontana, A., Frey, J.: The interview: from structured questions to negotiated text. Collect. Interpret. Qual. Mater. 66, 911–917 (2006) 31. Barbour, R.S.: Checklists for improving rigour in qualitative research: a case of the tail wagging the dog? BMJ 322, 1115–1117 (2001)

Measuring Knowledge Gains in an SMS m-Learning Intervention: The Case of ChildConnect South Africa Nicky Roberts(&), Ingrid Mostert, and Lydia-Ann Plaatjies University of Johannesburg, Johannesburg, South Africa [email protected], [email protected], [email protected]

Abstract. ChildConnect is an early childhood development programme which aims to use mobile learning to support and educate parents and other caregivers. The programme is based on an SMS curriculum focused on early childhood development content, underpinned by six core learning outcomes. In a pilot in 2017, 899 primary and secondary caregivers signed up to receive the messages. Caregivers were randomly assigned to either a treatment or control group. The treatment group received 3 messages a week for 24 weeks while the control group received one message every two weeks. Pre and post-tests were administered using ‘Unstructured Supplementary Service Data’ (USSD) surveys on mobile phones. Results of the tests were combined into a composite learning outcome score. This paper explores preliminary evidence of knowledge gains, comparing pre-test outcome scores to post-test outcome scores across treatment and control groups. The treatment group performed better on their learning outcome score than those in the control group by *0.7 points (Cohen’s d = 0.21, small effect size, but non-trivial). This shows that there was a small but measurable positive effect on the knowledge gains of the treatment group compared to the control. Keywords: m-learning Knowledge gains


South Africa
Parenting programme
Evaluation


1 Introduction The absence of public communications strategies to support primary caregivers at scale has been noted across 91 countries [1]. Harnessing m-learning - defined by [2] as ‘learning across multiple contexts, through social and content interactions, using personal electronic devices’ (p.4) - can be a cost effective way to offer a national early childhood parenting programme. To this end, short message service (SMS) is particularly attractive in contexts such as South Africa, where adults in low income communities tend to have access to feature (and not smart) mobile phones. [3] note that ‘various distance and higher education institutions in Africa pioneered the use of SMS on very basic mobile phones since the early 2000s’. Drawing on more recent literature [4] offer three examples of SMS use for public communication: Smoking cessation [5], diabetes prevention [6] and encouraging reading to children [7]. © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 69–80, 2021. https://doi.org/10.1007/978-3-030-49932-7_7

70

N. Roberts et al.

The simple SMS is a ‘very powerful tool to communicate easily and just-in-time with large groups of learners or even individually’ [3]. One of the key strengths of SMS is its broadcast functionality – the ability to send a single message to multiple recipients through bulk messaging. Mobile connectivity has created the opportunity for governments to send public service messages by the press of a button into the ‘palms’ of its citizens [8]. Sending an SMS is significantly cheaper than making a voice call [9]. SMS can be used for community engagement and development, as well as for public health initiatives and practice [10]. Historically a key weakness of broadcasting technology (such as radio, television or print media) has been its unidirectionality and the difficulties presented in obtaining meaningful audience feedback. While it is possible to reflect on uptake and usage (attrition and conversion rates) in relation to an SMS messaging programme (see for example [4]), it is far less common to measure shifts in learning with a rigorous experimental design which focuses on knowledge gains. To the best of our knowledge there seem to be very few studies on SMS programmes which report on knowledge gains in relation to specific learning outcomes – particularly when the intervention targets life-long learning of adults in informal settings in low income communities in Africa. This paper aims to make a contribution towards filling this gap.

2 The Case of ChildConnect ‘MomConnect’ is an SMS service that was established in South Africa in 2014 to provide pregnant woman with twice weekly health information text messages from when they registered their pregnancy until their child is 12 months of age. This paper focuses on the case of ‘ChildConnect’ which is a research study that seeks to pilot an extension of the MomConnect service for an additional 6 months (i.e. from 12 months of age to 18 months of age). While MomConnect focused on health [11], ChildConnect focuses on early language development. The research question in this paper pertains to shifts in caregiver knowledge. ChildConnect aimed to support and educate parents and other caregivers through an SMS curriculum of ECD content underpinned by six core learning outcomes. Each message addressed one or more of the six learning outcomes to support caregivers (mothers or secondary caregivers) to: 1. Know that they are their child’s first teacher (and have some knowledge and ideas of how to play this role). 2. Know that children learn from birth through meaningful play with a caring adult and know to follow 5 easy steps for meaningful play: LOOK, COPY, TALK, TAKE TURNS, ASK. 3. Know that there are many free resources available to them in their everyday environment that can be used to stimulate their children. 4. Know that it is important to talk in their home language using a wide range of vocabulary and full sentences with their child. 5. Know that there are many opportunities in the daily routine of a child which can be used for learning.

Measuring Knowledge Gains in an SMS m-Learning Intervention

71

6. Feel more motivated and supported as part of a community of other caregivers. The ChildConnect SMS curriculum was developed to be delivered in 72 messages over 3 cycles with each cycle lasting 8 weeks (3 cycles  8 weeks per cycle  3 messages per week). Each message was mapped onto one of the 6 core learning outcomes. A format of three different types of messages per week was trialed in this pilot. The three types of messages were: • Monday: “Inspire/Inform messages” motivate or inform mothers. • Wednesday: “Reply messages” ask mothers to respond by sending an SMS. • Friday: “Enable messages” suggest activities for mothers to do with their child. An additional 24 optional messages were developed for those mothers who responded to the Wednesday message. Some of these optional messages spanned over 2 or 3 individual SMS’s. To offer a flavor of the kind of messages sent via SMS for mothers and secondary caregivers of young children, these are the first four messages in the 6 month curriculum: • Monday (inspire/inform): Looking after a small child is tough. You may be tired, work long hours or feel overwhelmed. But every moment you play with your child teaches them something. • Wednesday (reply message): You are your child’s first teacher. To help your child’s brain grow clever and strong: LOOK, COPY, TALK, TAKE TURNS, ASK. Reply YES to receive more information (free SMS). The 5 easy steps to help your child’s brain grow clever and strong: STEP 1: LOOK at what your child is doing. STEP 2: COPY what your child is doing. STEP 3: TALK to your child out loud and in full sentences. STEP 4: TAKE TURNS by waiting for your child to make a sound or to move. STEP 5: ASK your child a question and make your special learning time together last longer. • Friday (enable message): Today at breakfast, listen and TALK to your child in your home language. Try to TALK in your home language about the smell, colour and type of food. • Monday (inspire/inform): To take care of your child, you must take care of yourself. Ask someone you trust to look after your child for 30 min. Take a break. The English SMS service included 59 invitation and survey messages, 74 content messages and 80 optional messages. Considering a person who requested all optional messages, this was a total of (at most) 213 SMS’s costing about R45 or US$3 per participant. This amount would be slightly more for Afrikaans and isiXhosa participants as the messages were slightly longer in these languages than in English. The ChildConnect intervention has reported on levels of engagement [4], and provided a multimedia narrative of a case study participant [12]. This paper explores preliminary evidence of knowledge gains, comparing pre-test outcome scores to post-test outcomes score across treatment and control groups for ChildConnect.

72

N. Roberts et al.

3 Methodology The overall ChildConnect study involved an iterative design-based research approach [13] with the purpose of improving uptake and impact through obtaining feedback during all the stages of the project. The research was undertaken with ethical clearance via the University of Johannesburg’s research ethics committee in the Faculty of Education (REC2017-038) and approved by the National Department of Health. 3.1

Research Question

The research question in focus for this paper is: To what extent, and in which areas, did the ChildConnect intervention improve caregiver knowledge in relation the learning outcomes? To answer this question an experimental design was adopted. The research team randomly assigned the mothers (with their secondary caregivers) to either a treatment or control group. Bivariate correlation between treatment and control confirmed that they were random (and therefore comparable) in terms of demographic categories. Both groups received and answered four incentivised surveys (R60 or $15 airtime over 6 months). The treatment group received all three messages per week. In contrast, the control group only received one Wednesday ‘reply’ message per fortnight. 3.2

Sampling

Participants were invited from the MomConnect database in the 5 provinces where there was research permission. As the ChildConnect pilot was only in English, Afrikaans and isiXhosa, invitations were limited to those three languages. The ChildConnect pilot targeted 200 mothers in each of three languages, English, Afrikaans and isiXhosa. Mothers who opted in were given the opportunity to invite a secondary caregiver. Because secondary caregivers were invited by mothers there was no control over how many were invited from each language. 899 caregivers (735 mothers from MomConnect and 164 invited secondary caregivers) joined ChildConnect. 3.3

Methods

The baseline survey focused its questions on biographical data and responses to pre-test statements mapped to each learning outcome. Six month later, in the endline survey, all participants responded to the same set of statements (now at post-test stage), again relating to the learning outcomes. As for the messages, all of the surveys were offered in the language chosen by the participant (isiXhosa, English or Afrikaans). The items were translated and back translated from Afrikaans into English and isiXhosa by researchers who were familiar with the home environments and contexts of the targeted mothers. Table 1 provides the survey questions posed to all participants at baseline and repeated at endline.

Measuring Knowledge Gains in an SMS m-Learning Intervention

73

Table 1. Survey questions Statement

Strongly disagree

Disagree Agree Strongly agree

I know how to help my child’s brain grow clever and strong My child is too young to learn (reverse coding) In my house there are many things my child can learn from My child understands only single words and baby talk (reverse coding) It is easy to think of ways to teach my child something everyday Statement How often does another adult look after your child (in the last week)? How often in the last week have you talked to someone you trust about your child’s learning Learning outcome score

1

2

3

4

n/a

1 1

2 2

3 3

4 4

n/a n/a

1

2

3

4

n/a

1

2

3

4

n/a

Never 1

2

3

4

Often 5

1

2

3

4

5

7

30

In order to develop a composite measure of learning gains, each response to a question was coded on a scale of 1 (worst) to 4 (best). For the last two questions the scale was from 1 (worst) to 5 (best). For each individual, we summed their item scores to calculate a ‘learning outcome score’ on a scale of 7 (worst) to 30 (best). Results reported on include only those caregivers who completed the questions in focus (the biggest possible n in each case). Completion of the baseline survey was a prerequisite for inclusion in the ChildConnect pilot hence the 100% response rate for the baseline survey. There are several limitations to this study. Firstly, as SMS was being used, the surveys were administered remotely and had to be kept very short to ensure participant completion. This meant that the study was exploratory of possible ways to measure knowledge gains solely using an SMS intervention. It would be preferable to administer longer written tests, where participants could spend more time engaged with the content (and potentially completing more open ended tasks). Secondly, there were only 7 items in each of the tests, meaning that the items had to be grouped together to reflect a composite learning outcome score. The SMS curriculum included tightly defined learning outcomes, but it would have been preferable to approach the assessment of each outcome with multiple items which could be validated. Thirdly, learning outcomes are probably best evaluated by testing knowledge in practice (and this was done relating to 12 case study mothers). To measure knowledge gains at scale, we relied on the short mobile surveys – as a way of measuring changes in response to the survey questions (which we took as proxy for suggested knowledge gains). Finally, even though the survey questions were piloted and improved upon in relation to phrasing of questions and technical administration, the survey test instruments were exploratory (as is appropriate at the initial phases of a design-based research study) and were not developed by following an agreed validation process.

74

3.4

N. Roberts et al.

Analysis

To gauge the significance of changes in responses to each learning outcome statements from Pre-Test to Post-Test, Paired Samples T-Tests were conducted for each group (treatment and control). Where significant changes were detected, effect size was calculated using Cohen’s d. This approach however had serious limitations, as testing a single outcome based on one or two questions is tenuous. To overcome this weakness, we reflect on overall learning (considering all 7 statements relating to the learning outcomes). A ‘learning outcome score’ was calculated at baseline and endline for each participant. This could range between 7 (worst) and 30 (best) on each survey. We explored differences between the treatment and control groups, as well as the influence of demographic and engagement factors, using exploratory factor analysis. We investigated the following hypotheses: Hypothesis 1: Learning outcome scores were higher for the treatment group than for the control group Hypothesis 2: More engagement (measured in responses to the closed questions), correlated with higher outcome scores Hypothesis 3: More engagement by the treatment group (measured in responses to content offerings), correlated with higher outcome scores Hypothesis 4: Mothers (primary caregivers) choosing to opt in a secondary caregiver to receive the same messages as her, correlated with higher outcome scores.

4 Findings 4.1

Changes from Pre-test to Post-test

The observed changes from pre-test to post on individual questions are evident in Fig. 1.

Treatment Control

Change from pre to post Change from pre to post

1 (worst)

2

3

4 (best)

I know how to help my child’s brain grow clever and strong.

Effect size n/a

My child is too young to learn.

d=0.2

In my house there are many things my child can learn from.

d=0.5

My child understands only single words and baby talk.

d=0.2

It is easy to think of ways to teach my child something everyday.

n/a

How often does another adult help you look after your child?

n/a

How often in the last week have you talked to someone you trust about your child's learning?

d=0.2

Fig. 1. Results of t-tests on changes from pre-test to post-test

Measuring Knowledge Gains in an SMS m-Learning Intervention

75

The changes observed were slight. Both the treatment and the control groups showed a significant improvement from pre-test to post-test for: ‘My child is too young to learn’ (small effect, d = 0.2); and ‘My child only understands single words and baby talk’ (small effect, d = 0.2). Only the treatment group showed a significant improvement for: ‘Availability of resources a child can learn from’ (large effect, d = 0.5). The treatment group did not change, while the control group declined (small effect, d = 0.2) for: ‘Frequency of talk about the child’s learning’. As mentioned as a limitation, these findings ought to be taken with caution; given that measurement of a learning gain based on a single item is tenuous. The factor analysis (discussed below) - which grouped all of the items together - offers a more robust analysis. 4.2

Exploratory Factor Analysis

In order to explore differences between the treatment and control groups, as well as the influence of demographic and engagement factors; correlations were run to identify possible subdomains (where it was hoped that a few items would correlate to suggest that each measured the same underlying trait). Exploratory factor analysis suggested three domains: (1) beliefs about child; (2) beliefs about self and (3) community support (see Table 2). Table 2. Domains and learning outcome statements Domain 1: beliefs about child (reverse coding) My child is too young to learn My child understands only single words and baby talk Domain 2: beliefs about self I know how to help my child’s brain grow clever and strong In my house there are many things my child can learn from It is easy to think of ways to teach my child something every day Domain 3: community support How often does another adult help you look after your child? How often in the last week have you talked to someone you trust about your child’s

It was assumed that these three factors would be correlated, so a Promax rotation was used (a procedure that doesn’t expect item factors to be totally distinct). The factors grouped into three sub domains, with ‘I know how to help my child’s brain grow clever and strong’ being identified as contributing to two domains, but having a stronger influence on domain 2 (see Fig. 2).

76

N. Roberts et al.

Fig. 2. Factor matrix for domains

We turn now to consider the total learning outcome score (and the gains from pretest to post-test in this score) in relation to each hypothesis. The first hypothesis was found to be true. The learning outcome scores were higher for the treatment group than for the control group (Fig. 3). The treatment group performed better than those in the control group by *0.7 points (Cohen’s d = 0.21, small effect size, but non-trivial). This is *2.3% points (considering a test with 30 as a total).

Fig. 3. Treatment group performed better on the learning outcome scores

Measuring Knowledge Gains in an SMS m-Learning Intervention

77

Testing for factors that influenced the learning outcome score, ‘home language’ (not the language of the messages) was found to be slightly related to Learning Outcome score. isiZulu, Setswana, and Afrikaans language were an indicator of higher scores. This had a very small effect size (Cohen’s d < 0.20). The second hypothesis was also found to be true. More engagement (measured in responses to the closed questions), correlated with higher outcome scores. This was a small effect size. Those who responded to closed questions performed better by 0.14 points (Cohen’s d = 0.24, small effect size, non-trivial) for every question answered. This is only *0.5 pp. The third hypothesis was not confirmed. Engagement by the treatment group (measured in responses to content offerings), correlated with higher outcome scores (but only very slightly). This requires further investigation, as the correlation is very small. There was a very slight, but significant, positive correlation (r = 0.13) between treatment participants learning outcome scores and their number of requests for content offerings. The fourth hypothesis could also not be confirmed or refuted. Mothers (primary caregivers) choosing to opt in a secondary caregiver to receive the same messages as her, correlated very slightly with higher outcome scores. This requires further investigation, as the correlation is very small. There was a very slight, but significant, positive correlation (r = 0.15) between treatment participants learning outcome scores and whether they were paired with a secondary caregiver. As the 7 items grouped into the three domains, where there was more than 1 item in each domain, it was possible consider each domain independently. This provided some important information about the ways in which participants learnt from ChildConnect – as some domains saw greater shifts than others. Domain 1: ‘Beliefs about child’ displayed no relationship between treatment or control group. This was only associated with home language, which suggests that different cultural beliefs view child’s capabilities differently. There were generally positive beliefs (disagreeing with negatively worded statements): ‘My child is too young to learn’ and ‘My child understands only single words and baby talk’. Domain 2: ‘Beliefs about self’ displayed no relationship between any of the variables included in the model, and treatment or control group. There were generally positive beliefs about self (agreeing with positively worded statements): ‘In my house there are many things my child can learn from’ and ‘I know how to help my child’s brain grow clever & strong.’ and ‘It is easy to think of ways to teach my child something everyday.’ Domain 3: ‘Community Support’ (Fig. 4) showed significant differences in treatment and control conditions. The treatment group showed higher scores than the control group (by 0.51 points; Cohen’s d = 0.22, small effect size).

78

N. Roberts et al.

Fig. 4. Community support – significant difference

Although there was a significant difference between the two groups for this domain, it is interesting that for ‘How often does another adult help you look after your child?’ both treatment and control groups reported that another adult helps them 1–3 times per week (estimated marginal mean). Most mothers (75%) reported that they talk to a trusted adult about their child at least twice a week.

5 Conclusions There was a small but measurable positive effect on the knowledge gains of the ChildConnect treatment group compared to the control. Also, asking for more information had a small but measurable positive effect on knowledge (as measured by the short pre-test and post-test survey). Therefore the participants who requested additional information (via the fortnightly invitations to do so) showed greater improvements in their responses to the survey questions, compared to those who did not ask for more information (or requested information less frequently). The factor analysis for each domain within the ChildConnect learning outcomes found that the ‘belief about child’ and ‘belief about self’ domains showed no significant difference between treatment and control, but that the ‘community support’ domain showed a significance difference, with the treatment groups scoring lower (better) than the control group. This was a small effect size, with Cohen’s d = 0.22. While the first two domains did not show an effect on their own, they contributed to the effect evident for the ‘learning outcome score’ which combined all 3 domains. Of interest was the possible effect of home language in relation to ‘belief about child’. This would require further investigation, as the correlation on this was very small. This study has provided preliminary evidence of knowledge gains, comparing pretest learning outcome scores to post-test learning outcomes scores across treatment and control groups for ChildConnect. While the evidence of knowledge gains is slight, it is in the right direction. The exploratory process adopted demonstrated how to measure shifts in learning with a rigorous experimental design. In future cycles a more extensive test (administered perhaps in parts at baseline and endline) should be included. The study contributes to the small set of studies on SMS programmes which report on knowledge gains in relation to specific learning outcomes, and where the intervention targets lifelong learning of adults in informal settings in low income communities in Africa.

Measuring Knowledge Gains in an SMS m-Learning Intervention

79

Acknowledgment. All three authors were associated with the University of Johannesburg. ChildConnect was sanctioned by the South African National Department of Health and funded by Innovation Edge. The technology partner was Praekelt.org, and the content design and evaluation research partner was Kelello, in collaboration with the Centre for Education Practice Research (CEPR), University of Johannesburg. This research was funded by Innovation Edge, and undertaken by Kelello in collaboration with University of Johannesburg’s CEPR. We thank Matthew Snelling for his factor analysis. We acknowledge the support and enabling environment created by Prof Henning (South African Research Chair on the ‘integration of mathematics, science and language in the Primary School’), and the PhD research grants of the National Research Foundation (NRF).

References 1. Anderson, C., Behrman, J., Bhutta, Z., Black, M., Britto, P., Cerezo, A., Fink, G.: Country profiles for early childhood development. (2019). https://hilton-production.s3.amazonaws. com/documents/400/attachments/2030-Countdown.pdf?1555091241 2. Crompton, H.: A historical overview of mobile learning: toward learner-centered education. In: Berge, Z.L., Muilenburg, L.Y. (eds.) Handbook of Mobile Learning, pp. 3–14. Routledge, Florence, KY (2013) 3. Brown, T.H., Mbati, L.S.: Mobile learning: moving past the myths and embracing the opportunities. Int. Rev. Res. Open Distance Learn. 16(2), 115–135 (2015). https://doi.org/ 10.1017/CBO9781107415324.004 4. Mostert, I., Roberts, N., Plaatjies, L.: Measuring uptake and engagement in an m-learning intervention: the case of ChildConnect South Africa. In: Auer, M.E. (ed.) Internet of Things, Infrastructures and Mobile Applications, pp. 81–90. SpringerNature (2021) 5. Rodgers, A., Corbett, T., Bramley, D., Riddell, T., Wills, M., Lin, R.-B., Jones, M.: Do u smoke after txt? Results of a randomised trial of smoking cessation using mobile phone text messaging. Tobacco Control 14, 255–261 (2005). https://doi.org/10.1136/tc.2005.011577 6. Wong, C., Fung, C., Siu, S., Lo, Y., Wong K., Fong, D., Lam, C.: A short message service (SMS) intervention to prevent diabetes in Chinese professional drivers with pre-diabetes: a pilot single-blinded randomized controlled trial. Diabetes Res. Clin. Pract. 102, 158–166 (2013) 7. Crane, A., Smith, W.: Leveraging Mobile Technology For Parental Engagement In The Early Years: Findings From The Read to Kids India Pilot 2015–2017, WorldReader, San Francisco, pp. 1–46 (2018) 8. Seipold, J.: Mobile Learning Structures, Concepts and Practices of the British and German Mobile Learning Discussion from a Media Education Perspective (2012). www.medienpaed. com/24/#seipold1408 9. Villanueva, K. Vaidya, J.: Transforming learning with mobile games: learning with mGames. In: Mentor, D. (ed.) Handbook of Research on Mobile Learning in Contemporary Classrooms, IGI-Global: Hershey, pp. 1–475 (2016). https://doi.org/10.4018/978-1-52250251-7 10. Murphy, F.: Community Engagement, Organization, and Development for Public Health Practice. Springer, New York (2012) 11. Barron P., Peter, J., LeFevre A., Sebidi, J., Bekker, M., Allen, R., Parsons, A., Benjamin, P. Pillay, Y.: Mobile health messaging service and helpdesk for South African mothers (MomConnect): history, successes and challenges. BMJ Glob. Health 2018; 3(Suppl 2), e000559 (2018). http://dx.doi.org/10.1136/bmjgh-2017-000559

80

N. Roberts et al.

12. Plaatjies, L.: Teach to love my grandchild. In: 2019 International Conference on Interactive Mobile Communication, Technologies and Learning, 31 Oct–1 Nov 2019, Thessaloniki, Greece (2019). (this issue) 13. Wang, F., Hannafin, M.: Design-based research and technology-enhanced learning environments. Educ. Technol. Res. Dev. 53(4), 5–23 (2005)

Measuring Uptake and Engagement in an m-Learning Intervention: The Case of ChildConnect South Africa Ingrid Mostert(&), Nicky Roberts, and Lydia-Anne Plaatjies University of Johannesburg, Johannesburg, South Africa [email protected], [email protected], [email protected]

Abstract. ChildConnect is a messaging service which aims to use mobile learning to support and educate parents and caregivers through an SMS curriculum of Early Childhood Development content. This paper reports on initial uptake and subsequent engagement with the ChildConnect messages. An experimental design was adopted where caregivers were randomly assigned to either a treatment or a control group. The uptake for ChildConnect from those targeted via public clinic data on pregnant women was 24%, this increased to 66% for secondary caregivers invited to join the service by a primary caregiver. Engagement was measured in terms of appetite, as well as in terms of response rates to both unincentivised and incentivised opportunities to engage. For both control and treatment groups there was clear appetite to receive more than three messages per week, with most (*78%) participants indicating that they would like to receive messages every day or every weekday. Three incentivised surveys (midline 1, midline 2 and endline) were completed by 72%, 70% and 69% of participants respectively. Between 25−55% of participants in the treatment group responded to unincentivised weekly ‘reply’ messages. Higher response rates were seen for the control group who received fewer messages and only fortnightly opportunities to reply. Keywords: m-Learning
South Africa Early childhood
Feature phones


Parenting programme
Evaluation


1 Introduction The dearth of Early Childhood Development (ECD) programs to support parents in South Africa is noted in [1], which acknowledges that citizens living in poverty can benefit from parent support programmes. Seven different kinds of parenting and family intervention programmes found in South Africa that provided various services offered by the health sector are described by [2]. In contrast to health, for the education sector parent programmes for ECD in South Africa are currently small in scale, typically provided by the non-profit sector and only a small number of parents are reached because of the cost of such an intervention.

© Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 81–90, 2021. https://doi.org/10.1007/978-3-030-49932-7_8

82

I. Mostert et al.

Mass communication, which refers to the ability to communicate a specific message instantly to a population with the understanding that the message comes from a source of power [3], offers the possibility to reach parents of young children at scale. A mass communication broadcast from a government can be received by a parent of a young child, thereby harnessing the potentials of mobile learning (m-learning) – defined by [4] as ‘learning across multiple contexts, through social and content interactions, using personal electronic devices’ (p.4). Short Message Service (SMS) broadcasts in particular can be used to deliver text-based communication directly to parents of young children. SMS is ‘a text messaging service component of a phone, web or mobile communication system which uses standardized communications protocols’ [5]. The pricing structure for mobile phones in many parts of South America and Africa makes mobile phone ownership much more prevalent, and sending an SMS is significantly cheaper than making a voice call [6]. SMS on mobile phones can be used for community engagement and development for public health initiatives and practice [7]. In other countries, SMS messaging services have shown good results. For example, a randomized control trial on SMS intervention on smoking cessation in New Zealand, showed higher quit rates at both six weeks and six months [8]. A Health intervention in China with a single blind randomized control showed positive results for diabetes prevention [9]. Using mobile phones for mass communication to parents relating to ECD has been tried by the government of India in the ‘Read to Kids’ India pilot (2015 −2017) [10]. The information was accessible on a mobile application that was built to give parents access to the content on their Internet enabled (smart) mobile phones. Added to this was a messaging campaign encouraging parents to create a habit of reading with their children. A key finding was that the behaviour change campaign was most effective when supported by community development partner. Although the purely digital programme had a lower conversion rate to reader or frequent reader (than the blended programme), it was significantly cheaper than the blended offering with a far wider reach [10]. In 2014 the national department of health (NDoH) in South Africa launched a SMS messaging service, called MomConnect, to provide stage-based messages to pregnant women and new mothers. The MomConnect service was established in 2012 ‘to register pregnancies and provide pregnant and post-partum women with twice weekly health information text messages as well as access to a health desk for patient queries and feedback’ [11]. MomConnect revealed that it was possible for the South African government to communicate directly with pregnant women using a mobile phone. At the time of the research, the MomConnect messaging service terminated when the baby turns one. In 2016 a study, known as ChildConnect, was initiated to pilot an extension of the MomConnect service for 6 months (i.e. from when the child was 12 months of age to 18 months of age). While MomConnect focused on health, ChildConnect focused on early language development. The messaging service commenced in April 2017 and concluded in December 2017. One of the big challenges in any messaging service, including MomConnect, is to ascertain the extent to which users are engaged with the service over the period of delivery. Unlike web-based services that allow for analytics to see how many people visited a website, there is no simple way to know whether an SMS has been read or engaged with. This study aimed to address this particular problem faced by the

Measuring Uptake and Engagement in an m-Learning Intervention

83

ChildConnect messaging service by including messages prompting a response from the user. It therefore offers a case study which contributes to assessment, evaluation and research methods in mobile learning.

2 Methodology The ChildConnect study involved an iterative design-based research approach with the purpose of improving uptake and impact through obtaining feedback during all the stages of the project. Design assumptions were made explicit at the outset of the intervention, with a view to revise these, if necessary, with each message cycle. The participants answered short (incentivized) baseline, midline and endline surveys from which we could gather their demographic data, perceptions and preferences for key aspects of the messaging service, including their appetite for the messages. Participants were able to engage with the service by responding to a question or prompt each Wednesday. At the project outset, the following three assumptions were made about participant engagement. Assumption 1: most parents will not tolerate more than 3 messages a week. Assumption 2: most parents will choose to ‘opt in’ to receive additional optional messages and provide positive feedback on this messaging format. Assumption 3: it is educationally appropriate to include ‘closed’ questions (with YES or NO answers, as opposed to open ended questions), and that most parents will tolerate and respond to these. The research was undertaken with ethical clearance via the University of Johannesburg’s research ethics committee in the Faculty of Education (REC2017-038) and approved by the National Department of Health. 2.1

Research Questions

This paper reports on the following research questions: (1) What was the uptake for the messaging service? (2) What were the response rates for incentivised and unincentivised opportunities to engage with the messaging service? 2.2

Sampling

Participants were invited from the MomConnect database in the 5 provinces where there was research permission. As the ChildConnect pilot was only in English, Afrikaans and isiXhosa, invitations were limited to these three languages. The ChildConnect pilot targeted 200 mothers in each of the three languages. In order to reach a target of 600 mothers, 3 015 mothers were invited. Mothers who opted in were given the opportunity to invite a secondary caregiver to join ChildConnect. Because secondary caregivers were invited by mothers there was no control over how many were invited from each language. 899 caregivers (735 mothers

84

I. Mostert et al.

from MomConnect and 164 invited secondary caregivers) joined ChildConnect. The invitation process and random allocation to treatment and control groups is shown in Fig. 1.

Fig. 1. Invitation process

The Child Connect sample included 982 primary and secondary caregivers at first data collection. Primary and secondary caregivers were then given the opportunity to “opt-out” of the programme, and after removing these records, we were left with a final sample of 899 participants. When comparing a treatment group and a control group, it is important that these two groups be equivalent at the start. Treatment and control groups were compared with all other demographic categories: primary/secondary caregiver; paired/not paired to another caregiver, relation to child (mother, father grandparent, aunt/uncle/cousin, nanny/day-mother, friend, unknown); chosen language (English, Afrikaans and isiXhosa) and province (Western Cape, Western Cape, Northern Cape, North West, KwaZulu-Natal) using bivariate correlation. No associations were found between any demographic factor and selection into the treatment group or control group and, as such, the two groups were equivalent. 2.3

SMS Message Design Included Opportunities for Data Collection

Messages were delivered via the RapidPro platform. RapidPro is an Open Source platform that allows developers to build interactive messaging systems using an easy visual interface. A core benefit of RapidPro (as compared to other messaging platforms) is the ability to manage Contacts, Flows and Campaigns in a rich user interface environment. Both groups received all four surveys – at baseline, at midline 1 (after cycle 1), at midline 2 (after cycle 2), and at endline (after message cycle 3). The total incentive for the surveys, over the 6 months, was R60 (U$4). The SMS curriculum was developed to be delivered in 72 messages over 3 cycles with each cycle lasting 8 weeks (3 cycles  8 weeks per cycle  3 messages per week). A format of three different types of messages per week was trialed in this pilot. The three types of messages were:

Measuring Uptake and Engagement in an m-Learning Intervention

85

• Monday: “Inspire/Inform messages” motivate or inform mothers. • Wednesday: “Reply messages” ask mothers to respond by sending an SMS. • Friday: “Enable messages” suggest activities for mothers to do with their child. The treatment group received all three messages per week. In contrast, the control group received the four surveys, but only received one Wednesday ‘reply’ message per fortnight. For both groups there were no incentives for answering questions or requesting more information (as offered on Wednesdays). 2.4

Analysis

In order to answer the research questions, data from the messaging service was analysed in terms of uptake and engagement. To determine uptake the number of users who opted in was compared with the number users invited. To determine engagement, three indicators were used. Firstly, we considered the appetite for messages as indicated in the responses to survey questions relating to messaging preferences. Secondly, we considered incentivised opportunities to respond, namely two midline and one endline survey. Thirdly, we considered response rates to unincentivised opportunities to respond, namely the Wednesday ‘reply’ messages. Wednesday messages alternated between content offers, where caregivers could answer YES to receive more information, and closed questions, where caregivers could respond with an answer. A caregiver was considered ‘engaged’ if they responded to a Wednesday message. This was a novel and important feature of the messaging service as it mitigated against not knowing whether caregivers were in fact reading and engaging with the messages.

3 Findings The findings are discussed in terms of uptake and engagement. For engagement the three indicators are discussed separately. 3.1

Uptake

3 015 mothers were invited. Out of these 735 (24.4%) chose to join. A number of the mobile numbers used for the invitation were likely to have been out of date due to high numbers of users changing numbers. The percentage uptake is therefore likely to be an underestimation. Of the 735 who chose to join, 247 (33.6%) chose to invite a secondary caregiver. Of those invited, 164 (66.4%) chose to join. Potential reasons for the higher opt-in rate for secondary caregivers include possible additional communication about the service from the mother who invited the secondary caregiver, as well as greater likelihood of an up to date cell phone number. There are very few examples where SMS is used to target parents in the same manner as was done for ChildConnect. Although not entirely comparable, the Read To Kids programme [10] provides some similar data, targeting a similar demographic of low income parents relating to ECD. Uptake for the Read to Kids programme can be calculated from their report. They estimated that 17 million people “saw or heard

86

I. Mostert et al.

behavior change messages around the benefits of shared reading” [10]. Of their 203 000 users, the majority (189 066) were reached through a media campaign, with 15 000 households reached traditionally through community-based partners in Delhi [10]. So, the uptake rate using a media campaign (and excluding the households reached through community-based partners) was 1.1%. This is much lower than ChildConnect where uptake was via a database from pregnant women using public health clinics. Interestingly in ChildConnect, the uptake rate was higher when a secondary caregiver was invited by a primary caregiver (66%), which indicates that using a viral campaign where new participants are invited by adults who they already know, may hold promise. 3.2

Engagement

The three indicators that were used to measure engagement were appetite, response rates for incentivised opportunities to reply and response rates for unincentivised opportunities to reply. Firstly, appetite was measured through short (incentivized) surveys. The survey data provided us with information about participant preferences for the format, time and frequency of messages. Most participants in ChildConnect would like messages every day or every weekday (*78%), prefer to receive a message before noon (*82%), would like to receive messages with a variety of content or messages of encouragement (*78%), prefer m-learning engagement via SMS (*86%) over WhatsApp, Facebook messenger or Snapchat. This indicates a clear appetite for more messages than the number provided. Secondly, in terms of response rates for incentivised engagement, the midline 1 survey was completed by 646 out of 899 participants (71.9%), the midline survey 2 by 594 out of 843 participants (70.4%), and the final survey by 583 out of 841 participants (69.3%). Note that the response rate for the baseline survey was 100% as signing up for the messaging service was dependent on completing the survey. Thirdly, in terms of response rates for unincentivised engagement, an engagement ratio was calculated by dividing the total number of caregivers who responded to a message by the number of caregivers who were receiving messages at the beginning of the cycle. Cycle 1: 899; Cycle 2: 843; Cycle 3: 841. Figure 2 shows a drop off in number of users responding to Wednesday messages. This is expected as there tends to be initial novelty value which Attwell, quoted in [12], refers to as “the initial ‘gadget honeymoon’” and which wears off over time. In both the control and treatment groups, more mothers respond than secondary caregivers (on average at least 10% points more), except for the treatment group in cycle 3 where response rates were similar. Interestingly, users in the control group (50 −75%) were more likely to respond to closed questions (possibly due to limited number of messages received) than the treatment group (25−55%). The ChildConnect Wednesday messages provided a weekly opportunity for ‘marketing conversion’ (to take the desired action of responding to a question or requesting more information). Once again, the dearth of documented cases of SMS interventions meant that finding industry benchmarks for engagement or conversion, relating to SMS was challenging. We drew once again on Read to Kids given its

Measuring Uptake and Engagement in an m-Learning Intervention

87

Fig. 2. Percentage of users requesting information, by group (treatment vs control) and by role (mother vs sec caregiver)

context in a low-income community, and content focus on an ECD topic. In the case of the ‘Read to Kids India’ campaign two industry level benchmarks were considered for conversion rates. The first related to click through rates from a WhatsApp push notification with “an industry standard of 3.05%” [13]. The second related to purely digital interventions using “the 2017 US retail e-commerce conversion rate of about 0.9% to 1.4%” a benchmark [14]. The Read for Kids India pilot found that using digital only strategies, 2% of total users became frequent readers, and that their average clickthrough rate for push notifications (where this was part of a wider campaign including community based organisation contact) was 5%. As ChildConnect was digital only, we adopted 2% as a benchmark conversion rate for unincentivised weekly responses to opportunities to act (by responding to a question or asking for more information). The data for the control group (25−55% responding to weekly messages) and treatment groups (55−75% to fortnightly messages) shows that a much higher response rate was obtained than the benchmark. This suggests ChildConnect’s SMS-based intervention targeting parents in low income communities in South Africa (which made use of feature phones and directly targeting mothers who had a young child), was more successful in attracting participants than the

88

I. Mostert et al.

Read to Kids intervention targeting parents in low income communities in India (and which made use of an app-based intervention which requires a smartphone, and involved a broad media campaign to the public). In both interventions the data costs were carried by the funder and for ChildConnect there was an additional incentive of R60 ($4) airtime for six months.

4 Conclusions ChildConnect attained very high uptake and engagement rates, compared to the Read to Kids campaign in India (and their comparison to industry benchmarks of 2−5%). In relation to uptake: • 3 015 mothers were invited. Out of these 735 (24.4%) chose to join (with an incentive of a possible R60 (about $4) over 6 months). • Of the 735 who chose to join, 247 (33.6%) chose to invite a secondary caregiver. • Of the secondary caregivers invited, 164 (66.4%) chose to join. We see very high opt-in rates (24%) for joining ChildConnect. The opt-in rate was even higher (66%) when the invitation was offered by someone who is known to the participant (as in the case of secondary caregivers receiving invitations from someone who nominated them). As mentioned previously, the Read to Kids programme in India had a much smaller uptake (1.1%). In relation to engagement, the actions which were incentivised with airtime are separated from those that were not and the latter are considered to be a better reflection of engagement. In terms of actions which were incentivised, ChildConnect achieved high response rates, 72%, 70% and 69% respectively, for three incentivised SMS surveys over the 6 month period. In terms of actions which were not incentivised, ChildConnect sustained engagement levels of above 25% over the 6 month period for both groups. This is a minimum of 5 times (500%) higher than the 5% ‘frequent reader’ conversion rate of the blended Read to Kids campaign. It is 10 times (1000%) higher than the conversion rate seen for push notifications. Users in the control group (50−75%) were more likely to respond to closed questions (possibly due to limited number of messages received, and few opportunities to respond) than the treatment group (25−55%). Noting that both programmes target parents of young children in low income communities, we found that the ChildConnect campaign (a digital only, SMS campaign which can be accessed by feature phones) had far higher engagement rates than Read To Kids (a blended intervention, application campaign which required a smart phone). Assumption 1, that most parents will not tolerate more than 3 messages a week, was incorrect. There was clear appetite for more messages than those provided. ‘Pull systems’, such as from a Frequently Asked Questions database, allow users to pull information, rather than all users receiving a broadcast. In the light of the appetite for messages, ‘pull systems’ could be cost effective and seemingly still reach the majority of users. Assumption 2, that most parents will choose to ‘opt in’ to receive optional messages, was correct for the control, but not met for the experiment group. *45% of

Measuring Uptake and Engagement in an m-Learning Intervention

89

treatment group opted in each week whereas *60% of control group opted in each week. From this we conclude that even when there is an appetite for more messages, when more messages are sent and more opportunities for engagement are provided, there could be a lower engagement rate (10−15% lower response rate for treatment than for control in this case). Assumption 3, that it is educationally appropriate to include closed questions (focused on obtaining feedback on how/or whether key concepts in the educational intent of the parent support messages are being understood), and that most parents will tolerate and respond to these, was not quite correct. The closed questions were responded to, but not by the majority: *45% of treatment group each week (recall that the control group did not receive closed questions). As a result of this finding in the first message cycle, the closed questions were changed in two main ways. First, instead of ‘testing knowledge’, which resulted in numerous unsolicited messages, the closed questions were framed as ‘soliciting opinions’ on particular topics. Second, the feedback given on responses to closed questions was changed to use the response message as a means of providing more information to caregivers. While we originally designed the closed questions simply to measure participant engagement, the closed questions were later used as a hook to send more information and test whether users are still engaged. By including messages that prompt a response from the user (either to solicit more information, or to respond to a question), the ChildConnect SMS message design overcame the problem evident in MomConnect of only being able to report on message delivery. This allowed for data to be collected, not just about uptake (and hence reach of the service) but also, about engagement. As each opportunity to respond was viewed as a request to act, this provided a conversion rate (or weekly activity for the treatment group, and fortnightly activity for the treatment group). Analysing these engagement rates, revealed that receiving fewer opportunities resulted in higher response rates. We trust that the methods used and the rates obtained will contribute to evaluation and research methods in mobile learning. In particular we hope this makes a contribution towards establishing m-learning industry benchmarks for conversion rates in low income communities where both smart and feature phones are used. We open these findings for critical review and application in other contexts, in an effort to provide good practices for measuring engagement in mobile learning research, making use of SMS technology. Acknowledgement. All three authors are affiliated to the University of Johannesburg. Child Connect was sanctioned by the South African National Department of Health and funded by Innovation Edge. The technology partner was Praekelt.org, and the content design and evaluation research partner was Kelello, in collaboration with the Centre for Education Practice Research (CEPR), University of Johannesburg. This research was funded by Innovation Edge, and undertaken by Kelello in collaboration with University of Johannesburg’s CEPR. We acknowledge the support and enabling environment created by Prof Henning through the South African Research Chair on the ‘Integration of Mathematics, Science and Languages in the Primary School’, funded by the National Research Fund (NRF), grant number 98573.

90

I. Mostert et al.

References 1. Republic of South Africa. National Integrated Early Childhood Development Policy. Government Printers: Pretoria, pp. 1–140 (2015). https://www.gov.za/sites/default/files/gcis_ document/201610/national-integrated-ecd-policy-web-version-final-01-08-2016a.pdf. Accessed 5 July 2019 2. Britto, P.R., Ponguta, L.A., Reyes, C., Karnati, R.: A systematic review of parenting programmes for young children in low and middle income countries. UNICEF (2015) 3. McQuail, D. (ed.): SAGE Benchmarks in Communication: Mass Communication. SAGE Publication Ltd., London (2010). https://doi.org/10.4135/9781446262467 4. Crompton, H.: A historical overview of mobile learning: toward learner-centered education. In: Berge, Z.L., Muilenburg, L.Y. (eds.) Handbook of Mobile Learning, pp. 3–14. Routledge, Florence (2013) 5. Shahrokhi, M., Kambayi, M.: The impact of mobile assisted language learning (MALL) on phrasal verbs of Iranian intermediate EFL students. In: Mentor, D. (ed.) Handbook of Research on Mobile Learning in Contemporary Classrooms. IGI-Global, Hershey (2016). https://doi.org/10.4018/978-1-5225-0251-7 6. Villanueva, K., Vaidya, J.: Transforming learning with mobile games: learning with mGames. In: Mentor, D. (ed.) Handbook of Research on Mobile Learning in Contemporary. IGI-Global, Hershey (2016). https://doi.org/10.4018/978-1-5225-0251-7 7. Murphy, F.: Community Engagement, Organization, and Development for Public Health Practice. Springer, New York (2012) 8. Rodgers, A., Corbett, T., Bramley, D., Riddell, T., Wills, M., Lin, R.-B., Jones, M.: Do u smoke after txt? Results of a randomised trial of smoking cessation using mobile phone text messaging. Tob. Control 14, 255–261 (2005). https://doi.org/10.1136/tc.2005.011577 9. Wong, C., Fung, C., Siu, S., Lo, Y., Wong, K., Fong, D., Lam, C.: A short message service (SMS) intervention to prevent diabetes in Chinese professional drivers with pre-diabetes: a pilot single-blinded randomized controlled trial. Diab. Res. Clin. Pract. 102(3), 158–166 (2013) 10. Crane, A., Smith, W.: Leveraging mobile technology for parental engagement in the early years: findings from the read to kids India pilot 2015–2017, WorldReader (2018) 11. Barron, P., Peter, J., LeFevre, A., Sebidi, J., Bekker, M., Allen, R., Parsons, A., Benjamin, P., Pillay, Y.: Mobile health messaging service and helpdesk for South African mothers (MomConnect): history, successes and challenges. BMJ Glob. Health 3(e000559) (2018). https://doi.org/10.1136/bmjgh-2017-000559 12. Hlodan, O.: Mobile learning anytime, anywhere. BioScience 60(9), 682 (2010). https://doi. org/10.1525/bio.2010.60.9.4 13. Takahashi, D.: 7 surprising facts about open rates for push notifications (2016). https:// venturebeat.com/2016/09/29/tapjoy-unveils-7-surprising-facts-about-who-opens-pushnotifications/ 14. Chaffey, D.: E-commerce conversion rates. Smart Insights (2018). https://www. smartinsights.com/ecommerce/ecommerce-analytics/ecommerce-conversion-rates/

Development of a Classroom Response System: A Web-Based Approach Used in SEPT Dan Centea(&), Konstantinos Apostolou, and Moein Mehrtash McMaster University, Hamilton, ON, Canada {centeadn,apostol,mehrtam}@mcmaster.ca

Abstract. The paper proposes an active-learning strategy aimed to increase class engagement and enhance student learning through the use of a Classroom Response System (CRS). A list of the commercially available CRS is reviewed. The motivation for designing an in-house CRS that addresses the limitations of the commercial CRS and solves specific needs of several automotive engineering courses is provided. The design and development of a mobile-based CRS at McMaster University in the School of Engineering Practice and Technology (SEPT) is presented. The CRS includes several modules that deal with the instructor and student interfaces, and with the infrastructure needed for a client-server application. The modules of developed CRS are described in detail. The paper presents the challenges that had to be mitigated during the development of the web-based applications and during its integration with the university’s Learning Management System. The benefits compared with commercial CRS justify the efforts to develop an in-house mobile web-based CRS software package. Keywords: Classroom response system learning
Web-based app


CRS
Active learning
Mobile

1 Introduction The traditional university practices based on lecture-intensive approaches have evolved by introducing alternative teaching practices that enable deep learning and retention, and engage students in various experiential learning activities. This evolution has resulted in several teaching approaches that can be applied in the classroom. Many of these approaches use active-learning strategies. There is significant published literature related to active-learning strategies. Various authors show improvements in student learning that are produced by different active-learning approaches [1–11]. However, given that each student is different in his/her learning ability, there is no unique way of conducting the classroom sessions that is equally applicable for all courses and all students. This paper focuses on an active-learning strategy based on an interactive Classroom Response System (CRS), also referred to as Student Response System (SRS); Mobile Response System (MRS); classroom performance system; Personal Responses Systems (PRS); Electronic Response Systems (ERS), Audience Response Systems (ARS), or Clickers. This active learning strategy encourages student participation in class while addressing personal concerns such as passive personalities, © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 91–101, 2021. https://doi.org/10.1007/978-3-030-49932-7_9

92

D. Centea et al.

cultural norms, lack of knowledge, or fear of humiliation. The use of a CRS assures a high level of active participation in the classroom. The efficiency of using a CRS in the classroom has been assessed by several authors. The use of an CRS has been shown to increase student engagement and achievement compared to traditional lecture format instruction [12], improve test scores, provide a positive and significant impact on student final grades, and reduce attrition [13–16]. They improve the classroom experience and classroom environment [17, 18]. However, the efficacy of the CRS depends strongly on the quality of the questions. Creating effective questions is difficult and differs from creating exam and homework problems [19]. When combined with collaborative peer-aided learning, clickers have shown promising results for high-order thinking [20]. Students perceive high levels of the constructs when using the clickers and especially high levels of learning performance; they perceive that using clickers in the class facilitates the understanding of the concepts and class materials and significantly improves their learning process [18]. There are several types of CRS currently used by the academic community [21]. The collection of the student responses can be accomplished using: – Hardware CRS, commonly referred to as clickers; examples include eInstruction Classroom Performance System, Qwizdom, TurningPoint, Interwrite PRS, iClicker, and H-ITT [22, 23]. – Commercial smartphone-enabled CRS, sometimes referred to as soft clickers; some examples of specialised software packages include Socrative [24–28]; Top Hat Monocle [29, 30]; ResponseWare [31, 32]; Poll Everywhere [33, 34]; and iClicker Cloud [35, 36]. – Online CRS included in Learning Management Systems, also referred to as online quizzes; – Self-designed software CRS [37]. All of the above listed systems have advantages, disadvantages and limitations yet none of these systems address all the expectations of instructors, students, and vendors [38]. The goal of an instructor teaching a course that uses a CRS is to select the type of system that addresses most of the stakeholder’s concerns, has the greatest impact on the course delivery, and ultimately leads to a better student learning. This paper presents the development of a software CRS designed in house with the purpose of being used for several courses in Canada at McMaster University by the School of Engineering Practice and Technology. Section 2 presents the motivation to design in house a CRS, and Sect. 3 presents in details the design and development of the CRS. A summary is presented in Sect. 4.

2 Motivation to Design in House a CRS A CRS can be used to assess student preparation and provide prompt feedback. Immediate feedback is extremely important to both instructors and students. Students can compare their responses with the responses of their peers and with the correct answer. When answers are discussed in class, the allocated time is longer if the percentage of the students who provided the correct answer is low. This approach helps

Development of a Classroom Response System: A Web-Based Approach

93

students understand and not memorize an answer, and allows the instructor to determine which content needs more explanation and discussion to be focused on during lectures. Although any type of CRS can accomplish the activities listed above, student feedback would allow the instructor to identify the topics that, from the student perspective, are difficult. Student feedback can be easily integrated in self-designed CRS. In many engineering application, in addition of knowing the course concept the students are required to do proper mathematical calculation to have reasonable answer. The following example demonstrates progressive steps that engage students, course named Modeling and simulation, in assessing of engineering concept and mathematical calculation. The engineering question is required to be break down to several progressive steps to provide valid insights for instructors. As a practical example, the instructor wants to assess students after he/she explained the concept of finding the velocity and acceleration of point on a rotating body, Fig. 1. Students need to do perform three steps: 1) define engineering terms from the provided information, 2) use proper formula for velocity and acceleration, and 3) do the math. It is possible that students make mistake in any steps that result in wrong final answer. This right and wrong status for final answer will confuse the instructor about students’ lack of knowledge. However, this can be done progressive steps and use of CRS. As shown in Fig. xxx, two questions were first shown for students to make sure that they can define engineering terms for their calculations. Students first tried question 1, 42% answered correct. However after the instructor explanation, 73% of students could find the correct answer in question 2, Fig. 2. This progressive approach using CRS create promising learning and teaching environment for both students and instructors.

Fig. 1. Progressive steps using CRS

94

D. Centea et al.

Fig. 2. Progressive steps using CRS for two questions, A) Question 1 and B) Question 2

A CRS can be used either to reinforce a topic taught in a lecture or to connect a theoretical concept with a real-world application. Providing students a list of possible applications and asking them to select the one that has the strongest connection with the theoretical concept can be accomplished with a matching type of question. The matching exercise allows students to understand better new concepts. On the other hand, asking them to think and suggest a possible application instead of asking them to select one from a predefined list encourages them to think creatively. Furthermore, asking students to analyze relationships between multiple concepts and make evaluations based on certain criteria is expected to develop their critical thinking skills. The last two cases provide a better learning environment; they can be accomplished if the CRS allows students to enter text. The traditional way of asking students to copy a detailed solution does not encourage deep thinking. If the students are encouraged to suggest the next steps during the development of a technical problem they will have a better understanding of the approach, and will have better abilities to develop a similar solution by themselves. A self-designed CRS can be used to ask students to suggest these steps, to identify the correct units, and to accept/decline the numerical values of the results. If students could provide answers that include numbers or a short text, the scope of the questions could be highly expanded. The CRS presented in this paper is used in three automotive engineering technology courses for different active learning strategies. Examples of such strategies are quizzes, application questions, critical thinking questions, feedback from students, matching exercises, and a significant amount of problem solving exercises. These courses are very specific for the automotive education, and their vendors of commercial CRS have no incentives to develop custom CRS for courses with low level of generality that cannot be sold to other customers. Consequently, a CRS designed in-house is the best solution for addressing the needs specified above. The design of the CRS should not be specific to a limited number of courses. Although initially designed for three automotive courses, the CRS can be used for other courses with more users or more functionality. To address these requirements, the CRS should be scalable in terms of users or functionality.

Development of a Classroom Response System: A Web-Based Approach

95

3 Design of a CRS for Mobile Devices A CRS used in the classroom improves student learning. Students can be engaged in active learning exercises, while the instructor acquires real-time evidence of student learning. Researchers in mobile learning report that the use of mobile devices in the classroom produces positive educational outcomes [39]. There limited literature related to the development of self-designed functional CRS. The authors of reference [37] report the design, deployment and evaluation of a MRS software package that facilitates execution and assessment of multi-step in-class interactive problem-solving activities using mobile devices. The CRS described in this research is fully designed in-house in the School of Engineering Practice and Technology at McMaster University. This client-server application includes an instructor module, a polling module, an assessment module and an infrastructure module. The instructor module is a web-based application developed for computers. The polling module is an application developed primarily for mobile devices; it uses a webinterface to avoid platform-specified challenges, and therefore can be used on mobile devices, such a smartphones or tablets, or on full screen computers. The instructor module includes two components: “manage questions” and “manage quizzes”. A quiz is defined, for the purpose of this software application, as a collection of questions, and it does not imply a formal assessment. The manage questions component allows the instructor to create new questions, edit existing questions while saving everything into a database, To create a question, the instructor will first select the type of each question: true or false; multiple-choice with one or more possible answers; ordering; numerical; short answer; fill in the blanks; and column matching questions. For each type of question the instructor can manage relevant information such as the time limit for the question, the image (if needed) that will be displayed in the question, the text of the question, and the correct answer(s). Figure 3 shows the interface for creating a new column-matching question in which students will be asked to match the left column and the right column based on the three figures shown on the page. Figure 4 shows the instructor interface for creating a multiple-choice question. The manage quizzes component allows the instructor to create new quizzes, save them in a test bank, edit quizzes, and delete quizzes. The functionality also enables the instructor to create a new quiz by modifying previously saved quizzes. The instructor can also access quizzes that have already been deployed to students. In order to create a new quiz, the instructor enters the name of the quiz, its weight (expand on weight), the course code, and the type of the quiz, as shown in Fig. 5. The type of the quiz can be mid-term or interactive, based on the order in which the students answer the questions and the allocated time. Midterm quizzes allow students to answer each question in any order with a time limit for the entire quiz, while the interactive quizzes ask students to answer the questions in an imposed order with a time limit for each question.

96

D. Centea et al.

Fig. 3. Creating a column-matching question

Fig. 4. Creating a multiple-choice question

Development of a Classroom Response System: A Web-Based Approach

97

Fig. 5. Instructor module - create a new quiz

The polling module is a web-based application that connects students’ mobile devices with the quizzes database running on a server. The screen is scaled differently for smartphones, tablets or computers. Answers can be polled from students only when the quiz is deployed by the instructor, who has his/her own interface to administer it. The answers provided by students are saved on the server and handled separately by the assessment module. The assessment module works differently for different types of quizzes. If a quiz is used to assess student knowledge, the results, provided at the end of the quiz, demonstrate a graphical representation of student responses, which can be discussed If a quiz is used to reinforce a topic taught in a lecture or to connect a theoretical concept with a real-world application, the correct answer is shown on the screen after students’ answers are polled. These answers can be used by the instructor to assess students’ critical thinking. Some quizzes include text entered by the students. This text can be in the form of a short answer or used for feedback. Short answers can be automatically assessed by the app if the text includes one or more of the keywords specified by the instructor when the quiz was created. Feedback, however, is not assessed. It is saved by the app in a file that can be viewed by the instructor at any time. The infrastructure module deals with resources, as the app is deployed on the network on a virtual machine. A challenge of the client-server approach is to ensure that the design of the instructor and polling modules, which are the front of the application, work simultaneous with the polling interface and with the assessment

98

D. Centea et al.

module that runs on the server on the back-end of the application. The original plan was to write the back-end completely from scratch. The challenge that was encountered was that the app is very complex and the user authentication (login) is difficult and raises security concerns. To mitigate this issue, the app designers decided to use a Laravel framework that handles security and user authentication. This framework is used to incorporate the front and back-end in an efficient manner. It allows the simultaneous development of the front and back-end without worrying about the server. The advantage of using a Laravel framework is that a lot of the generic parts of a web app, such as user authentication, guards, database-related programs and debugging environments are already implemented and there is no need to reprogram them. One of the biggest challenges is the integration of the virtual machine with the university’s Learning Management Systems, which currently is D2L, formerly called Desire2Learn, and named Avenue to Learn at McMaster University. The development of the infrastructure module, particularly the secure integration with D2L, took the largest amount of the time dedicated for developing the app. The infrastructure module of the application presented in this paper works together with the assessment module and uses the Laravel framework for authentication and access to the Gradebook in D2L. The grades calculated by the assessment module are entered directly in the Gradebook, allowing the students to see them at the end of the class. This immediate feedback is fully appreciated by students. The mobile-based application addresses the limitations of several courses offered in SEPT and can be adapted to a large list of courses. Extending the CRS to more courses is another advantage of the mobile-based CRS.

4 Summary There is significant published literature showing that alternative teaching approaches that use active-learning strategies engage students, enable retention, and create a better learning environment. A strategy reported by several authors is the use of an interactive CRS. Several types of CRS are currently used by the academic community. The collection of student responses can be accomplished using hardware CRS, commonly referred to as clickers; commercial software-enabled CRS, sometimes referred to as soft clickers; online CRS included in Learning Management Systems, also referred to as online quizzes; and self-designed software CRS. A comparison of the existing commercial hardware- and software-based CRS shows that each system provides a series of advantages and limitations. The paper presents a motivation for the developing a self-designed CRS. When used for quizzes that test student knowledge, the CRS can be designed to ask students to provide written feedback related to the difficulty of the quiz. When used for quizzes that reinforce a topic taught in a lecture or connect a theoretical concept with a realworld application, students’ critical thinking skills can be developed by asking them to analyze relationships between multiple concepts and making evaluations based on certain criteria. When used for quizzes that develop problem-solving skills, students can be asked to suggest the next steps during the development of a technical problem, to identify the correct units, and to accept or not the numerical values of the results. All

Development of a Classroom Response System: A Web-Based Approach

99

these features cannot be reasonably provided by commercial CRS packages. Furthermore, the CRS presented in this paper is used in courses are very specific for the automotive education, and the vendors of commercial CRS have no incentives to develop custom CRS for courses with low level of generality that cannot be sold to other customers. A CRS designed in-house seems to be the best solution for addressing the needs specified above. The paper presents the design and development of a software CRS application designed in house in Canada at McMaster University with the purpose of being used is several courses offered by the School of Engineering Practice and Technology. A design that addresses the limitations and needs specified above justifies the efforts to develop the software systems. The CRS includes several modules that deal with the instructor and student interfaces, and with the infrastructure needed for a client-server application. The paper presents the challenges were overcome during the development of the applications and during its integration with the University’s learning Management System. Acknowledgements. The authors would like to acknowledge the students Mohamed Maklad, Biya Kazmi, and Harrison Ossias for the software development, Jan Boer for helping with the network infrastructure, and Dr. Ishwar Singh for coordinating the software development of the web-based app with mobile-enabled interfaces.

References 1. Froyd, J.E. (2008). Evidence for the efficacy of student-active learning pedagogies, https:// www.semanticscholar.org/paper/Evidence-for-the-Efficacy-of-Student-active-Froyd/ 83146105daf257cac8c66bba2066157e48221bff. Accessed 4 July 2019 2. Hake, R.R.: Interactive-engagement versus traditional methods: a six-thousand student survey of mechanics test data for introductory physics courses. Am. J. Phys. 66, 64–76 (1998). https://doi.org/10.1119/1.18809 3. O’Mahony, T.K., Vye, N.J., Bransford, J.D., Sanders, E.A., Stevens, R., Stephens, R.D., Soleiman, M.K.: A comparison of lecture-based and challenge-based learning in a workplace setting: Course designs, patterns of interactivity, and learning outcomes. J. Learn. Sci. 21(1), 182–206 (2012) 4. Slavin, R.E.: Co-operative learning: what makes group-work work, In: Dumont, H., Istance, D., Benavides, F. (eds), The nature of learning: Using research to inspire practice, OECD Publishing, 161–178 (2010). http://dx.doi.org/10.1787/9789264086487-9-en 5. Freeman, S., Eddy, S.L., McDonough, M., Smith, M.K., Okoroafor, N., Jordt, H., Wenderoth, M.P.: Active learning increases student performance in science, engineering, and mathematics. Proc. Natl. Acad. Sci. 111(23), 8410–8415 (2014) 6. Centea, D., Srinivasan, S.: A comprehensive assessment strategy for a PBL environment. Int. J. Innov. Res. Educ. Sci. (IJIRES) 3(6), 2349–5219 (2016) 7. Kolb, D.A.: Experiential Learning: Experience as the Source of Learning and Development (2nd ed.). FT press, Pearson Education, Inc (2015) 8. Yang, Y., Qian, J.: Cooperative learning practices in university classroom. 3rd Annual International Conference on Education Science and Education Management (ESEM 2018), 31–36 (2018). https://doi.org/10.12783/dtssehs/esem2018/23868

100

D. Centea et al.

9. Koretsky, M., Keeler, J., Ivanovitch, J.: The role of pedagogical tools in active learning: a case for sense-making. Int. J. STEM Educ. 5(1), 18 (2018). https://doi.org/10.1186/s40594018-0116-5 10. Srinivasan, S., Centea, D.: An active learning strategy for programming courses. In: Auer, M., Tsiatsos, T. (eds) Mobile Technologies and Applications for the Internet of Things. IMCL 2018. Advances in Intelligent Systems and Computing. Springer, Cham, 909, pp. 327–336 (2019). https://doi.org/10.1007/978-3-030-11434-3_36 11. Awidi, I.T., Paynter, M.: The impact of a flipped classroom approach on student learning experience. Comput. Educ. 128, 269–283 (2019). https://doi.org/10.1016/j.compedu.2018. 09.013 12. Bojinova, E., Oigara, J.: Teaching and learning with clickers: are clickers good for students? Interdisc. J. E-Learn. Learn. Objects 7(1), 169–184 (2011) 13. Bojinova, E., Oigara, J.: Teaching and learning with clickers in higher education. Int. J. Teach. Learn. High. Educ. 25(2), 154–165 (2013) 14. Kennedy, G.E., Cutts, Q.I.: The association between students’ use of an electronic voting system and their learning outcomes. J. Comp. As. Learn. 21, 260–268 (2005) 15. Cheng, L.T.W., Wang, J.W.: Enhancing learning performance through classroom response systems: the effect of knowledge type and social presence. Int. J. Manage. Educ. 17(1), 103– 118 (2019) 16. Ma, S., Steger, D.G., Doolittle, P.E., Stewart, A.: Improved academic performance and student perceptions of learning through use of a cell phone-based personal response system. Res. Food Sci. Educ. 17(1), 27–32 (2018). https://doi.org/10.1111/1541-4329.12131 17. Caldwell, J.: Clicker in the large classroom: current research and best-practice tips. Life Sci. Educ. 6(1), 9–20 (2007) 18. Blasco-Arcas, L., Buil, I., Hernandez-Ortega, B., Sesse, F.J.: Using clickers in class. the role of interactivity, active collaborative learning and engagement in learning performance. Comput. Educ. 62, 102–110 (2013) 19. Beatty, I.D., Gerace, W.J., Leonard, W.J., Dufresne, R.J.: Designing effective questions for classroom response system teaching. Am. J. Phys. 74(1), 31–39 (2006). https://doi.org/10. 1119/1.2121753 20. Liu, C., Chen, S., Chi, C., Chien, K.-P., Liu, Y., Chou, T.-L.: The effects of clickers with different teaching strategies. J. Educ. Comput. Res. 55(5), 603–628 (2017) 21. Petto, A.: Technology meets pedagogy: comparing classroom response systems. J. Coll. Sci. Teach. 48(4), 55–63 (2019) 22. Katz, L., Hallam, M.C., Duvall, M.M., Polsky, Z.: Considerations for using personal Wi-Fi enabled devices as “clickers” in a large university class. Act. Learn. High Educ. 18(1), 25–35 (2017) 23. Johnson, D., McLeod, S.: Get answers: using student response system to see students’ thinking. Learn. Lead. Technol. 32(4), 18–23 (2005) 24. Coca, D., Sliško, J.: Software socrative and smartphones as tools for implementation of basic processes of active physics learning in classroom: an initial feasibility study with prospective teachers. Eur. J. Phys. Educ. 4(2), 17–24 (2013) 25. Dervan, P.: Increasing in-class student engagement using socrative (an online student response system). All Irel. J. Teach. Learn. High. Educ. (AISHE-J) 6(3), 1801–1813 (2014) 26. Awedh, M., Mueen, A., Zafar, B., Manzoor, U.: Using socrative and smartphones for the support of collaborative learning. Int. J. Integrating Technol. Educ. (IJITE) 3(4), 17–24 (2015) 27. Wash, P.: Taking advantage of mobile devices: using socrative in the classroom. J. Teach. Learn. Technol. 3(1), 99–101 (2014)

Development of a Classroom Response System: A Web-Based Approach

101

28. Dakka, S.M.: Using socrative to enhance in-class student engagement and collaboration. Int. J. Integrating Technol. Educ. (IJITE) 4(3), 13–19 (2015) 29. Alemohammad, H., Shahini, M.: Use of mobile devices as an interactive method in a mechatronics engineering course: a case study. Proceedings of the ASME International Mechanical Engineering Congress and Exposition IMECE2013, Education and Globalization 5, November 15–21, Can Diego, California, USA (2013) 30. Lucke, T., Keyssner, U., Dunn, P.: The use of a classroom response system to more effectively flip the classroom. Frontiers in Educ. Conf. 2013 IEEE, pp. 491–495 (2013) 31. De Vos, M. (2018). Using Electronic Voting Systems with ResponseWare to Improve Student Learning and Enhance the Student Learning Experience - Final Report, https://www. researchgate.net/publication/267250288. Accessed 14 Aug 2018 32. Gong, Z., Wallace, J.D.: A comparative analysis of iPad and other m-learning technologies: exploring students’ view of adoption, potentials, and challenges. J. Literacy Technol. 13(1), 2–29 (2012) 33. Popescu, O., Chezan, L.C., Jovanovic, V.M., Ayala, O.M.: The use of poll everywhere in engineering technology classes to student stimulates critical thinking and motivation. 122nd ASEE Annual Conference and Exposition, Making Value for Society, June 14 – 17, Seattle, WA, USA (2015) 34. Tregonning, A.M., Doherty, D.A., Hornbuckle, J., Dickinson, J.: The audience response system and knowledge gain: a prospective study. Med. Teach. 34(4), 269–274 (2012) 35. Wu, X., Gao, Y.: Applying the extended technology acceptance model to the use of clickers in student learning: some evidence from macroeconomics classes. Am. J. Bus. Educ. 4(7), 43–50 (2011) 36. Whitehead, C., Ray, L. (2018). Using the iClicker Classroom Response System to Enhance Student Involvement Learning. www.researchgate.net/publication/265192585. Accessed 15 Aug 2018 37. Fuad, M., Deb, D., Etim, J., Gloster, C.: Mobile response system: a novel approach to interactive and hands-on activity in the classroom. Educ. Technol. Res. Devel. 66(2), 493– 514 (2018). https://doi.org/10.1007/s11423-018-9570-5 38. Centea, D., Mehrtash, M.: Active learning strategy using mobile technologies. In: Auer M., Tsiatsos T. (eds) Mobile Technologies and Applications for the Internet of Things. Advances in Intelligent Systems and Computing, 909: 359–367. Springer, Cham (2019). https://doi. org/10.1007/978-3-030-11434-3_3 39. Wu, W.H., Wu, Y.C.J., Chen, C.Y., Kao, H.Y., Lin, C.H., Huang, S.H.: Review of trends from mobile learning studies: a meta-analysis. Comput. Educ. 59(2), 817–827 (2012). https://doi.org/10.1016/j.compedu.2012.03.016

Analysis of the Perception of Students of the Autonomous University of Baja California Sur for the Use of m-Learning Jesús Andrés Sandoval Bringas1(&), Mónica Adriana Carreño León1, and Francisco Javier Alvarez Rodriguez2 1

Universidad Autónoma de Baja California Sur, La Paz, B.C.S, Mexico {sandoval,mcarreno}@uabcs.mx 2 Universidad Autónoma de Aguascalientes, Aguascalientes, Mexico [email protected]

Abstract. Currently there is a fairly widespread tendency to investigate the use of mobile devices in education, since this type of device makes learning more flexible due to the possibility of using it anywhere and anytime. There are numerous projects that refer to the use of m-Learning and its incorporation into educational processes. This paper presents the results of an experience that was applied to a group of students of the Academic Department of Computational Systems (DASC) of the Autonomous University of Baja California Sur (UABCS) with the objective of analyzing the use of mobile devices as support in the classroom. The results show that the students improved the achievement rate, showing availability for the use of m-Learning. In terms of perception, the recognition of the benefits of using m-Learning environments stands out. Keywords: m-Learning


Mobile devices
ICT

1 Introduction Mobile devices are present in almost all areas, and are used to increase productivity in many sectors. The development of educational software is no longer limited to a personal computer, it has extended to the use of mobile devices to achieve greater reach and obtain the benefits that mobile computing offers the educational sector, resulting in the creation of a technological model called m-Learning or mobile learning [1]. Mobile Learning is the process that links the use of mobile devices to teachinglearning practices in the classroom or at a distance [2]. Mobile learning (m-learning), “is an innovative form of teaching-learning that shows the applicability of mobile learning through a broad spectrum of activity” [2, 3] since, it allows the design of activities in a real context and greater interaction between “student-teacher”. Considering that currently many students have a mobile device, and that both students and teachers are developing in the digital age, it is possible in an important way that the teaching-learning process at a higher level can be carried out through the use of mobile devices. M-Learning is an area of growing scientific, academic and economic interest due, among numerous causes, to the strong expansion in orderly and © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 102–108, 2021. https://doi.org/10.1007/978-3-030-49932-7_10

Analysis of the Perception of Students

103

permanent training and the need to ensure the effectiveness of economic investments in learning technologies. The United Nations Educational, Scientific and Cultural Organization (UNESCO) refers to Information and Communication Technologies (ICT) in the educational context and states that these can help students acquire certain skills to get there to be competent in them, like being searchers, analyzers and evaluators of that information; have the ability to solve problems; be creative, communicators, collaborators, publishers, producers and capable of contributing to society [4]. Several researchers consider the use of ICT in higher education as a means to improve the educational quality of their programs. In recent years the use of mobile technology for educational purposes has had a great development in higher education, since there are universities in Europe and America that have mobile education systems [5]. The characteristics of current students, who have been growing with the development of technology, allow the interaction with information through various mechanisms, that is, they are multitasking students, which allows them many advantages using ICT since they can encourage this type of contact with information to turn it into learning [6, 7].

2 Methodology A study was carried out in order to obtain information and analyze the perspective of the students that, within the teaching-learning process, grant to m-Learning environments and their implementation at the higher level. University students of the Educational Programs (PE) of the DASC participated in the development of the study: Software Development Engineering (IDS), Computational Technology Engineering (ITC), Degree in Information Technology Management (LATI) and Bachelor’s Degree in Computing (LC). From the total enrollment of the DASC (496) a representative sample of size (n) was randomly selected. To calculate the sample size, Eq. 1 was used, which considers a finite population, since the study population is known. n¼

Nz2/=2 Pð1  PÞ ðN  1Þe2 þ z2/=2 Pð1  PÞ

Where: n: Estimated sample size N: Size of the population Z: The value of z corresponding to the chosen level of confidence P: Frequency/probability of the factor to be studied e: maximum error estimate

ð1Þ

104

J. A. Sandoval Bringas et al.

The estimated sample size was 115 based on the following values: [N = 496; Z = 1.96; P = 0.5; e = 0.08]. The sample size represents 23.18% of the DASC. The members of this population have as main characteristic to have a professional profile focused on the use of technology, i.e. Software Development Engineers and Computational Technology, and Graduates in Computing and Information Technology Management. As part of the methodology, a change applied to the dynamics of the classes was initially included, with the objective that students use mobile devices in a pedagogical way. For which the activities that were presented to the students were designed, which were carried out during the sessions of the course. The activities that the students had to carry out were focused on developing skills for the search of information on the internet, reading and selecting the most convenient for their learning; in the same way, work together, in the solution of practical exercises proposed in the classroom, and end with the use of the mobile device for its delivery. The study was conducted during the months of January to March of 2018, the survey being the instrument used to collect the assessments and perceptions of the students analyzed. The survey consists of a set of questions regarding one or more variables to be measured [8]. The survey was designed with a quantitative approach and with Likert scales in order to have greater affinity with the student and that this is not discouraged when filling the instrument. A series of affirmations and points on m-Learning in higher education are included, with the purpose of gathering information and obtaining the opinions and value judgments of the students on: knowledge of the subject, the implementation of m-Learning environments as a tool of motivation for the teachinglearning process, as well as the interest of the actors for the creation of a possible application proposal in their classes. Each of the items offers a Likert scale of 4 levels for the response, divided on one side into: 1) Completely agree, 2) Agree, 3) Disagree and 4) Completely disagree, and on the other hand: 1) A lot, 2) Fairly, 3) Little and 4) Nothing. The dimensions that were used in the survey are: 1. Knowledge about m-Learning and its use in the teaching-learning process. Questions to know if students have heard about m-Learning and the benefits obtained. 2. Utility of m-Learning. It includes statements about the opinion of students about the possibilities that m-Learning has, to be used in the teaching-learning process. 3. Attitude before m-Learning. Affirmations to know the opinion of the students about the possibilities of m-Learning before some key questions, that involve the behavior and the emotions of the students as: memory, motivation, commitment among others. 4. Attitude m-Learning against other resources and/or techniques. Know if the students identify some advantages of m-Learning or if they prefer that other didactic resources are implemented in their classroom. 5. Available technological resources: it seeks to know if it has a mobile device, hardware and software features.

Analysis of the Perception of Students

105

6. Interest in the creation of an application proposal in class. It includes affirmations to know the disposition and interest of the students before a proposal of use of mLearning environments in their classroom and the degree of said interest.

3 Results During the test period, the students were given three exams, which allowed them to evaluate the knowledge acquired during the course that they used m-Learning. Below is shown in Fig. 1 the graph that highlights the significant progress in the rate of use, where you can see the decrease in the percentage of failing to 9.7%, in the last partial compared to 29% in the first partial.

Fig. 1. Progress of the results obtained. Source: Authors

The results of the study applied to the students are classified according to the established dimensions, in most of the statements the students’ desire to participate in projects that involve the use of m-Learning environments in their academic activities is evidenced. Below are some graphs obtained with the application of the survey. For the dimension “Knowledge on m-Learning and its use in the teaching-learning process”, it can be seen in Fig. 2 that 58.3% of respondents said that m-Learning motivates participation in class a lot, while only 4.3% answered negatively. The expressed by the students serves to contrast one of the great benefits of the use of mLearning environments which is the motivation, because its use enhances the motivation and participation of the student.

106

J. A. Sandoval Bringas et al.

Fig. 2. Chart of the affirmation: m-Learning motivates in the student the active participation in the teaching-learning process. Source: Authors

For the dimension “Usefulness of m-Learning”, it can be seen in Fig. 3 that 54.8% of respondents said that explanations through m-Learning environments are more easily assimilated, while only 7% answered negatively. Attracting students’ attention in the development of the classes contributes to the success of the same, and it would be evident as a result, the acquired learning.

Fig. 3. Chart of the affirmation: Explanations through m-Learning environments are more easily assimilated. Source: Authors

The lack of knowledge on the subject does not allow students to know the full potential of the use of m-Learning environments; that is why to the statement: “You learn more than the traditional way in the classroom than using m-Learning environments”, there is almost a similarity of affirmative and negative responses, 25.2% consider that they learn better in the same way traditional compared to 23.5% that it would be better to use the m-Learning environment (Fig. 4).

Analysis of the Perception of Students

107

Fig. 4. Chart of the affirmation: Learn more in the traditional way in the classroom than using m-Learning environments. Source: Authors

4 Conclusions The development of ICT has positively benefited many sectors of society, including education. These resources contribute quality to the development of the teachinglearning process. Its use allows it to be carried out in a more personalized, independent and automated way, supporting its formative nature by facilitating an immediate feedback. Given the need to adapt to a new educational system that includes the use and implementation of new technologies in the classroom, it is important to keep updated and explore how to insert innovative techniques that support existing methodologies in education and achieve a significant learning. The results analyzed show that students approximately 88% would feel motivated to use m-Learning environments in the classroom and almost 81% assure that explanations would be more easily assimilated using m-Learning environments. Additionally, it can be highlighted that 100% of the students participating in the study had a mobile device, at least with the indispensable minimum characteristics of a Smartphone or tablet. These results make possible the use of m-Learning in the courses of the DASC’s educational programs.

References 1. Cruz-Flores, R.: Framework On Mobile Learning in Latin America. UNESCO, pp. 1–69 (2007) 2. UNESCO: Turning On Mobile Learning In Latin America. UNESCO, pp. 1–69 (2012) 3. Naismith, L., Lonsdale, P., Vavoula, G., Sharples, M.: Literature review in mobile technologies and learning (2004). http://www.futurelab.org.uk/sites 4. UNESCO: Estandares de competencia en TIC para docentes (2008) 5. Zamarripa, R.: M-learning: El aprendizaje a través de la tecnología móvil, desde la perspectiva de los alumnos de educación superior, pp. 1–15 (2015)

108

J. A. Sandoval Bringas et al.

6. Díaz-Maroto, I., Del Valle, A., Parrón, S.: M-Learning: experiencias actuales y clasificación de tendencias. America, Learning & Media (2012) 7. Barragán, A., Martín, A., Peralta, A.: Análisis del Smartphone como herramienta de apoyo en la formación académica de alumnos universitarios. Pistas Educativas 38, 135–155 (2016) 8. Hernández, Fernández, Baptista: Metodología de la Investigación. Sexta Edición, Mc Graw Hill (2014)

Work-in-Progress: Development of a Framework for Incorporating Usability Aspects with Digital Didactical Design for Mobile/Tablet Based Learning in Pre-primary Education Uthpala Samarakoon1(&) and Hakim Usoof2 1

2

University of Colombo School of Computing, Colombo, Sri Lanka [email protected] Department of Statistics and Computer Science, University of Peradeniya, Peradeniya, Sri Lanka [email protected]

Abstract. Mobile mediated learning is becoming more popular among small children with the advancement of technology. In early days learning process was completely teacher centric. Once the technology is coupled with learning processes, enhanced teaching-learning activities based on mobile or tablet can be presented even outside the classroom. This helps small children to get hands-on experience on various educational activities in learner-centered educational environment which allow them to become active learners. Hence, development of mobile learning systems which will be able to use both within the classroom environment as well as outside that would be more useful for deeper learning. Due to existing research, usability issues take prominence in today’s mobile mediated applications that targeting small children. Small children put more effort on interactions rather than achieving intended objectives of the application. This leads to a reduction of performance of those applications. Hence, the proposed research aims to develop a framework which incorporates usability aspects with digital didactical design for mobile mediated learning in preprimary education. The framework would provide a set of guidelines for incorporating usability aspects with digital didactical design in mobile based learning applications. Keywords: Digital didactical design
Usability
Pre-primary education
Child computer interaction
Mobile based learning
Tablet mediated learning

1 Introduction With the advancement of technology, e-learning has become popular universally. In early days learning process was completely teacher-centric with limited student interactions. The teacher delivered all the theoretical instructions by converting students into passive learners. With technology coupling with the learning processes, enhanced teaching-learning activities can be presented; the students can obtain not only © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 109–116, 2021. https://doi.org/10.1007/978-3-030-49932-7_11

110

U. Samarakoon and H. Usoof

the theoretical knowledge but hands-on experience on various educational activities in learner-centred educational environment which allow them to become active learners. The term “learner-centred” used in educational context is quite analogous to the “usercentred” design approaches of software design, which focuses on the needs, skills, and interests of the learner (Norman and Spohrer 1996). In today’s networked world, learners use traditional classroom environment for asking questions, sharing information and learning from others as well as they use off and on-line channels to engage with the world around them (Jahnke 2015). Hence, development of learning systems to use within as well as without classroom environment would be more useful for deeper learning. Moreover, that kind of learning systems should be designed considering digital didactical concepts. A didactical design includes the design elements such as teaching objectives, learning activities, different forms of process-based feedback, social relations, and ICT, mobile technology or media tablets. The digital didactical design (DDD) is when these elements are visible in a “constructive alignment” where learning really takes places regarding the intended learning outcomes (Jahnke et al. 2014). This would be further improved, when DDD is supported with user experience perspectives to provide students with more effective and efficient interaction mechanism. However, at present DDD lacks in usability and user experience component. Integration of usability aspects within digital didactical design approaches would be an ideal environment to teach in pre-school education. Therefore, this study is mainly focusing on filling this gap by incorporating usability with digital didactical design in pre-primary education.

2 Background E-learning offers numerous opportunities to both learner and the mentor to intensify their teaching learning experience through virtual environments that support delivery, exploration and application of information (Holmes and Gardner 2006). Among all the other stages of education, introduction of e-learning to primary schools has been one of the most significant developments for both teachers and the students and improve students’ self-direction towards teaching learning activities (Etherington 2008). Today, tablet mediated learning plays a major role in e-learning pedagogy and is popular specially among small children. Crescenzi-Lanna and Grané-Oró (2016) identified the problem of poor interaction design in children targeted educational apps. They evaluated 100 educational applications recommended by international experts for children from six months to eight years old. Then they evaluated design parameters such as visual design, adaptability, interaction design, layout and navigation. They found that there are clear issues related to visual and interaction design, adaptability, layout and navigation making it evident that there is a lack of quality and adaptation in terms of children focused application development. Katie Sherwi and Jakob Nielsen conducted several usability studies related to small children over several years. According to them, they have conducted three separate rounds of usability studies, testing a total of 125 children (59 girls and 66 boys). In their first study in 2001, they tested 27 websites with 55 children, aged 6–11 from Israel and

Work-in-Progress: Development of a Framework for Incorporating

111

U.S. In a second study conducted in 2010 they tested 29 websites with 35 children from U.S. between the ages 3 and 12 years. Their latest study in 2019 tested 39 sites and 36 apps with 35 children, aged 3–12 years from U.S. and China. Katie Sherwi and Jakob Nielsenhave done their initial study in 2010 and very recently in January 2019 they have revised and updated their findings in 2010 and published as “Children’s UX: Usability Issues in Designing for Young People”. They conclude that research with users aged 3–12 shows that children have gained substantial proficiency in using websites and apps since their last study in 2010, though many designs are still not optimized for younger users. Designing for children requires distinct usability approaches, including targeting content narrowly for children of different ages. There are several existing models for technology integration in teaching and learning such as SAMR (Substitution Augmentation Modification Redefinition) model by Puentedura (2014), TPACK (Technological Pedagogical Content Knowledge) model by Koehler et al. (2007) and TIM (Technology Integration Matrix) by FCIT, (2013). All of the above models mainly focused on stages of technology integration and the ways in which teachers make use of technological, pedagogical, and content for teaching. This research focused on Digital Didactical Design (DDD) which is mainly focused on learning, that views teaching, learning, and technology integration as a system of three components and encourage collaboration. The original word “didaktik” coming from a German and Scandinavian background and views that “didactic” is a learning activity. In the English form it is Instructional/Pedagogical design. It is the theory of designing teaching and learning in social media world using ICT.

3 Research Objectives The context of this research is framed in three main aspects; Digital Didactical Design, pre-school education and usability aspects of children focused tablet mediated learning. Considering challenges and research concerns, this research would mainly focus on defining a framework for incorporating usability aspects with digital didactical design for tablet-based learning in pre-primary education. The main objective is subdivided into sub-objectives for the feasibility of understanding and realizing. One sub-objective is to identify what usability aspects affect tablet-mediated learning by early primary children. A literature survey found that there was only a smaller number of researches carried on usability aspects of apps for children of this age group (Pre-school level). Furthermore, most of the existing researches are oriented towards children in developed countries and the work done on usability aspects of children in developing countries like Sri Lanka is sparse. As the next objective, the research would analyze whether the aspects found in above research questions can make learning more effective in Pre-school education in Sri Lanka. The combination of outcome of above objective would lead to design a framework for incorporating usability aspects with digital Didactical Design using multi-touch tablet-based learning in pre-primary education. Here Mathematics will be used as the sample domain.

112

U. Samarakoon and H. Usoof

4 Approach Identifying defining a framework for incorporating usability aspects with digital didactical design for tablet-based learning in pre-primary education is the main objective of this research and to achieve this main objective, this research would follow design science research as a methodology. Design science is a methodology that could be exercised on improving human performance of information and communication systems research where the focus is on researching the application of computer technology and methodology in other disciplines. According to design science research, artifacts are created and evaluated by following a rigorous iterative process to solve existing problems (Peffers et al. 2007). This aspect of design science research shares similarities such as iterative nature of cycles when creating a solution as in designbased research, a methodology frequently used in education and learning sciences and interdisciplinary research (Amiel and Reeves 2008). This process includes six stages as problem identification and motivation; define objectives for solution, design and development, demonstration, evaluation and communication of end results. Furthermore, this study will use a mixed method approach which uses both quantitative and qualitative data gathered using different objective and subjective instruments and mainly will focus on the qualitative aspects of the domain. Number of correct/incorrect attempts and number of total repeat attempts will take as quantitative data and data that will gather from teacher interviews, questionnaires for parents and classroom observations will be used for qualitative measures. Sri Lankan pre-school level Mathematics curriculum was selected as the sample curriculum and age 3 to 4 (Pre-school) children were selected as the target group for the study. The Sri Lankan Mathematics curriculum was taken for this research. Figure 1 illustrates the overall research plan and methodology of the proposed research. The process was initiated by familiarizing and identifying prevailing usability problems in the primary e-Learning Apps. Subsequently a thorough literature review was conducted around the main research question to formulate the research objectives. The existing research gaps like exclusion of addressing usability aspects in digital didactical design of children (age 3–5) target apps were recognized. In next phase a methodical literature review was conducted to identify interface designing factors and touch gesture performance of children of age 2–7 years. The review identified different factors that should consider when designing interfaces and systems for children of preoperational stage (Age 2 to 7). Further, the literature survey on touch gesture performance of children was used to identify most feasible gestures that can be perform easily by children and further identified some best practices that should be followed when selecting gestures for small children. As the next phase an experiment was conducted to verify the findings of the literature review on touch gesture performance of pre-school children (Age 3 to 5) to evaluate actual performance of children in local (Sri Lankan) context. Existing educational apps in play store were used for the experiment and analyzed the gesture performances of local children and their preferences when designing interfaces. Finally, a complete set of touch gestures that can be performed and most preferable interface design factors for children of age three to four were identified. In addition, information was gathered related to teaching

Work-in-Progress: Development of a Framework for Incorporating

113

and learning patterns in pre-school using classroom observations and interviews with teachers. Information related to children’s socio-economic background and technology usage at home was also identified through a questionnaire for parents of the children of the study. Here the Mathematics was taken as the sample curriculum. The digital didactical design elements will be mapped to selected Math lessons. Then a prototype digital artifact will be developed by considering digital didactical design and findings of the previous steps of the research for these selected topics. Children will also be a part of the design process where their likes and dislikes will be considered when designing the digital artifact. The evaluation process of the prototype would focus on different aspects such as the usability of proposed system like user friendliness, clarity of functionality and components, easy navigation, gesture performance etc. To assess the usability issues and concerns related to usability and didactical design of the developed prototype, a systematic user testing phase will be conducted. The developed digital artifact will be evaluated by small children until an acceptable level of refinement is achieved. This would be done through an iterative process. Subsequently, the prototype will be converted into a comprehensive tablet-based e-Learning application for pre-school. A prototype framework would be developed based on factors and best practices identified during the research process. Familiarising and idenƟfying prevailing problems in the e-Learning of primary educaƟon

Apply digital didacƟcal design in Kindergarten mathemaƟcs curriculum

IniƟal literature review and idenƟfy research gap

Design and develop a prototype tablet-based applicaƟon

Evaluate

Refine

User tesƟng of prototype for EvaluaƟon of usability

Define research quesƟon and formulate research objecƟves

Gather background informaƟon through teacher interviews, quesƟonnaire for parents and classroom observaƟons

Touch gesture performance and interface design preferences of children in local context by experiment

IdenƟfy usability aspects in children’s’ soŌware development of preoperaƟonal stage children

Finalize the tablet-based e-Learning applicaƟon

Refine

Touch gesture performance from literature review

Interface designing factors from literature review

Design the prototype framework

Evaluate the prototype framework Evaluate

Finalize the framework for an e-Learning System which Incorporates usability aspects with Digital DidacƟcal Design in the area of MulƟ-touch tablet-based Learning in primary educaƟon

Fig. 1. Overall research plan and methodology of the proposed research

114

U. Samarakoon and H. Usoof

The framework would be evaluated through experts in subject domain and this will lead to the ultimate contribution of the research, formulation of a framework which incorporates usability aspects with Digital Didactical Design in the area of Multi-touch tablet-based Learning in primary education.

5 Analysis of Current Findings An experimental study was conducted to identify touch gesture performance of preschool children and their preferences in interface designing and navigation of preschool children. One major intention of the experiment was to investigate the ability of pre-school children (aged 3 to 5) in using the nine gestures in tablet applications. At the same time, the observations were used to identify most suitable design criteria for small children. Thirty-five children of age three to five in Sri Lankan pre-school level took part in the experiment. The study prepared a comfortable environment for the children to use the tablet one at a time. A digital video camera was carefully used to record images and video of gestures made by each child. The researcher guided the child to use each application at least once. The items recorded were child’s age, prior experience with touch screen devices and how well they use the app in the tablet. The details include gestures they could use, the finger movements while using each gesture and the problems faced by the child while using them.

100 90 80

94

89

89 70

70 60 50

54

49

43

40

37

30 20

11

10 0 Tap (T) Flick (F) Point & Spread Drag & Drag (D) Pinch Click (S) Drop (P) (P&C) (D&D)

Long Double press Tap (DT) (LP)

Fig. 2. Gesture performance of pre-school children in local context

The findings (Fig. 2) show that tap, flick and point & click having high success rates compared to other gestures. Spread gesture had the next highest success percentage, while drag & drop, pinch and drag are having considerably low percentages

Work-in-Progress: Development of a Framework for Incorporating

115

than other gestures. Long press and double tap gesture were having the least success percentages which show that, children at this age struggle when perform those gestures in tablet-based applications. Only one research is available (Nacher et al. 2014) which experimented these two gestures and also indicate low performance such as 62% for long press and 50% for double tap. The overall results found that gestures like tap, point & click and flick are most comfortable with small children while drag, drag & drop and spread gestures are better to use only in necessary situations. In the other hand gestures like double tap and long press better to avoid in children targeted applications. At present the research is in the stage of design and develops a prototype tabletbased application as per Fig. 1. Then the developed application will be tested through pre-school children and the process shown in Fig. 1 will be followed until achieve the outcome of the research.

6 Anticipated Outcomes This research will focus on the areas that are not covered by the research done in this area with respect to improving usability aspects in children targeted applications while applying digital didactical design in tablet/mobile based pre-primary education. The outcome of this research will be a framework for incorporating digital didactical design with usability in pre-school education. The digital artifact that will come out from this research will be enriched with usability aspects where pre-school children may more focus on activities rather struggling with interactions. Finally, the framework will be guiding and improving the children targeted software development industry towards enhanced usability.

7 Conclusion E-learning is popular among small children with advancement of mobile technologies. However, initial studies have shown that small children are ready for mobile based learning but need proper design based on their cognitive and psychomotor abilities and limitations. The designers of children targeted software need to consider the abilities of children such as gesture performance as well as interface designing factors feasible for small children as these directly will affect the usability of such applications. The findings of this research study would provide a framework and recommendations to enhance interactions for mobile based learning application for pre-school children, by incorporating digital didactical design with usability aspects in pre-primary education. Acknowledgement. The authors acknowledge the support received from pre-schoolchildren, teachers and parents in this research work.

116

U. Samarakoon and H. Usoof

References Amiel, T., Reeves, T.C.: Design-based research and educational technology: Rethinking technology and the research agenda. J. Educ. Technol. Soc. 11(4), 29–40 (2008) Crescenzi-Lanna, L., Grané-Oró, M.: An Analysis of the Interaction Design of the Best Educational Apps for Children Aged Zero to Eight = Análisis del diseñointeractivo de las mejores apps educativas para niños de cero aochoaños. Comunicar 24(46), 77–85 (2016) Etherington, M.: E-Learning pedagogy in the primary school classroom: the McDonaldization of education. Aust. J. Teacher Educ. 33(5), 29–54 (2008) Holmes, B., Gardner, J.: E-learning: Concepts and Practice. Sage, London (2006) Jahnke, I.: Digital Didactical Designs: Teaching and Learning in CrossActionSpaces. Routledge, New York (2015) Jahnke, I., Mårell-Olsson, E., Bergström, P., Norqvist, L., Olsson, A.: Designs Settings?. Skapaförutsättningarförforskning-generiskafärdigheterfördoktorander, 13 (2014) Koehler, M.J., Mishra, P., Yahya, K.: Tracing the development of teacher knowledge in a design seminar: Integrating content, pedagogy and technology. Comput. Educ. 49(3), 740–762 (2007) Nacher, V., Jaen, J., Catala, A., Navarro, E., Gonzalez, P.: Improving pre-kindergarten touch performance. In: Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces, pp. 163–166, November 2014 Norman, D.A., Spohrer, J.C.: Learner-centered education. Commun. ACM 39(4), 24–27 (1996) Peffers, K., Tuunanen, T., Rothenberger, M.A., Chatterjee, S.: A design science research methodology for information systems research. J. Manag. Inf. Syst. 24(3), 45–77 (2007) Puentedura, R.: Learning, technology, and the SAMR model: Goals, processes, and practice. Puentedura’s Weblog, Ruben R (2014)

Promoting Authentic Student Assessment for STEM Project-Based Learning Activities Andri Vrioni1(&), Anna Mavroudi2, and Ioannis Ioannou3 1

2

University of Nicosia, Nicosia, Cyprus [email protected] Norwegian University of Science and Technology, Trondheim, Norway 3 Ministry of Education, Strovolos, Cyprus

Abstract. This paper is a case study which discusses the development and the use of rubrics as means of assessing students in STEM project-based learning activities. The context of the case study involves the pilot phase of the STEMFREAK program, an out-of-school educational program which engaged students in such projects. The aim of the paper is twofold: 1) to present the systematic process that was employed for the design, development and application of rubrics and 2) to assess the effectiveness of the STEMFREAK program which, following the principle of constructive alignment between learning design and assessment strategy, was designed to develop the students’ competencies that the rubrics assess. The design and the development of the rubrics took place following an iterative and participatory approach, whereas their application engaged 15 students in STEM projects. The students were assessed using the ensuing rubrics in a pre-post single group research design. The results showed that all students made statistically significant progress in all aspects of their performance, as assessed by the rubrics made. The case study has the potential of informing future approaches towards the assessment of students in projects that cater for STEM project-based learning with young learners. Keywords: STEM education
Assessment rubrics Authentic assessment
Project-based learning


21st century skills


1 Introduction In recent years, there is an emphasis on further advancing STEM (Science, Technology, Engineering, Mathematics) education and the need to create today STEM educational opportunities among young learners that will help towards the growth of the skilled STEM workforce of tomorrow. Nowadays, these opportunities can be met not only in formal education through schooling, but in a variety of (informal or semiformal learning) settings. There is growing evidence that opportunities to learn STEM out of school can have a positive influence on STEM school performance [14]. The case study discussed here involves STEM education in the context of an out-of-school afternoon program in Cyprus. The aim of the study is to present a methodology of capitalising on what alternative assessment can offer in STEM education and provide

© Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 117–126, 2021. https://doi.org/10.1007/978-3-030-49932-7_12

118

A. Vrioni et al.

some preliminary empirical findings on its applicability in project-based learning settings.

2 Background 2.1

21st Century Skills Frameworks

Several organizations have made attempts to list 21st century skills. The aim of this section is to summarise those attempts that were considered in the planning phase of the rubrics used. The Partnership for 21st Century Skills [15], proposed a set of 21st century student outcomes, including both core subjects and transversal skills. The core subjects refer to reading, writing, and arithmetic, whereas the 21st century skills refer to critical thinking, communication, collaboration and creativity. The P21 partnership also proposed that these 21st century student outcomes could affect standards and assessment, curriculum and instruction, professional development, and learning environments. The Assessing and Teaching of 21st Century Skills (ATC21S) organization commissioned a white paper to define 21st century skills [6], in which an expert group was used to propose a taxonomy organized as follows: “Ways of Thinking” (creativity and innovation; critical thinking, problem solving and decision making; learning to learn and metacognition), “Ways of Working” (communication; collaboration and teamwork), “Tools for Working” (information literacy; ICT literacy) and “Living in the World” (global and local citizenship; life and career; personal and social responsibility). The Valid Assessment of Learning in Undergraduate Education (VALUE) rubrics were developed to evaluate authentic student work samples [11]. They assess student learning at four distinct levels, where the first one measures student performance at entrance into university and the final one measures performance upon completion of an undergraduate degree. There are 16 VALUE rubrics grouped under three main categories: intellectual and practical skills, personal and social responsibility, and integrative and applied learning. The most relevant VALUE rubrics for the purposes of this work were those that touched upon written communication, oral communication, teamwork, problem solving, and inquiry and analysis [2]. The Engineering Graduate Attributes framework [7] was developed by the University of Toronto in response to the need of an outcome based accreditation system that places emphasis on competencies, such as: design skills (e.g. design solutions for open‐ended engineering problems), communication skills (e.g. communicate engineering concepts, comprehend and write effective reports and design documentation, give and effectively respond to clear instructions), individual- and team-work, problem analysis (identify, formulate, and analyse engineering problems), investigation (through appropriate experiments, analysis and interpretation of data, and synthesis of information in order to reach valid conclusions).

Promoting Authentic Student Assessment

2.2

119

The Use of Rubrics as an Assessment Tool in STEM

Literature review suggests that rubrics is an authentic tool of assessment that can help students reach their learning objectives [19]. Some educators have developed rubrics to provide measurement guidelines, to assess performance and to facilitate feedback to students. Andrade [4] provides background on the structure, purpose and benefits of rubrics as teaching and grading tools while adding that effective rubrics are considered those which are valid, reliable and fair. Stevens and Levi [16] believe that collaborating to develop a rubric offers an opportunity to discuss and evaluate and validate grading practices. In the field of STEM, rubrics have been developed and used as an assessment mechanism by several researchers in the context of school education. Cateté and Barne [10] developed rubrics to help K-12 teachers to grade computer science education programs and labs. The authors used the Delphi Method to create learning-oriented rubrics for the “Beauty and Joy of Computing curriculum” [17]. Among their findings, the authors conclude that a small expert panel with active teachers shows minimal drawbacks when compared to a larger heterogeneous panel in the use of Delphi method. They accentuate the importance of engaging proper stakeholders with timeavailability, knowledge on the subject, and willingness to participate. Another example of using rubrics in STEM education for school pupils involves the MarineTech project [18], which developed and integrated project-based learning activities for the middle and high school STEM curriculum. Although the rubrics description is brief in their article, the authors mention that students were working in groups and that each group activity was assessed based on rubrics criteria which mostly touched upon the final student product (i.e. the ship).

3 Theoretical Framework: Project-Based Learning Project-Based Learning (PBL) is a student-driven and teacher-facilitated approach that organises learning into student group projects [5]. In the literature, there is no generally accepted model or theory regarding PBL. Yet, it has been argued [5] that in its generic form, students in the PBL model: initially use organizers to isolate an inquiry question; then they brainstorm about the procedure that they will follow in their inquiry and identify materials they might need; finally, students select a way to demonstrate what they have learned in the form of a project. Perhaps PBL can be better understood by the defining characteristics of its processes. For instance, PBL entails the active engagement of students who [3]: a) cooperate and collaborate (e.g. share knowledge, work in groups), b) solve authentic problems (e.g. set project goals, conduct investigations), and c) construct a final solution in the form of some tangible artefact(s). It has been frequently suggested – see for example Bell [5] - that PBL can be used as a means of cultivating students’ 21st century skills such as communication, negotiation, collaboration and creativity. Karaçalli and Korur [12] conducted a quasi-experimental study in Turkey with fourth-grade science students working in PBL tasks within the topic of electricity in our lives. The authors found statistically significant affects in terms of academic achievement, students’ attitude and retention of knowledge. In another case

120

A. Vrioni et al.

study which took place in the context of secondary high school, Al-Balushi and AlAamri [1] studied the effects of PBL on students’ environmental knowledge and environmental science attitudes. The study followed a quasi-experimental pre-post control group design and used two ad-hoc research instruments for assessing the knowledge gain and the attitude change of the students, respectively. The results indicated that students’ involvement in PBL had a statistically significant positive effect on both aspects. Also, that the projects that had the most impact were those that required students to produce enjoyable and unusual final products, such as a documentary movie.

4 Method 4.1

Rubrics Development Procedure

The Planning Phase started with reviewing the literature and was carried out in order to create a comprehensive list of learning outcomes. The goal was to seek what criteria are common among existing STEM program rubrics. The sources of input for this phase are described in Sect. 2.1 above. Regarding the researchers’ decision to also consider sources with assessment criteria that apply for university students (as opposed to school pupils), the justification comes from the fact that recently there is an emphasis on STEM education with respect to linking secondary education with post-secondary practices [8]. At the end of this phase, a list of more specific, measurable learning outcomes in the form of indicators, was compiled. In the Development Phase, rubric descriptors were drafted and modified through consultation with STEM teachers and students. Especially the co-development of the rubrics with the students before the beginning of the actual project work is documented as a good practice in the STEM PBL literature [8]. The outcomes and indicators were also validated through a systematic Delphi method with the STEM teachers. The Delphi method [3] is used to collect, synthesize and augment knowledge from a group of experts through a series of questions posed in consecutive rounds. At the end of each round, the anonymous responses of the experts and their justification are collected by the researchers. Then, the researchers summarise the responses and as the rounds continue, the responses are refined with the purposes of achieving consensus, as much as possible. In line with the Delphi method process, multiple rounds of questionnaires were sent to a panel of expert STEM teachers and their anonymous responses were aggregated and shared with the group after each round. The experts were allowed to adjust their answers in subsequent rounds, based on how they interpret the “group response” that has been provided to them. Through the implementation of Delphi method consensus was achieved with respect to the development of rubrics criteria and performance standards across them. The third and final phase was the Evaluation Phase focusing on testing the rubrics. In particular, “assessors” were requested to evaluate students and their work while students were implementing STEM PBL activities. The “assessors”, who were the same experts that were used in the previous phase, were using the rubrics and provided feedback about them through focus groups. The objective of testing was to further

Promoting Authentic Student Assessment

121

validate the learning objectives, indicators and rubric descriptors and to obtain feedback on how they could be improved. Based on analysis of discussions with the assessors, the rubrics underwent revision to address the feedback provided by the “assessors”. 4.2

The STEMFREAK Educational Program Methodology

The STEMFREAK is an out-of-school educational program of 100 h duration; 90 of them involve school pupils actively working in STEM PBL activities. The remaining 10 h involve welcoming, pretraining activities, and other support. For instance, the STEM class teacher has to familiarise the newcoming students with the logic and procedures of the program e.g. to introduce them to the program expectations, rubrics, and timelines. The remaining 90 h are comprised of 45 sessions (approximately 2 h each) during which students are typically working in small groups with challenges described in the form of STEM scenarios. These scenarios align with the PBL methodology. For instance, the challenges are designed in a way to: 1) capture students’ interest by relating to real-world issues, 2) draw on students’ previous learning and experience, 3) integrate content objectives with problem-solving skills, 4) require a cooperative, multi-staged method to solve, 5) enable students to do some independent research in order to gather all information relevant to the problem. During the sessions, the students are working with their peers and the class teacher (who has a mentor role) in a space that resembles a makerspace. It is especially designed for kids, where (in addition to the usual STEM makerspace equipment and materials) laptops, tablets and smartphones are available to help pupils carry out their investigations. Since the locus of control of the learning process is towards the pupils’ side, they can choose for themselves which kind of devices they would like to work with. Some students can even bring their own devices, if they wish. The majority of pupils prefer to work with tablets. Near the end of the session, the students create their artefacts and, in addition, they present their findings. Again, there are given several options which can be combined as the students wish, e.g. the creation of movies/documentaries, posters, concepts maps. Typically, students create a small documentary movie using the tables and/or the smartphones, where they explain how they worked along the process and demonstrate their final product. 4.3

Participants’ Profile

Regarding the profile of the STEM experts, they are all active experienced STEM teachers. They hold a bachelors’ degree in some STEM area; also, they all hold a relevant masters’ degree (i.e. in the intersection of education and some STEM topic). For the purpose of this work, they should additionally participate in a training workshop of 36 h focusing on STEM education and authentic assessment methods. From the pool of these experts, we selected three of those that already had three years teaching experience in teaching STEM within a PBL approach. Also, the experts had time-availability, knowledge on the subject, and willingness to participate [10]. The STEM PBL process was facilitated by a STEM teacher who was not part of the expert panel.

122

A. Vrioni et al.

There are two types of participant students: those that affected the development of the rubrics (as explained in Sect. 2.1) and those that participated in the STEM PBL program (which is described in Sect. 4.3). The students that were chosen to participate in the development of rubrics had at least two years’ experience in a STEM programme. We choose four students, two from primary school education and two from secondary school education with good communication skills so that they could fluently express their beliefs, ideas and opinions about the rubrics. Regarding the students that participated in the STEM PBL program, they were purposively selected from a larger sample of students participating in the after-school program activities so that 1) the distribution across boys and girls was fair, 2) they all had never before participated in an out-of-school STEM experience, and 3) they covered a wide range of age distribution (from 5 to 15 years old). Fifteen students participated in this research. Students within the same age group were working together in groups of three. The five groups of students that were formed were of ages 5–6, 7–8, 9–10, 11–12 and 13–15 years old, respectively. Eight of them were boys and seven of them were girls. 4.4

Data Collection and Analysis

Regarding the development of the rubrics and the Delphi method, initially the experts were asked about indicators of STEM learning in a PBL class using an anonymous survey. The survey consisted of a single open-ended question that asked experts to list all the criteria/indicators that are necessary to assess students learning outcomes when they are involved in STEM PBL activities. Duplicates were removed as well as criteria that were deemed irrelevant through discussion and consensus, while taking into account the literature review that had been conducted previously. Subsequently, experts were presented with the remaining list of criteria (after filtering the ones initially gathered), and they were asked anonymously to rate 1) how likely they were to use each criterion in their teaching, and 2) the (perceived) importance of each criterion in the STEM PBL curriculum using a 4-point Likert type scale. A score of 4 indicated the greatest likelihood, and highest level of importance. Finally, the participants were given an opportunity to make any suggestions or additional comments on the consolidated list of criteria. The same procedure was followed for the performance standards across criteria. The STEM expert teachers were observing the STEM PBL process using the rubrics to assess each student. These observations were conducted by two independent evaluators. In case of disagreement, the opinion of the class teacher was taken into consideration. The results were analysed by calculating the average performance scores in each of the two points in time (i.e. pre- and post- test) for each student. Each average performance score considered the performance scores in all the criteria that were included in a rubric. The pre-test was completed at the end of the first session and the post-test at the end of the lass session of the STEMFREAK program. Due to the small number of participants a non-parametric test, namely the Wilcoxon signed-rank test, was employed to understand whether there were performance differences in the matched pairs of the repeated measures and whether they were statistically significant. The Wilcoxon test does not assume normality in the data as its equivalent paired sample

Promoting Authentic Student Assessment

123

t-test. It was used in tandem with the average student performance scores and other basic descriptive statistics measures.

5 Results 5.1

The STEM PBL Assessment Rubrics

The STEM PBL assessment rubrics were different across pupils’ age groups. The main reason is that the assessment has to be aligned with the developmentally appropriate learning activities of the STEM PBL curriculum. Regarding their development with the Delphi method, Table 1 presents an indicative example. It involves part of the results obtained at the first rounds of the Delphi method regarding rubrics’ design for 5–6 years old students: a list of some of the criteria that were gathered initially by the two experts, and some of the results of the intermediate phase (when the criteria list was elaborated and the experts gave scores in a 4-point likert scale). Table 1. Part of an indicative example of Delphi study results (ages 5–6 years old) Criteria listed by experts (round 1)

Consolidated list of criteria probably using when assessing students STEM PBL work (round 2)

Grades from experts (round 2)

Presentation skills Creativity Teamwork skills Time management Scientific reasoning Effectiveness Completion

Stay on task Justifies the solution Cognitive domain Troubleshooting Conflict resolution skills Quality of student artefact Metacognitive skills

2 3 3 1 1 4 3

3 4 4 1 4 4 4

Table 2 below summarises the main dimensions for all the assessment rubrics used. Each dimension corresponds to more than one assessment criterion. For each criterion, four levels of behaviour indicators were described. In that way, the different scores (ranging from 1 to 4) across the several criteria, could be aggregated to one final performance score for the student. As can be seen from the table, the only common dimension across the different age groups was the assessment of the criteria that were associated to the cognitive domain; furthermore, one common criterion included in that dimension was referring to student’s understanding of subject-matter specific concepts and principles.

124

A. Vrioni et al. Table 2. Overview of the assessment rubrics design Student age group Dimensions assessed in the rubric 5–6 years old Design solution-students’ artefact Teamwork Cognitive domain 7–8 years old Design solution-students’ artefact Teamwork Communication Cognitive domain 9–10 years old Design solution-students’ artefact Teamwork Oral final presentation Cognitive domain 10–12 years old Design solution-students’ artefact Teamwork Written final project report Cognitive domain 13–15 years old Project work Written final project report Oral final presentation Cognitive domain

5.2

Number of criteria 4 4 2 7 4 2 2 5 5 3 2 7 5 4 2 3 4 5 2

Students’ Performance Results

Table 3 shows descriptive statistics for the pre-test and the post-test. The average and the median scores of students’ performance at the end of the final challenge is higher than those at the end of the first challenge. Table 3. Descriptive statistics of student performance Pre-test Mean 1,62 Median 1,59 Std. Deviation 0,24

Post-test 3,48 3,43 0,42

The difference scores were calculated and then, they were visually inspected in a histogram. Through visual inspection of the histogram, it emerged that the difference scores were approximately symmetrically distributed, thus the assumptions of the Wilcoxon ranked test were met and consequently, the analysis could be performed appropriately. The results of the test showed that all 15 students had a significant improvement in their performance. Also, the differences between the pre- and the posttest scores were statistically significant (z = 3,408, p = 0,001).

Promoting Authentic Student Assessment

125

6 Discussion This case study presents preliminary findings of an ongoing research work on authentic pupils’ assessment in the context of STEM PBL. The study was focusing on the development of rubrics in STEM PBL settings and their application to understand whether there was an improvement of the students’ scores. Regarding the rubrics, their development was based on the relevant literature, but it also was participatory and iterative. With respect to the performance of students, we followed a repeated measures single group research design using the criteria of the assessment rubrics. The increase in their performance was found to be statistically significant. It should be noted that the students were assessed each session using the rubrics created, but for the purpose of this work we considered only with the pre-test (at the end of the first session) and the post-test (at the end of the last session) data.

7 Conclusions The contribution of this work touches upon the fact that STEM PBL requires new perspectives on assessment [9]. A good idea -which we followed in this work- would be to capitalise on what alternative types of assessment can offer. Such alternatives can embrace authentic assessment that which focuses on students’ projects. Specifically, “just-in-time” assessment is recommended, as a form of authentic assessment that capitalises on mobile devices affordances [9]. Mobile devices or tables can easily capture student performance via a video and audio file, which can be used later by the student to present their work as well as by the teacher to digitally record information into rubrics and made immediately available to the students [9]. This is also the case for the STEM empirical research presented herein. Although the mobile devices were not central to all the PBL phases, they were used extensively by the students near the end of each challenge to help them to create and share the documentaries or videos that demonstrate their artefacts and the ways in which the students worked to create these artefacts. As can be seen from Table 2, in terms of rubrics design this work expands previous work on STEM PBL conducted by Verma, Dickerson and McKinney [18], which mostly used rubrics criteria that touched upon the final student artefact. Limitations of this study pertain to the small number of participating students as well as to the fact that all participants live in the same country, which in turn has an effect on the generalisability of the results. Future research plans include the use of rubrics in tandem with other means of program evaluation, such as student interviews and field notes in order to triangulate the findings and to promote authentic assessment in STEM PBL settings.

References 1. Al-Balushi, S.M., Al-Aamri, S.S.: The effect of environmental science projects on students’ environmental knowledge and science attitudes. Int. Res. Geograph. Environ. Educ. 23, 213–227 (2014)

126

A. Vrioni et al.

2. Association of American Colleges & Universities: VALUE: Valid Assessment of Learning In Undergraduate Education (2013). https://www.aacu.org/value/rubrics. Accessed 22 Aug 2019 3. Adler, M., Ziglio, E.: Gazing into the Oracle. Jessica Kingsley Publishers, Bristol (1996) 4. Andrade, L.: Teaching with rubrics: the good, the bad, and the ugly. Coll. Teach. 53, 27–30 (2005) 5. Bell, S.: Project-based learning for the 21st century: Skills for the future. Clearing House 83(2), 39–43 (2010) 6. Binkley, M., Erstad, O., Herman, J., Raizen, S., Ripley, M. (with M. Rumble): Defining 21st century skills and assessments (Draft White Paper 1). Melbourne, Australia: Assessment and Teaching of 21st Century Skills. University of Melbourne (2010). http://hdl.voced.edu.au/ 10707/260031. Accessed 22 Aug 2019 7. Canadian Engineering Accreditation Board. Accreditation Criteria and Procedures 2008, September 2008. http://www.engineerscanada.ca/e/files/report_ceab_08_txt_only.pdf. Accessed 22 Aug 2019 8. Capraro, R.M., Slough, S.W.: Why PBL? Why STEM? why now? an introduction to STEM project-based learning: an integrated science, technology, engineering, and mathematics (STEM) approach. In: STEM Project-Based Learning, pp. 1–5. Brill Sense (2013) 9. Capraro, R.M., Corlu, M.S.: Changing views on assessment for STEM project-based learning. In: STEM Project-Based Learning, pp. 109–118. Sense Publishers, Rotterdam (2013) 10. Cateté, V., Barnes, T.: Application of the delphi method in computer science principles rubric creation. In: Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education, pp. 164–169. ACM, June 2017 11. Gray, J.S., Brown, M.A., Connolly, J.P.: Examining construct validity of the quantitative literacy VALUE rubric in college-level STEM assignments. Res. Pract. Assess. 12, 20–31 (2017) 12. Karaçalli, S., Korur, F.: The effects of project-based learning on students’ academic achievement, attitude, and retention of knowledge: The subject of ‘electricity in our lives’. School Sci. Math. 114, 224–235 (2014) 13. Kokotsaki, D., Menzies, V., Wiggins, A.: Project-based learning: a review of the literature. Improving Schools 19(3), 267–277 (2016) 14. National Research Council: Identifying and supporting productive STEM programs in outof-school settings. National Academies Press (2015) 15. Partnership for 21st Century Skills: Framework for 21st century learning (2019). http:// www.battelleforkids.org/networks/p. Accessed 22 Aug 2019, 21 16. Stevens, D.D., Levi, A.: Introduction to Rubrics: An Assessment Tool to save Grading Time, Convey Effective Feedback, and Promote Student Learning. Stylus Publishing, Sterling (2005). Tucker, T., Derelian, D., & 17. University of California, Berkeley and Education Development Center, Inc. 2017. Beauty and Joy of Computing. https://bjc.berkeley.edu/. Accessed 22 August 2019 18. Verma, A.K., Dickerson, D., McKinney, S.: Engaging students in STEM careers with project-based learning—MarineTech project. Technology and Engineering Teacher, vol. 71 (1) (2011) 19. Yoshina, J.M., Harada, V.H.: Involving students in learning through rubrics. Library Media Coordinator 25(5), 10–14 (2007)

Predictive Modeling Concerning Mobile Learning Advance Malinka Ivanova(&) College of Energy and Electronics, Technical University of Sofia, Sofia, Bulgaria [email protected] Abstract. The paper treats an application of predictive modeling in the field of mobile learning. A methodology to facilitate the realization of a model predicting the most utilized research topics that are close to the term mobile learning is developed. The constructed model is based on machine learning technique and fuzzy logic method and it predicts the implementation of mobile learning in different educational context. The results point out the found dependency and tendency for future advance of mobile learning. Keywords: Mobile learning
eLearning informatics Linear regression
Fuzzy logic


Machine learning


1 Introduction One brunch of eLearning Informatics as a scientific field explores the possibility of Informatics statements and theories how to be applied in the context of eLearning. Informatics attainments propose a huge pool of knowledge in different topics, including in modeling of concepts, events and processes in eLearning. Modeling algorithms and techniques contribute to better understanding the static and dynamic features of a system, preparing views from different perspectives. For the purposes of modeling a wide variety of machine learning approaches are utilized to automate identification of patterns and trends in the domain of teaching and learning [1]. Predictive modeling with machine learning algorithms allows complex systems to be explored and studied with opportunity of algorithms for self-learning and self-evolving. Predictive analysis facilitates understanding the challenging issues, assumptions permission and decision making based on precise data processing and training. Mobile learning (mLearning) is seen as the future of eLearning proposing new virtual learning environment that stimulates students to learn from any geographical location and at suitable for them time [2, 3]. Mobile technology is also a driving force for open teaching achieving extreme flexibility and efficacy. eLearning imperceptibly converts to mLearning uncovering multiple advantages of mobile technologies like: devices portability and integration of smart functions, wearability and networking as well as its applications in different learning context. The state of mobile learning in Europe is summarized in [4], giving its main characteristics related to: bridging formal and informal learning, improving collaborative and conversational learning, stimulating self-directed and personalized learning. © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 127–135, 2021. https://doi.org/10.1007/978-3-030-49932-7_13

128

M. Ivanova

Predictive modeling based on machine learning techniques in mobile learning is well accepted approach for predicting the students’ performance and effectiveness, for identifying the students’ at-risk and their drop-out rate, to improve retention and engagement. This work presents a methodology for forecasting the directions for evolvement of mobile learning and its relationships with contextual learning, based on extracted terms from abstract and citation database Scopus and construction and visualization the bibliographic networks as well as applying linear regression and fuzzy logic techniques. A model based on the proposed methodology is created to predict the future state of mLearning and its further implementation in a wide variety of scenarios and situations.

2 Methodology This methodology is developed to facilitate the implementation of a predictive model concerning the mLearning evolvement in order to point out the well explored topics and topics that need more attention by researchers. The methodology consists of the following procedures: I. Data extraction from abstract and citation database Scopus and construction of bibliometric networks through usage of VOSviewer software for scientific visualizations.; II. Creation of preparatory matrixes with extracted terms containing information about terms’ occurrences in the used set of documents and the terms’ total link strengths as well as the dependences between occurrences and year of publication.; III. Applying linear regression algorithm to forecast the effect of changes in the term’s occurrences and the term’s total link strengths during eleven consecutive years – from 2008 year to 2018 year as well as to predict the trends trough utilization of Octave software for numerical computations.; IV. Constructing a fuzzy inference system (FIS) through usage of software VisPro for predicting the connection of the term mobile technology to the terms teaching and learning. I Procedure: Data extraction and bibliometric networks construction. 1. Gathering data about the term mLearning. The starting point is query construction in Scopus search engine regarding the keyword mobile learning (and its equivalents keywords mLearning and m-learning) and performance of results limitation according to documents relevance, year of publication – consecutive eleven years – from 2008 to 2018 year, document type – conference paper, article and review, source type - conference papers and journals, language – English. The query is applied to search in Article title, Abstract and Keywords of documents. The obtained bibliographic results (citation information, bibliographical information and abstract and keywords) separately for each year are exported in csv format. 2. Construction of bibliometric networks. To find the connections among the term mLearning and other extracted terms the bibliometric networks over the selected years separately are constructed. For this purpose the exported .csv file from Scopus is imported in VOSviewer [5]. Several settings are adjusted like: type of analysis is chosen to co-occurrence and unit of analysis is selected to all keywords. The applied method is full counting and minimal numbers of occurrences of a keyword

Predictive Modeling Concerning Mobile Learning Advance

129

is limited to 5. A co-occurrence link between two terms shows the number of documents that at the same time include these both terms. Just one link connects two terms. Each link characterizes with a strength that is defined with a positive number. The strength is greater when the number of co-occurrences is higher. Full counting method takes into account the assigned number of occurrences of a keyword in documents. Then the software calculated the total strength of the cooccurrence links from one term to other terms through text-mining method and linguistic filtering. The result is a list with terms and assigned weights related to frequency of occurrences and total links strength. The final set with terms is used for creation of bibliometric network that presents the terms, links and distance among them. II Procedure: Creation of preparatory matrixes. The preparatory matrices for each year and for every extracted term with values of occurrences and total links strength are prepared for machine learning analysis in Octave software. Two types of data files are created. The first type of data files contains measurements of occurrences and total links strength. The y-values are occurrences of a keyword in documents and x-values are total links strength corresponding to the occurrences. The second type of data files points out the x-values which are the years and y-values show the corresponding occurrences. III Procedure: Building a predictive model forecasting the effect of changes in the terms regarding the values of occurrences and total link strengths and dependences between years and occurrences. Applying Supervised Learning. The sets with the preparatory matrices are used for input data to Octave software. The linear regression algorithm with gradient descent is applied to the training sets according to the equations [6]: y ¼ b0 þ b1 x þ
;

ð1Þ

where b0 is a coefficient that represent the intercept and b1 is the coefficient showing the slope,
is the error. The prediction of the future y value is based on x value: ^ þb ^ x; ^y ¼ b 0 1

ð2Þ

^ are estimated coefficients for the model. ^ and b where b 0 1 The residual sum of squares RSS is: RSS ¼ e21 þ e22 þ


þ e2n
2
2
2 ^ þb ^ x1 þ y2  b ^ þb ^ x2 þ


þ yn  b ^ þb ^ xn ; ¼ y1  b 0 1 0 1 0 1 where ei ¼ yi  ybi is the ith residual.

ð3Þ

130

M. Ivanova

^ ^ To minimize Pn the residual sum of squares RSS, the coefficients b0 and b1 are chosen P ðx xi Þðyi yi Þ i¼1 i ^ ¼ y  b ^ x, where x ¼ 1 n xi and y ¼ 1 Pn yi ^ ¼ P and b to be: b n 2 1 0 1 i¼1 i¼1 n n i¼1

ðxi xi Þ

are samples. IV Procedure: Construction of a fuzzy inference system for predicting the context of mobile learning usage. The FIS is created for predicting the context of mobile technology usage: for teaching, learning or for both. It is an important issue describing the most common exploitation of mobile technology – whether mobile technology mainly supports teachers or mainly facilitates learners or the applied scenarios are balanced, assisting at equal level the teachers and learners. The numerical data of the extracted terms during the whole examined period with VOSviewer software are used as input for FisPro software. The last one is utilized for FIS construction with three input values: mobile technology, teaching and learning and one output value: context. All values (input and output) are defined using standardized fuzzy partitioning approach that is described P through the following equation: lAij ð xÞ ¼ 1 for every x 2 Xi , where x is a point from j

a fuzzy set A with a membership degree 0  lA ð xÞ  1; Aj are fuzzy sets formed after partitioning, lA is the membership function [7]. The created fuzzy standardized partitions are characterized with linguistic variables that are chosen to be: very low, low, average, high and very high. The Mamdani conjunctive fuzzy rules are applied in the following form: IF x1 is Ai1 AND x2 is Ai2 AND. . .AND xn is Ain THEN y1 is Bi1 ;

ð4Þ

where Ai1 ; Ai2 ; . . .Ain and Bi1 are fuzzy sets that present the input and output space partitioning. The utilized rule aggregation concerns disjunction of defined conjunctive rules and it is described through the max operation: W j ¼ fmaxðwr ð xÞÞjCr ¼ jg

ð5Þ

for 8j ¼ 1; 2; . . .; m and where r is the number of rules, m is the number of labels of the partitioned space.

3 Creating a Predictive Model For identification of the key terms connected to the main explored term mLearning (mobile learning, m-learning) the software for scientific visualizations VOSviewer is exploited. The extracted terms with their corresponding values of occurrences and total link strengths are classified in tables for each year. Table 1 is just one example showing the selected terms with non-zero values of occurrences (O) and total link strengths

Predictive Modeling Concerning Mobile Learning Advance

131

(TLS) for 2018 year. The Table 2 includes the values of occurrences and total link strengths of the term mLearning during the explored years – from 2008 to 2018 year. Similar tables to Table 2 containing data for the extracted terms are used as data sources for performance of linear regression algorithm. The constructed bibliometric network for the term mLearning, pointing out its connection to the other terms and the strength of each connective link, is presented on Fig. 1. Table 1. Connected terms to the term mLearning in 2018 year mLearning in context mLearning

O TLS mLearning in context 506 1886 Collaborative/ Cooperative learning eLearning 252 1394 Experimental learning Engineering education 43 290 Adaptive learning Higher education 34 165 Online learning Learning through augmented reality 34 164 Ubiquitous learning Game-based learning 28 154 Interactive learning Personalized learning 7 39 Secondary schools Learning in virtual reality 18 112 Distance education Language learning 15 63 Informal learning Technology enhanced learning 14 84 Flipped classroom Problem-based learning 5 25 MOOC Blended learning 12 66

O TLS 12 49 5 7 7 21 7 5 5 9 5 13

34 33 35 92 43 32 23 38 12 77

Table 2. Occurences and total link strengths for the term mLearning during the examined period 2018 O TLS 506 1886 2014 442 1902 2010 463 2423

2017 O TLS 433 1985 2013 383 1580 2009 326 1779

2016 O TLS 452 2279 2012 543 2821 2008 253 728

2015 O TLS 441 2344 2011 419 1886

The results after applying the algorithm for linear regression for selected terms are summarized through graphics on Fig. 2. The created patterns with trained data express the usage of the terms and show the future trends. The constructed approximate lines are characterized with intersect and slope which parameters talk about the usage state of a given term at the examined period and about the frequency of the term usage during the years. For example, if a comparison related to the usage of mobile learning in context of learning through augmented reality and learning through games is

132

M. Ivanova

Fig. 1. Bibliometric network for the term mLearning for 2018 year

performed, then the analysis shows very close intersect and slope coefficients that reflect on the similar line steepness. It leads to the conclusion that the topics related to mobile learning through augmented reality and games take the similar attention of researchers, including these terms in their publications. Also, the lines steepness is positive that outlines an increasing tendency for utilization of these terms in the scientific production. The summarized results after applying linear regression method to the extracted terms indicate one positive tendency of the research topics concerning mobile learning. The detailed analysis of the gathered data for 2018 year outlines that the mobile learning is explored in different context and multiple learning scenarios. The highest interest addresses the connection between mLearning and eLearning. Then the explorations are focused on applications of mLearning in engineering education, Higher education, learning through augmented reality, game-based learning, ubiquitous learning, learning in virtual reality, language learning, technology enhanced learning, MOOC, blended learning, collaborative/cooperative learning, informal learning, adaptive learning, online learning, personalized learning, interactive learning. The small number of research papers connects mLearning to problem-based learning, flipped classroom, experimental learning, distance education and learning in secondary schools. With zero values of occurrences for 2018 year are the terms: elementary schools, learning through web 2.0 technologies, seamless learning, life-long learning, inquiry-based learning, situated learning, authentic learning, outdoor learning, workplace learning, self-regulated learning/self-directed, pervasive learning, intelligent tutoring, micro-learning.

Predictive Modeling Concerning Mobile Learning Advance

b) O/TLS for the term blended learning

c) O/TLS for the term learning with augmented reality

a) O/TLS mLearning

for

the

term

d) O/Year mlearning

for

the

term

e) O/Year for the term blended learning

f) O/Year for the term learning with augmented reality

g) O/TLS for the term gamebased learning

h) O/TLS for the term language learning

i) O/TLS for the term informal learning

j) O/Year for the term gamebased learning

k) O/Year for the term language learning

l) O/Year for the term informal learning

m) O/TLS for the term Higher education

n) O/TLS for the term engineering education

o) O/TLS for the term personalized learning

p) O/Year for the term Higher education

q) O/Year for the term engineering education

r) O/Year for the term personalized learning

Fig. 2. Results from linear regression

133

134

M. Ivanova

In order to understand the future utilization of mobile technology for teaching and learning a FIS is constructed with 125 rules and the part of the inference is shown on Fig. 3. The terms mobile technology, learning and teaching are examined for the period of eleven years – from 2008 to 2018 year. The final result shows different usage of the terms teaching and learning in the context of mobile learning during the examined years. The extracted tendency is that the term mobile technology will be closer to the term learning than to the term teaching. Also, according to the selected values of the terms teaching and learning in the constructed FIS could be found a solution that is closer to teaching or to learning as well as an approach for balanced utilization of mobile technology in teaching and learning. The surface view regarding the FIS response at two input variables mtechnology and learning is presented on Fig. 4. The red color indicates the minimum of the output value corresponding to the input values.

Fig. 3. The constructed fuzzy inference system

Fig. 4. The FIS response with surface view

4 Conclusion The current research uses extracted terms from Scopus bibliographic data to describe the current state regarding the connection between the term mobile learning and other examined terms as well as to find the tendency concerning the future of mobile learning

Predictive Modeling Concerning Mobile Learning Advance

135

implementation. For these purposes the machine learning algorithm of linear regression and fuzzy logic method are utilized. Also, a research methodology and a predictive model are developed. The finding point out that the term mobile learning is closer to the terms eLearning, engineering education, higher education, learning through augmented reality, game-based learning, ubiquitous learning and with big distance from the term mobile learning are the terms: cooperative learning, environmental education, elementary education, secondary education, experimental learning, location-based learning. The found tendency about the usage of the term mLearning in scientific publications is characterized with an increasing line. Also, the term mobile learning is better connected to the term learning that to the term teaching. The proposed methodology and created predictive model are useful for: (1) gathering results about challenging issues and its further understanding, (2) for hypothesis construction and its acceptance/rejection and (3) for decision making taking account the found tendency. In the context of this work, the reached findings outline the more explored topics by researchers and the tendency of their examination during the years. Such findings can give orientation to teachers and researchers about the current state and can indicate the future directions for research. Acknowledgements. The author would like to thank the Research and Development Sector at the Technical University of Sofia for the financial support.

References 1. Brooks, C., Thompson, C.: Predictive modelling in teaching and learning. In: Lang, C., Siemens, G., Wise, A., Gašević, D. (eds.) Handbook of Learning Analytics, pp. 61–68. Publisher Society for Learning Analytics Research (2017). https://doi.org/10.18608/hla17.005 2. Keegan, D.: The Future of Learning: From eLearning to mLearning (2002). ISSN-1435–9340, https://files.eric.ed.gov/fulltext/ED472435.pdf 3. Parsons, D.: Combining E-Learning and M-Learning: New Applications of Blended Educational Resources. Publisher IGI Global (2011). https://doi.org/10.4018/978-1-60960481-3 4. Kukulska-Hulme, A., Sharples, M., Milrad, M., Arnedillo-Sánchez, I., Vavoula, G.: The genesis and development of mobile learning in Europe. In: Parsons, D. (ed.) Combining ELearning and M-Learning: New Applications of Blended Educational Resources. Hershey, PA: Information Science Reference (an imprint of IGI Global), pp. 151–177 (2011) 5. Perianes-Rodriguez, A., Waltman, L., Van Eck, N.J.: Constructing bibliometric networks: a comparison between full and fractional counting. J. Informetrics 10(4), 1178–1195 (2016) 6. Linear regression, Statistical learning. https://lagunita.stanford.edu/c4x/HumanitiesScience/ StatLearning/asset/linear_regression.pdf 7. FisPro : An open source portable software for fuzzy inference systems (2013). https://pdfs. semanticscholar.org/7b90/9ffe94ed60af8a76a1b09d603ef007d0facb.pdf

Assessing Early Grade Mathematics Learner Outcomes Using m-Learning The Case of JumpStart South Africa Nicky Roberts(&) University of Johannesburg, Soweto, South Africa [email protected]

Abstract. JumpStart makes use of research-based structured learning material for mathematics together with in-class support by interns, teacher training, realtime formative and summative assessment, through m-learning. To assess its impact on learner outcomes, a cross sectional study design was adopted, whereby performance of a random selection for learners across Grades 1 to 3 in 18 schools was analysed. Grouped learner outcomes (at the level of a grade across groups of schools) prior to the intervention were compared to learner outcomes post the intervention and set alongside a description of the nature of the intervention. Improvements in learning outcomes, as measured by the Early Grade Mathematics Assessment (EGMA) are evident after one year of intervention, and appear to be strengthened after two years of intervention. Overall the JumpStart programme is having a significant impact on all the schools. The use of m-learning by interns for the efficient uptake and appropriate dosage of an early maths intervention; as well as the utility of a tablet interface for the efficient collection of early grade mathematics assessment data should be of interest to the m-learning and broader education community. Keywords: m-learning
South Africa
Mathematics
Early grades
Mobilebased assessment

1 Introduction The m-learning research agenda has grown and shifted away from Western Europe, which historically was characterized (albeit in a simplified way) by small scale interventions and those seeking to increase access to education in more remote areas [1]. Regional reviews of the mobile learning landscape have provided a clearer sense of the global inequity in distribution of mobile learning interventions, and highlight the diversity of use in different contexts. By way of example [2] identifies a ‘dearth of research, empirical evidence, and underlying theory development on mobile learning in the Africa and Middle East region’. There are very few mobile learning initiatives which focused on teacher development and Open and Distance Learning, and limited results on how these impacted on learning outcomes [3]. Reflecting on m-learning for international development, [1] argues that the ‘more diverse our global ecology of learning and its theories the richer the opportunities we offer to other cultures and © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 136–146, 2021. https://doi.org/10.1007/978-3-030-49932-7_14

Assessing Early Grade Mathematics Learner Outcomes Using m-Learning

137

communities and the more responsively we can engage’. This paper offers a case study which has been locally development and funded within South Africa in support of public primary schooling. Various studies have identified and lamented the poor assessment practices and knowledge of assessment literacy amongst South African Foundation Phase teachers [4–6]. There are large gaps in teacher knowledge of assessment practices in the early grades, and calls have been made for research on the reliability, validity and impact of assessment instruments on early learning [6]. South African Foundation Phase teachers seem to be unable to effectively use assessment to support their learners and address their learning needs, and seem unlikely to be able to use assessment information to improve their own teaching practices [3]. There is a potential for teachers to be supported in their assessment practices and assessment literacy by utilising m-learning. However a review by [7] found only 7% of studies on mobile devices in education examine their use for assessment. A meta analysis of ‘mobile-based assessment’ (MBA) found only 43 articles relating to assessment, the majority of which were drawn from 4 countries, and all of which (except 5% which did not disclose location) were from the Global North [8]. This paper contributes to teachers’ assessment in mobile learning as well as mobile learning models, theory and pedagogy in two ways. First, this paper focuses on ‘mobile-based assessment’ which is utilized in formal public primary schooling. The OECD definition of assessment as ‘the process of measuring and/or collecting and using evidence about the outcomes of students’ learning’ is followed [8]. There are gaps with regard to m-learning studies which focused on both formative and summative assessment using mobile devices [8]. The case study in this paper reflects on teachers and caring adults (teaching interns) using mobile based assessment for both formative and summative purposes. Second, this paper offers details with regard to assessing impact on early grade learning outcomes in the domain of mathematics. It does so by applying an analytical framework for describing m-learning interventions in the domain of mathematics [9] and setting this description alongside cross sectional data of learner attainment in a standardised Early Grade Mathematics Assessment (EGMA), as measured over 3 years.

2 Research Design To describe the JumpStart intervention, qualitative data about the JumpStart design and context was collected via structured interviews with the JumpStart team. To allow for comparability to other mathematics interventions, the interview instrument used for this was drawn from [11]. Available quantitative data (such as exists in the Education Management Information System (EMIS) for South African schools) was drawn upon. The adapted analytical framework for describing m-learning interventions [9] was used which considered various spectra: learning, kinetic, collaboration, device access, affordability and mathematics pedagogy. To analyze learner outcomes, a quazi-experimental, cross sectional study design was adopted, whereby performance of a random selection for learners across Grades 1

138

N. Roberts

to 3 in 18 schools was analysed. As shown in Fig. 1, there were three separate entry points for the JumpStart interventions with the most recent being in 2018.

Fig. 1. Simplified assessment and implementation timelines (by years from intervention start)

As such performance snapshots were taken for different populations at particular points in time: (1) for the control schools there was a snapshot at one point in time, (2) for the Phase 1 treatment group snapshots were available at 3 points in time (over a 2 year period), and (3) for the Phase 2 treatment group 2 snapshots were available (over a 1-year period). The learner outcome data was drawn from the Core Early Grade Mathematics Assessment (EGMA), which is an orally administered assessment of the core mathematical competencies taught in early primary grades. EGMA development was undertaken by RTI International and funded by the United States Agency for International Development (USAID). The Core EGMA offers an opportunity to determine whether children are developing the fundamental skills upon which other mathematical skills build, and, if not, where efforts might be best directed [12]. The Core EGMA is describe as “an assessment of early mathematics learning, with an emphasis on number and operations” [12]. It consists of six subtests (referred to as tasks in the instrument) that, taken together, can produce a snapshot of children’s knowledge of the competencies that are fundamental in early grade mathematics: number identification, reasoning about magnitude, recognition of number patterns, addition and subtraction (levels 1 and 2); and word problems [12]. The Core EGMA has been designed for both summative and formative assessments. It has been used to (1) determine how students in a country are performing overall compared to its stated curriculum and (2) examine the effectiveness of specific curricula, interventions, or teacher training programs. In this case, the EGMA was used for the latter. To the best of my knowledge, this is the only validated early grade mathematics instrument, which is freely available for use in developing country contexts. Its format is suitable for young children with the oral and individual administration mitigating

Assessing Early Grade Mathematics Learner Outcomes Using m-Learning

139

against the conflation of reading and writing ability with mathematics. That the assessment has been administered in the language of learning and teaching of the school, and administered by trained interns is a further strength of this assessment. The EGMA is administered orally in small groups by trained assessors. The test is short (30 min) which is appropriate for children in Foundation Phase. The collection of the EGMA assessments data, was done making use of a mobile application (JumpTrak digital platform from JumpCO Digital). The EGMA was administered annually to a random selection of about 10–15 learners per school per grade. The learners are grouped, into a classroom and each is given a tablet on which they complete the assessment. The tests are administered in the language of teaching and learning in foundation phase at the school (and so the languages for the tests vary). A detailed explanation of the core EGMA’s development, descriptions of its technical adequacy, as well as the details of its validity and reliability are offered by [12]. Processes for local adaptation and training of assessors are also prescribed. The core EGMA is meant to be locally adapted to fit the needs of the local context, particularly with regard to language where it is administered in the language of learning and teaching for mathematics at Foundation Phase in the school. The EGMA test was translated into some African languages (at least Afrikaans, isiXhosa). The translations pertain to the instruction prompts, word problems and the naming of numbers. The assessment is administered orally with an adult administrator reading out the instructions to a group of children. The children then make their choice of answer using a tablet. The administrator then moves on to the next question, reads this out and all children answer. This is the same way that Annual National Assessments were administered by teachers with their classes [13]. The script or guide was provided in English, and the adult administrator translated on the spot into the language of teaching and learning of the school. This is a design and data collection weakness as it means that the tests were not standardized and tightly scripted for each language. This places questions on the validity of data especially for the word problem questions. In order to minimize test familiarity over time, the test items were varied slightly for each year. These changes made were done in conjunction with Aarnout Brombacher (who has contributed to the RTI design team). Some concerns were evident about the quality of Grade 1 data, which may be reflective of the Grade 1s poor literacy and their lack of familiarity with the tablet.

3 Findings 3.1

Description of JumpStart as an m-Learning Case

Beginning in 2016, the JumpStart Numeracy Intervention targeted the geographical region of Ekurhuleni South, Gauteng Province, South Africa. By the end of 2018, it had reached 22 schools, with approximately 15 000 learners and 200 teachers. The Ekurhuleni South district is described by the Department of Basic Education as being “urban with some informal settlements” [3]. The predominant home languages are: English (30%), IsiZulu (26%), Sesotho (14%) and Afrikaans (14%) [14]. Relative to other districts in South Africa Ekurhuleni South is small in geographic area, caters

140

N. Roberts

for a large number of learners and has a very high population density. This is a relatively well performing district, within one of the top performing provinces in South Africa. Most of the schools in this district are classified as quintile 3 (no fee schools), with 52% of the school being no-fee schools [14]. JumpStart employs 56 full-time teaching assistants (or interns) to facilitate the one on one feedback with the leaner marking, which is captured using mobile devices. Table 1 offers synopsis of the staffing and project reach. Table 1. Synposis of JumpStart intervention Feature Project scale

Key team members

Duration of project

Intervention aims

Geographical focus Profile of schools targeted by intervention Targeted phase

Description 22 schools (of which 18 are considered for this evaluation) 15 000 learners 200 teachers 56 interns 4. Admin support 1.Project delivery 5. Robotics Admin manager support 2.Team Lead 3.Team Lead It is a rolling project. The current life cycle is that it has been running since 2016. The current strategy is planned for until 2020 To strengthen introductory mathematics skills at the foundation level so that learners have a much better chance of success in the next phase Ekurhuleni South Semi urban underperforming schools Foundation Phase

The schools selected to be part of the JumpStart intervention were semi-urban, underperforming schools. They were selected by the Gauteng Department of Education and its district office. The JumpStart intervention aimed to strengthen learners’ introductory mathematics skills at the Foundation Phase (Grades 1–3) so that learners had a better chance of success in the next phase. Given the Foundation Phase focus – where mathematics is taught in a wide range of languages - the Number Sense workbooks are available in the following languages: Afrikaans, English, Sepedi, Sesotho, Xitsonga, Setswana, isiXhosa and isiZulu. The primary inputs in this JumpStart intervention are the NumberSense workbooks, use of which is supported by school interns. JumpStart began using the paper-based workbooks in five Ekurhuleni South schools in 2016. To facilitate the smooth running of these classroom sessions, JumpStart’s project delivery manager and two team leaders engage in both formalised workshops and informal mentorship to support

Assessing Early Grade Mathematics Learner Outcomes Using m-Learning

141

teachers and interns in the classroom. The interns in the schools were further supported with school-based visits by a JumpStart project director: At an individual learner level, the Jumpstart lessons are held twice per week, with their normal class teacher, who is supported by an intern. The children each have a NumberSense workbook (graded to their level of development). The JumpStart intern moves around the class with a tablet and marks the learners work (using the Jump digital platform). The inclusion of interns in the programme has an important secondary benefit in the South African context of high youth unemployment. This is seen as the programme’s biggest community input. These interns were previously unemployed young South Africans who were employed for a year or two to perform mathematics tutoring in the classroom. Through the internship they gained valuable employment experience, developed a new skill set, and in some cases, found a path to higher education. Figure 2 locates the JumpStart intervention on the spectra put forward by [9] for an analytical framework to describe mobile learning (as applied to early grade mathematics [9] and for mobile pilot interventions in Africa [10]).

Fig. 2. JumpStart mapped to analytical framework for mobile learning

On the learning spectrum, JumpStart supports formal schooling with a focus on mathematics, and especially the formative and summative assessment of early grade

142

N. Roberts

mathematics using mobile devises. On the kinetic spectrum, the tablets are mobile. For formative assessment there is one tablet per teaching assistant who moves around the class of children, recording each of their progress in the workbook. The tablets are also moved from school to school, and class to class to assess learners for their summative assessment. In relation to the collaboration spectrum, the children work individually in their graded workbooks. However, depending on their progress recorded on the tablet, the children may be called into a small group to work collaboratively with the teaching intern, to address areas of observed difficulty. For device access, the mobile devises are provided by the school, and are not personally owned by the teacher, teacher assistant or learners. In terms of affordability the JumpStart model is cost effective: there are only 2 tablets per school, and about 9 Foundation Phase classes have the advantage of mobile-based assessment. The teacher assistants move from class to class with the devices, which is a key affordance of mobile devises. Data is stored on the tablet, and only uploaded to the database when the teacher assistants are in a free wifi-area. This model does however carry the cost of individual workbooks for each learner (and each learners uses 3 or 4 books per year). For the final spectrum – mathematical pedagogy – the JumpStart intervention is detailed and coherent as it builds on the research base and instructional design of the Number Sense Mathematics Programme. This has been created to support children’s development of a robust sense of number and a deep understanding of mathematics [15]. The materials consist of 12 work books (four for each grade level in Foundation Phase). Each workbook has 48 pages and focuses on the development of key mathematical concepts. The topics in the workbooks are curriculum relevant, however their sequence does not follow the suggested term-by-term learning programme. The workbooks are designed to complement the curriculum focusing on: working fluently and flexibly with numbers and number concepts; developing a rich understanding of the meaning of number; encouraging use of a wide range of effective strategies for solving a large variety of number problems. The development of these skills can be broken into three types of activities: counting, manipulating numbers, and solving problems. In the Grade 1 workbooks, counting has a majority focus, but by Grade 3, the focus has shifted predominantly to manipulating numbers and solving problems. In his interview, the JumpStart project manager articulated their explicit approach to mathematics as follows: ‘Mathematics is a problem-solving activity, and in the early grades, we want to teach learners to count with meaning not by rote. We want them to manipulate numbers, and we want them to start solving the problems.’ He described five pillars as being the foundation of their theory of change: (1) teachers, (2) books, (3) mentoring, (4) pedagogy and (5) feedback via monitoring and evaluation (and using a JumpTrak mobile application). In his interview the JumpStart project manager noted that the ‘real-time monitoring and evaluation’ facilitated via the mobile application was ‘a key component’. He emphases the importance of identifying areas of difficulty immediately through the formative assessment process. In relation to summative assessments, he commented: ‘We do not want to wait for the end of exams, we want to do it immediately. We want to gather [feedback] this week so we can start making an impact the next week.’

Assessing Early Grade Mathematics Learner Outcomes Using m-Learning

3.2

143

The Cross-Sectional Study on Learner Outcomes

The ultimate finding of this paper is that the JumpStart programme is having a statistically significant impact on the learners in Grades 1 to Grade 3 in terms of their Early Grade Mathematics Assessment (EGMA) performance. There are three groups of schools which are relevant to this analysis: Phase 1 schools: In 2016, the JumpStart intervention commenced in 2016 with schools 1–5 in Grades 1 to Grade 3. EGMA assessments were undertaken annually. Engagement with these phase 1 schools continued to 2018 when it was extended to Grade 4 (and where EGMA assessments were then included). Phase 2 schools: In 2017, the JumpStart intervention added schools 6-10 with attention to Grades 1 to Grade 3. EGMA assessments were undertaken annually. Control schools: In 2018, schools 11–18 were included as control schools, and the EGMA assessment were administered as for the Phase 1 and Phase 2 treatment schools. Attainment in an EGMA assessment is reported as a level (from level 1 to level 4) per individual child. In reporting to a school, the proportion of children at each level in a particular grade is provided. The mean EGMA levels for groups of schools were adjusted by making use of the control school data to calculate a monthly expected shift in attainment (Table 2). Table 2. EGMA mean results per Grade in control schools Grade Grade 1 mean Grade 2 mean Grade 3 mean Shift from Grade 1 to Grade 2 Shift from Grade 2 to Grade 3

Control schools 1.5 1.9 2.4 0.4 0.5

That the treatment schools had a lower attainment in the EGMA assessments is evident from the higher proportions of learners at levels 1 and 2, compared to the control schools. However after 1 year of the JumpStart intervention, the treatment schools have far more learners (about a quarter) performing at level 4 (Fig. 3).

Fig. 3. Comparison of treatment to control schools (EGMA results)

144

N. Roberts

These shifts are evident in summary form when considering the mean EGMA level across the individual learners in each school group. At baseline (T01) the treatment group (mean = 1.7 EGMA levels) as a whole performed lower than the control group (mean = 1.9 EGMA levels). However after 1 year of intervention (T13), the treatment schools (mean = 2.2 EGMA levels) were performing better than the control schools. Table 3 offers a tabulation of the overall results for the two treatment groups of schools (Phase 1 treatment and Phase 2 treatment where T1 data is used as a baseline): Table 3. Summary of significance and effect sizes (EGMA) Phase 1 Mean at Mean after 1 (Schools 1–5) baseline (T1) year (T13)

Mean after 2 years (T25)

Difference* Significant?

Effect size

Grade 1 Grade 2

1.32 1.75

n/a 1.95

n/a n/a

n/a +0.63a

n/a Yes

Grade 3

2.22

2.61

3.20

+0.86

Yes

Phase 2 (Schools 6 - 10) Grade 1 Grade 2

Mean at baseline

Mean after 1 year

Mean after 2 years

Difference* Significant difference?

n/a 0.33 (small) 1.14 (large) Effect size

1.38 1.72

n/a 2.67

n/a n/a

n/a 1.29b

n/a Yes

Grade 3

2.11

3.41

n/a

1.69

Yes

n/a 1.57 (very large) 1.71 (very large)

*

EGMA levels from one grade to the next Grade 2 after 1 year subtract Grade 1 at baseline: 1.95 – 1.32 = + 0.63 levels over one academic year for this cohort. b ‘Grade 2 after 1 year’ subtract ‘Grade 1 at baseline’: 2.67 – 1.38 = +1.29 levels over one academic year for this cohort. a

Based upon the above analysis it is clear that the JumpStart programme is having a significant impact on the 10 schools on which this evaluation focused. Every school has a statistically significant increase in their respective EGMA means. Overall the JumpStart programme is having a significant impact on all the schools. This study will be more robust once there is more data – to reflect on the effect of having been in the programme for two years; and having a second data point for the control schools. JumpStart makes use of research-based structured learning material for mathematics NumberSense workbooks [15], together with in-class support by interns, teacher training, real-time monitoring and evaluation (of uptake, use and learner attainment), through m-learning. The assessment and tracking of learner progress in the paper-based workbooks is collected by interns on tablets during the twice or thrice weekly 40-min NumberSense classroom sessions. For each grade in a school an intern with a tablet is provided to support the JumpStart implementation. To assess the effectiveness of the intervention, JumpStart implements progress assessments annually (making use of the Early Grade Maths Assessment administered on a tablet).

Assessing Early Grade Mathematics Learner Outcomes Using m-Learning

145

4 Conclusion This paper contributes to research on teachers’ assessment by focusing on ‘mobilebased assessment’ (MBA) which is utilized in a formal public primary schooling in South Africa. Its findings suggest ways to overcome some of the problem of teachers not being equipped to make use of digital learning (despite 80% of South African schools having access to the appropriate infrastructure) [2], and to improve teachers’ assessment literacy, a problem identified by [6]. The cross-sectional study of learning outcomes from the Early Grade Mathematics Assessment (EGMA) clear that the JumpStart programme is having a significant impact on the 10 schools on which this paper was focused. Every school has a statistically significant increase in their respective EGMA means (compared to their own baseline, and to the comparison schools). Overall the JumpStart programme is having a significant impact on all the schools. Improvements in learning outcomes, as measured by the EGMA are evident after one year of intervention, and appear to be strengthened after two years of intervention. This paper also contributes to research on mobile learning models, theory and pedagogy. The qualitative data is organized into an analytical framework for describing m-learning interventions (with a specific focus on mathematics). A cost effective model of integrating technology, using just two teaching assistants each with a tablet, a locally developed JumpStart application and access to data, to support both formative and summative assessment across Foundation Phase mathematics class in underperforming schools, is described. The use of m-learning by interns for the efficient uptake and appropriate dosage of an early mathematics intervention; as well as the utility of a tablet interface for the efficient collection of early grade mathematics assessment data should be of interest to the m-learning and broader education community. Acknowledgments. The JumpStart intervention was funded by the Michael & Susan Dell Foundation, and undertaken by JumpStart Foundation. It makes use of the NumberSense mathematics learning programme developed by Brombacher and Associates, together with the JumpTrak digital platform developed by JumpCO. The evaluation research on which this paper draws, was funded by Zenex Foundation, and undertaken by Kelello in collaboration with University of Johannesburg’s CEPR. I thank Dr Stacey Sommerdyk, Scott Hunt and Francesca Matthys for their contributions to the evaluation. Professor Elizabeth Henning (South African Research Chair on the ‘Integration of Mathematics, Science and Languages in the Primary School’ and director for CEPR), provided the support and enabling environment for this research.

References 1. Traxler, J.: m-Learning solutions for international development - rethinking the thinking. Digital Culture Educ. 5(2), 74–85 (2013). http://www.digitalcultureandeducation.com/cms/ wp-content/uploads/2013/12/traxler.pdf

146

N. Roberts

2. Isaacs, S.: A baseline study on technology-enabled learning in the African and Mediterranean countries of the Commonwealth (Report). Burnaby, Canada: Commonwealth of Learning (2015). http://oasis.col.org/bitstream/handle/11599/1674/2015_Isaacs_BaselineAfrica.pdf?sequence=1. Accessed 19 Aug 2019 3. Isaacs, S.: Mobile learning for teachers in Africa and the Middle East: Exploring the potential of mobile technologies to support teachers and improve practice. Paris, France: United Nations Educational, Scientific and Cultural Organization (UNESCO) (2012b). http://www.schoolnet.org.za/sharing/mobile_learning_AME.pdf. Accessed 19 August 2019. 08.24 4. Kanjee, A., Croft, C.: Enhancing the use of assessment for learning: addressing challenges facing South African teachers. Paper presented at the annual American Educational Research Conference, Vancouver, Canada, 13–17 April 2012 (2012) 5. Kanjee, A., Mthembu, J.: Assessment literacy of foundation phase teachers: an exploratory study. South Afr. J. Child. Educ. 5(1), 142–168 (2015). ISSN: 2223-7674 6. Sayed, Y., Kanjee, A., Rao, N.: Assessment and learning: problems and prospects. In: Wagner, D. (ed.) Learning and Education in Developing Countries: Research and Policy for the Post-2015 UN Development Goals, pp. 91–109. Palgrave Macmillan, New York (2014) 7. Cheung, W.S., Hew, K.F.: A review of research methodologies used in studies on mobile handheld devices in K-12 and higher education settings. Australas. J. Educ. Technol. 25(2), 153–183 (2009) 8. Nikou, S., Economides, A.: Mobile-based assessment: a literature review of publications in major referred journals from 2009 to 2018. Comput. Educ. 125(2018), 101–119 (2018). https://doi.org/10.1016/j.compedu.2018.06.006 9. Roberts, N., Spencer-Smith, G.: A modified analytical framework for describing m-learning (as applied to early grade Mathematics). South Afr. J. Child. Educ. 9(1), a523 (2019). https://sajce.co.za/index.php/sajce/article/view/532/1038 10. Isaacs, S., Roberts, N., Spencer-Smith, G.: Learning with mobile devices: a comparison of four mobile learning pilots in Africa. South Afr. J. Educ. (2019). https://sajce.co.za/index. php/sajce/article/view/532/1038 11. Roberts, N., Mostert, I., Takane, T.: Zenex Foundation Landscape Review of Mathematics Interventions in South African Schools, undertaken for Zenex Foundation by Kelello Consulting: Johannesburg, pp. 1–305 (2016) 12. Platas, L.M., Ketterlin-Gellar, L., Brombacher, A., Sitabkhan, Y.: Early Grade Mathematics Assessment (EGMA) Toolkit, pp. 1–49. Research Triangle Pa,rk, North Carolina, RTI (2014) 13. Department of Basic Education: Annual National Assessments 2011 Report. Department of Education, Pretoria (2012) 14. Department of Basic Education: Education Districts: Education district profiles 2015 (2015). https://www.education.gov.za/Informationfor/EducationDistricts.aspx. Accessed 4 Apr 2019, Government of South Africa: Pretoria 15. Brombacher and Associates: The NumberSense Mathematics Programme (2019). https:// www.numbersense.co.za/about/about-the-workbooks. Accessed 9 March 2019

Users’ and Experts’ Evaluation of TARGET: A Serious Game for Mitigating Performance Enhancement Culture in Youth Panagiotis Stylianidis, Agisilaos Chaldogeridis, Nikolaos Politopoulos, Vassilis Barkoukis, and Thrasyvoulos Tsiatsos(&) Aristotle University of Thessaloniki, Thessaloniki, Greece {pastylia,achaldog,npolitop,tsiatsos}@csd.auth.gr, [email protected]

Abstract. This paper presents both the experts and users evaluation results of the alpha version of TARGET (anTi-doping AwaReness Game for hEalthy lifestyle), a Serious Game for supporting the Anti-Doping Awareness for a Healthy Lifestyle. The game has been designed and implemented in the context of GAME (A serious Game Approach in Mitigating performance Enhancement culture in youth) which is an Erasmus + project. Particularly, the paper briefly presents the design and content of the serious game, along with the technological solution, which is based on the well-known Unity 3D platform in combination with Fungus plugin, and then presents the findings of the Alpha Version Users’ and Experts’ Evaluation of the game. According to the evaluation results, Lund’s results are very satisfying indicating a simple and easy to use and learn user interface and users believe that the game is useful for them, and on the other hand, EGameFlow results are also satisfying, despite the fact that Social Interaction is below average but this kind of games do not include interaction between users, although Immersion is low. Combining this result with the open-ended questions about the negative aspects the result might be the long dialogues that the user meets during the gaming experience. Keywords: Serious games


Anti-doping
Sports
Unity 3D

1 Introduction GAME project is motivated by the need to advance anti-doping education intervention targeting competitive and recreational athletes [1, 4–7, 9, 10]. Towards this goal, our team recognized the need to move forwards and transform the way anti-doping education is designed, delivered and evaluated and aimed to utilize: a) updated research from the social and behavioural sciences on doping use; b) state-of-art learning pedagogies; and c) cutting-edge serious gaming design and technology in order to deliver an innovative and impactful anti-doping educational intervention. The result was the development of a serious game consisting of problem-based learning scenarios that can realistically depict risk-conducive situations and contingencies in a serious game © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 147–157, 2021. https://doi.org/10.1007/978-3-030-49932-7_15

148

P. Stylianidis et al.

environment, applying behaviour change indicators (e.g., changes in beliefs, intentions, and actual behaviour) [3, 11] that allowed us to validate the effectiveness and impact of the envisaged anti-doping serious game, and also to recommend specific policy actions that are needed to optimize the effectiveness of anti-doping education.

2 Game Description TARGET is mainly a telltale serious game using realistic scenarios in order to engage users in critical situations [2, 12] which will help them to change behaviors and perspective towards anti-doping awareness. To this direction, the game classifies users into amateur and elite athletes, offering four different scenarios in each level, corresponding to four different learning indicators (knowledge, self-efficacy, moral displacement and attitudes). Each scenario describes a unique story where the user has to take critical decisions and make the appropriate choices, in order to succeed and score as many points as possible. The game consists of the login page, where each user must fill her/his personal credentials to login to the platform, the level selection screen (Fig. 1), where she/he can choose from different scenarios, the actual stories where the main games-scenarios take place (Fig. 2) and the score screen with user’s scores and users leaderboard (Fig. 3).

Fig. 1. Level selection screen

Users’ and Experts’ Evaluation of TARGET

149

Fig. 2. Sample scenario screen

Fig. 3. Scores - leaderboard screen

3 Evaluation TARGET’s evaluation consisted of the Heuristic Expert Evaluation (Technical Validation & Expert Evaluation) and also Preliminary User Evaluation (End Users’ Evaluation Report and Results). Each methodology is presented separately below.

150

P. Stylianidis et al.

3.1

Expert Evaluation

In this part a review of the evaluation methodology is presented as well as the evaluators’ profiles, the heuristic rules used and the evaluation process. After that for every group of interactions that a user can engage in: • The results of the heuristic evaluation are presented as well as the relevant comments from the evaluators • The positive points regarding system’s usability are mentioned • A brief description of the problems and suggested solutions is described 3.1.1 Methodology In order to evaluate applications and systems where non-novice users’ interactions cannot be accurately predicted, the use of general practices for the measurement of usability and design control is required. This requirement can be satisfied by the heuristic evaluation method (HE) [10]. Although this particular method is not analytical, it has a subjective character and is based on empirical rules and findings that are well known and are related to good interface design. An alternative term that is used to describe the heuristic evaluation is Usability Inspection (UI), since essentially is an inspection process conducted by experts on interface characteristics based on heuristic rules. The heuristic evaluation method focuses on two key points: • The general design of system screens. • The flow of dialogues, messages and actions required by the user to perform a specific task. The rules, that can be used for a heuristic evaluation, are not strictly specified, different experts might consider some rules more important than others. Furthermore, some problems require that greater emphasis is attributed to some of the rules. A widespread set of heuristic evaluation rules contain the following: a) Aesthetic and minimalist design b) Match between system and the real world c) Recognition rather than recall d) Consistency and standards e) Visibility of system status f) User control and freedom g) Flexibility and efficiency of use h) Help users recognize, diagnose, and recover from errors i) Error prevention j) Help and documentation. 3.1.2 Evaluators According to literature, the heuristic evaluation, conducted by external evaluators, excluding the system designers in order to ensure impartial judgement and different views on the design. Furthermore, an optimal number of evaluators has been adopted (5–6), whose opinions accumulate cumulatively. In Table 1, the evaluators’ demographics (sex, age and educational background) are being presented.

Users’ and Experts’ Evaluation of TARGET

151

Table 1. Evaluators’ Demographics Gender

Male Female Age 25–34 35–44 >45 Educational Level MSc Graduate PhD Graduate

Size 4 1 1 2 2 4 1

Percentage (%) 80 20 20 40 40 80 20

3.1.3 Heuristic Evaluation Results Table 2 presents the heuristic evaluation results and Table 3 the evaluator comments for each rule. Table 2. Heuristic evaluation results Evaluator 1 Sev. Fq. A/I Heuristic rule 1 Aesthetic and minimalist design 2 Match between system and the real world 3 Recognition rather than recall 4 Consistency and standards 5 Visibility of system status 6 User control and freedom 7 Flexibility and efficiency of use 8 Help users recognize, diagnose, and recover from errors 9 Error prevention 10 Help and documentation

3.2

Evaluator 2 Sev. Fq.

Evaluator 3 Sev. Fq.

Evaluator 4 Sev. Fq.

Evaluator 5 Sev. Fq.

(0–4) (0–4) (0–4) (0–4) (0–4) (0–4) (0–4) (0–4) (0–4) (0–4) 3 1 2 1 0 0 1 2 4 4 3

3

4

3

4

3

3

4

4

3

0

0

0

1

0

0

2

1

4

0

3 0 2 0

1 0 1 2

0 2 0 1

0 1 0 1

0 0 0 0

0 0 0 0

0 0 2 0

0 4 1 0

4 3 4 4

0 1 0 0

0

2

1

1

0

0

0

0

4

0

0 0

0 0

2 1

1 1

0 0

0 0

0 4

0 3

4 3

1 0

Preliminary User Evaluation

This section describes the Preliminary User Evaluation process and its findings.

152

P. Stylianidis et al.

3.2.1 Methodology For the evaluation of the game enjoyment and game usability a Questionnaire consisting of 93 questions was used, combined with 2 questions for personal information and 3 questions regarding gaming experience: In order to evaluate game’s usability, players were asked to complete an online questionnaire after the activity was concluded. The particular questionnaire was based on Lund’s “USE” questionnaire (Lund, 2001), which is a short questionnaire designed to measure effectively the most important aspects of a product’s usability. It consists of 32 questions grouped in four dimensions: (1) Usefulness, (2) Ease of use, (3) Ease of Learning, and (4) Satisfaction. The questions’ type is a 7-point Likert rating scale with the following anchors: 1 strongly disagree, 2 disagree, 3 slightly disagree, 4 neutral, 5 slightly agree, 6 agree and 7 strongly agree. In order to explore whether the game offers enjoyment to the players, the participants were asked to complete a second online questionnaire, the EGameFlow questionnaire (Fu et al., 2009). It consists of 56 items grouped in eight dimensions (1) Concentration, (2) Clear Goal, (3) Feedback, (4) Challenge, (5) Autonomy, (6) Immersion, (7) Social Interaction, (8) Knowledge Improvement. The questions’ type is a 7-point Likert rating scale respectively representing the lowest and the highest degree to which respondents agree with the items. 3.2.2 Participants For the user evaluation process 85 students selected from a Computer Science Department of a Greek University. There were 55 males and 30 females, having mean age 20.5 years. 44 of them consider themselves “casual gamers”, 32 are considered “core/mid core gamers” and the remaining 9 are “hardcore gamers”. Moreover, 11 of them prefer to play games on consoles (XBox, Playstation, Wii), 52 prefer PCs and 21 on smartphones/tablets and only one handheld console (PSP, PlayStation Vita, Nintendo 3DS etc.). 3.2.3 Descriptive Analysis of Responses to Use Questionnaire The statistical analysis was performed using Microsoft’s Excel. For each one of the four dimensions (usefulness, ease of use, ease of learning and satisfaction) descriptive measures of central tendency, such as mean, were estimated for each Likert-type item from one to seven [9]. The mean values of the four dimensions are being presented in Table 4. As indicated by the results of the analysis, the players’ opinion was positive, since the mean value (M) for the “Usefulness” dimension is 4.29, whereas M is 4.01, for the “Satisfaction” dimension. Moreover, it could be deduced that the players did not encounter any major difficulties in comprehending and using the tool, since the mean value for the “Ease of Use” dimension is equal to 5.43, while M = 6.12, for the dimension “Ease of Learning”. The frequency diagram presented the user interface satisfaction is being presented in Fig. 1 (Fig. 4).

Users’ and Experts’ Evaluation of TARGET

153

Table 3. Evaluators’ comments A/I 1

Heuristic rule Aesthetic and minimalist design

2

Match between system and the real world

3

Recognition rather than recall

4

Consistency and standards

5

Visibility of system status

6 7

User control and freedom Flexibility and efficiency of use Help users recognize, diagnose, and recover from errors Error prevention Help and documentation

8

9 10

Comments Evaluator 1: Leaderboard option doesn’t fit to the environment and no “return to previous” button Lack of “Return to Home” button after clicking on “Play” Evaluator 4: Lengthy text in player’s buttons Evaluator 5: Repetition of information Evaluator 4: Too much terminology Evaluator 5: Very informal language in some cases (e.g. dude, mate, yeah man, get a good pump etc.) – Used language more close to a male player e.g. “your testicles” and “also for women…” Evaluator 4: Too much text Evaluator 5: Very few items on screen. Lack of guidance/directions/controls during the game (who talks each moment, timer) Evaluator 1: Lack of “Home” button in every screen. Evaluator 5: Leaderboard screen has no “return to previous” button. Evaluator 5: No “Pause game” option – Evaluator 4: Less text in user’s answers Evaluator 5: Answers should be adopted to user’s sex – –

Table 4. Mean values of the four dimensions Usefulness Ease of Use Ease of Learning Satisfaction Mean 4.29 5.43 6.12 4.01

154

P. Stylianidis et al.

Fig. 4. Frequency diagram of User Interface Satisfaction

The users’ answers to the open-ended questions revealed positive and negative issues:’ • Positive – Easy to use – Educational – Simple – Nice U/I • Negative – Wall of text in the game dialogs – Too much terminology in the game dialogs – Unrealistic conversations Lund’s results are very satisfying since they indicate a U/I which is simple and easy to use and learn, while users believe that the game is useful for them. 3.2.4 Descriptive Analysis of Responses to EGameFlow Questionnaire For each one of the eight dimensions (concentration, clear goal, feedback, challenge, autonomy, immersion, social interaction and knowledge improvement) descriptive measures of central tendency, such as mean, were estimated for each Likert-type item from one to seven [8]. The mean values of the eight dimensions are being presented in Table 5. Table 5. Mean values of the eight dimensions Concentration Clear goal Mean 4.67

4.84

Feedback Challenge Autonomy Immersion Social Knowledge interaction improvement 4.89

4.02

4.54

3.2

3.29

5.36

Users’ and Experts’ Evaluation of TARGET

155

Fig. 5. Frequency diagram of gamer enjoyment

As indicated by the results of the analysis, the players’ opinion was positive, since the mean value (M) for the “Concentration” dimension is 4.67, whereas M is 4.84, for the “Clear Goal” dimension and 4.89 for the “Feedback” dimension. Moreover, the M for the “Challenge” dimension is 4.02 while the M for the “Autonomy” dimension is 4.54. It could be deduced that the game offers social interaction and immersion since the means values are 3.29 and 3.2 respectively. Furthermore the players believe that the game improves their knowledge since the M is 5.36. The frequency diagram presented the gamer enjoyment is being presented in Fig. 5. EGameFlow results are also satisfying since Social Interaction is below average (3.50) but this kind of games do not include interaction between users’ and also, Immersion is low. If we combine this result with the open-ended questions about the negative aspects the result might be the long dialogues that the user meets during the gaming experience.

4 Conclusions – Future Work This paper presents the Alpha Version Users and Experts Evaluation of serious game platform for anti-doping education. This platform has been deployed in the context of GAME Erasmus + Sport project. The development of the alpha version of the platform has taken into account the needs analysis and the design conducted in previous steps of the GAME project. The heuristic evaluation results show that TARGET game is easy to use, consistent, with a simple user interface and the overall attitude of the evaluators is positive.

156

P. Stylianidis et al.

It is worth mentioning that in the expert evaluation most of the rules had few problems. Most of the comments were about the lack of specific controls and buttons, like “Back” and “Return to home/previous” in the majority of the game’s screens. Also, the reviewers agreed on the size of the text that’s being used, since all of them found it too lengthy to read and thus tracking the dialogues was difficult. Finally, the language being used was found to be informal in some cases and needs to be revisited. These issues can be addressed by adding appropriate controls and buttons that will help the user to have more options and guidance/instructions, decreasing the text amount in screens where there is a lot of it, and correcting the language where is considered too informal. On the other hand, the evaluation results of the users’ evaluation are encouraging since the student’s opinion was positive. They found the game easy to use, easy to learn and satisfactory. Moreover, the results indicate that the game offers enjoyment to the players. The next step is to resolve any issues that rose from the heuristic evaluation as well as the users’ evaluation and conduct a technical evaluation of the platform with a larger user base. Acknowledgment. This project has been funded with support from the European Commission. This publication reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein. The authors of this research would like to thank GAME team who generously shared their time, experience, and materials for the purposes of this project.

References 1. Baron, D.A., Martin, D.M., Abol Magd, S.: Doping in sports and its spread to at-risk populations: an international review. World Psychiatry Off. J. World Psychiatr. Assoc. WPA 6, 118–123 (2007) 2. Caperton, I.: Video Games and Education. OECD Background Paper for OECD-ENLACES Expert Meeting (2007) 3. Chan, D.KC., Lentillon-Kaestner, V., Dimmock, J., Donovan, R., Keatley, D. Hardcastle, S., Hagger, M.: Self-Control, self-regulation, and doping in sport: a test of the strength-energy model. J. Sport Exerc. Psychol. 37, 199–206 (2015). https://doi.org/10.1123/jsep.2014-0250 4. Christou, M., A. Christou, P., Markozannes, G., Tsatsoulis, A., Mastorakos, G., Tigas, S.: Effects of anabolic androgenic steroids on the reproductive system of athletes and recreational users: a systematic review and meta-analysis. Sports Med. 47 (2017). https://doi. org/10.1007/s40279-017-0709-z 5. De Hon, O., Kuipers, H., van Bottenburg, M.: Prevalence of doping use in elite sports: a review of numbers and methods. Sports Med 45(1), 57–69 (2015) 6. Laure, P.: Epidemiologic approach of doping in sport: a review. J. Sports Med. Phys. Fitness 37(3): 218–224 (1997) 7. Frati, P., Busardò, F.P., Cipolloni, L., Dominicis, E.D., Fineschi, V.: Anabolic androgenic steroid (AAS) related deaths: autoptic, histopathological and toxicological findings. Curr. Neuropharmacol. 2015(13), 146–159 (2015) 8. Nielsen, J., Molich, R.: Heuristic evaluation of user interfaces. In: Proceedings of the ACM CHI 1990 Conference, Seattle, WA, 1–5 April, pp. 249–256 (1990)

Users’ and Experts’ Evaluation of TARGET

157

9. Ntoumanis, N., Ng, J.Y.Y., Barkoukis, V., Backhouse, S.: Personal and Psychosocial Predictors of Doping Use in Physical Activity Settings: A Meta-Analysis. Sports medicine (Auckland, N.Z.). 44 (2014). https://doi.org/10.1007/s40279-014-0240-4 10. Erickson, K., McKenna, J., Backhouse, S.H.: A qualitative analysis of the factors that protect athletes against doping in sport. Psychol. Sport Exerc. 16, 149–155 (2015) 11. Michie, S., West, R., Campbell, R., Brown, J., Gainforth, H.L.: ABC of Behaviour Change Theories. Silverback Publishing, London (2014) 12. Treviño-Guzmán, Pomales-García: How can a serious game impact student motivation and learning? In: Industrial and Systems Engineering Research Conference, 2014, Montreal, IIE, Norcross (2014)

Poster: Exploring the Educational Affordances of an Academic ePortfolio for Engineer Students Through a Self-regulated Learning Framework Foteini Paraskeva(&), Eleni Neofotistou, Angeliki Alafouzou, and Aikaterini Alexiou Department of Digital Systems, University of Piraeus, Piraeus, Greece [email protected], [email protected], [email protected], [email protected]

Abstract. This paper outlines the process of integrating an ePortfolio as a tool of academic development for students in tertiary education. The construction of the ePortfolio was based on the Self-Regulated Learning strategies which lead students’ performance towards constructing the artifacts for their ePortfolios in order to better present reflect and improve one of their academic ongoing projects by reflecting on their prior knowledge. The evaluation of the research is based on qualitative and quantitative analysis. The results prove the correlation between the enhancement of students’ self-regulation and the construction of an academic ePortfolio for the promotion of their academic development. Keywords: ePortfolio development


Self-regulated learning
Web 2.0
Academic

1 Introduction Digital technologies as approaches that enable educational innovations offer many opportunities of interaction, collaboration and sharing in order students to meet the needs of today’s globalized and technology-oriented world. Such an example is ePortfolios which refer to online environments that are built with the use of Web 2.0 technology in order to support the creation and the sharing of self-authored contents and interactions among the participants (Clark and Eynon 2009). However, technology cannot offer pedagogical change without the integration of a pedagogical and studentoriented framework (Batson and Chen 2008). Self-Regulated Learning theory (SRL) can be suggested as a learning process of setting goals, monitoring towards them and applying appropriate study strategies in order to lead students to become autonomous and effective learners so for their academic as for their professional performance. This paper presents the design of the “apT2ePortfolio” e-lab course, as an ePSRL system-oriented e-learning framework for academic development and analyses the results of its implementation throughout an IT related academic course. The goal of this research is to show the affordances of an academic ePortfolio constructed by the pillars of SRL in the enhancement of learners’ self-regulation and academic skills. © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 158–166, 2021. https://doi.org/10.1007/978-3-030-49932-7_16

Poster: Exploring the Educational Affordances of an Academic ePortfolio

159

The mostly used SRL strategies for the construction of the ePortfolio are going to be analyzed. Moreover, the structure of the system and how much does it affect the improvement of SRL efficacy of learners will be outlined. Finally, throughout this paper it will be explored how the SRL strategies are related to the tasks and the affordances of the ePortfolio content.

2 Theoretical Background Definition of ePortfolios: EPortfolio is defined as a collection of “work that a learner has collected, reflected, selected and presented to show his evolution over time” (Barrett 2010). According to Stefani et al. (2007) the best way to develop an ePortfolio is by integrating it into a course because as an institutional device it can be used to demonstrate learners’ progress and also to assess their learning outcomes. Portfolios contain an organized collection of content, such as text, files, photos, videos, and more, which are generically referred to as artifacts, providing evidence of what have been learnt. The use of multimedia as artifacts in the online interface of e- portfolio can be a way of flexible and customizable representation of students’ learning outcomes and experiences (Dunbar-Hall et al. 2015). Artifacts could also be created via online collaboration and this interaction would force learners to self-explain, communicate, elaborate and integrate their ideas to a group (Klenowski et al. 2006). The affordances of ePortfolios: EPortfolios can promote the understanding of “Where am I”, “Where I want to be”, “What can I be” for both students and instructors. Consequently, ePortfolios can be effectively used in teaching, learning and assessment as they are student-centered and allow students to organize, control and personalize the content. Moreover, ePortfolios’ construction leads learners to assess their learning and reflect on it in order to be improved. Consequently, learners’ independence via feedback and social support can lead students to become autonomous learners and reduce their dependence on teacher support (Sadler 2010). According to literature, an effective ePortfolio should be designed in a way that it will incorporate the four affordances of ePortfolio assessment (Barbera 2009): – Task authenticity: learners are able to use the new knowledge through authentic tasks and also reflect on their prior knowledge (Darling-Hammond and Snyder, 2000). The providence of explicit guidelines on the construction of artifacts that will represent the learning outcomes, is considered as a path to lead students to develop authentic tasks (Handley and Williams 2011). – Self-regulated and reflective learning: in order students to be goal-oriented, selfmanaged, self-evaluated and self-improved during the learning process, they should be provided with a facilitative learning environment which will enable their independence in learning (Boekaerts and Corno 2005; Zimmerman 2008). Moreover, during the construction of artifacts, learners should reflect on their learning experiences and evaluate their progress in regard to their goals (Fernsten and Fernsten 2005). While tasks are self-managing and self-reflective allow learners to selfreflect and take responsibility of their own learning.

160

F. Paraskeva et al.

– Constructive feedback: by providing feedback interactions, productive learning can be promoted via instructor’s guidance and students’ contribution which supports peer-learning that strengths learners’ cognitive process through the critical evaluation of peers’ ePortfolios against the assessment standards (Nicol et al. 2014). Moreover, Web 2.0 technologies that are used for the construction of an ePortfolio facilitate the provision of interactive feedback and also promote the collective knowledge building through peer-learning (Luchoomun et al. 2010). – Student autonomy: this affordance can be analyzed as the sense of independence during the construction of new knowledge (Barnett 2007). Learners’ autonomy is promoted in collaborative knowledge building (Klenowski et al. 2006). The Self-regulated Learning Theory: according to Zimmerman (1986), a learner is self-regulated when he is cognitively, motivationally and behaviorally active participant in the learning process. He developed a cyclical model of self-regulation from social-cognitive theory (Zimmerman 2000). Zimmerman’s (2000) cyclical model of self-regulation includes three phases: ‘Forethought phase’ (learning task, goal-setting, task analysis, motivation, reinforcements & benefits, self-efficacy). ‘Performance Control phase’ (project-planning, monitor control, help-seeking, organize information, elaborate on cognitive, meta-cognitive and management skills), ‘Self-Reflection phase’ (performance evaluation, strategies’ effectiveness assessment, performance improving, planning the next tasks). In each of the strategies mentioned above, cognitive, motivational and social aspects could be depicted.

3 The Methodology The Research Goal: the goal of the instructional design presented below is to show the affordances of an academic ePortfolio constructed by the pillars of Self-Regulated Learning theory in the enhancement of learners’ self-regulation and academic skills and delivered through a web2.0 e-learning system called “apT2ePortfolio”. This instructional design is used in an undergraduate computer science program in tertiary education, in a course titled ‘IT -centric Professional Development’ (ITcPD) as a framework to help students design and create their own ePortfolios. The research conducted was driven by the following leading questions: RQ1: Which SRL strategies are mostly used in the construction of the ePortfolio? RQ2: How do SRL strategies evident related to the task and affordances of the ePortfolio content? RQ3: How much does the system affect the improvement of SRL efficacy of learners? The Theoretical Approach: research has shown that via ePortfolios, students are engaged to a goal-oriented, self-managed, self-evaluated, and self-improving learning process, while they should be provided with a facilitative learning environment to

Poster: Exploring the Educational Affordances of an Academic ePortfolio

161

foster their independence in learning (Boekaerts and Corno 2005; Zimmerman 2008). Both the above statements make it clear that the SRL process should be embedded in ePortfolio tasks. As a result, the “apT2ePortfolio” site was designed by the pillars of SRL theory, enabling students to follow the theory’s components to construct their own ePortfolios. Participants follow the SRL phases in a row, performing a continuous circle as the procedure starts over when it comes to re-design and improvements after the reflection phase. The construction process was articulated by the SRL phases and guided by the ePortfolio key-elements that were introduced to students as characteristics that their ePortfolio should have in order to be successful (Liu and Burt 2015). These keyelements are the following: – A. Critical thinking through reflection – The ePortfolio should include organized artifacts accompanied by reflective journals. – B. Technical competency – The tools and multimedia features selected should be compatible to the platform and fully functional. – C. Visual literacy – Visual elements and the whole aesthetic should be carefully chosen in order to support ePortfolio’s objectives. – D. Effective communication – The ePortfolio illustrates the message that the creator needs to express and supports a specific identity, via the design and its original content. The above characteristics were included as Key Performance Indicators (KPIs) in each component of the SRL theory. More specifically, students had the opportunity via self-assessment multiple-choice forms to reflect on their progress and make the necessary changes and improve their work. Through this procedure they got more familiar to the reflection process as a whole, so they were prepared to reflect on their artifacts as well. The key-elements were operationally matched to the SRL phases and both were supported by the e-learning system that was designed in order to host and deliver the ecourse. The E-learning System: in terms of this research, a web-based lab course, named “apT2ePortfolio” was designed and hosted by Weebly web-site builder, as it was important to engage and familiarize students in a web-based environment, similar to the one they were asked to develop in order to create and host their ePortfolio. The flexibility in the design procedure that such tools offer was a valuable advantage when it comes to ePortfolio designing as each creator needs to adapt the tool’s affordances to his own wills. Furthermore, instructor-led paths that could exceed the options given by a standard Web 2.0 tool could better support the ePortfolio’s creation. Also, there is a wide variety of collaborative and scaffolding tools which could be embedded in the site in addition to many multi-media tools that are compatible to it. Finally, Weebly facilitates asynchronous communication via e-mail forms and blog feed. The e-learning system affordances that enabled its use in such content are analyzed in the Table 1 below:

162

F. Paraskeva et al.

Table 1. System affordances combined to the SRL phases and ePortfolio key-elements SRL phases Forethought

“apT2ePortfolio” web2.0 system affordances • Clear index and content • Ease of navigation

• • • • • • Self-reflection •

Performance

ePortfolio key-elements *Introductory tasks: ePortfolio planning Self-motivation Task analysis Critical thinking Visualize the strategies in use Wide variety of compatible tools Technical competency Familiarize with the multimedia tools embedded Visual literacy Consistency in units’ organizing Ease of navigation KPI self-assessment forms KPI self-assessment forms Coherent message

As it is proven from the table above, there are correlations among the system’s affordances, the SRL phases and the ePortfolio’s key-elements. Additionally, Weebly is a mobile- friendly Web 2.0 technology. This system’s affordance may lead to the construction of a mPortfolio which can be used anywhere and anytime.

4 Results At the end of the ePortfolio project, the participants (N = 59) answered three questionnaires (self-reflection phase) in order the research to be evaluated. Concerning the first research question (RQ1_ Which SRL strategies are mostly used in the construction of the ePortfolio?) data collection was via the use of a parametrized questionnaire that was derived by the research of Sebesta and Speth in their study ‘How should I study for the exams? Self-Regulated Learning strategies and achievements in introductory Biology’ (2017). This questionnaire contains 14 close-ended Likert-scale questions (Likert scale, from 1-never to 5-very often). The results that occurred are described in the Table 2 below.

Table 2. The evaluation of the SRL strategies’ usefulness by the students SRL strategies Self-evaluation Organize and transforming information Goal- setting and planning Seeking information Keeping records and monitoring Environmental structuring Self-consequating Seeking assistance from peers

Never – – – – 0,02 – 0,02 –

Rarely – 0,03 0,02 0,03 0,05 0,02 0,10 0,12

Sometimes 0,08 0,29 0,31 0,14 0,24 0,42 0,42 0,24

Often 0,73 0,44 0,42 0,51 0,46 0,36 0,31 0,49

Very often 0,19 0,24 0,25 0,32 0,24 0,20 0,15 0,15 (continued)

Poster: Exploring the Educational Affordances of an Academic ePortfolio

163

Table 2. (continued) SRL strategies Seeking instructor assistance Seeking assistance from other resources Reviewing notes Reviewing examples Reviewing textbooks and course material Reviewing graded work

Never 0,02 – – – – 0,02

Rarely 0,02 0,03 0,05 0,07 0,08 0,03

Sometimes 0,31 0,31 0,20 0,41 0,34 0,32

Often 0,54 0,54 0,56 0,34 0,49 0,56

Very often 0,12 0,12 0,19 0,19 0,08 0,07

As it is depicted in the table above, reviewing notes and reviewing graded work had the higher score, as almost half of the students claimed they often use them. On the other hand, a 2% of them mentioned that they never review their graded work. Students also claimed that they very often spend time on setting goals and editing timelines. Finally, students on the average agreed that the arrangement of their working environments could sometimes enhance effective learning. Additionally, concerning the second research question (RQ2_ How do SRL strategies evident related to the task and affordances of the ePortfolio content?), we constructed for the data collection, a questionnaire based on the ePortfolio’s affordances (Barbera 2009). The questionnaire included 16 close-ended Likert scale questions (Likert scale, from 1-strongly disagree to 5-strongly agree). The results of this research are described in the Table 3 below. Table 3. The ePortfolio affordances evaluated by the students during “apT2ePortfolio” e-course ePortfolio affordances Task authenticity Reflective and Selfregulated Learning Constructive feedback Student autonomy

Strongly disagree – –

Disagree

Agree

0,01 0,01

Neither agree nor disagree 0,10 0,14

0,62 0,61

Strongly agree 0,28 0,24

– –

– –

0,15 0,18

0,67 0,68

0,17 0,32

Students evaluated higher the advantages of the ePortfolio creation process, such as the opportunity to reflect on their progress and perform any kind of assessment or reflect on a course or module in general. On the other hand, they evaluated lower the technical advantages of the ePortfolios, such as the open-access and portability features, possibly because they were not yet quite familiar with the system, as they were still working on it. A great high percentage agreed that ePortfolio provides an opportunity for learning which can serve both a formative and/or summative assessment purpose. Moreover, learners evaluated in a positive way the SRL strategies under which the construction of an ePortfolio was orchestrated. For instance, the 60% of students agreed that ePortfolios help them plan their academic pathways by

164

F. Paraskeva et al.

comprehending what they know and what they still need to learn. Finally, a 65% of students claimed that ePortfolios helped them build their personal and academic identities as they complete multifaceted projects and reflect on their capabilities and development. Finally, regarding the third research question (RQ3_ How much does the system affect the improvement of SRL efficacy of learners?), data collection was based on the construction of a questionnaire that included 18 close-ended Likert scale questions (Likert scale, from 1-strongly disagree to 5-strongly agree). The e-learning system’s characteristics that were evaluated, were categorized into four types: interface and design quality, e-course studying material, activities’ arrangement and schedule and micro-scenarios developed and learning paths. The results of this research are described in the Table 4 below. Table 4. The e-learning system evaluated towards quality, schedule, scenarios and material provided e-learning system characteristics evaluated (general) Interface and design quality E-course studying material Activities’ arrangement and schedule Micro-scenarios developed and learning paths

Agree

Strongly agree

0,02

Neither agree nor disagree 0,23

0,63

0,12

0,00 –

0,01 0,02

0,19 0,27

0,61 0,57

0,19 0,14

–

0,01

0,21

0,66

0,12

Strongly disagree

Disagree

–

As it is proven by the results, a great percentage of the participants agreed that the ePortfolio system was accessible and its interface was user-friendly. Concerning the elearning material, the 67% of learners claimed that the ePortfolio course system was well-organized and that it provided them the necessary information to schedule their learning. They also evaluated positively the activities’ arrangement and schedule as they declared that the ePortfolio course system helped them in time management and also in the setting of the appropriate learning goals and in the plan setting. Finally, it is proven that collaborative learning helped students improve their learning design and amend their learning strategies.

5 Conclusion This study examines the implementation of an ePortfolio in order the self-regulated skills of learners to be enhanced. The proposed design for an academic and system-oriented ePortfolio that was structured under the Self-Regulated Learning Theory, seems to have positive effects on students’ self-regulated and academic skills. The results of the research

Poster: Exploring the Educational Affordances of an Academic ePortfolio

165

prove that the structure of the ePortfolio course system under the SRL theory and the ePortfolio’s affordances, help learners in the construction of their ePortfolios, in the organization of learning and in the enhancement of their self-regulated skills. However, a major limitation of this research is the small number of the participants. From this point of view, a suggestion could be the use of this research as a pilot study for the further examination of the proposed design, the process and the tool. Acknowledgements. This work has been partly supported by the Research Center of the University of Piraeus.

References Barbera, E.: Mutual feedback in ePortfolio assessment: an approach to the netfolio system. Br. J. Educ. Technol. 40(2), 342–357 (2009). https://doi.org/10.1111/j.1467-8535.2007.00803.x Barret, H.: Electronic Portfolios in STEM - What is an Electronic Portfolio (2010). http://www. scribd.com/doc/40206175/E-Portfolio-Definition Barnett, R.: Assessment in higher education: an impossible mission? In: Boud, D., Falchikov, N. (eds.) Rethinking Assessment in Higher Education, pp. 29–40. Routledge, London (2007) Batson, T., Chen, L.: Next-Generation ePortfolio, Academic Impressions (2008) Boekaerts, M., Corno, L.: Self-regulation in the classroom: a perspective on assessment and intervention. Appl. Psychol. Int. Rev. 54(2), 199–231 (2005) Clark, J.E., Eynon, B.: EPortfolios at 2.0 – surveying the field. Peer Rev. 11(1), 18–23 (2009) Darling-Hammond, L., Snyder, J.: Authentic assessment of teaching in context. Teaching Teacher Educ. 16, 523–545 (2000) Dunbar-Hall, P., Rowley, J., Brooks, W., Cotton, H., Lill, A.: EPortfolios in music and other performing arts education: history through a critique of literature. J. Hist. Res. Music Educ. 36(2), 139–154 (2015) Fernsten, L., Fernsten, J.: Portfolio assessment and reflection: enhancing learning through effective practice. Reflective Pract. Int. Multi. Perspect. 6(2), 303–309 (2005) Handley, K., Williams, L.: From copying to learning: using exemplars to engage students with assessment criteria and feedback. Assess. Eval. High. Educ. 36(1), 95–108 (2011) Klenowski, V., Askew, S., Carnell, E.: Portfolios for learning, assessment and professional development in higher education. Assess. Eval. High. Educ. 31(3), 267–286 (2006) Liu, J., Burt, R.: Introducing ePortfolios to construction management undergraduate students. In: 51st ASC Annual International Conference, Texas, USA (2015) Luchoomun, D., McLuckie, J., van Wesel, M.: Collaborative e-learning: EPortfolios for assessment, teaching and learning. Electron. J. e-Learning 8(1), 21–30 (2010) Nicol, D., Thomson, A., Breslin, C.: Rethinking feedback practices in higher education: a peer review perspective. Assess. Eval. High. Educ. 39(1), 102–122 (2014) Sadler, D.R.: Beyond feedback: developing student capability in complex appraisal. Assess. Eval. High. Educ. 35(5), 535–550 (2010) Sebesta, A.J., Speth, E.B.: How should i study for the exam? self-regulated learning strategies and achievement in introductory biology. CBE—Life Sci. Educ. 16(2) (2017) https://doi.org/ 10.1187/cbe.16-09-0269 Stefani, L., Mason, R., Pegler, C.: The Educational Potential of ePortfolios, Supporting Personal Development and Reflective Learning. Routledge, London (2007)

166

F. Paraskeva et al.

Zimmerman, B.J.: Investigating self-regulation and motivation: historical background, methodological developments, and future prospects. Am. Educ. Res. J. 45(1), 166–183 (2008). https://doi.org/10.3102/0002831207312909 Zimmerman, B.J.: Development of self-regulated learning: which are the key subprocesses? Contemp. Educ. Psychol. 16, 307–313 (1986) Zimmerman, B.J.: Attaining self-regulation: a social-cognitive perspective. In: Boekaerts, M., Pintrich, P., Seidner, M. (eds.) Selfregulation: Theory, Research, and Applications, pp. 13–39. Academic Press, Orlando (2000)

Mobile Learning Models, Theory and Pedagogy

M-Health as a Tool in the Cognitive Flexibility of the Elderly Cristina Páez-Quinde(&), Sonia Armas-Arias, Dorys Cumbe-Coraizaca, and Santiago Velastegui-Hernández Facultad de Ciencias Humanas y de la Educación, Universidad Técnica de Ambato, Ambato, Ecuador {mc.paez,sp.armas,dm.cumbe,rs.velastegui}@uta.edu.ec

Abstract. This research is based on verifying the changes in cognitive function through the use of a mobile application designed to work in the mental flexibility processes of the elderly. It is worth mentioning that the presence of Information and Communication Technologies (ICT) within society is a reality that manifests a special reflection in continuous evolution, and it has been increased in the field of health. Health mobile or M-health is the use of portable devices such as smartphones or tablets exclusively for medical use including diagnosis, support or treatment or even having a report on general health and well-being. All users (doctors/patients) can interact with these devices through a software application (App), which generally gather information interactively. This research aims to identify the support of Apps in the mental flexibility of the elderly. Mental flexibility is part of the executive functions that get deteriorated due to several causes, like the death of neurons. It was verified that there are new ways of learning in the elderly through the use of cognitive stimulation, taking advantage of the technology as a cognitive stimulation resource by using an App in patients of the Vida en los Años Nursing Home. The methodology used for this research is experimental, because the variable cognitive flexibility was manipulated through the use of a mobile application that modified its operation. Also, the changes were verified through a pre and posttest called Stroop Test. The result of Cronbach’s alpha is (0.854) of the questions posed in the structured survey. Using the Wilcoxon statistic, the silver hypothesis in the research was verified by checking the confidence ranges in the Gaussian bell. Keywords: ICT learning


mHealth
Mental stimulation
Cognitive flexibility
m-

1 Introduction The objective of this research was to determine the influence of mobile applications on mental flexibility in the elderly and also how this type of technological tools help through a non-invasive process in the quality of life of the elderly by promoting an impact on the use of Apps that generate a great impact on health, known as M-Health. The mental flexibility [1] is part of the executive functions that deteriorate due to several causes, such as; the death of neurons that occurs when people get older. It is innovative considering that mobile applications help neuronal restructuring in different © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 169–177, 2021. https://doi.org/10.1007/978-3-030-49932-7_17

170

C. Páez-Quinde et al.

pedagogical processes and even with the elderly; the importance of the project in cognitive stimulation in order to improve retentiveness in the elderly caused by aging through the use of mobile applications, generate a change in the process of neuronal motivation. The interest of this project was to generate and disseminate that mobile applications promote new ways of learning in the elderly by taking advantage of technology as a resource for cognitive stimulation in the Vida en los Años Nursing Home. The elderly of the Vida en los Años Nursing Home were the beneficiaries of this work. They were evaluated with this type of tools and based on the results that were found, some actions were recommended for their treatment. Also these results promote more suitable trainings for the elderly. Research itself also provides important data that will serve as the basis for new research within the field of educational informatics, health and the development of new technologies. It should be considered that cognitive rehabilitation instruments exist, but they are still not used in Ecuador due to economic limitations and lack of knowledge of this field. This research is based on the contribution to the cognitive stimulation of the elderly of the Vida en los Años Nursing Home by contributing in the field of computer sciences and health sciences, which allow the integration of technology with human sciences and behavior as well as the improvement in the quality of life and advances in science, which have meant that life expectancy has increased considerably in most countries. The elderly has been one of the groups where this change has been most noticed, because this group has been in continuous growth, so new services must be designed to meet the new technological and informative needs that arise in this group. There is a wide variety of lines of research related to the elderly (usefulness of the Internet to find information that serves their particular interests, application of technology in healthcare for this social group, information literacy or its relationship with social ethics) and the real possibility of designing specific contents thus promoting new opportunities. The presence of information and communication technologies (ICT) in society is a constant reality and a field of special reflection in continuous evolution that has been increased in recent years by the speed with which technological developments take place, and for its presence and impact in the educational and health world. In the United States, 85% of American adults own a cell phone and 53% of them own smartphones. All information is available wherever and whenever people need it. In addition, 31% of cell phone owners say they use their phone to search for medical or health information online. The growth of mobile applications in smartphones and tablets is one of the most surprising technological developments in recent years. More than 700,000 applications are available for immediate download from the application markets. These markets are a significant disruptive change in the way content is created and consumed. On the supply side, they provide content creators with direct, instant and popular distribution systems where they can implement their own marketing and pricing policies so that they eliminate intermediaries. Mobile health, or “mHealth,” [2] is the use of portable devices such as smartphones and tablets for medical use, including diagnosis, treatment or general health and welfare support. Users can interact with mobile devices through software applications “applications” that usually gather information from interactive

M-Health as a Tool in the Cognitive Flexibility of the Elderly

171

questionnaires, separate medical devices connected to the mobile device, or device features themselves, such as; camera, motion sensor, or microphone.

2 State of the Art Cognitive flexibility can be defined as the ability of our brain to adapt to unexpected, novel, changing circumstances, as well as the ability to solve problems immediately and provide an alternative solution, so through these actions we can modify our behavior and adapt to the environment that we have to live in a patient and consistent way by guaranteeing a mental balance. The theory of cognitive flexibility (TFC) [3] is responsible for the acquisition of complex and unstructured knowledge, presenting the complexity in small units analyzed through multiple options that will help in the research topic. Also, it is characterized by the intervention of many ideas or concepts on a case, so its combination is inconsistent in equal cases. This is a “holistic integrative” domain of knowledge. The main features of the FCT discuss: crossing of conceptual landscapes, structuring knowledge domains, advanced learning and complex concepts, structure in cases and mini cases, flexible, alternative conceptions, and random learning teaching. Cognitive flexibility is defined as the ability to adapt performance to environmental conditions in front of a task or as a component of executive functions [4]. There is consensus to consider as components of executive functions to the following things: the ability to plan behaviors aimed at a goal, the programming of actions necessary to achieve that goal, the monitoring of execution, the ability to control the interference of irrelevant stimuli, the flexibility to correct errors, incorporate new behaviors and to complete a task when it has been completed. The following postulates that arise from the TFC are: The subject needs different representations and interpretations to produce complex learning. Subjects that receive knowledge from cognitive flexibility are able to solve problems as an adaptive response to changes that occur in a given situation. The repetition of information in different contexts helps to improve the transfer of knowledge. The use of multiple perspectives in educational programs is an example of one of the most important recommendations of the theory of cognitive flexibility. Cognitive flexibility usually develops and is stimulated from the day we are born and develops throughout our lives [5]. However, when we become older this ability goes down. That is why during aging the inhibitory processes that regulate the attention that is stored in the memory bring different consequences on a wide range of cognitive processes affecting the well-being and quality of life of the elderly. Its presence sometimes includes symptoms of depression, loss of memory, inattention, among others. The changes usually associated with the aging process generate a greater number of complaints and daily problems among the elderly related to the objective and subjective loss of memory, as well as a deficit in motor skills, a decrease in processing speed, among other shortcomings are accompanied by the little cognitive flexibility. Therefore, it can be pointed out that the risk factors derived from the lack of cognitive flexibility concerning to the performance of tasks that require a high level of attention

172

C. Páez-Quinde et al.

and controlled processing show differences in the individuals who present inhibitory disorder and memory loss. The general cognitive deficit is noticeable during aging [6]; this is considered harmful for the elderly, so this will complicate the components of cognitive inhibition, the general slowness and the prevalence of errors. On the other hand, the symptoms derived from alterations in some of the cognitive functions constitute a central characteristic of the affective disorders that involve depression and anxiety. Ultimately, aging has not only become a social problem but also a global one as the number of elderly increases every day. The elderly will suffer from cognitive decline and dementia in the coming decades, causing costs at the family, personal level and social and above all a poor quality of life. For this situation, we must look for alternatives that allow to stimulate the cognitive development and prevent the memory of the elderly from being disabled. The quality of life of the elderly can be improved through memory stimulation with cognitive therapies and manual and technological exercises, such as the application of virtual reality for the ones cannot leave home or bed. By using this strategy, we can transport them to wonderful worlds to improve their levels of depression and anxiety. It can be used to remind them that they should take medicines or perform some specific activity, i.e., give them routines that they must fulfill, so the elderly will have a more bearable lifestyle [7]. There are some recommendations that we can apply immediately to people who are close to become elderly in order to stimulate their cognitive flexibility: • • • • •

Manage cardiovascular values such as hypertension with adequate treatment. Perform activities of physical exercise and a balanced diet. Carry out social and cultural activities. Challenge the brain that is learning something new, reading is crucial. Avoid sedentary lifestyle and routine.

Mobile Learning as an extension of e-learning is characterized by its independence in relation to location in space and time [8]. Quinn’s vision of mobile computing is based on portable computing with high interactivity, total and high connectivity and processing; a small device that is always networked allowing easy entry of data through pens, dictations or a keyboard if necessary. It also has the ability to view images with high resolution and high sound quality, i.e. a mobile application or app, is software designed to work on smartphones and other mobile devices. In recent years, these apps have suffered a boom in offers to their users and diversity, entering the field of medicine, both for professionals and for patients [9]. Of all the “health apps”, there is a part dedicated to the field of nutrition. It is estimated that in the category of “diet and fitness” there are more than 5,400 apps. 95 apps were analyzed in addition to the ones reviewed in the research studies. The apps turned out to be an option in the choice of strategies to improve and prevent certain diseases related to nutrition, exercise and daily habits, both from the individual point of view, as well as by professionals, although the unreliability of the great majority, 51.57% were qualified as “low quality”. Although most of the applications are not useful or safe. If in the future they are normalized and improved, they could be a very useful tool for society and the health system.

M-Health as a Tool in the Cognitive Flexibility of the Elderly

173

Mobile learning includes the use of a cell phone, a PDA, so that the user has access to available course resources wherever they are, i.e., they can consult learning (guides or readings) [10], verify learning activities, find information updated about a course that you are doing or take a test that allows you to assess your level of knowledge. In fact, it is possible that the user can download any type of material, work offline (if desired), reconnect to send the results and receive feedback. In summary, it can be said that mobile learning is constituted by the use of “mobile technologies” at the service of the processes associated with teaching and learning. In recent years, the development of wireless data networks has allowed the connection of devices such as; tablets and electronic smartphones to the Internet with the ability to access educational content at any time and place, without the need to be physically in the classroom. This phenomenon gives rise to a new modality of distance learning called “mobile learning” (known as m-learning) [11]. [12] Define m-learning as a combination of e-learning and mobile computing that mixes mobile and wireless technology to provide learning experiences. Currently, m-learning software developers have offered educators applications of various topics to be used as support in the learning process (inside or outside the classroom). However, there is little documentation (design guides, best practices or scientific studies) [13] of how an m-learning application should be projected and how the correct integration of the different educational components (knowledge and skills) should be done, so that an application meets the didactic objective established. Therefore, a combination of intuition, skill and luck led programmers or software designers to develop a new application [14].

3 Methodology This research was carried out with the Elderly of the Vida en los Años Nursing Home in order to know the preferences in the use of mobile devices and the information requirements of the users. It was found that the great majority of Seniors use their mobile devices a lot, often using them for various activities, such as; searching for information, communicating with friends, family and others, and also for fun and entertainment activities. This study allowed to identify that the elderly had acceptance when using this type of tools within their therapies in the nursing home. In addition, it was possible to carry out several activities mainly in the cognitive part in order to improve this process by means of a non-invasive method. Training was done for a prudential time using an App based on the Stroop test, so that it is accessible and immediate for all its users. It was also possible to identify that the mobile applications for the Elderly caused a great commotion among them when the survey was conducted, also it caused high interest to know what this technological application was about. After observing and analyzing everything related to the use of mobile applications and devices in the Vida en los Años Nursing Home, these activities supported the design and programming of a mobile application on an Android platform, which improves the dissemination of information through recreational activities. The results from the research demonstrate the importance of mobile applications in the mental health of the elderly. The little access to them has caused them not to be

174

C. Páez-Quinde et al.

used frequently, among the main ones are the cost as well as the lack of knowledge of new technologies by the professionals of this field. The following proposal has a technological impact since by developing a free mobile application with the aim of improving the mental flexibility of the elderly contributes significantly to scientific progress in the field of technology and rehabilitation in health. For this reason, the proposal is technologically justified because there is knowledge of programming and the equipment necessary to perform the relevant functional tests of the mobile application. In addition, there are adequate and qualified personnel to advice on this proposal, as well as institutional and economic resources. By developing this application, it will be possible to verify the activities that the elderly must meet in the different applications created for cognitive stimulation and the activities that the elderly must perform to improve cognitive.

Fig. 1. AppMemoria

Figure 1 shows the logo of the application in the upper part and a message of Welcome to the user of the application in the lower part. A framework of options is also shown with the 4 games of the Application.

Fig. 2. Scheme menu

M-Health as a Tool in the Cognitive Flexibility of the Elderly

175

Figure 2 shows the options menu of the Application in the upper part where users will find the Start Page, the games available in the application and the Help page that will be explained later.

Fig. 3. Scheme game

Figure 3 shows the scheme and the modality of the game where the methodology of the game is shown and a table of successes that the user obtained are shown at the end.

Fig. 4. Application coding

In Fig. 4, part of the development environment of Adobe Dreamweaver is shown. You can see the code used and the file panel where the source files of the Application are located (Fig. 5).

176

C. Páez-Quinde et al.

Fig. 5. Website 2 APK Builder

To design the App, the WebSite 2 APK Builder framework was used. This is a useful tool because it allows you to transform a web app to a mobile app adapted to Android devices. It should be noted that this tool is paid, so a license was used for its use.

4 Conclusions Mobile applications influence the cognitive flexibility. After a time of using the App, it was determined that there are significant changes in the cognitive function of the elderly through non-invasive methods for the patient. In addition, it was possible to demonstrate that the frequent use of these types of resources are important in this new technological era. It should be mentioned that mobile applications designed for health stimulation and rehabilitation are used by health professionals in a minority due to lack of knowledge and access to them; 93.4% of elderly presented alterations in cognitive flexibility, and 6.6% did not present alterations. These results were obtained in the first evaluation by means of the Stroop Test. After verifying that mobile applications influence cognitive flexibility, a free app designed with mental exercises was created aiming to stimulate functions involved in the cognitive flexibility.

References 1. Leung, C., Cheng, L., Yu, J., Yiend, J., Lee, T.: Six-month longitudinal associations between cognitive functioning and distress among the community-based elderly in Hong Kong: a cross-lagged panel analysis. Psychiatry Res. 265, 77–81 (2018) 2. Eze, E., Gleasure, R., Heavin, C.: Planning and positioning mHealth interventions in developing countries. Health Policy Technol. 8(2), 137–142 (2019) 3. Gu, L., Chen, J., Gao, L., Shu, H., Wang, Z., Liu, D., Yan, Y., Zhang, Z.: Cognitive reserve modulates attention processes in healthy elderly and amnestic mild cognitive impairment: an event-related potential study. Clin. Neurophysiol. 129(1), 198–207 (2018)

M-Health as a Tool in the Cognitive Flexibility of the Elderly

177

4. Laubach, M., Lammers, F., Zacharias, N., Feinkohl, I., Psichon, T., Borchers, F., Slooter, A., Kuhn, S., Spies, C., Winterer, G., BioCog Consortium: Size matters: grey matter brain reserve predicts executive functioning in the elderly. Neuropsychologia 119, 172–181 (2018) 5. Berrocal, J., Garcia-Alonso, J., Murillo, J., Canal, C.: Rich contextual information for monitoring the elderly in an early stage of cognitive impairment. Pervasive Mob. Comput. 34, 106–125 (2017) 6. Lemercier, C., Simoes-Perlant, A., Schmidt, J., Boujon, C.: Stroop interference and development: Influence of expectation on color-naming response times Interférence et développement: influence du processus d’expectation sur les temps de dénomination de la couleur. Revue Européenne de Psychologie Appliquée 67(1), 43–50 (2017) 7. Witterkind, C., Muhtz, C., Moritz, S., Jelinek, L.: Performance in a blocked versus randomized emotional Stroop task in an aged, early traumatized group with and without posttraumatic stress symptoms. J. Behav. Ther. Exp. Psychiatry 54, 35–43 (2017) 8. Greene, E., Proctor, P., Kotz, D.: Secure sharing of mHealth data streams through cryptographically-enforced access control. Smart Health 12, 49–65 (2019) 9. Ki-Hun, K., Kwang-Jae, K., Dae-Ho, L., Min-Geun, K.: Identification of critical quality dimensions for continuance intention in mHealth services: case study of onecare service. Int. J. Inf. Manage. 46, 187–197 (2019) 10. Kosse, R., Murray, E., Bouvy, M., de Vries, T.W., Stevenson, F.: Potential normalization of an asthma mHealth intervention in community pharmacies: applying a theory-based framework. Res. Soc. Adm. Pharm. 16, 195–201 (2020) 11. Zhang, X., Lai, K., Guo, X.: Promoting China’s mHealth market: a policy perspective. Health Policy Technol. 6(4), 383–388 (2017) 12. Bakirci-Taylor, A., Reed, D., McCool, B., Dawson, J.: mHealth improved fruit and vegetable accessibility and intake in young children. J. Nutr. Educ. Behav. 51(5), 556–566 (2019) 13. Wang, L., Wu, T., Guo, X., Zhang, X., Wang, W.: Exploring mHealth monitoring service acceptance from a service characteristics perspective. Electron. Commer. Res. Appl. 30, 159–168 (2018) 14. Onodera, R., Sengoku, S.: Innovation process of mHealth: an overview of FDA-approved mobile medical applications. Int. J. Med. Inform. 118, 65–71 (2018)

Critical Categorization of Android and IOS Applications Available for STEAM Education in Early Childhood Tharrenos Bratitsis(&), Michalis Ioannou, and George Palaigeorgiou University of Western Macedonia, Florina, Greece [email protected], [email protected], [email protected] Abstract. Nowadays, children have their first experience of using a mobile device very early in their life (even under their 2nd year). Furthermore, STEAM in Early Childhood Education (ECE) draws more attention lately, while a lot of discussion is being made regarding the integration of mobile applications in educational settings, formal and informal. A very common concern of both educators and parents of children aged 4–6 years old is “what mobile applications are appropriate for the children?”. Especially focusing in STEAM education questions are raised regarding the corresponding competences’ cultivation for these ages and the availability of applications that can facilitate this process. Although the issue of examining available mobile applications is being dealt with for several years for older ages and other disciplinary areas (e.g. mathematics, language), there is not much work regarding STEAM mobile applications for 4–6 ages. A survey in both Android and IOS environments was carried out in order to record and attempt to categorize the existing mobile applications which fall under the STEAM umbrella, in order to gain perspective on what is available for in-classroom utilization in this regard. Keywords: STEAM


Kindergarten
Mobile applications

1 Introduction Science, Technology, Engineering and Mathematics (STEM) education is described as a center of integrated disciplines, as an interdisciplinary connection among distinct disciplines or as an entity [1]. The integrated STEM education includes approaches that explore teaching and learning among any two or more of the STEM subject areas, or between a STEM subject and at least one more other school subject [2]. So, it can be defined as an integrative approach to curriculum and instruction, content and skills, approaching all the STEM areas as one, without any boundaries [3]. Generally, through STEM education, students can develop 21st Century skills like problem solving, complex communication, adaptability and system thinking [4]. Especially for young children, arts can be added to the STEM acronym, leading to STEAM. In this case, all forms of art can influence and enhance the STEM content areas [5]. Overall, STEAM is evolving as a big trend in contemporary educational research and many studies are conducted constantly. © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 178–188, 2021. https://doi.org/10.1007/978-3-030-49932-7_18

Critical Categorization of Android and IOS Applications Available for STEAM

179

On the other hand, the vast economy growth and societal evolvement over the past years has led to the ubiquitous presence of technologies [6]. Information access and communication have become easier and widely available for the ever-growing part of the population that is technology fluent [7]. Mobile devices are being increasingly utilized within this vein [8], especially by young people, “digital natives” as Prensky described them [9]. Likewise, Marsh et al. [10] stated that young children (up to 6 years of age) are “immersed in practices related to popular culture, media and new technologies from birth”. Thus, they are operate machines very efficiently while developing skills, knowledge and understandings about the world they live in, at the same time [7]. The focus of this paper is Early Childhood Education (ECE) in the context of STEAM Education through mobile applications. Following the clearly aforementioned, stated research trends and following unofficial discussions with educators (mainly) and parents who often pose questions about identifying appropriate mobile applications for their students and children accordingly, this paper attempts to investigate this issue. A thorough search in Google Play and App Store was carried out in order to distinguish STEAM related applications for ECE. An attempt to initially categorize them in order to reach some useful conclusions is then made. The rest of the paper is structured as follows: initially, a brief overview of the literature regarding mobile devices in Kindergarten and Early Childhood Education is presented. Then, the research approach is described and the results are presented. The last section is a concluding discussion which attempts to sum up the findings and explore the anticipated frustration of teachers (mainly) when they are involved in discussions regarding the topic under investigation.

2 Background of the Study This section presents the background of this study which lies on two pillars, mobile devices and STEAM in ECE. It should be noted that there is not much work exists for the designated age group, especially when considering that the term mobile devices nowadays refers to smartphones and tablets/ipads, as opposed to earlier studies which incorporated iPods, PDAs and Tablet PCs. This study follows the contemporary perception of mobile devices. 2.1

STEAM and ECE

In 1990’s the National Science Foundation introduced the term STEM as a generic label for any actions, policies, programs or practices than involve one or more of the terms disciplines: Science, Technology, Engineering and Mathematics [11]. In recent years, Arts are proposed as an additional discipline, leading to STEAM (Science, Technology, Engineering, Arts and Mathematics) [12]. Hands-on and creative approaches to STEM education, using many of the methods proposed for creative arts seem to attract interest of young people in the STEM fields. Art, like Engineering, deals with finding, troubleshooting and looking for visual solutions using design processes [13].

180

T. Bratitsis et al.

Many scientists, mathematicians and engineers recognize that the arts are important for their success and they use art-related skills as scientific tools. Those skills include: 1) relying on someone’s curiosity, b) to observe with precision, c) to perceive an object in different form and perspective, d) to construct meaning and express remarks to someone with precision, e) to cooperate effectively with others, f) to think objects spatially, and g) to perceive kinesthetically. These skills are not usually taught in STEM education but are encountered in writing, dancing, music, painting, theater, etc. The arts facilitate the development of creativity, problem solving skills, critical thinking, communication, initiative and cooperation [14]. So, adding/incorporating arts to STEM-based subjects, such as engineering and programming, may enhance learning of students by offering opportunities for innovation and creativity [15]. STEAM educational programs can be implemented in Kindergarten as children, even before they officially enter the formal educational system, form ideas and beliefs for physics’ phenomena within the world surrounding them and construct definitions about them [16]. These early ideas and theories play a significant role in their future learning experience [17]. So, children must develop their interest in STEAM at a young age in order to excel at these disciplines when they grow up [18]. There is, also, an increased focus from researchers and policy-makers in early childhood as the importance of exposing young children to STEAM activities seems to help avoiding stereotypes or other disciplines in later years [19]. According to Chesloff [20] children are born as scientists and engineers. Like STEAM education, investing in ECE is a very important issue. It seems that high quality ECE reduces the low performance of children in their future education. The high quality learning environments in ECE offer children the fundamentals on which they can rely on during their investigations, their knowledge constructions and their doubts’ resolution. Also, children’s engagement with the sciences at early ages, combined with other disciplines such as Technology, helps them become aware of and increase their interest in sciences [21]. An important benefit of STEM education is the opportunities it offers to learners to cultivate the ability to transfer learning to different contexts. Children can solve new problems and draw conclusions, based on their previous knowledge and principles which they already came to possess through science, technology, engineering and mathematics. It seems that teaching strategies such as problem solving through STEM curriculums can enhance the interest of the children to understand the world around them and strengthen their participation in the classroom [22]. STEAM is adopted by early childhood educators as an authentic approach to prepare children in the areas of science, mathematics, engineering, and technology within an inquiry-based, developmentally appropriate and measurable context [23]. However, the idea of teaching STEAM to young children is still not clear to teachers and STEAM in ECE seems to be marginalized, despite the justified necessity to start STEAM education in preschool years [24]. 2.2

Mobile Devices in Kindergarten

Mobile devices have a number of unique characteristics which totally change the utilization of educational applications, far beyond the way traditional ICTs are used

Critical Categorization of Android and IOS Applications Available for STEAM

181

[25]. Mobile learning refers to the kind of learning which occurs in mobile settings, considering the mobility of technology, learners or even learning itself [26]. In this paper, the term is related to the use of mobile devices, namely Android and IOS tablets in classroom settings. For that matter, simple reference to the existing literature is considered as needed, only in order to highlight the research trend. The ease of use of mobile devices with touch screens which imitate physical writing but also incorporate multi point touch and gesture-based interaction are significant factors for deciding to use mobile devices in education. Furthermore, accelerometers and other sensors allow interaction through physical movement and automated screen rotation, thus adding value to the interactivity of a tablet [27]. All those are feature that allows for young children (under the age of 6 years old) to start their journey within the digital world much earlier in their developmental process than ever before [28, 29]. There are several studies in the literature which pinpoint the participation and collaboration enhancement of preschool children [30], opposing the long existing resistance of ECE for incorporating technology in class [31]. Sahin et al., [32] only found 18 papers in their literature review for tablet use in Kindergarten which were categorized in highlighting the teachers’ perspectives, the children’s perspectives, the utilized software and other topics (e.g. parents’ perspectives). Kjällander & Moinan [33] showed that children in Kindergarten can be very creative when using mobile applications and they actually make new meanings in a playful manner, thus learning by shifting from learning consumers to playing producers. Furthermore, there are several case studies which regard the exploitation of tablets in Kindergarten, focusing on specific disciplinary areas [34–36, 38]. Also, recently there are interventions in the literature which exploit non-educational applications in educational contexts, such as Augmented Reality apps [36]. Overall, mobile devices’ utilization in ECE is gaining momentum over the last few years, in various contexts. Thus, the need to examine relevant applications in the STEAM context can be characterized as meaningful.

3 The Study: Searching for STEAM Apps The authors conducted a systematics research in two mobile applications databases, Google Play and App Store. The aim was to examine the available STEAM applications for Android and IOS devices, appropriate for in school and out of school educational purposes for Kindergarten children. For this purpose, an extensive search was carried out in both online databases, using the keywords “STEM Education”, “STEAM Education”, “Engineering”, “Kindergarten” and “Early Childhood Education”. The keywords were written in Greek and English. Avery novel filtering approach was used to select applications in both languages which do not present textual information and thus do not require reading skills. This correlation with textual information was more loosely followed in the native Greek applications. Then, the available applications are categorized based on their nature, the manipulation skills required to work with them, the specified ages within the target age span for which they are addressed and the STEAM areas that they can cover. A critical evaluation of the application availability is to be made through this paper, also taking

182

T. Bratitsis et al.

into account the actual value of each application and providing the keywords which were used to locate it. 3.1

Aim

The aim was to find all the possible STEAM applications for children 4–6 years old and an attempt to categorize them. All the app data from Google Play were stored in an Excel file, including the used keywords, app name, PEGI level, downloads, evaluation by users (stars), suggested age, language, category and access (free). These are features incorporated in Google Play. In addition, all the app data from the App Store were stored also in the same Excel file, including the used keywords, app name, evaluation, number of evaluations, suggested ages, language, category and access. In the App Store there are filters according to the price (Free/Any), the classification (Relevance/Popularity/Ratings/Release Date) and Ages (All/5-/6-8/9-11). All this information was analyzed in a quantitative manned in order to draw some preliminary conclusions. 3.2

The Search Procedure

During the Search procedure in both databases, all the application data were stored in an Excel file manually, one by one. The search returned a total of 810 apps in Google Play and 78 in App Store. As these databases do not offer enough filters to minimize the amount of the applications, each result was recorder and examined manually. The next step was to identify which STEAM applications are available for the ages 4–6. For this purpose, each application was downloaded and tested separately. In this way, an initial filtering approach was conducted by the researchers. During this step, the researchers tested the functionality of the applications, the age appropriateness, the educational disciplinary areas of every application, the user interface interactions, the application availability (free or paid) and the integrated STEAM characteristics. Also, all the applications which only presented data/information were excluded by the results. The reasoning for this choice was that in these cases the users (children) would not be required to perform any tasks and/or activities in order to learn but would rather be provided with information (in a lecture-like manner). Thus, they were excluded for being behavioristic in nature and not allowing knowledge construction through action. Finally, the total amount of the potential STEAM applications for kindergarten children was 86; 72 in Google Play and 14 in App Store.

4 Findings This section is divided in two parts. In the first one, the analysis of the App Store results are presented. The second contains the analysis results for Google Play. The analysis approach was very novel, as basic characteristics of the apps were examined in a qualitative and manual approach.

Critical Categorization of Android and IOS Applications Available for STEAM

4.1

183

App Store Results

In this section, the App Store data are analyzed. Out of the 78 applications initially recorded, only 14 were finally selected as STEAM-related applications, appropriate for ages of 4–6 years old. The main criteria for this selection was the free availability, the age appropriateness and the functionality of the app as a standalone software, as many applications required robots and other additional material which would work with or through the application. Also, the main criteria were the STEAM characteristics of the apps; that is their correspondence to the STEAM disciplinary areas and the way they are supported, as the definition of the term describes. All the applications (14/14) were suggested for ages “4+” by the developers (as stated in the App Store) and were freely available. Free availability corresponds to at least a significant portion of the application being freely available in order for it to be used for teaching and learning, as opposed to limited-day trial applications which can only be used for certain days. Specifically, 4 out of 14 (28.6%) were in the “5-” age category and 10/14 (71.4%) in the category “6–8” (although in the application description they are described as “4+” appropriate), regarding the age appropriateness. Regarding the language, 13 (92.9%) of them were in English and 1 (7,1%) in Japanese. Also, 12/14 (92.9%) belonged to the “Education” category of the App Store, while 1 was in the “Games: Education” category. Thus, one can observer the lack of applications for other language settings, although it was already mentioned that the authors selected applications that do not use textual prompts, as children of these ages usually are not able to read. All 14 applications were returned when STEM EDUCATION were used as search keywords. Only 1 out of 14 had an evaluation of 4, 5 (out of 5) which derived from only 6 evaluations. Thus, it is safe to say that these applications seem to be well underused and under-examined in the context of this paper. Regarding the STEAM disciplines, 8 out of 14 applications (57.2%) were related to Technology (coding), 4 were related to Science (28.6%), 1 (7.1%) was related to Mathematics and 1 (7.1%) related to Engineering. As already mentioned, there is a number of applications which accompany external devices and material, such as OSMO and Robotic devices. They were not included in this search, as they cannot be used in a standalone mode for teaching STEAM subjects. 4.2

Google Play Results

In this section, the Google Play data are analyzed. Out of the 810 applications initially recorded, only 72 were finally selected as STEAM-related applications, appropriate for ages of 4–6 years old. The main criteria for this selection was the free availability, the age appropriateness and the functionality of the app as a standalone software, as many applications required robots and other additional material which would work with or through the application. Also, the main criteria were the STEAM characteristics of the apps; that is their correspondence to the STEAM disciplinary areas and the way they are supported, as the definition of the term describes. All the applications (72/72) were suggested for ages under examination, by being characterized as PEGI 3 application in Google Play. PEGI stands for “Pan-European

184

T. Bratitsis et al.

Game Information” and it is a classification system which indicates the age groups for which a mobile app is appropriate for. The main aim for creating this system was to assist mainly parents to identify apps which are appropriate for their children. This system is used by 30 European countries. Developers choose the appropriate PEGI class of their applications which is further evaluated by a committee, which approves or not this choice. The classification is determined by a number which corresponds to the age of the end-user that the app is appropriate for. For example a PEGI 3 application is appropriate to be used by children of 3 years of age or more. Usually, the PEGI classifications are: 3, 7, 12, 16, 18 and “Parental guidance recommended”. Thus, in the case of Kindergarten, PEGI 3 applications were examined. Other countries, outside EU use similar classification systems which characterize the age appropriateness of the content. On the other hand, despite the PEGI classification, the developers are free to indicate in the description of their application the age appropriateness of it. Overall, 31 out of 72 (43.1%) were suggested for ages “8-”, 8 (11.1%) for ages “5-”, 7 (9.7%) for ages “6–12”, 1 (1.4%) for ages “6–8”, 1 (1.4%) for ages “9–12”, 1 (1.4%) for ages “8-18” and 23 of the applications (31.9%) had no age reference. Although there seems to be some inconsistency with the PEGI classification, the authors decided to include the apps that are proposed for older ages that 8 years old after having tried them. Examining the number of downloads the apps had been downloaded from 50 (Min value) to 1,000,000 (Max value) times. The average amount of downloads was over 500,000 indicating applications that were tried out by users a lot. Regarding the star evaluation (the user rating system incorporates in Google Play) of the applications the average evaluation was 4.3 (spanning from 0 to 5 stars). The number of evaluating users spanned from 1 to 57030 and the average number was 5,620.8. Thus, it is evident that the Google Play applications were used more heavily than the App Store ones. In addition, 54/72 (75%) of the applications fell under the category “Education”, 33 of them (45.8%) in “Educational” category, 4 (5.6) in “Mind Games” and in “Medical”, 3 (4.2%) in “Creativity”, 2 (2.8%) in “Puzzle Games” and 1 (1.4%) in “Entertainment”. These categories are mainly keywords used by Google Play to classify applications in order for them to be included in searches which include these keywords. Many applications appeared as belonging in more than one category. Examining the disciplinary area of the applications, 28 out of 72 (38.9%) can be considered as related to Science, while 8 of these apps (28.6%) related to Biology. Also, 25 (34.7%) applications were related to Mathematics, while 2 (2.8%) of them combined Mathematics with Language. Additionally, 9/72 (12.5%) applications combined Science with Mathematics, 7 (9.7%) dealt with Technology (mainly coding), 1 (1.4%) was related to Engineering, 1 (1.4%) combined Mathematics with Technology (coding) and 1 (1.4%) was related to the combination of Science, Technology, Engineering and Mathematics as a whole. Not all of them had adequate descriptions, but they were tested by the authors. This manual testing allowed to examine also the nature of the apps. The results revealed that 45 out of 72 (62.5%) were games, 19 (26.4%) were non-games, 5 (6.9%) applications presented science experiments with low engagement of the user, 2 (2.8%) of the apps were in form of worksheets and 1 (1.4%) application was in the form of an encyclopedia. Finally, 71/72 (98.6%) were in English and 1 (1.4%) in Greek language.

Critical Categorization of Android and IOS Applications Available for STEAM

185

As in the case of the App Store, in this cases there is a number of applications which accompany external devices and material, such as Robotic devices. They were not included in this search, as they cannot be used in a standalone mode for teaching STEAM subjects. It is important to mention that there are applications of this type which are not available for Android Devices (e.g. OSMO).

5 Discussion The issues of the emerging STEAM Education field and that of mobile learning were discussed in this paper, focusing on their convergence; namely using mobile devices for STEAM approaches in ECE. The authors attempted to search for and attempt to categorize the available applications for the 2 main mobile platforms, namely Android and IOS. For that matter, an extensive search in the corresponding application databases was carried out. Regarding the IOS platform, the number of available applications is rather limited. Furthermore, the user rankings and the number of installations reported by the App Store indicate that there are still no popular STEAM applications for ECE in IOS. Of course there are applications which are used but were not included in this study, as they fail to meet all the set criteria. The OSMO application is a representative example, as it requires external material (tangram pieces and a special reflector which is positioned on an iPad’s camera). On the other hand, there are far many more applications available for the Android platform. Moreover, these are used more that the IOS applications, as it can be concluded by the number of downloads and the user grading participation. It is important to mention that Google Play provides the number of downloads but not the installations. One can download an application and never install it or install it in multiple devices. The number of downloads corresponds to the google accounts connected to the application [39]. One could argue that there are many more applications which can be considered as STEAM related for ECE. Indeed, one of the authors did a similar study for Android applications available for the same age group but in a national context (that of Greece) [39]. The difference is that those were related to mainly one disciplinary area, if any. The latter comment refers to the fact that many of them were game-like applications which were registered in Google Play as educational applications and that wasn’t always the case. Furthermore, most of them were related to Language or Mathematics. At this point, any attempt to critically categorize the findings seems to be vein. The reason is that the existing applications are of a rather simple and yet unclear nature. They treat two or more of the STEAM areas, but mainly coding applications (thus related to Technology) are available and most of them are of a playful nature. While the latter is not a disadvantage, considering the age group, it still requires support by an adult (usually the teacher, but also a parent) for successfully utilizing such applications. In [39] it was mentioned that in the literature one can find just a few application categorization attempts. Probably the most known one is that of Goodwin [8] who proposes 3 categories: Instructive, Manipulable and Constructive. This is based on usage experience and not in the disciplinary area or the content of the applications.

186

T. Bratitsis et al.

Furthermore, Goodwin claimed that teachers prefer to utilize constructive apps for young children, as the allow them to be more creative. In the case of the applications examined in this study, many of them were of a more instructive nature. This could also explain the low usage rated, especially for the IOS applications. There are other categorizations (e.g. see Ebner’s proposal [40]), but still such attempts seem inconclusive. This is still in line with the already stated claim [32, 39, 40] that principles for selecting appropriate applications and rubrics for their evaluation are still needed. Clarke & Abbot [42] highlight that this is more evident in ECE, as research in younger ages is still lacking. Concluding, there is still not a solid framework for categorizing and evaluating mobile applications for educational use, in the literature. This is more evident in STEAM contexts, as the approaches are rather new and still under-researched. The possibility of building up from user reported experiences and creating some short of database, which teachers and maybe parents can consult when searching for applications seems a way to follow. This is also highlighted by [8, 39] but is still far from being implemented. Towards this direction, collaboration among researchers seems necessary by sharing information, as there are many approaches that utilize mobile applications which are custom built for a specific research activity or ones that are more generic in nature (e.g. mathematics applications) which act as additional teaching material within wider interventions. Still, though, standalone STEAM applications are not evident in the two application databases and probably we are still too far from achieving such a condition. Thus, a next step, following this study would be the construction of a more solid categorization framework and the initiation of a data collection process from case studies, worldwide, in order to record the apps used in every case, along with the main educational outcomes. The “bigger picture” is the construction of a database which will be open and will support keyword based search to facilitate searches in the context of the one conducted in this study. The way the application platforms (Google Play and iTunes/AppStore) are operating at the moment increases the difficulty of such an attempt, but also highlights the need for it to be realized.

References 1. Morrison, J.: TIES STEM education monograph series, Attributes of STEM education. TIES, Baltimore (2006) 2. Morrison, J., Bartlett, B.: STEM as a curriculum: An experimental approach (2009) 3. Sanders, M.: STEM, STEM education, STEMmania. Technol. Teach. 68(4), 20–26 (2009) 4. Bybee, R.: Advancing STEM education: a 2020 vision. Technol. Eng. Teach. 70(1), 30–35 (2010) 5. Sherapan, H.: From STEM to STEAM: how early childhood educators can apply Fred Roger’s approach. Young Child. 67(1), 36–40 (2012) 6. Yelland, N.J.: New technologies, playful experiences, and multimodal learning. In: Berson, I.R., Berson, M.J. (eds.) High Tech Tots: Childhood in a Digital World, pp. 5–22. Information Age, Charlotte (2010) 7. Yelland, N.J., Gilbert, C.L.: iPlay, iLearn, iGrow (external link). A report for IBM. Victoria University, Melbourne (2013)

Critical Categorization of Android and IOS Applications Available for STEAM

187

8. Goodwin, K.: Use of tablet technology in the classroom. Curriculum and Learning Innovation Centre, NSW Department of Education and Communities, Strathfield (2012) 9. Prensky, M.: Digital natives, digital immigrants. Horizon 9(5), 1–6 (2001) 10. Marsh, J., Brooks, G., Hughes, J., Ritchie, L., Roberts, S., Wright, K.: Digital beginnings: young children’s use of popular culture, media and new technologies. University of Sheffield, Sheffield (2005) 11. National Research Council (NRC): Exploring the Intersection of Science Education and 21st Century Skills: A Workshop Summary. National Academies Press, Washington, DC (2010) 12. Stemtosteam: What is STEAM? (n.d.). www.stemtosteam.org 13. ASL (the Art of Science Learning) (n.d.). Art, Science, Innovation. http://www. artofsciencelearning.org/ 14. Sousa, D., Pilecki, T.: From STEM to STEAM: Integrating the Arts (2015) 15. Robelen, E.W.: STEAM: experts make case for adding arts to STEM. Educ. Week 31(13), 8 (2011) 16. Driver, R.: Children’s Ideas in Science (2000) 17. Jos, E.: The right question at the right time. In: Harlan, W. (ed.) Primary Science: Taking the Plunge. Heinemann, Oxford (1985) 18. Hunter, J.: STEM education: Kindergarten is where it should begin (2015). http://www.smh. com.au/comment/stem-education-kindergarten-is-where-it-should-begin-20150816-gj00p5. html 19. Elkin, M., Sullivan, A., Bers, M.U.: Implementing a robotics curriculum in an early childhood Montessori classroom. J. Inf. Technol. Educ. Innov. Pract. 13, 153–169 (2014) 20. Chesloff, J.D.: STEM education must start in early childhood. Educ. Week (edweek.org) (2013). www.edweek.org/ew/articles/2013/03/06/23chesloff.h32.html 21. Mantzicopoulos, P., Samarapungavan, A., Patrick, H.: We learn how to predict and be a scientist: early science experiences and kindergarten children’s social meanings about science. Cogn. Instr. 27(4), 312–369 (2009) 22. Havice, W.: The power and promise of a STEM education: thriving in a complex technological world. In: ITEEA (ed.) The Overlooked STEM Imperatives: Technology and Engineering, pp. 10–17. ITEEA, Reston (2009) 23. Jamil, F., Linder, S., Stegelin, D.: Early childhood teacher beliefs about STEAM education after a professional development conference. Early Child. Educ. J. 46, 409–417 (2017) 24. Parette, H., Quesenberry, A., Blum, C.: Missing the boat with technology usage in early childhood settings: a 21st century view of developmentally appropriate practice. Early Child. Educ. J. 37, 335–343 (2010) 25. Terras, M., Ramsay, J.: The five central psychological challenges facing effective mobile learning. Br. J. Edu. Technol. 43(5), 20–832 (2012) 26. Sharples, M., Taylor, J., Vavoula, G.: A theory of learning for the mobile age. In: Andrews, R., Haythornthwaite, C. (eds.) The Sage Handbook of E-Learning Research, pp. 221–247. Sage, London (2007) 27. Newmann, M., Newmann, D.: Touch screen tablets and emergent literacy. Early Child. Educ. J. 42, 231–239 (2014) 28. Orlando, J.: How young is too young? Mobile technologies and young children August 21, 2011 Posted by Editor21C in Directions in Education, Early Childhood Education, Engaging Learning Environments. University of Western Sydney (2011). http://learning21c.wordpress. com/2011/08 29. Plowman, L., Stevenson, O., Stephen, C., McPake, J.: Preschool children’s learning with technology at home. Comput. Educ. 59(1), 30–37 (2012) 30. Lindahl, G., Folkesson, A.: ICT in preschool: friend or foe? The significance of norms in a changing practice. Int. J. Early Years Educ. 20, 422–436 (2012)

188

T. Bratitsis et al.

31. Egan, M., Hengst, R.: Software on demand: an early child-hood numeracy partnership. Contemp. Issues Technol. Teach. Educ. 12, 328–342 (2012) 32. Sahin, M.C., Tas, I., Ogul, I.G., Cilingir, E., Keles, O.: Literature review on the use of tablet computers in preschool education. Eur. J. Res. Soc. Stud. 1(Special Issue 1), 80–83 (2014) 33. Kjällander, S., Moinan, F.: Digital tablets and applications in preschool – preschoolers creative transformation of didactic design. Des. Learn. 7(1), 10–33 (2014) 34. Zaranis, N., Kalogiannakis, M., Papadakis, S.: Using mobile devices for teaching realistic mathematics in kindergarten education. Creative Educ. 4(7A1), 1–10 (2013) 35. Spencer, P.: iPads: improving numeracy learning in the early years. In: Steinle, V., Ball, L., Bardini, C. (eds.) Mathematics Education: Yesterday, Today and Tomorrow (Proceedings of the 36th Annual Conference of the Mathematics Education Research Group of Australasia). MERGA, Melbourne (2013) 36. Chiong, C., Shuler, C.: Learning: is there an app for that? Investigations of young children’s usage and learning with mobile devices and apps. The Joan Ganz Cooney Center at Sesame Workshop, New York (2010) 37. Beschorner, B., Hutchison, A.: iPads as a literacy teaching tool in early childhood. Int. J. Educ. Math. Sci. Technol. 1(1), 16–24 (2013) 38. Bratitsis, T., Bardanika, P., Ioannou, M.: Science education and augmented reality content: the case of the water circle. In: 17th IEEE International Conference on Advanced Learning Technologies - ICALT 2017, pp. 311–312. IEEE (2017) 39. Ebner, M.: Mobile applications for math education – how should they be done? In: Crompton, H., Traxler, J. (eds.) Mobile Learning and Mathematics. Foundations, Design, and Case Studies, pp. 20–32. Routledge, New York, London (2015) 40. Clarke, L., Abbot, L.: Young pupils’, their teacher’s and classroom assistants’ experiences of iPads in a Northern Ireland school: “four and five years old, who would have thought they could do that?”. Br. J. Edu. Technol. 47(6), 1051–1064 (2015)

Museum Exhibits that Interact with Pupils’ Mobile Devices. The Case of Hellenic Maritime Museum Dimitris Rammos and Tharrenos Bratitsis(&) University of Western Macedonia, Kozani, Greece [email protected], [email protected]

Abstract. Educational visits to museums are beneficial for pupils in terms of differentiated teaching and cognitive objectives accomplishment. Pupils’ direct contact with historical and cultural evidence activates visual perception and observation skills. Moreover, when activities during such visits are gamified, learning process becomes more attractive. Therefore, pupils’ interest is increased. Equally important is the active involvement of children in educational activities’ preparation. Tutorials production for peers promotes motivation and participation willingness. Active involvement opposes the discouraging impression of young pupils, who often think themselves as passive receivers of obscure and uninspiring information during educational museums visits. In recent years, interactive educational activities in museums are developed, based on the utilization of Information and Communications Technology (ICT), such as playful applications for pupils’ mobile devices. The development of an interactive Augmented Reality (AR) playful activity in Hellenic Maritime Museum (HMM) is presented in this article. A small scale-study was conducted to measure the cognitive and attitude benefits for pupils after the completion of the study. The results are briefly presented in the article. Keywords: Digital storytelling teaching


Interactive museum exhibits
Differentiated

1 Introduction During the year, schools organise educational visits to destinations of historical and cultural interest, such as museums. Students like these visits since school routine is differentiated and the external learning environment is changed [1]. Transferring pupils and learning in places other than the school offers attractiveness in relation to traditional teaching [2]. On the other hand, educational visits must be linked to the cognitive evolution of pupils and the achievement of specific learning objectives [3]. A specific research strategy should be designed by teachers in order to check the attainment of these objectives [4]. This strategy depends on the age of the students, the type of educational visit and the overall teaching plan [2]. Interest and participation of pupils during educational visits is also increased through the production and utilization of educational material by pupils, for pupils [5]. After the material is checked for its validity and value, it can be utilized in organising © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 189–201, 2021. https://doi.org/10.1007/978-3-030-49932-7_19

190

D. Rammos and T. Bratitsis

playful activities inside museum premises. In this case, preparatory sessions must be carried out in the classroom. Educational material for peers can be digitally formatted through ICT [6]. Cutting edge technology applications downloaded into mobile electronic devices are already being increasingly exploited in museum educational programs [7]. This article follows this content structure. Initially the theoretical background of the study is presented. Then, the research methodology and the implementation stages of the educational activity are described. Finally, the research results and the relevant discussion are presented.

2 Background of the Study 2.1

Linking School Context with Activities Outside the Classroom

Modern teaching methodology must be characterized by innovation and originality [8]. Pupils, especially in elementary education, become more engaged when teaching includes a variety of methods that activate their imagination and promote creation [9]. A form of differentiated teaching is educational activities outside school, away from the usual classroom environment [6, 8]. These activities differ from simple visits to cultural and sports venues [1]. They concern targeted utilization of an outdoor space or organisation for educational reasons. Thus, they primarily do not aim at entertaining but at conducting teaching activities with alternative character [9]. So, the duration of these visits is relatively short, with a specific timetable and scheduled activities [6]. The destination choice depends on many factors. Initially, the geographic location of the school and the ability of organising visits [10]. Then, on pupils’ age and their capacity to participate in such activities [8]. Finally, on the nature of the teaching session and its content relativity with the outdoor area [10]. Museums and cultural organisations are seemingly more suited for organizing a teaching visit, especially when it comes to teaching History and Art [11]. However, any school subject can be combined with outdoor learning [9]. The external environment stimuli may trigger the observation capacity of pupils [11]. It can also make them feel free and disconnect learning from the usual classroom routine [6]. The organisation of outdoor teaching activities must be done very carefully regarding learning objectives, duration, the material to be used and the prediction of possible difficulties [5]. 2.2

Educational Visits in Museums – Active Learning Through Contact with Exhibits

The value of museum educational visits is known and widespread at an international level [11]. It is part of the general perception that contact with works of art benefits pupils in gaining knowledge and developing skills such as critical thinking and artistic perception [6]. Museums promote interaction with authentic objects which are an important learning tool [12]. Children construct knowledge about their environment, primarily when interacting with objects [6]. These trigger their senses, cause interest, enhance

Museum Exhibits that Interact with Pupils’ Mobile Devices

191

experimentation and encourage participation in the learning process [7, 12]. Children are strangely congenital and exhibit spontaneous exploratory behaviour when inspired by an educational environment [9]. Experiential and active learning through museum exhibits interaction is an educational need, since children understand the real world that surrounds them [6, 7, 12]. Museum exhibits distinguish four educational values that are beneficial for students: • Materiality: Their material dimension supports learning process through personal experience and perception of the senses [11]. • Authenticity: Having been salvaged from the past, they create tangible links to it [9]. Thus, they trigger imaginary journeys in space and time, as well as stimuli for historical experiences and personal interpretations [6]. • Openness: They are characterized by a wide variety of information and associated narratives [11]. Thus, interdisciplinary teaching approaches and flexible “open” learning processes can be developed [4]. A variety of differentiated educational objectives related to historical learning, natural sciences, technology, language, cultural and aesthetics education, can be promoted through organised activities [8]. • Aesthetics: Their aesthetic value is related to the instant interaction with morphological, colour and shape characteristics and the three-dimensional form in real environment [9, 11]. Teaching approaches may promote exhibits’ comparison as for aesthetic characteristics or study as for the social influences of the artists [8]. Therefore, learning can no longer be a process of transferring information [6], but one of knowledge building, involving a number of cognitive processes. Such processes closely related with educational museum visits are: a) the motivation to learn and gain knowledge during the visit [1], b) the interest in the museum type and the exhibition subject [12], c) the building of learning objectives concerning the visit activities [2], d) the desire to approach and utilize specific exhibits [6], e) the adaption of appropriate cognitive strategies to link educational activities with prior knowledge [5]. 2.3

Interaction with Museum Exhibits Through the Use of ICT

There is a number of studies that attempt to record the efforts to utilize digital technologies to support learning in museums and cultural sites [13–18]. The digital applications and media used in educational museum visits differ in their origin and developer [12]. In this study, a teaching activity developed by the educator is presented. Activities developed by school or academic staff usually use open source and free digital applications in contrary to ICT educational services developed by museums that are mainly professional and commercial oriented. However, in both cases, the activation of innovative leaning principles such as participation, collaboration, narration, personalization and adaptation, is being attempted [17]. In school-developed activities, mobile electronic devices during a museum visit are often used to record pupils’ digital storytelling while digital treasure hunt games are organised in museum premises [6]. Treasure hunt games may utilize existing digital material developed by the museum or school-developed material embedded in museum exhibits thanks to Augmented Reality applications [6, 7, 13].

192

2.4

D. Rammos and T. Bratitsis

Authentic and Innovative Tutorials Developed by Pupils for Pupils

Traditional educational practices mainly follow strategies which lean on school books with ready-to-use material and activities [2]. In differentiated teaching, educators provide pupils with links in order to search for material and decide the sources to use for developing their own tutorials [7, 13]. The latter are presented and discussed in the classroom. Moreover, they are often utilized in teaching by educators. This approach increases active participation and pupils’ interest for the teaching process [4]. Interest increase promotes lateral thinking which guarantees ingenuity and innovation [9]. Tutorials are mainly produced through pupils’ team work in the classroom [5]. Thus, collaboration and communication skills are developed [19]. The stories presented in tutorials may have oral, written or digital form and may be accompanied by audiovisual material. In addition, digital skills are also developed [6]. More significantly, authentic tutorials produced by pupils gives them the opportunity to take initiatives and form the educational process themselves [5, 19]. Information validity checking and greater expressiveness are only two of pupils’ main priorities during the tutorial production [19]. Responsibility and diligence skills are, therefore, promoted [8]. Tutorials’ content may vary according to the School Subject or the teaching activity in which they correspond to [5]. In case of educational visits preparation, pupils’ digital tutorials and stories may be used as accompanying and explanatory material for peers [19].

3 Presentation of Educational Activity and Research Study During Hellenic Maritime Museum Visit 3.1

Hellenic Maritime Museum Selection

Hellenic Maritime Museum (HMM) was founded in 1949 when an association memorandum called “Maritime Museum Society and collection of national relics at sea” was signed between citizens of Piraeus and the Government. The museum exhibits, more than 2500, are related to Hellenic maritime History. They include many ship models and a number of paintings from 20th century. The 3d ship models from Hellenic Revolution against Turks and a great number of other exhibits related to History School Subject were the main reason for selecting HMM as this educational visit’s destination. Moreover, the HMM website hosts significant digital information and educational material, which was very helpful in terms of classroom pre-visit preparation. The visit was organised in coordination with the HMM personnel both during teacher preparatory visits and two-hour pupils’ visit and activity in the museum premises. 3.2

General Description of the Educational Activity in HMM Premises

The main concept of the HMM educational activity was about discovering tutorials made by peers during classroom work which were hidden in museum exhibits using Augmented Reality Technology. Linking museum exhibits with pupils’ digital tutorials

Museum Exhibits that Interact with Pupils’ Mobile Devices

193

was done in classroom entering exhibits’ photos in the ‘Augment’ application. Each team had chosen a couple of museum exhibits to ‘hide’ their stories in. This choice was only known by team members and the teacher. In addition, secret interactive museum exhibits were developed before the educational visit in HMM. After discovering the ‘enriched’ exhibits, pupil-teams would hear the tutorials through their devices and complete a worksheet provided to them. Six 4-member teams with two mobile devices each were assigned to deliver the completed worksheets to the Director of the Museum who was supposed to award the best team with a mystery prize. A 30-min guided tour in the museum areas was carried out before the playful activity. After that, 1.5 h was given to all teams in order to return their worksheet completed. Pupils had already been informed about the activity rules and guidelines of the whole educational museum visit at a preparatory meeting in school. All twelve mobile devices belong to school. Thus, they were promptly checked as far as the battery level and the operational use of the required applications. Wireless Internet connection was available in all museum areas. 3.3

General Purpose of the Study

The purpose of this study was threefold. Regarding knowledge, the aim was to check pupils’ performance on historical meanings after participating in the playful HMM activity. As for digital skills, the aim was to monitor the effectiveness of cutting-edge technology utilization in producing tutorials for peers and formulating a playful activity using AR applications. Also, the evaluation of the mobile devices’ use by pupils’ during the activity. On the attitudes change level, the question was whether this teaching activity contributed to the improvement of pupils’ attitude towards visiting museums. Cultural and History exhibits conceal and combine a lot of information presented in the school curriculum and pupils should be able to discover this connection [8]. All research considerations of this study highlight the general notion that differentiated teaching practices should be clearly oriented to the greatest possible pupils’ involvement [4, 8]. Their interests and abilities should be taken into account in the design of inclusive teaching activities [9]. The utilization of mobile devices during such activities is identified as one of their most obvious, contemporary interests [7]. 3.4

Research Plan

In order to evaluate pupils’ performance in terms of historical knowledge presented in tutorials, a small-scale, qualitative ethnographical research was designed. The development of collaboration and communication skills were also examined. Systematic observation (SO) during activity implementation as well as a testing worksheet at the end were selected for data collection. The former was chosen since the researcher was the classroom teacher of the sample group for the whole school year. Therefore, recording and understanding of performance and attitude changes of pupils during the playful activity were much easier. On the other hand, pupils’ performance was more accurately reflected by the use of a testing worksheet since it allowed teacher to make

194

D. Rammos and T. Bratitsis

comparisons with previous data and draw safe conclusions. Digital literacy skills improvement was also monitored trough SO and the completion of personalised rubrics. All pupils’ involvement with digital sources, devices or applications were recorded in detail. Those rubrics’ data was examined both individually and as a result of team work. Results, though are only presented individually. Furthermore, a set of semi-structured interviews with pupils’ teams was conducted at the end of the activity so as to evaluate their attitude change towards educational visits in museums (Fig. 1). Team interviews were preferred since individual ones often discourage young pupils from expressing themselves freely.

Fig. 1. HMM playful activity implementation stages

A short questionnaire with three closed and one open question (Fig. 2) was distributed to the parents of the sample group. It also aimed at monitoring pupils’ attitude change towards museum visits. Closed questions examined if any kind of experience gained from the playful activity was transferred at home while the open question asked parents to identify possible children’s attitude change after the visit in HMM, if any.

Fig. 2. HMM playful activity implementation stages

The population of the study consisted of 24 students, 11–12 years old. The duration was 3 weeks, including eight 45-min sessions and a daily educational visit in total. 3.5

Stages of the Educational Activity

The playful educational activity presented in this article was carried out in three main stages in terms of development, implementation and evaluation. Each one contained sub-stages as shown in Fig. 3.

Museum Exhibits that Interact with Pupils’ Mobile Devices

195

Fig. 3. HMM playful activity implementation stages

In the development stage the teacher collected all printed and digital material for the activity, including high resolution photographs of the HMM main exhibits. Also, a visit to the museum was also organized to inform museum personnel about the school project. Mobile devices were also prepared, as for the installation of required applications. After the preparatory session, the forthcoming educational activity was presented in classroom. Pupils were informed about its general characteristics and purpose, in detail. Teams were also formed and asked to organise their future work. In the implementation stage, pupils’ teams worked autonomously to produce digital tutorials. Initially, their work included detailed searching of teacher-provided sources in printed and digital form. Access to their mobile devices was also provided, both for searching more information in the HMM website and to begin their tutorial recordings. After the tutorial production, Augment application was used to enrich photos of the selected exhibits with the corresponding recordings. This procedure enabled pupils to discover the hidden stories with their mobile devices during the HMM playful activity. Furthermore, after this phase, the accompanying worksheet of the playful activity was prepared by the teacher. It contained mystery questions and blanks-filling exercises. The responses for them were included in the hidden tutorials (augmentation videos). After completing the playful activity development, the educational visit in HMM was organised. Pupils’ teams were first tour-guided in the museum and then allowed to move and work autonomously for the activity implementation (Fig. 4). In all the activity stages, SO was conducted by the teacher following the research methodology. Indicative snapshot photo and video shooting together with detailed notes keeping was continuously carried out. The evaluation stage included the measurement of the gained knowledge after the HMM playful activity, as well as the estimation of skills developed. More specifically, the testing worksheet results were compared to previous data records, after relevant teaching activities of the History school subject. Furthermore, pupils’ collaboratively and communicative willingness during this activity was compared to the average

196

D. Rammos and T. Bratitsis

Fig. 4. Pupils working as groups with the augmented exhibits and the worksheets

attitude during team work. Video recordings and pupils’ own evaluative descriptions about their team’s performance were taken into consideration in this comparison. Finally, short team interviews were carried out and questionnaires were distributed to parents, so as to record pupils’ possible attitude change towards educational visits in museums. All data was gathered and analysed at the end of the evaluation stage.

4 Research Findings 4.1

Level of History Knowledge Gained

The results of the testing worksheet showed that the majority of the 24 pupils performed better than their average scores. As shown in Fig. 5, although in most cases no impressive changes are applied, pupils proved to gain and absorb historical knowledge easily through the gamified teaching methodology.

Fig. 5. Comparison of history testing records in this study and average History performance

Testing worksheet results were crosschecked and completed with relevant teacher notes through systematic observation. Significantly remarkable was the critical analysis and synthesis of information during tutorial narration. Critical thinking was promptly connected with better lessons understanding and higher performance.

Museum Exhibits that Interact with Pupils’ Mobile Devices

197

Although testing didn’t follow the traditional standard operating procedure, pupils performed great after a totally different teaching plan. Methodologically, it was the first time that pupils’ tests were based on tutorials, processed and produced themselves. 4.2

Collaboration and Communication Level During the HMM Activity

Greater employment of communication and collaboration skills was also recorded through SO. The level of team members’ communication was graded 9.3 (following the ordinary grading approach carried out in Greek Primary School) while the average grade level during the school year was 7.4 (Fig. 6). This practically means that teams’ initial objectives as set during the organisation stage were accomplished to a very satisfactory extent thanks to equal participation and common action.

Fig. 6. Communication and collaboration level in HMM activity

Collaboration skills were also graded with a 7,5 while average score was 6.4. The successful production of tutorials together with AR application utilization demanded quality team work, especially in difficulties’ resolution. Also, team collaboration was tested during the HMM playful activity prosecution where quickness and accurate cooperation between all members was required and integrated in the overall grading. 4.3

Digital Literacy Skills

ICT contribution was crucial in the described educational activity. One of the study’s research questions was whether digital literacy skills were developed to pupils. The pupils’ capacity to use mobile devices, digital tutorial production and familiarity with AR application use were mainly examined. Records’ keeping and performance grading was based on teacher’s SO throughout both autonomous team work in the classroom and the activity implementation during the HMM visit. A total grade on the scale of 10 came out after linking individual performance and team collaboration. Different grades for each pupil were applied in all three skill sub categories. Content quality and technical characteristics of tutorials were also taken into consideration, while evaluating skills improvement. A comparison between present and average pupils’ records showed a slight improvement, as mobile devices’ use was

198

D. Rammos and T. Bratitsis

concerned. A more notable improvement was recorded both to tutorial production and AR application utilization skills. As shown in Fig. 7, most pupils already performed high in activities involving ICT use. In addition, digital skills improvement seemed more evident in pupils with lower average performance.

Fig. 7. Digital literacy skills development

4.4

Change of Attitudes Towards Museum Educational Visits

A very positive evaluation of the HMM playful activity emerged form the team interviews results’ analysis. As shown in questions one, three and four records (Fig. 1 and Table 1), pupils’ satisfaction was clearly expressed. A more receptive and positive attitude of pupils can be concluded after a comparison with relevant short interviews’ results after educational museum visits in the past. Table 1. Team interviews results Question Team 1 1 Totally

2

3 4 5

Surprising Curious Interesting

2 More than expected

Mindblowing Interesting Unusual 10 10 Definitely We will do it right now Educational Inspiring Entertaining Entertaining

3 Definitely

4 Totally

5 For sure

Gamifying Pleasant Interesting

Surprising Pleasant Useful

6 Better than higher expectations Interesting Amazing Educational Interesting Gamifying Gamifying

10 Already have Educational Entertaining

10 Of course

10 Definitely

10 Definitely

Educational Entertaining Entertaining Entertaining Aesthetic Inspiring

Museum Exhibits that Interact with Pupils’ Mobile Devices

199

The entertainment level during the activity seemed to be the main criterion for noticing attitude change, as proven by question 5 answers (Table 1). Equally important though, is the fact that half teams highlighted the educational character of the activity. Pupils’ positive attitude change towards museum visits also emerged from parents’ questionnaires analysis. Educational visits are mostly discussed at home when pupils’ interest was high. HMM visit activated pupils’ interest as indicated by both parents’ answers (Fig. 8) and descriptions in the open-ended question (Fig. 2). The latter brought out the parents’ surprise, that their children shared with them detailed information about the activity and they expressed their willing to visit the HMM again, very strongly.

Fig. 8. Parents’ questionnaire results

5 Conclusions In line with previous research, this study highlighted the beneficial role of mobile devices use during educational visits in museums. The increase of pupils’ interest and motivation are some of the most remarkable, common findings. Similarly, pupils’ knowledge performance was also reinforced through educational ICT use in museums as shown in the corresponding evaluation. In the case of the HMM educational visit, pupils’ History Knowledge was increased through exhibits examination. This study has also demonstrated pupils’ contribution in designing an interactive playful activity for peers within a museum’s premises. In producing their own digital tutorials based on History information, pupils critically processed historical sources. Since tutorials were embedded to real museum exhibits using AR in order to develop the core activity content, the activation, engagement and positive attitude of pupils were importantly increased. More importantly, the study highlighted that capability of students of these ages to design and perform activities of this kind, thus significantly contributing to differentiated teaching approaches’ design. Of course, there are limitations in this study, as it was of a small scale and restricted to only one class and one museum. On the other hand, the time needed to organize the material, the visit and all the activities was rather demanding and thus prevented the researchers from incorporating additional participants and/or visiting sites. Moreover, multimedia material like the ones created by the pupils in this study are easily reusable,

200

D. Rammos and T. Bratitsis

something that counterbalances the required effort when occurring for the first time. Also, enrichment of the material, both in matters of worksheets and multimedia augmentations is easier in the future. What was obvious though, was that the combination of authentic teaching material production by pupils with the development of gamified school visits is museums forms a remarkable promising educational process which must object to further research.

References 1. Bossavit, B., Pina, A., Sanchez-Gil, I., Urtasun, A.: Educational games to enhance museum visits for schools. J. Educ. Technol. Soc. 21(4), 171–186 (2018) 2. Chapman, J.T.: Teaching outside to engage student learning (2017) 3. Anderson, D., Piscitelli, B., Weier, K., Everett, M., Tayler, C.: Children’s museum experiences: identifying powerful mediators of learning. Curator Mus. J. 45(3), 213–231 (2002) 4. Vallance, E.: Museum education as curriculum: four models, leading to a fifth. Stud. Art Educ. 45(4), 343–358 (2004) 5. Lockwood, F.: The Design and Production of Self-instructional Materials. Routledge, London (2018) 6. Rammos, D., Bratitsis, T.: Inclusive strategies for the history subject in 6th grade of Greek Primary School: gamifying the curriculum with digital storytelling and augmented reality. In: Proceedings of the 8th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-Exclusion. ACM (2018) 7. Maragos, K., Grigoriadou, M.: Towards the design of intelligent educational gaming systems. In: Proceedings of the AIED05 WORKSHOP5: Educational Games as Intelligent Learning Environments, pp. 35–38 (2005) 8. Van Laar, E., Van Deursen, A.J., Van Dijk, J.A., De Haan, J.: The relation between 21stcentury skills and digital skills: a systematic literature review. Comput. Hum. Behav. 72, 577–588 (2017) 9. Chalkiadaki, A.: A systematic literature review of 21st century skills and competencies in primary education. Int. J. Instr. 11(3), 1–16 (2018) 10. Castle, M.C.: Interpreters, docents and educators, ways of knowing, ways of teaching in a history museum, an art gallery, and a nature centre. Doctoral dissertation, National Library of Canada (2001) 11. Demircioglu, I.H.: History teachers’ attitudes towards museum visits: a Turkish perspective. Teach. Hist. 41(3), 24–30 (2007) 12. Schwartz, J.P.: Object lessons: teaching multiliteracies through the museum. Coll. English 71(1), 27–47 (2008) 13. Bouta, H., Retalis, S., Paraskeva, F.: Utilising a collaborative macro-script to enhance student engagement: a mixed method study in a 3D virtual environment. Comput. Educ. 58 (1), 501–517 (2012) 14. Bickmore, T., Pfeifer, L., Schulman, D.: Relational agents improve engagement and learning in science museum visitors. In: Vilhjálmsson, H.H., Kopp, S., Marsella, S., Thόrisson, K.R. (eds.) Intelligent Virtual Agents, pp. 55–67. Springer, Heidelberg (2011) 15. Michael, D., Pelekanos, N., Chrysanthou, I., Zaharias, P., Hadjigavriel, L.L., Chrysanthou, Y.: Comparative study of interactive systems in a museum. In: Ioannides, M., Fellner, D., Georgopoulos, A., Hadjimitsis, D.G. (eds.) EuroMed 2010. LNCS, vol. 6436, pp. 250–261. Springer, Heidelberg (2010)

Museum Exhibits that Interact with Pupils’ Mobile Devices

201

16. Pierroux, P., Kaptelinin, V., Hall, T., Walker, K., Bannon, L., Stuedahl, D.: MUSTEL: framing the design of technology-enhanced learning activities for museum visitors. In: ICHIM 2007, International Cultural Heritage Informatics Meeting, Toronto (2007) 17. Tallon, L., Walker, K.: Digital Technologies and the Museum Experience: Handheld Guides and Other Media. AltaMira Press, Lanham (2008) 18. Falk, J.H., Dierking, L.: Enhancing visitor interaction and learning with mobile technologies. In: Tallon, L., Walker, K. (eds.) Digital Technologies and the Museum Experience: Handheld Devices and Other Media, pp. 19–34. Altamira Press, Lanham (2008) 19. Breyer, Y.: Learning and teaching with corpora: reflections by student teachers. Comput. Assist. Lang. Learn. 22(2), 153–172 (2009) 20. Bratitsis, T., Bardanika, P., Ioannou, M.: Science education and augmented reality content: the case of the water circle. In: Proceedings of the 17th IEEE International Conference on Advanced Learning Technologies (ICALT 2017), pp. 451–453 (2017)

Can Elementary Students Co-design the Learning Content of Educational Apps: The We!Design!Fractions Participatory Design Approach George Palaigeorgiou(&) and Vasiliki Sidiropoulou University of Western Macedonia, Florina, Greece [email protected], [email protected]

Abstract. Although there is a growing interest in engaging children in the design of interactive learning technology, young learners are mostly asked to codesign features and apps that lie on the peripheral of the learning processes, such as websites, e-portfolios or assessment applications. In this article, we will try to investigate whether experienced elementary students can offer useful insights about the design of the content of an educational app, its learning representations and its learning interactions. For this purpose, we have developed a collaborative board design game called We!Design!Fractions, for designing apps about fractions. Fractions constitute a demanding learning domain in elementary education and, hence, an appropriate testbed for our research aims. The design game consists of a game board, cards, dice and pawns and it was developed by exploiting theories about board games, collaborative design games and idea generation. A study was conducted with 27 6th grade students who participated in 7 design sessions of three to four students. Each session lasted for about 3 h. By playing the game, students proposed a total of 199 ideas about the educational app that were later evaluated by themselves regarding their learning usefulness and their innovation. Students claimed that the board game stimulated their idea generation process, offered an enjoyable experience and that their proposals could enable designers develop an educational application about fractions that would fulfil better their needs and preferences. Keywords: Participatory design


Interaction design with children
Fractions

1 Introduction During the last two decades, participatory design with children has gained wide acceptance as a design process that can lead to new technologies that correspond to the needs, interests and potential of today’s independent, interactive and information active children [1]. Many studies have been conducted, in which students are co-designers of their own educational tools, such as e.g. an educational newspaper [2], a digital bag (eBag) for storing images, videos, music or documents [3], a school website [4] or a digital badge system for use in an after-school science program [5]. Students’ participation however, has not been confined only to the school context and students have © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 202–214, 2021. https://doi.org/10.1007/978-3-030-49932-7_20

Can Elementary Students Co-design the Learning Content of Educational Apps

203

become co-designers of educational tools such as e.g. a three dimensional interactive book for a museum exhibition [6], an educational website about art conservation in a museum [7], digital interactive installations in a museum exhibition [8], or even “positive” computing technology [9]. By taking a closer look into all previous design products, we can conclude that children do have a voice in the design of their own technologies, but the participatory design products are mainly related to the periphery of the learning processes and are less concerned with the interactions with the learning content. In this study, we attempt to explore how to co-design with children the learning interactions and representations of an educational app. In particular, we present and evaluate a participatory design approach for enabling elementary students to design a tablet-based educational app about fractions.

2 Learning About Fractions In order to support students’ idea generation about a specific learning domain, it is of imperative importance to exploit and embed the well-known related learning difficulties into the design cycle. In that way, student-designers will orient themselves more towards generating new insights on the most demanding issues rather than on rediscovering interaction alternatives for the most trivial topics. We selected the subject matter of fractions because it is challenging for elementary students with complex conceptual content. Fraction understanding includes the facts that fractions represent a part of an object or parts of a set of objects, they can be represented by fractional symbols, and that they are numbers that reflect numerical magnitudes [10]. Students often meet difficulties, as they cannot overcome the belief that the whole number properties are not applicable for all numbers in all cases and, even in high-school, many students are unable to understand that there are infinite numbers between any two fractions [11]. The role of learning representations for understanding mathematic concepts is fundamental, whereas several studies focus on the development of fractions representational systems to enhance learning within digital environments. For example, Morgan et al. [12], presented two microworlds that offer novel fractions representations to support students’ learning, whereas Rau et al. [13] presented Fractions Tutor, an intelligent tutoring system for fractions that uses multiple, interactive graphical representations.

3 The We!Design!Fractions Game In order to extract students views and perspectives about fractions learning in a tablet app, we developed a design game based on We!Design&Play framework [14]. We! Design&Play provides guidance on how to create a board, cards, rules and props so as to develop collaborative board games for the design of educational applications. The theoretical background of We!Design&Play derives from idea generation theory and design games. Idea generation theory aims at helping participants to produce a rich

204

G. Palaigeorgiou and V. Sidiropoulou

pool of ideas while it underlines and addresses well-known social influences on idea generation such as production blocking, social loafing, evaluation apprehension, the attention/incubation hypotheses and the cognitive inertia [15–19]. Design games try to make the exploration of the design space playful and fruitful, they aim at establishing a design environment which is familiar and relaxed and balance the power relations between participants while providing a shared language for eliciting participants’ knowledge and perspectives [20–23]. We!Design&Play proposes three perspectives on pursuing the exploration of the design space: a convergence perspective, which helps to elicit needs and solutions about the design problem with an introspection into participant’s personal past experiences; a divergent perspective, which aims at a creative and imaginary exploration of new, unconventional ideas through the combination of incompatible stimuli; a contextual perspective which aims at the exploration of ideas derived from the everyday activity context of the participants. We!Design&Play have two inefficiencies in regards to the design of an educational app for familiarizing with fractions: a) it is not supposed to support the design of the interactions and representations for a specific learning domain but to address general learning needs and affordances, b) it has not been applied with primary school students but only with undergraduates. In our view, the design exploration perspectives should be organically integrated with the types of the design results aimed for, while the props, the cards and the rules should be adjusted according to the age of the participants. Hence, the participatory design game We!Design!Fractions, aims at eliciting: • interactive learning representations about fractions that are convenient to students, • interactive learning sequences about fractions which enhance the efficiency of their learning, • examples and contexts of use that are authentic to them. A significant prerequisite of We!Design!Fractions, is that the participants must have explicit experience in the learning domain in order to reveal and introspect into the ways they mastered it. By conducting several design sessions with different experienced students, a final list of needs and ideas should be produced which will enable the designers to propose an educational app for fractions. The We!Design!Fractions consists of a game board on which students move around their pawns, design alter egos, dices, colorful post-it notes and idea cards which provide stimuli for creating ideas. The Board Game The game board consists of 22 positions that correspond to each one of the 22 game cards, depicting the card’s category and provided stimuli. The Design Alter Ego Before the game begins, there is in introductory phase where the coordinators clarify the design process’ objectives and ask the students to create a ‘fictional character’ so as to be their communicative agent throughout the design process [14, 24]. The “design alter ego” liberates students from the fear of straightforward exposing themselves and enhances their introspection and creativity. Each student fills up a ‘game character’ form in which they create a persona by giving her a face, a name, personality traits, and

Can Elementary Students Co-design the Learning Content of Educational Apps

205

psychological and sociological attributes. From the moment the students create their design alter ego, they play-at its role during the design session (Fig. 1).

Fig. 1. The board game and a photo from a design session

The Game Cards Game cards are the basic tools offering stimuli for the exploration of students’ original ideas on learning about fractions. Card prompts were developed based on the three design exploration perspectives described previously and on a combination of several idea generation techniques such as super heroes [25], random stimulus [26], break the rules [27], future workshops [28]. The game cards can be organized into the following categories: The Convergent Perspective Cards Learning experience (1 card): the card aims at provoking students to introspect to their prior learning experience with fractions. Design Alter Ego experience (1 card): students have to think about how their design alter ego would like to learn and practice about fractions. Collaborative Playing experience (2 cards): the cards aim to elicit students’ views on how they could learn together with their colleagues while playing on a tablet. Domain learning difficulties (6 cards): the cards ask students to propose learning interventions for overcoming typical learning difficulties about fractions The Divergent Perspective Cards The future (2 cards): students have to explore learning representations and interactions on an imaginary future technological landscape. Breaking the rules (2 cards): the cards aim at investigating learning ideas that defy logical thought, rules or standards. My super hero (2 cards): students have to describe how their super hero’s powers would take advantage of fractions in his/her landscape.

206

G. Palaigeorgiou and V. Sidiropoulou

The Contextual Perspective Cards Place/Space (1 card): the card aims at eliciting students’ projections of fractions on their personal space and their daily activities. Time (3 cards): the cards provoke students to think about practical applications of fractions in their daily routine in three different time periods of a day. Verbs (2 cards): the cards provide verbs as stimuli for producing authentic experiences with fractions in the context of the imaginary app.

Fig. 2. Sample cards

The Rules of the Game Each player has his own post-it notes, colored differently from the other players, and his own dice and pawn. Additionally, coordinators give a switched off tablet to each player as a visual prompt for conceptualizing their ideas on the screen. In every round of the game, players throw their own dice and move around their pawn on the corresponding board position. Each player reads the related card and has about 10 min to think and write down as many ideas as possible on his post-it notes. Immediately after, children are asked to present their ideas to the team, one at a time and their colleagues are asked to discuss, extend or argue on them. Before the next round, each player sticks his ideas on the corresponding board position.

4 Methodology This study aims at evaluating the participatory board game We!Design!Fractions, in order to answer the following questions: • Can elementary students co-design the learning representations and interactions of an educational application? • Can the We!Design!Fractions game provoke students’ satisfaction, creativity and productivity? • Are there new perspectives from students’ suggestions on learning about fractions on a tablet?

Can Elementary Students Co-design the Learning Content of Educational Apps

207

Twenty-seven 6th grade students (19 girls and 8 boys) participated in the study, selected from 7 different schools of a provincial city in Greece, fulfilling the requirement for mastery in fractions. Their experience on fractions was confirmed by their teachers and by a pre-test on fractions completed exactly before the design sessions. We conducted 7 participatory design sessions with the students which lasted approximately 3 h. Three to four students and two experienced coordinators in participatory design joined each design session. At the end of the sessions, the students were presented with the final list of ideas generated, and were asked to evaluate them in terms of their perceived usefulness and originality on a 5-point Likert scale. Students also completed a questionnaire for assessing the following aspects: the design process as a whole (3 questions), the final products (2 questions), their experience with the design game (4 questions), their freedom on expressing their thoughts (1 question), their perceived satisfaction and creativity through the different stages of the game (2 questions), the importance of the game’s elements and props (1 question). In the last question, students evaluated the usefulness of the different cards categories stimuli (1 question: “Which do you think were the 3 most helpful cards categories in the game?”). A semi-formal discussion was also conducted at the end of each session, concerning students’ experience, the effect of game’s props on their idea generation and the evaluation of their ideas. All informal discussions were recorded and transcribed.

5 Results Students’ Attitudes Towards We!Design!Fractions Process and Products As shown in Table 1, students considered the We!Design!Fractions game as a very interesting, pleasant and efficient experience. Elementary students claimed that they were satisfied by the list of ideas that they had produced and believed that these ideas can become the basis of an educational app that will make fractions learning more enjoying. The students supported that they felt like playing a real game and that the game helped them to produce and share many ideas without hesitation. Table 1. Students attitudes for We!Design!Fractions process and products Question The process was very interesting and original The process was not very pleasant The process was particularly productive I feel very satisfied with the final set of ideas we generated, me and my teammates I had the feeling of playing a real game I believe that the game helped me a lot in generating ideas I would prefer to generate ideas without the game The game was well structured and had clear rules I think that some teammates have spoken a lot and impeded the rest from participating in equal terms I felt the ultimate freedom for expressing my thoughts, with no restrictions

AVG 4.74 1.41 4.30 4.67

SD 0.59 1.08 1.10 0.48

4.41 4.23 1.19 4.48 1.15

0.75 1.03 0.48 0.80 0.46

4.22

1.50

208

G. Palaigeorgiou and V. Sidiropoulou

Students Evaluation for the Different Stages As presented in Fig. 2, students considered the We!Design!Fractions process as both satisfying and creative in all of each phases. Design alter ego technique worked exceptionally well as an entertaining, warm up activity which introduced the elementary school students to the design process. It is interesting that the perceived creativity and satisfaction were reduced in the starting round of the game as students were required to familiarize themselves with an activity with rules, restrictions and unexpected requirements for ideation and introspection. By playing the game, they quickly got acquainted with the process, their productivity was increased and they engaged themselves with their team ideas, retaining their satisfaction and creativity at the end of the game. It is of special importance to note, that these high levels of satisfaction were achieved while students were working on designing an educational application for one of the most unpopular learning domains in primary school and for 3 h without a break (Fig. 3).

5 4 3 2 1 0 Design alter egos' creation

1st round

midle round

Satisfaction

final round Creativity

Fig. 3. Perceived satisfaction and creativity during We!Design!Fractions

Game Elements Evaluation As expected, students evaluated the design cards as the most useful game prop, since they provided the stimuli for generating ideas along with the post-it, on which their ideas were recorded. The game board was considered as an important element of the game which, along with the dices, contributed to the unpredictability of the experience (Table 2). Table 2. We!Design!Fractions elements’ assessment Game elements Cards Post-it Board Dice Pawns

Average 4,78 4,56 4,30 4,22 3,89

S.D. 0,506 0,847 0,775 1,013 0,934

Can Elementary Students Co-design the Learning Content of Educational Apps

209

Design Cards Evaluation When students were asked to indicate the three design cards which they considered as the most helpful ones, all the cards were designated as important by at least one participant. That essentially means that all of cards fitted the generative thinking process of at least some of the participants (Table 3). Additionally, it is interesting that the students indicated as most useful cards, one from each perspective: Time from the context perspective, My Hero from the divergent perspective and Known Learning Difficulties from the convergent perspective. This underlines that a balanced exploration of the design space from the three perspectives is more than needed. Finally, it is worth mentioning that the convergent cards which are probably closer to more conventional inquiry methods, were indicated as useful by less students. The other two perspectives (contextual and divergent) proved to be more attractive, challenging and provocative for students who seem to seek for more inventive ways for exploring needs and ideas. Table 3. Most useful design cards Design card Time My hero Known Learning Difficulties Verbs Future All Design Alter ego experience Breaking the rules Learning experience Place/space Collaborative Playing experience

N 15 12 11 11 8 6 4 4 4 3 1

% 19.0% 15.2% 13.9% 13.9% 10.1% 7.6% 5.1% 5.1% 5.1% 3.8% 1.3%

The Proposed List of Ideas Students produced a total of 199 ideas - we counted only the proposals which were considered as of value to a potential tablet app - with an average of 28 ideas per session. In Table 4, the count of ideas per participatory session and per round are presented. In most sessions, the ideas were more in the first rounds since it was easier to produce some convergent ideas and were reduced in the final rounds where the students had to be more inventive. As shown in Table 5, students though that their ideas were quite useful and original. The usefulness of their ideas has a slightly higher rating than their originality because many of the recorded ideas were derived or transposed from their prior learning experience, and these ideas were assessed as useful but not as original.

210

G. Palaigeorgiou and V. Sidiropoulou Table 4. Ideas produced in each session and round

Session 1 2 3 4 5 6 7

st

1 round 2nd round 3nd round 4th round 5th round 6th round 7th round Sum 6 4 5 6 5 6 – 32 4 5 6 3 5 3 3 29 3 6 3 3 3 8 – 26 9 6 8 6 4 – – 33 7 7 3 4 – – – 21 5 8 4 6 3 – – 26 7 7 4 9 5 – – 32

Table 5. Students perceived usefulness and innovativeness of their own ideas Average S.D. Usefulness 3,93 1,04 Innovativeness 3,83 1,35

An analysis of students’ ideas is not provided in this specific article but some demonstrative ideas are presented. Representations: “A clown is walking on a tightrope. The user has to describe with fractions the point in which the clown is, otherwise the clown falls to the pool”, “Fractions indicate the water’s height in a swimming pool and we have to estimate how much the sun evaporates the water every day”. Authentic context: “we could create a game like Minecraft where the player has to select the materials needed to build a house or other constructions with fractions.” Learning Sequence: “The user has to put different amounts of ice cream described in fractions terms in same sized cups and afterwards compare them in order to understand their size”. Students managed to offer novel and intimate representations and contexts of fractions usage but were less able to provide interesting interactive learning sequences. Most of their related ideas were behavioristic and tended to look like as an assessment game. Students’ undeveloped metacognitive awareness or the inefficiency of the design cards prevented them from imagining guided or explorative interactions with the learning domain, and this issue should be further investigated. Qualitative Measures The Game Experience Many of the students’ comments surpassed the best researchers’ expectations. Participants were thrilled with their experience e.g. “I had a wonderful time”, “it felt like going to the supermarket, buying gums, blew bubbles, get into them and fly with them”, and the majority of students stated that they wished to participate again in such an activity, e.g. “I wish I did it everyday”, “when the game ended, the first thing I thought was that it had never ended”. All children, with no exception, experienced the whole process as a game, e.g. “we were playing a game”, “you forget that this is all about

Can Elementary Students Co-design the Learning Content of Educational Apps

211

maths, you wouldn’t say it’s about math, this is a board game”, and as a creative process that stimulated their thinking and their creative potential, e.g. “we trained our brain”, “you create things here.. you train your brain to think”, “we freed up our imagination”. Some children highlighted the value of collaborative team work and the freedom in expressing their thoughts, e.g. “we worked together and talked to each other and expressed our ideas freely”. The Game Elements Students described the design alter egos technique as a playful and enjoyable introduction to the game which established a pleasant atmosphere e.g. “I had a great time creating the character, I was excited”, “the character helped us a lot to get into the role and to focus on what we have to think and write,” “you thought of another self that could do these things..” Design alter ego also supported efficiently students’ idea generation e.g. “my character had my own characteristics, but he had his own too, in that way I was able to think of more ideas”. Students also supported that the game board, the dice and the pawns created unpredictability of the stimuli distribution between them e.g. “in this way, you could take cards at random, and challenge yourself”, “you don’t know where you will go… it offers suspense in regards to which card you get”. Evaluating Their Ideas Most of the children stated that the quantity of ideas generated was impressive and unexpected e.g. “I didn’t expect to produce so many ideas”, and that these ideas could generate an original and entertaining application e.g. “our ideas would be very pleasant to play with”, “it [the app] would have been more imaginative, than an adults’ design, who don’t have enough creativity”, “application designers couldn’t design an application as fancy as ours”. Students claimed that they could guide better the design of a game-based learning app, e.g. “the ideas are created by children, so they follow children’s state of mind… that we learn through playing”, “it’s worth making an application because the ideas are original and we do learn by playing”. Students identified the proposed examples as the most important added value to the educational app e.g. “it would help a lot, using simpler examples than teachers”, “at school, examples are more ‘scientific’, more complicated… these examples are more helpful”. Designing an Application also Means Learning A side effect of students’ participation in the game was that they familiarize better with fractions concepts and usage, and improved their attitudes towards the specific learning domain. Students experienced the idea generation and their discussion as a leverage for deeper understanding of fractions, e.g. “it helped me on understanding fractions better because I was thinking how to explain my examples”, “From now on, I will work on fractions more pleasantly”, “now that I dealt with fractions in this way, they seem to be like a game”, “now fractions have a nice place in my mind and in my heart”.

212

G. Palaigeorgiou and V. Sidiropoulou

6 Conclusions This study attempted to investigate the possibilities of co-designing an educational app’s content, its learning representations and interactions with experienced primary school students. Students, guided by the participatory design game We!Design!Fractions, produced a large set of 199 ideas, which, according to them, are representative of how fractions should be learned through a playful tablet application. By studying the proposed ideas, we can identify that students achieved two of the three initial goals: they were able to create their own representations and to find original contexts of use for fractions, but not to offer interesting or original learning interaction sequences. Students proposals about learning sequences were mostly behavioristic. The interesting conclusion that has emerged, is the clarity with which the elementary students stated that they want, they should, and that it would be better for them to be the designers of the applications that concern them. Students demonstrated that they could commit themselves in a design process for one of the most difficult subjects of the elementary school curriculum with the support of the appropriate participatory design methodology, to think creatively and to produce a significant set of different ideas. The We!Design!Fractions game cultivated a fruitful environment for producing useful and original ideas in a pleasant cooperative atmosphere for several hours. Students experienced the design process as a learning opportunity in which they were emotionally involved and which resulted in gaining more familiarity with the subject. The final list of students’ ideas opens up fresh perspectives on how students want to learn about fractions. Nevertheless, the material produced needs further analysis by subject specialists in order to clarify in detail which of the proposed elements can be incorporated into educational applications and which incorporate original interactions and representations. Ideally, the evaluation of We!Design!Fractions should have included the development of the tablet application and its assessment but this goes beyond the context of this study. We!Design!Fractions can become just the first step towards participatory design methodologies that would target specific learning domains.

References 1. Druin, A.: Mobile Technology for Children: Designing for Interaction and Learning. Morgan Kaufmann, Burlington (2009) 2. Bekker, M., Beusmans, J., Keyson, D., Lloyd, P.: KidReporter: a user requirements gathering technique for designing with children. Interact. Comput. 15(2), 187–202 (2003) 3. Dindler, C., Eriksson, E., Iversen, O.S., Lykke-Olesen, A., Ludvigsen, M.: Mission from Mars: a method for exploring user requirements for children in a narrative space. In: Proceedings of the 2005 Conference on Interaction Design and Children, pp. 40–47. ACM (2005) 4. Read, J.C., Gregory, P., MacFarlane, S., McManus, B., Gray, P., Patel, R.: An investigation of participatory design with children-informant, balanced and facilitated design. In: Proceedings of Interaction Design and Children International Workshop, pp. 53–64. Shaker, Maastricht (2002)

Can Elementary Students Co-design the Learning Content of Educational Apps

213

5. Bell, A., Davis, K.: Learning through participatory design: designing digital badges for and with teens. In: Proceedings of the 15th International Conference on Interaction Design and Children, pp. 218–229. ACM (2016) 6. Culén, A.L., Bratteteig, T., Pandey, S., Srivastava, S.: The Child-to-Child (C2C) method: participatory design for, with and by children in a children’s museum. IADIS Int. J. WWW/Internet 11(2), 92–113 (2013) 7. Roussou, M., Kavalieratou, E., Doulgeridis, M.: Children designers in the museum: applying participatory design for the development of an art education program. In: Proceedings of the 6th International Conference on Interaction Design and Children, pp. 77–80. ACM (2007) 8. Iversen, O.S., Smith, R.C.: Scandinavian participatory design: dialogic curation with teenagers. In: Proceedings of the 11th International Conference on Interaction Design and Children, pp. 106–115. ACM (2012) 9. Yarosh, S., Schueller, S.M.: “Happiness inventors”: informing positive computing technologies through participatory design with children. J. Med. Internet Res. 19(1), e14 (2017) 10. Jordan, N.C., et al.: Developmental predictors of fraction concepts and procedures. J. Exp. Child Psychol. 116(1), 45–58 (2013) 11. Vamvakoussi, X., Vosniadou, S.: How many decimals are there between two fractions? Aspects of secondary school students’ understanding of rational numbers and their notation. Cogn. Instr. 28(2), 181–209 (2010) 12. Morgan, C., Mariotti, M.A., Maffei, L.: Representation in computational environments: epistemological and social distance. Int. J. Comput. Math. Learn. 14(3), 241–263 (2009) 13. Rau, M.A., Aleven, V., Rummel, N., Pardos, Z.: How should intelligent tutoring systems sequence multiple graphical representations of fractions? A multi-methods study. Int. J. Artif. Intell. Educ. 24(2), 125–161 (2014) 14. Triantafyllakos, G., Palaigeorgiou, G., Tsoukalas, I.A.: Designing educational software with students through collaborative design games: the We!Design&Play framework. Comput. Educ. 56(1), 227–242 (2011) 15. Nijstad, B.A., Stroebe, W.: How the group affects the mind: a cognitive model of idea generation in groups. Pers. Soc. Psychol. Rev. 10(3), 186–213 (2006) 16. Liikkanen, L., Perttula, M.: Contextual cueing and verbal stimuli in design idea generation. In: Proceeding of the Design Computing and Cognition 2006, pp. 619–631 (2006) 17. Paulus, P.B., Yang, H.C.: Idea generation in groups: a basis for creativity in organizations. Organ. Behav. Hum. Decis. Process. 82(1), 76–87 (2000) 18. Shih, P.C., Nguyen, D.H., Hirano, S.H., Redmiles, D.F., Hayes, G.R.: GroupMind: supporting idea generation through a collaborative mind-mapping tool. In: Proceedings of the ACM 2009 International Conference on Supporting Group Work, pp. 139–148. ACM (2009) 19. Warr, A., O’Neill, E.: Understanding design as a social creative process. In: Proceedings of the 5th Conference on Creativity & Cognition, pp. 118–127. ACM, April 2005 20. Brandt, E., Messeter, J., Binder, T.: Formatting design dialogues–games and participation. Co-Design 4(1), 51–64 (2008) 21. Pedersen, J., Buur, J.: Games and movies: towards innovative co-design with users. In: Scrivener, S.A.R., Ball, L.J., Woodcock, A. (eds.) Collaborative Design, pp. 93–100. Springer, London (2000) 22. Iversen, O.S., Buur, J.: Design is a game: developing design competence in a game setting. In: Proceedings of the 7th Biennial Participatory Design Conference, pp. 22–28 (2002) 23. Kultima, A., Niemelä, J., Paavilainen, J., Saarenpää, H.: Designing game idea generation games. In: Proceedings of the 2008 Conference on Future Play: Research, Play, Share, pp. 137–144. ACM (2008)

214

G. Palaigeorgiou and V. Sidiropoulou

24. Triantafyllakos, G., Palaigeorgiou, G., Tsoukalas, I.A.: Design alter egos: constructing and employing fictional characters in collaborative design sessions. In: Proceedings of the 23rd British HCI Group Annual Conference on People and Computers: Celebrating People and Technology, pp. 20–28. British Computer Society (2009) 25. Grossman, S., Catlin, K.: SuperHeroes. In: 31st Annual Creative Problem Solving Institute, Buffalo, NY. The Creative Education Foundation (1985) 26. Kowaltowski, D.C., Bianchi, G., De Paiva, V.T.: Methods that may stimulate creativity and their use in architectural design education. Int. J. Technol. Des. Educ. 20(4), 453–476 (2010) 27. Silver, M.: Exploring Interface Design. Cengage Learning, Boston (2013) 28. Kensing, F., Madsen, K.H.: Generating visions: future workshops and metaphorical design, pp. 155–168. L. Erlbaum Associates Inc (1992)

Pedagogical Considerations for Mobile-Based Augmented Reality Learning Environments Betul Czerkawski(&) and Margherita Berti University of Arizona, Tucson, USA {bcozkan,berti}@email.arizona.edu

Abstract. As teaching methods become increasingly digitized, mobile augmented reality has taken a leading position in education due to its affordances, for instance the anywhere-anytime accessibility, the availability of augmented reality applications for students’ smartphones, and the effective and collaborative capabilities. Although there is a growing number of studies investigating the use of augmented reality in various subjects, a set of guidelines describing learning theories and pedagogical considerations regarding the development and use of mobile-based augmented reality environments is clearly needed. This paper first reviews empirical studies in the domain of learning with mobilebased augmented reality and then proposes situated cognition as a pedagogical framework for the design and use of this technology in educational contexts. Keywords: Augmented reality cognition


Mobile-based augmented reality
Situated

1 Background Augmented reality (AR) refers to the enhancement of real-life objects and environments by multimodal digital technologies. AR offers numerous opportunities to expand students’ learning experiences by adding interactivity, visualization, authenticity, and engagement to instructional content. In recent years, AR has been employed successfully in educational environments. The reason for this success can be seen in the considerable research on the benefits of AR technologies, as illustrated in Table 1. Although in these studies AR technologies were beneficial to student learning, the literature also reveals some problems regarding the pedagogical approaches used in the design of AR activities and scenarios. For instance, Wu et al. (2013) point out the internal tension between actual reality and reality created by AR technologies, and state that “a set of set of design guidelines based on learning theories (e.g., distributed cognition and situated learning) and empirical evidence would be useful for educators and designers to resolve this tension” (p. 47). In other words, a set of guidelines using adequate learning theories and instructional design approaches would provide a pedagogically sound effectiveness.

© Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 215–222, 2021. https://doi.org/10.1007/978-3-030-49932-7_21

216

B. Czerkawski and M. Berti Table 1. Benefits of augmented reality technologies

Authors Buchner and Zumbach (2018) Chen and Tsai (2012) Chiang et al. (2014) Costa et al. (2018)

Benefits Increased intrinsic motivation Improved learning performance Increased interaction

Participants High school students

Methodology Experimental Experimental

Quantitative survey Experimental

Dunleavy et al. (2009) Garrett et al. (2018) Kamarainen et al. (2013) Kurniawan and Witjaksono (2018) Liu et al. (2018)

Deeper understanding of content Deeper understanding of content

Elementary school students Elementary school students Elementary school students Middle school and high school students Undergraduate students Middle school students Undergraduate students

Better performance

High school students

Radosavljevic et al. (2018)

Increased learning efficiency and faster performance

Undergraduate students

Increased attention and participation Engagement and participation Self-directed learning

Experimental Qualitative case study Qualitative case study Mixed methods Quantitative survey Quantitative survey

The purpose of this paper is to review the AR literature on such pedagogical knowledge base and apply situated cognition to learning with AR. More specifically, the following research questions will guide this paper: (1) What learning theories in support of AR for education have been used in empirical studies? a. What pedagogical design considerations can be extrapolated from these learning theories? (2) How can situated cognition learning theory be used in the design of AR learning environments? Wu et al. (2013) identified various subsets of AR technologies: mobile-based, game-based, and multi-player AR. These subsets allude to the technologies used as AR may require additional hardware such as headsets, glasses, displays or other portable and wearable technologies. In fact, most of the criticism about AR is related to the technologies used. Their interface may cause various compatibility issues, vision problems or simply could be too expensive to be used ubiquitously. However, in education, mobile-based AR technologies are becoming more common due to their availability and ease of use. In this paper, we will explicitly focus on mobile-based AR technologies.

Pedagogical Considerations for Mobile-Based AR Learning Environments

217

2 Methodology To answer the first research question and to further understand how mobile-based AR has been used in educational contexts, empirical studies that implemented learning theories in support of AR were examined. To achieve this purpose, articles indexed in the Academic Search Ultimate database were identified using the following keyword search terms: “mobile augmented reality”, “mobile augmented learning”, and “mobile augmented reality in education”. Empirical articles published in journals in 2009 or later were selected for inclusion in this paper with the goal of identifying learning theories and theoretical frameworks, which can then inform instructors who desire to include mobile AR technologies in their own classroom, as well as designers eager to develop AR environments based on pedagogical design considerations.

3 Augmented Reality in Education AR has been identified as one of the most promising technologies in the field of education. Through a combination of multimedia elements (e.g., texts, images, videos, and animations) and real word environments, AR enables the merging of the digital world with the real world, transforming content and traditional learning. The use of AR technologies in education has been increasing in recent years with a growing number of studies exploring its potential to make learning more engaging and effective. Despite research showing the potential of AR systems to enhance student learning (e.g., Chen and Tsai 2012; Kamarainen et al. 2013; Tee et al. 2018), clear pedagogical considerations addressing its usefulness in educational environments are needed (Radu 2014). As with other innovative educational technologies, the value of AR also lays in how it is used and integrated in the educational setting. The next section briefly describes mobile-based AR, follows with a review of studies using mobile-based AR environments for educational purposes, and discusses pedagogical design considerations extrapolated from learning theories implemented in empirical research. 3.1

Mobile-Based Augmented Reality

Mobile phones have developed into powerful computational devices equipped with advanced color displays, high resolution digital cameras, and GPS features. Ownership of mobile devices has constantly increased over the years. A recent survey conducted by the Pew Research Center (2019) has found that in the United States smartphone ownership has reached 81% among adults, from 35% in 2011, proving that mobile device use continues to grow. The number of mobile applications utilizing AR technologies has also increased (Adhani and Awang 2012). Scholars explain that mobilebased AR is more appealing to users since it allows easy access to AR environments with hand-held mobile devices, as compared to cumbersome head-mounted displays (Scrivner et al. 2019). Mobile AR is especially well suited for ubiquitous learning, supporting the idea that every person can learn wherever they are, whenever they want. Mobile AR has enormous potential and it is suitable for wider use. The literature distinguishes between marker-based AR, markerless AR, and location-based AR (Chen

218

B. Czerkawski and M. Berti

and Tsai 2012). Marker-based AR utilizes artificial markers to assist object recognition. In this instance, the camera of a device recognizes a distinctive shape or image (i.e., the markers) in the real world, and follows with an animation or digital overlay. On the other hand, markerless AR places virtual objects in real environments. Markerless AR is used by those applications that do not need an “anchor” and can place digital content right in front of the user. Last, location-based AR utilizes a GPS sensor to determine position and orientation and enables easy navigation thanks to virtual cues overlaid in the real world. These diverse types of AR technologies have been used for various purposes (e.g., games, navigation, and object information) in educational contexts. 3.2

Research on Mobile-Based Augmented Reality Environments

The studies presented in Table 1 show the value and potential of mobile-based AR environments for education. In Kamarainen et al. (2013), during a field trip, students used mobile devices with an AR application to navigate a local pond and observed virtual media and information overlaid onto the real environment. Results show that students’ understanding of specific content (i.e., water quality) improved, suggesting that implementing AR in out-of-class activities holds potential for creating connections between what students are learning and new situations. The theoretical framework adopted in the study is situated learning theory, “in which cognition is seen as situated within both a physical and a psychosocial context and as distributed between a person and the tools that person is using” (p. 545). Similarly, Chen and Tsai (2012) utilized situated learning theory as the theoretical framework for the use of an educational AR system in the library’s learning environment. In this study, students learning performance improved, and according to questionnaire results the AR technology promoted learner motivation and willingness to learn. In another study by Dunleavy et al. (2009), student engagement in an inquiry-based AR simulation was measured with the use interviews, direct observations, web site posts, and site documents. Participants worked in teams, collected digital items, solved math, language, and scientific literacy puzzles by using AR GPS technology to link their real-world location to their virtual location. Results show that using handheld devices to navigate and collect data increased motivation; however, the learning task created concerns in terms of students ignoring their environment while walking in the urban setting. Chiang et al. (2014) also utilized an inquiry-based framework and multimedia design theory in support of learning activities with mobile AR. The authors explain that through AR activities, students’ learning performance improved due the connections created between real-world contexts and the digital learning resources. Mobile-based AR environments produced learning scenarios with relevant and organized materials. Radosavljevic et al. (2018) utilized multimedia learning theory to support the use of mobile AR technologies with college students in a vocational training program. The goal was to improve the process of learning content as well as task performance. Results show that participants demonstrated adaptation in professional environment, faster reaction to the tasks, and a more accurate fulfillment of said tasks. In another study (Garrett et al. 2018), the use of mobile-based AR by students in three different disciplines (i.e., nursing, physical therapy, and occupational therapy) led to

Pedagogical Considerations for Mobile-Based AR Learning Environments

219

self-directed learning, as participants made conceptual links between multimedia AR resources and physical equipment. The authors explain that AR mobile learning promoted a constructivist approach, through which educators provide students with AR activities that enable alternative ways to engage with content, thereby promoting more active and self-directed learning. From the analysis of the literature, the majority of studies only focuses on initial implementations or case studies, with no long-term use of AR technologies. Overall, empirical studies investigating the use of mobile-base AR environments adopted various theoretical frameworks, including situated learning theory, inquiry-based learning, multimedia learning theory, and constructivism. These theories support the use of AR technologies in education, which brought significant benefits to students and their learning experiences. Nonetheless, most studies mainly focus on the learning activity at hand. Furthermore, it is important to note that most AR research is conducted with small-scale samples with limited validated instrumentation. 3.3

Pedagogical Considerations

As seen in the literature, the use of AR in education focused mostly on the design of AR technologies, activities and tasks, or students’ content attainment, skill training or attitudes. For instance, there is abundant research that used AR as a teaching aid, additional resource for students, or activities that required their active engagement. Fowler (2015) cautions against research “that starts with an analysis of the technology then seeks to derive learning benefits, often from loosely defined or implicit learning approaches (particularly constructivism)” (p. 420). It is imperative to note that little to no research considers the pedagogical methods of designing and developing an ARbased learning environment holistically. A holistic approach, on the other hand, starts from the learning objectives and goals and entails the design, development and evaluation of learning activities while considering a learning theory. In other words, many research topics and issues regarding instructional design and development are waiting to be explored (Wu et al. 2013). Many researchers are also aware of the lack of sound pedagogical foundation in AR literature. For instance, some scholars (Chen 2010; Dunleavy et al. 2009; Kamarainen et al. 2013) pointed out the importance of learning theories, instructional design principles and teaching strategies, which often were ignored when AR technologies are used. Wu et al. (2013) mentioned that current AR interfaces are not flexible enough to accommodate learning objectives and implement a true learner-centered pedagogy. The use of AR in informal learning environments is most common, so lack of this pedagogical foundation is partly expected. However, Garrett et al. (2018) state that a pedagogic integration strategy as well as clearly stated instructional goals are key to the success of AR technologies in formal settings. 3.4

Application of Situated Cognition

Learning theories provide the pedagogical framework for educational technologies, so that their use in the learning and teaching environment promotes understanding and student success. Of the various learning theories, several researchers (Chen and Tsai

220

B. Czerkawski and M. Berti

2012; Huang et al. 2010; Kamarainen, et al. 2013; Wasko 2013; Wu et al. 2013) point out that constructivism and related approaches are the most effective in the use of AR technologies. Although many of these approaches can be useful, in order to provide an example where an actual learning theory is applied to the use of AR technologies, the present paper specifically focuses on situated cognition or situated learning theory. Situated cognition is a theory which emphasizes that people’s knowledge is constructed within and linked to the activity, context, and culture in which it was learned (Brown et al. 1989). In this view, learning is social, contextual and collaborative. Authentic and problem-based activities, active learning, and apprenticeship models are especially emphasized. Since the late 20th century, situated cognition has been of interest to many educators and instructional designers, as it provides direction to those who aim at creating meaningful learning experiences for students. Therefore, in the last two decades many examples of situated cognition have been applied to various content areas, using different educational technologies. In the context of AR technologies, the question is how to develop learning environments using situated cognition. What do they look like? What considerations educators and instructional designers should take into account when utilizing AR? Contextual: First, as an interactive medium AR enables for knowledge or content to be situated in a particular context. Traditional education systems rely on decontextualized and explicit disciplines and content arrangements. However, in AR-based environments learning is situated and tacit (Dunleavy et al. 2009). Distributed: AR supports distributed and community-based learning and teaching activities rather than isolated, individual activities. Therefore, collaboration and group methods are most appropriate for AR. However, collaboration does not necessarily mean group work, either. For instance, students can start with individual AR projects, but in the development process, they can work with their peers to improve their understanding and gain new insights. Collaborative work can involve peer review (i.e., reflection) and social negotiation skills (i.e., articulation) that will eventually improve students’ collaborative work skills. Location-based AR methods especially work well for promoting collaborative learning. Cognitive Apprenticeship and Facilitation/Coaching: The majority of students have no prior knowledge on how to use AR technology. By embedding the learning task in a familiar activity, students gain access to their implicit knowledge. The instructor’s job is to facilitate and guide students while they are trying AR for the first time and then provide a mental structure of how to progress gradually. According to Brown et al. (1989), students learn through such cognitive scaffolding exercises. When designing an AR based activity, instructors can start from familiar activities, situate the tasks into authentic contexts and emphasize students’ social interaction with each other. Authentic Tasks/Problems: The real-life problems or tasks presented to students via AR technologies should be ill-defined in contrast to the well-defined academic or school problems. Ill-defined here refers to the broad, unstructured, activities where students take charge of a complex problem or task and define it themselves. This way, students gain “ordinary experiences of their culture” (Brown et al., 1989 p. 34) via AR activities, and develop expertise in the content area.

Pedagogical Considerations for Mobile-Based AR Learning Environments

221

4 Discussion and Conclusion The use of AR technologies is very popular (Horizon Report 2019) in education, but the utilization of mobile technologies, and more specifically smartphones, made them more ubiquitous in higher education institutions. Most research reviewed in this paper focuses on the activity or interface design, while also pointing out lack of pedagogical approaches incorporated in AR environments. In the present paper, we discussed situated cognition as a possible pedagogical framework that will guide educators when using mobile AR technologies. However, there is little empirical research that tests the appropriateness of such framework. Chen (2010) asserts that studies that focus on deriving the underlying theoretical framework also guide the instructional design process, so they are much needed. In addition, task design and teaching presence processes need to be defined to avoid cognitive overload while providing an authentic and contextual learning experience to the students. Finally, when discussing possible cognitive frameworks for mobile-based AR, it is important to describe instructional design (ID) procedures that will guide the development process. These ID procedures should be empirically proven (Akcayir and Akcayir 2017). According to Reigeluth and Frick (1999) more instructional design theories (or models) are needed to provide guidance on the use of new information technology tools. In the AR literature, no ID research exists. While most studies used small-scale samples, ID models may support bigger scale studies with sound implications. Clemens, Purcell, and Clements et al. (1960) point out another implication of using ID guidelines (p. 1964): The educator has to first evaluate learning objectives and make a professional and informed judgment regarding implementing an instructional technology tool in a pedagogically sound way to meet these objectives. Just because AR shows potential for learning and could be an innovative addition to your classroom, if it does not meet instructional goals, add value, efficiency or activity not available with other accessible tools, the educator is not encouraged to implement AR or any other innovative tool that is a new or emerging.

References Adhani, N.I., Awang, R.D.R.: A survey of mobile augmented reality applications. In: 1st International Conference on Future Trends in Computing and Communication Technologies, pp. 89–96 (2012) Akcayir, M., Akcayir, G.: Advantages and challenges associated with augmented reality in education: a systematic review of the literature. Educ. Res. Rev. 20, 1–11 (2017) Brown, J.S., Collins, A., Duguid, P.: Situated cognition and the culture of learning. Educ. Researcher 18(1), 32–42 (1989) Buchner, J., Zumbach, J.: Promoting intrinsic motivation with a mobile augmented reality learning environment. In: 14th International Conference Mobile Learning, pp. 55–61 (2018) Chen, C.J.: Theoretical bases for using virtual reality in education. Themes Sci. Technol. Educ. 2(1–2), 71–90 (2010) Chen, C.M., Tsai, Y.N.: Interactive augmented reality system for enhancing library instruction in elementary schools. Comput. Educ. 59(2), 638–652 (2012)

222

B. Czerkawski and M. Berti

Chiang, T.H., Yang, S.J., Hwang, G.J.: Students’ online interactive patterns in augmented realitybased inquiry activities. Comput. Educ. 78, 97–108 (2014) Clemens, R., Purcell, S., Slykhuis, D.: Implementing augmented reality in K-12 education – analyzing current trends. In: Chamblee, G., Langub, L. (eds.) Proceedings of Society for Information Technology & Teacher Education International Conference, pp. 1960–1967. Association for the Advancement of Computing in Education (AACE), Savannah, GA, United States (2016) Costa, M.C., Patricio, J.M., Carranca, J.A., Farropo, B.: Augmented reality technologies to promote STEM learning. In: 2018 13th Iberian Conference on Information Systems and Technologies (CISTI), pp. 1–4. IEEE, June 2018 Dunleavy, M., Dede, C., Mitchell, R.: Affordances and limitations of immersive participatory augmented reality simulations for teaching and learning. J. Sci. Educ. Technol. 18(1), 7–22 (2009) Garrett, B.M., Anthony, J., Jackson, C.: Using mobile augmented reality to enhance health professional practice education. Curr. Issues Emerg. eLearning 4(1), 10 (2018) Huang, H.M., Rauch, U., Liaw, S.S.: Investigating learners’ attitudes toward virtual reality learning environments: based on a constructivist approach. Comput. Educ. 55(3), 1171–1182 (2010) Horizon Report: EDUCAUSE Horizon Report. Higher Education Edition (2019). https://library. educause.edu/-/media/files/library/2019/4/2019horizonreport.pdf?la=en&hash=C8E8D444A F372E705FA1BF9D4FF0DD4CC6F0FDD1 Kamarainen, A.M., Metcalf, S., Grotzer, T., Browne, A., Mazzuca, D., Tutwiler, M.S., Dede, C.: EcoMOBILE: integrating augmented reality and probeware with environmental education field trips. Comput. Educ. 68, 545–556 (2013) Kurniawan, M.H., Witjaksono, G.: Human anatomy learning systems using augmented reality on mobile application. Procedia Comput. Sci. 135, 80–88 (2018) Liu, E., Li, Y., Cai, S., Li, X.: The effect of augmented reality in solid geometry class on students’ learning performance and attitudes. In: International Conference on Remote Engineering and Virtual Instrumentation, pp. 549–558. Springer, Cham, March 2018 Pew Research Center: Mobile Fact Sheet (2019). http://www.pewinternet.org/fact-sheet/mobile Radosavljevic, S., Radosavljevic, V., Grgurovic, B.: The potential of implementing augmented reality into vocational higher education through mobile learning. Interact. Learn. Environ. 1– 15 (2018) Radu, I.: Augmented reality in education: a meta-review and cross-media analysis. Pers. Ubiquit. Comput. 18(6), 1533–1543 (2014) Reigeluth, C.M., Frick, T.W.: Formative research: a methodology for creating and improving design theories. In: Reigeluth, C.M. (ed.) Instructional-Design Theories and Models – A New Paradigm of Instructional Theory, pp. 633–652. Lawrence Erlbaum, New Jersey (1999) Scrivner, O., Madewell, J., Buckley, C., Perez, N.: Best practices in the use of augmented and virtual reality technologies for SLA: design, implementation, and feedback. In: Carrió-Pastor, M.L. (ed.) Teaching Language and Teaching Literature in Virtual Environments, pp. 55–72. Springer, New York (2019) Tee, N.Y.K., Gan, H.S., Li, J., Cheong, B.H.P., Tan, H.Y., Liew, O.W., Ng, T.W.: Developing and demonstrating an augmented reality colorimetric titration tool. J. Chem. Educ. 95(3), 393–399 (2018) Wasko, C.: What teachers need to know about augmented reality enhanced learning environments. TechTrends 57(4), 17–21 (2013) Wu, H.K., Lee, S.W.Y., Chang, H.Y., Liang, J.C.: Current status, opportunities and challenges of augmented reality in education. Comput. Educ. 62, 41–49 (2013)

Machine Learning and Deep Learning: Recent Overview in Medical Care Nour Elhouda Chalabi1(&), Abdelouahab Attia1, and Samir Akrouf2 1

Mohamed El Bachir El Ibrahimi University of Bordj Bou Arreridj, El Anseur, Algeria [email protected], [email protected] 2 Mohamed Boudiaf University of M’sila, M’sila, Algeria [email protected]

Abstract. Medical care has always presented quite wide ranged and challenging problems. However, machine learning techniques and methods as well as deep learning never stopped evolving and tackling those challenges issued by medicine, medical and health care. In order to have a more close up look on how machine learning and deep learning has been affecting medical care in general, we review in this paper some machine learning and deep learning techniques used in a variety of medical care sections such as medical imaging, medical decision, diagnostic, medical records and big data, and disease prediction. Keywords: Machine databases


Learning
Deep learning
Medical care
Medical

1 Introduction Machine learning is a branch of artificial intelligence (AI) and it consists of methods that allow computers to act without the human interaction. In other words, it is a program that can learn from data and take decisions. Machine learning as a field of research in computer science has been used widely in different areas such as speech recognition, computer vision, robot control, accelerating empirical sciences, biosurveillance and product recommendations [25]. Machine learning methods can be divided into four categories: supervised, nonsupervised, semi-supervised and reinforcement. Some well-known algorithms that belong to those categories are: Nearest Neighbor, Naïve Bayes, Decision Tree, Linear Regression, SVM (Support Vector Machine), k-means clustering, Association Rules, Neural Network Q-Learning [1]. On the other hand, deep learning is a subcategory of machine learning that has showcased a high potential in solving problems in different areas compared to other machine learning methods. Deep learning usually consisted in a multilayered neural network algorithm that enables extracting universal features in very complex datasets [2] it happens that it outperforms various techniques [26]. Deep learning has presented extremely promising results for different tasks such as: natural language understanding, particularly topic classification, sentiment analysis, question answering and language translation [2]. © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 223–231, 2021. https://doi.org/10.1007/978-3-030-49932-7_22

224

N. E. Chalabi et al.

In this paper, we survey the effect of machine and deep learning in the medical care sector and discuss the most used methods and the most affected area in medicine in general by those two disciplines and the challenges faced in the road toward a more effective medical care. 1.1

Motivation

Almost every aspect of life has been affected by machine learning or deep learning in a way, so it is not surprising if medical care can’t be excluded from this mass of changes brought by those two approaches as well as their big success in different fields. Therefore, we hope by presenting this paper, we can project the spotlight on the most recent works in medical care using machine or deep learning and the results they bring in the midst of this awash of fast development.

2 Overview Depending on the medical section and the type of the input there are different steps of machine learning and deep learning algorithms used, Fig. 1 shows the different phases. Note that those phases do not apply to all databases and medical sections.

Medical Database input

Medical Database input

Preprocessing Features extraction

Machine Learning Deep Learning

Features selection Classification Decision output

Decision output

Fig. 1. Machine and deep learning algorithms workflow

2.1

Machine Learning in Medical Care

In this subsection, we focus on machine learning methods that have been applied on various medical care sections in the literature.

Machine Learning and Deep Learning: Recent Overview in Medical Care

225

One of the medical sections that machine learning has been used is medical diagnosis Samant and Agarwal [3] have proposed a diabetes diagnostic model based on the infrared images of the iris of both eyes of the patient, after selecting the region of interest (ROI) that corresponds to the position of pancreas organ. Features extraction was performed according to the order statistics, textural features and wavelet features. As results, the RF classifier (Random Forest classifier) was proved to have the best accuracy results among other six classifiers noted: Binary Tree Model (BT), Support Vector Machine (SVM), Adaptive Boosting Model (AB), Generalized Linear Models (GL), Neural Network Model (NN) and Random Forest (RF). The experiment was done on the total of 338 subjects (180 diabetic of type II and 158 non-diabetics). In medical disease prediction, Mathur et al. [4] have presented a comparative work for Parkinson disease prediction in a big data framework. This latter was layered in three components: First Hadoop was used to analyze and sort the unstructured dataset. Secondly, a probabilistic platform called Predictive Analytics that uses different statistical modeling, data mining techniques, and machine learning techniques have been used to give an insight about the data. Lastly, Prediction Models, different machine learning techniques were used to classify the data such as: Support Vector Machine (SVM) and ANN algorithm called multilayer perceptron (MLP). The data used is UCI machine learning library with 195 instances and 24 attributes, the techniques were compared based on accuracy obtained from SMO, KNN, Random Forest, AdaBoost. M1, Bagging, MLP, and DT algorithms. K-Nearest Neighbors KNN algorithm. It appears that ANN showed better results compared to other algorithms. In medical imaging, Amin et al. [5] presented in a recent work, a brain tumor detection from Magnetic Resonance Imaging (MRI) using statistical and machine learning methods. The proposed methodology has three main phases. First, the MRI images were de-noised using Weiner filter, then segmentation was performed using Potential Field (PF) clustering to find the subset of tumor pixels in MRI. Lastly, feature extraction and classification was performed, the features were extracted using Local Binary Pattern (LBP) and Gabor Wavelet Transform (GWT) and the results were fused in one vector of features. For classification, various methods were applied such as: Support vector machine (SVM) with quadratic kernel function, K-Nearest Neighbors (KNN), Decision Tree (DT), Random forest (RF), Naive Bayes (NB). The experiment was done on local database of 86 images (49 with tumors, 37 without tumors). The results showed that features fusion and KNN outperformed other classifiers. In medical disease classification, Garg et al. [6] proposed recently an Ischemic Stroke (IS) subtype classification using machine learning and natural language processing. In this work the authors, analyzed unstructured text-based data including neurology progress notes and neuroradiology reports obtained from electronic health records (HER) and transformed it to a format to give to machine learning algorithms. The feature selection method used was Principal Component Analysis (PCA) and XGBoost and the features obtained from both methods are combined, for the training. Variety of machine learning methods were employed such as: K-nearest neighbors (KNN), Support vector machines (SVM), Random Forests (RF), extra randomized trees classifiers, gradient boosting machines, extreme gradient boosting (XGBoost) and

226

N. E. Chalabi et al.

stacking using logistic regression. The experiment was conducted on 1091 patient records; the comparison was performed between machine learning methods and classic classification method by calculating kappa value. The results showed an agreement between the proposed model and the manual TOAST classification. Table 1 summarize the already discussed work in order to give a better insight on the different machine learning methods used in medical care. From Table 1 it can be seen that KNN and SVM are the most used machine learning methods, it also reveilles the use of a variety of other methods which allows to compare them and find the best method depending on the application, medical section and the used data base.

Table 1. Summary of machine learning in medical care Medical section

Application

Authors

Disease diagnosis

diabetes disease diagnostic model

P. Samant and R. Agarwal [3]

Disease prediction

Parkinson disease prediction model Medical Brain tumor Imaging and detection disease from MRI detection images

Machine learning methods

Random Forest classifier (RF) Binary Tree Model (BT), Support Vector Machine (SVM), Adaptive Boosting Model (AB), Generalized Linear Models (GL), Neural Network Model (NN) Mathur, R., Pathak, SMO, KNN, Random Forest, V, and Bandil [4] AdaBoost.M1, Bagging, MLP, and DT algorithms

Amin, J et al. [5]

Disease Ischemic Garg, R et al. [6] classification Stroke (IS) subtype classification

Potential Field (PF) clustering, Support vector machine (SVM) with quadratic kernel function, KNearest Neighbors (KNN), Decision Tree (DT), Random forest (RF), Naive Bayes (NB) K-nearest neighbors (KNN), Support vector machines (SVM), Random Forests (RF), extra randomized trees classifiers, gradient boosting machines, extreme gradient boosting (XGBoost), Stacking using logistic regression

Database Infra-red images of 338 subjects (180 diabetic and 158 non-diabetic)

UCI machine learning library database with of 195 instances and 24 attributes local database of 86 images (49 tumor, 37 non-tumor)

1091 patient records from electronic health records (HER) between 2012 and 2014

Machine Learning and Deep Learning: Recent Overview in Medical Care

2.2

227

Deep Learning in Medical Care

In this subsection, we focus on deep learning methods that have been applied on various medical care sections in the literature. Like machine learning, deep learning methods have been used in medical diagnosis as well. Bi et al. [7] proposed a Computer Aided Alzheimer’s Disease Diagnosis using unsupervised deep Learning technology, the presented work use Principal Component Analysis network (PCA Net) for features extraction applied on three MRI images (orthogonal planes (TOP)) and K-means clustering for prediction. The experiment was done on 1075 subjects MRI image grouped in three classes AD (Alzheimer’s disease), NC (Normal cognitive) and MCI (Mild cognitive impairment). For disease prediction, Xia et al. [8] proposed a convolutional neural network (CNN) based ensemble method for cancer prediction using DNA methylation. First, feature selection was conducted using a t-test, in the next stage features were transmitted to five different machine learning classifiers: Naive Bayesian Classifier, kNearest Neighbor, Decision Tree, Random Forest and Gradient Boosting Decision Tree. For the first stage classification was performed in an attempt to distinguish cancer from normal samples, then S-fold cross validation was performed too. After that the authors has introduced a CNN based ensemble model structure to ensemble the predicted results of various classifiers and predict a result. The experiment was conducted on DNA Methylation 450K data sets which includes three common types of cancer data: Lung Adenocarcinoma (LUAD), liver hepatocellular carcinoma (LIHC) and Kidney Clear Cell Carcinoma (KIRC). Herent et al. [9] proposed in the section of medical imaging a model for detection and characterization of breast lesions from MRI images using deep learning. In this work, 50-layer residual neural network (ResNet-50) was used first to extract the features from the images resulting in feature vector. Next the detection of lesion (localization) and classification was done into two steps respectively using for the first a single convolution to result in a local prediction with a score between 0 & 1. The second was performed by a densely connected layer with five neurons one for each class: lesion malignancy classification, normal tissue, other benign lesion, IDC, and other malignant lesions. The experiment was performed on 335 two-dimensional T1weighted gadolinium chelate-enhanced MR images of 335 patients. For disease classification, Hasan and Bhatta charjee [10], proposed a deep learning approach for cardiovascular disease classification using modified ECG signal from Empirical Mode Decomposition (EMD). First, the ECG signal was modified using the empirical mode decomposition and the resulting first three IMF signals are summed to give the new ECG signal, then the convolution neural network (CNN) which was used as a one dimensional model was feed the signal. The CNN in the proposed work has five convolution layers and three fully connected layers, the final output of the network is softmax regressor with a determined number of classes depending on the database to classify the disease. The model was evaluated using three different databases: PTB Diagnostic ECG Database, MIT-BIH Arrhythmia Database, St.-Petersburg Institute of Cardiological Technique’s 12-Lead Arrhythmia Database. Each database has different number of classes and data recording criteria.

228

N. E. Chalabi et al.

Table 2 summarize the already discussed work in order to give a better insight on the different deep learning methods used in medical care. From Table 2 it can be seen that neural network is used in all the application since it’s the core of deep learning,, it also reveilles the use of a variety of other methods which allows to compare them and find the best method depending on the application, medical section and the used data base.

Table 2. Summary of deep learning in medical care Medical section Disease diagnosis

Application

Authors

Features extraction

Bi, X et al. [7]

Disease prediction

Cancer Prediction

Xia, C et al. [8]

Medical imaging

Features extraction

Herent, P et al. [9]

Disease classification

Cardiovascular disease classification

Hasan, N. I., and Bhattacharjee, A [10]

Deep learning methods Principal Component Analysis network (PCA Net) convolutional neural network (CNN) 50-layer residual neural network (ResNet-50) convolutional neural network (CNN)

Database Alzheimer’s disease Neuroimaging Initiative (ADNI) database MRI with 1075 subjects DNA Methylation450K datasets 335 twodimensional T1weighted gadolinium chelate-enhanced MR images PTB Diagnostic ECG Database MIT-BIH Arrhythmia Database St.-Petersburg Institute of Cardiological Technique’s 12Lead Arrhythmia Database

Machine Learning and Deep Learning: Recent Overview in Medical Care

2.3

229

Some Medical Databases

Databases, are an important element for any experiment in order to test any system and give a relevant results, literature has provided a wide range of accessible databases in almost every medical field to contribute to medical care development. Table 3 shows some of the existent databases in different diseases. From Table 3 it can be seen that the amount of effort put in order to help develop medical care in general also it shows the variety of data basses collected and used over the years which reveals the amount of research has been dedicated to the area.

Table 3. Medical databases Field Retinal

Database name DRIONS-DB [11] STARE [12] MESSIDOR [13] DRIVE [14] Alzheimer ADNI [15] Parkinson PDmutDB [16, 17] MGH/MF Waveform [18] Epilepsy DEAP [19] Continuous EEG [20] Chest X-Ray NIH [21] SCR [22] Heart ECG Apnea-ECG Database [23] PTB Diagnostic ECG Database [24]

3 Conclusion In this paper, we have provided an overview on some of the recent frameworks that employed machine learning or deep learning methods in medical care. This latter has exploited different sections such as diseases diagnostic, diseases prediction, medical imaging, diseases classification, machine learning and deep learning has shown a growth in medical care in general proving their robustness against some of the mainstream approach by maintaining a high profile on the provided results in addition the discussed work was summarized in Table 1 and Table 2 in order to provide a more clear insight on the used methods, we also provided in Table 3 some of the most known databases since this latter is a very important part of any experiment, furthermore we can tell that the future can be promising for machine learning and deep learning in such a research area. As future contribution, we aim to explore machine learning and deep learning in medical care more deeply and widely to cover as much as possible of the two disciplines influence in medical and health care in general.

230

N. E. Chalabi et al.

References 1. Divide Fumo 5 Jun 2017. http://towardsdatascience.com. Accessed 23 Jun 2019 2. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436 (2015) 3. Samant, P., Agarwal, R.: Machine learning techniques for medical diagnosis of diabetes using iris images. Comput. Methods Programs Biomed. 157, 121–128 (2018) 4. Mathur, R., Pathak, V., Bandil, D.: Parkinson disease prediction using machine learning algorithm. In: Rathore, V.S., Worring, M., Mishra, D.K., Joshi, A., Maheshwari, S. (eds.) Emerging Trends in Expert Applications and Security. AISC, vol. 841, pp. 357–363. Springer, Singapore (2019) 5. Amin, J., Sharif, M., Raza, M., Saba, T., Anjum, M.A.: Brain tumor detection using statistical and machine learning method. Comput. Methods Programs Biomed. 177, 69–79 (2019) 6. Garg, R., Oh, E., Naidech, A., Kording, K., Prabhakaran, S.: Automating ischemic stroke subtype classification using machine learning and natural language processing. J. Stroke Cerebrovasc. Dis. 28(7), 2045–2051 (2019) 7. Bi, X., Li, S., Xiao, B., Li, Y., Wang, G., Ma, X.: Computer aided Alzheimer’s disease diagnosis by an unsupervised deep learning technology. Neurocomputing (2019) 8. Xia, C., Xiao, Y., Wu, J., Zhao, X., Li, H.: A convolutional neural network based ensemble method for cancer prediction using DNA methylation data. In: Proceedings of the 2019 11th International Conference on Machine Learning and Computing, pp. 191–196. ACM, February 2019 9. Herent, P., Schmauch, B., Jehanno, P., Dehaene, O., Saillard, C., Balleyguier, C., ArfiRouche, J., Jégou, S.: Detection and characterization of MRI breast lesions using deep learning. Diagn. Interv. Imaging 100(4), 219–225 (2019) 10. Hasan, N.I., Bhattacharjee, A.: Deep learning approach to cardiovascular disease classification employing modified ECG signal from empirical mode decomposition. Biomed. Signal Process. Control 52, 128–140 (2019) 11. Carmona, J., Rincón, M., García-Feijoo, J., Martínez-de-la-Casa, J.M.: Identification of the optic nerve head with genetic algorithms. Artif. Intell. Med. 43(3), 243–259 (2008). [DOI] [Results and Demos] 12. Hoover, A.: Structured analysis of the retina. 11 April 2000 (2000). http://www.ces.clemson. edu/-ahoover/stare 13. Decencière, et al.: Feedback on a publicly distributed database: the Messidor database. Image Anal. Stereology 33(3), 231–234 (2014). ISSN 1854-5165 14. Staal, J.J., Abramoff, M.D., Niemeijer, M., Viergever, M.A., van Ginneken, B.: Ridge based vessel segmentation in color images of the retina. IEEE Trans. Med. Imaging 23, 501–509 (2004) 15. Jack Jr., C.R., et al.: The Alzheimer’s disease neuroimaging initiative (ADNI): MRI methods. J. Magn. Reson. Imaging Off. J. Int. Soc. Magn. Reson. Med. 27(4), 685–691 (2008) 16. Cruts, M., Theuns, J., Van Broeckhoven, C.: Locus-specific mutation databases for neurodegenerative brain diseases. Hum. Mutat. 33(9), 1340–1344 (2012) 17. Nuytemans, K., Theuns, J., Cruts, M., Van Broeckhoven, C.: Genetic etiology of Parkinson disease associated with mutations in the SNCA, PARK2, PINK1, PARK7, and LRRK2 genes: a mutation update. Hum. Mutat. 31(7), 763–780 (2010) 18. Welch, J.P., Ford, P.J., Teplick, R.S., Rubsamen, R.M.: The massachusetts general hospitalmarquette foundation hemodynamic and electrocardiographic database – comprehensive collection of critical care waveforms. J. Clin. Monit. 7(1), 96–97 (1991)

Machine Learning and Deep Learning: Recent Overview in Medical Care

231

19. Koelstra, S., Muhl, C., Soleymani, M., Lee, J.S., Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A., Patras, I.: Deap: a database for emotion analysis; using physiological signals. IEEE Trans. Affect. Comput. 3(1), 18–31 (2011) 20. Zhang, X.L., Begleiter, H., Porjesz, B., Wang, W., Litke, A.: Event related potentials during object recognition tasks. Brain Res. Bull. 38(6), 531–538 (1995) 21. Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.M.: Chestx-ray8: hospitalscale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2097–2106 (2017) 22. van Ginneken, B., Stegmann, M.B., Loog, M.: Segmentation of anatomical structures in chest radiographs using supervised methods: a comparative study on a public database. Med. Image Anal. 10, 19–40 (2006) 23. Penzel, T., Moody, G.B., Mark, R.G., Goldberger, A.L., Peter, J.H.: The apnea-ECG database. In: Computers in Cardiology 2000, vol. 27, (Cat. 00CH37163), pp. 255–258. IEEE, September 2000 24. Bousseljot, R., Kreiseler, D., Schnabel, A.: Nutzung der EKG-Signaldatenbank CARDIODAT der PTB über das Internet. Biomed. Technik/Biomed. Eng. 40(s1), 317–318 (1995) 25. Mitchell, T.M.: The discipline of machine learning, vol. 9. Carnegie Mellon University, School of Computer Science, Machine Learning Department, Pittsburgh, PA (2006) 26. Hua, K.L., Hsu, C.H., Hidayati, S.C., Cheng, W.H., Chen, Y.J.: Computer-aided classification of lung nodules on computed tomography images via deep learning technique. OncoTargets Therapy 8, 2015–2022. (2015)

Learn to Code, an Interactive Application to Promote Mobile Student-Centred Learning Anne-Gaelle Colom and Wendy Purdy(&) University of Westminster, 115 New Cavendish Street, London W1W 6UW, UK {A.Colom,W.Purdy}@westminster.ac.uk Abstract. This paper discusses the first two phases in the development of a Learn to Code online web-based application, presents our initial findings and outlines the next stage of the research. First year Computer Science students start university with a wide range of qualifications, competencies, learning styles and experience with respect to computer programming. Research has found that the way programming is taught impacts on students’ confidence, their level of interest for the subject and whether they continue with programming. A common criticism of the teaching of computer programming is that it is too ‘abstract’ and hard to grasp, leading to students losing interest in the subject. The application is a digital resource developed to build students’ confidence in three common programming languages. The application uses code visualisation, web animation, lessons and video. The code visualiser analyses the code entered by the student and shows them graphically - in real-time - the code in the form of a flowchart. The application promotes learner autonomy and was designed to be accessible from multiple platforms and devices to enable students to be mobile learners. Feedback has been positive. Future work will extend the functionality of the application and the collaboration with students as co-creators. Keywords: Learn to Code learning
Mobile learning


Dynamic flowcharts
Technology enhanced-

1 Introduction Our large cohort of undergraduate computer science students start University with a diverse range of qualifications, competencies and prior programming experience and this presents challenges for the teaching of programming. There is criticism that computer programming teaching is too ‘abstract’ for many novice learners [1]. Students find learning to program difficult, and it is associated with high dropout rates [2]. This paper discusses the design and development of a Learn to Code (hereafter L2C) application and outlines how it aims to support the teaching and learning of programming.

© Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 232–241, 2021. https://doi.org/10.1007/978-3-030-49932-7_23

Learn to Code, an Interactive Application to Promote

233

2 L2C Design Principles and Justification Our initial proposal was to develop a platform or framework that would enable the development of programming teaching material for first year students with no programming background. To ensure the tool would work on as many platforms as possible (desktop and mobile devices) a web solution with progressive enhancements was decided as the best solution. This would ensure the tool would be usable beyond the release of new OS versions. The L2C application aids students learning preferences in relation to the visual, auditory and kinaesthetic (VAK) classification of learning styles defined by Sarasin [3]. In the VAK classification the visual learners engage best when they see the learning resources, while auditory learners engage best when hearing the material, and kinaesthetic learners prefer to engage by doing [4]. To accommodate these learning styles, the platform would consists of: – – – –

a graphical representation of previously prepared code examples; screencasts and interactive videos, including tests and quizzes; text-based and interactive lessons; and self-marking tutorials.

Student-centred teaching is designed to encourage students to engage in learning by relating to how students learn through activity and engagement [5]. The L2C application resources are accessible in any order, enabling students to choose what they will learn, how they learn, and how they assess their learning.

3 Methodology To evaluate the proposed application, we conducted an initial survey with 20 student respondents drawn from the three years of undergraduate study. The survey was used to gather feedback on the features of the application and to help prioritise our requirements. The students were sent a questionnaire of 22 questions with single choice, multiple choice, and free text answers, to gather data of how useful they would find various potential features of the tool, which features were essential to them, and which were good to have. We also asked them about their year of study, and their experience and confidence in programming. The data was collected in a spreadsheet as part of the Google form answers to analyse the results. The findings of this initial survey are discussed in Sect. 3.1. At the end of the code visualiser development (Phase One), the tool was demonstrated to 50 first year students during a class and the students were surveyed about the usefulness of the L2C application in order to ensure that the tool in development matched their learning needs. This survey will be referred to as the ‘Post Phase-One Survey’ and the results are discussed in Sect. 5. Finally, during the development of Phase Two, we created a small ‘Learn to Code’ community composed of staff and students, and we used an iterative methodology whereby we presented progress and prototypes to the community for feedback on functionality, interaction, user experience and style.

234

3.1

A.-G. Colom and W. Purdy

Pre-design Findings

At the initial survey stage, the students rated the proposed graphical tool as useful (85.7% 1st year, 87.5% 2nd year, and 100% 3rd year students). In addition, 90% of all the respondents stated that it was essential to be able to view changes in the graphical representation when editing code. This was not part of our original proposal. We had planned to incorporate animating flowcharts for given examples, but the students stated they would prefer a tool that would automatically enable them to visualise code as they edited it. Therefore, the graphical Visualising tool (code visualiser) became a priority and Phase One requirements were derived from student requests. 3.2

Review of Existing Tools

A range of existing code visualising tools were reviewed. Python Tutor [6] is a visualiser tool that shows a user how a computer executes each line of code in programs. It allows the user to edit the code and review the output but does not produce flowcharts. Visustin [7] is a flowchart generator that visualises source code. The flowchart produced can be saved in the common image formats (GIF, JPEG, PNG) and is a static image that cannot be animated. Visustin is not web-based, requires a Windows Operating System and a paid license. Graphviz [8] is a Graph Visualization Software that draws static graph layouts via a range of interfaces to include web browsers. The graph formats include images, Scalable Vector Graphics (SVG) and PDF. It provides options for colours, fonts and custom shapes. The graph descriptions are entered as a text language and so it does not convert source code. Dynagraph [9] is a related application that allows for interactive and incremental layouts [10]. From this review, it was specified that the proposed code visualiser would convert source code to a flowchart, be web-based with the flowcharts produced in SVG so that they could be animated. Ideally, the flowcharts would be customisable so that the colours and fonts can be changed.

4 The L2C Environment The application is a digital Web-based learning resource and available on any device enabling students to be mobile learners. Students can use the web application as a selflearning tool outside the University. Once a working version of the code visualiser was completed (Phase One), we moved to Phase Two and introduced a full web-based application skeleton with coding lessons and illustrative videos. We used the ‘Learn to Code’ community to give us feedback and seek improvements to ensure all users would be satisfied with the final application. Phase Two is underway. Phase Three will involve students as co-creators to develop the content, enhance the lessons and create a framework to incorporate selfassessing tutorial exercises.

Learn to Code, an Interactive Application to Promote

235

The first two phases of the L2C application address learning preferences by using a visualiser to present information in diagram form (visual) and tutoring videos (auditory and visual). Features to allow the user to modify and view updated visualiser content, and an interactive area where users can experiment with the lesson code and view the corresponding code structure, encourage learning by doing (kinaesthetic). 4.1

Phase One: Real-Time Code Visualiser

The code visualiser was developed to address student’s confidence in programming by explaining the basic concepts visually (see Fig. 1). The benefits of using flowcharts to visually represent algorithms in contrast to text-based solutions such as pseudocode has been documented. For example, Scanlan [11] concluded that flowcharts gave students more confidence in their understanding of an algorithm. The code visualiser is composed of two parts: a text editor on the left-hand side of the screen, and the flowchart section on the right-hand side. The text editor has been developed using codeMirror, an open-source text editor released under an MIT License, and developed in JavaScript for the browser. It is supported on all modern browsers and benefits from addons to support the syntax highlighting of all the languages we aim to cover, and comes with a wide range of theming options. CodeMirror also supports the main text editing commands for cut, copy, paste, edit and delete, native to the platform used by the user, and also benefits from a rich programming API. The code entered in the text editor being on the webpage, means that it is directly accessible from the DOM for analysis using JavaScript. The flowchart section consists of an area in which the flowchart shapes are drawn in real-time when the user saves their code. This is carried out by a suite of JavaScript libraries developed by the team. – SVG3.js, which allows the creation of an SVG that represents primitive objects such as lines, arrows, text, rectangle with text, diamond with text and ellipse with text. – generateFlowChart.js which translates code entered by the user into a set of instructions (intermediary language) to produce the flowchart. These instructions are read by the drawChart library. – drawChart.js allows the creation of an SVG that represents a flow chart corresponding to given JavaScript code, and the grouping of its graphical elements and their corresponding code. This library is dependent on the SVG3.js and produces the SVG corresponding to the instruction produced by the generateFlowChart.js library. – animateCode.js which allows the animation of an SVG while code is executed step by step, as well as and its corresponding lines of code. The code visualiser analyses code entered by the student and shows graphically in real-time the flow of the code. It enhances learning by enabling students to view a graphical representation of their code, thereby increasing their understanding of how code flows, and appreciate the processes illustrated by using the application. The flowchart is produced in SVG, which allows for scalability, interactivity and animation. To enhance the student understanding of the code, each graphical element

236

A.-G. Colom and W. Purdy

can be highlighted to see its corresponding line of code and vice versa. Students are able to modify the content and view graphically the effect of the programmed manipulation. In this way, computer code is made more ‘concrete’. The user can ‘see’ the code and how it flows. It is anticipated this will help a student design and understand algorithms, and help the learner in gaining confidence and understanding in how computer code works. As an addition, we are developing an Object Visualiser to graphically show arrays and objects and the effect of running commands to modify their content or access their data.

Fig. 1. The code visualiser

4.2

Phase Two: Lessons and Videos to Support Learning of Key Programming Concepts

The main difference between computer science, in particular when looking at programming, and other scientific subjects such as biology, physics or chemistry, is that it does not follow the ‘rule of nature’, but the rules that one computer scientist has defined. As students learn a programming language, the person that invented that particular language has defined a vocabulary, a language structure, and rules of how the instructions get translated into low-level programming language and electrical signals. The challenge is to accept that the grammar and vocabulary of every new programming language encountered will be different and may contradict those of previously learnt languages, and constitutes ‘troublesome knowledge’, i.e. knowledge that is conceptually difficult or counter-intuitive [12]. When teaching this discipline, we have to get students to learn the programming language that they will use on the module, and get them into the habit to carry out extensive testing, write elegantly, and comment their code to ensure good software

Learn to Code, an Interactive Application to Promote

237

quality and maintainability. Computer science and software engineering is almost a new philosophy of working that students must adopt. The second development phase added lessons in fundamental programming concepts such as variable assignment, selection, repetition. Each lesson is available in the three core programming languages taught during the first year: Python, Java and JavaScript, and scales to the available viewport (see Fig. 2). Each lesson relates to the design and development of a program to solve a certain task which is an area identified as difficult for novice programmers by Lahtinen et al. [1]. When developing the L2C Lessons, we identify the threshold concepts and ‘troublesome knowledge’ [12] for the topic covered. The student selects the language they wish to view the lesson in and can switch between the programming languages to compare syntax differences. Following feedback, an enhanced comparison feature was added (see Fig. 3) to show code examples for selected languages adjacent to each other. This enables students who have experience of one or more of the programming languages covered to learn a new language and code structure by applying prior learning and threshold concepts to help them identify the ‘troublesome knowledge’ associated with the new language easily, and hence facilitating their learning. Students can experiment with the lesson code in an interactive area and view code structure in the visualiser. Stage two has extended the collaboration between staff in the design and development of the application, and collaboration between staff and students in testing and the development stages.

Fig. 2. Example lesson on mobile landscape and portrait

238

A.-G. Colom and W. Purdy

Fig. 3. Comparison feature on desktop/tablet and smaller device

For each programming concept covered in the lesson, L2C offers corresponding videos. There is a video that explains the concept and then a series of videos of worked examples is offered. The students can either watch the video to understand the concept covered, or use it once they have attempted to code a solution for the problem given, putting them again in the driving seat of their learning and making them autonomous learners. The videos are based around the other L2C resources, and consist of screencasts with voice over, and make use of the code visualiser and lessons to show to the students also how these resources can be used to enhance their learning (see Fig. 4).

Fig. 4. Video example

Learn to Code, an Interactive Application to Promote

239

5 Findings and Discussion The code visualiser (Phase One) was demonstrated to first year students at the end of their first semester and then surveyed using an in-class Nearpod clicker style quiz. In this ‘Post Phase One Survey’, students were asked 3 questions. The first question aimed at finding out the usefulness of the developed code visualiser, and out of the 50 respondents, 96% reported that they felt the L2C application would help them gain a better understanding of programming (68% said it would help them greatly, 28% said it would help them a little, 4% said they already had a good understanding of programming). The students were shown a first attempt at the array visualiser and were then asked whether this tool would help them understand objects and arrays. Again, 96% of the students confirmed that they felt it would help them get a better understanding, the remaining 4% saying they already had a good understanding and therefore the tool would not help them. Finally, the students were asked to provide feedback in the form of free text, and again the responses were positive, with many students asking to have access to the tool as soon as possible. After further development of the application to include lessons, videos, interactive examples and code that opens in the code visualiser, Alpha testing was carried out with a small pool of students from the ‘Learn to Code’ community to gather feedback on the L2C application. Feedback has been positive, and students expressed how useful the tool was to grasp new concepts in computer programming. Final year students felt it would “help teaching new coders immensely”, and the tool would have helped them grasp the initial concepts of programming quicker and they would have gained a deeper understanding of the subject “a lot sooner in their coding life”. In response to staff feedback, an enhanced comparison feature was added. A cut down version of the tool was released online under an academic license, and was presented at the Bett 2017 education technology industry show. The tool received positive feedback and colleagues from secondary schools and Higher Education institutions expressed an interest in using it as part of their teaching. The tool is now being used by 1230 unique users across 53 countries with 53% of our users in the United States of America.

6 Conclusions and Future Work Our initial findings are that students find the application useful for learning programming. To date, we have not collected data to measure performance after using the application due to a change in the programming language learnt by 1st year students this academic year. The application has been designed as a self-learning tool and as a solution to large student cohorts arriving with diverse programming backgrounds. It presents learning materials in different format so that students can select their preferred learning style, i.e., visual, auditory or kinaesthetic. We will endeavour to integrate introductions to the

240

A.-G. Colom and W. Purdy

tool into core first year programming modules and encourage students to use the tool as part of their programming self-study. The application is a digital Web-based learning resource and available on any device enabling students to be mobile learners. The code visualiser tool differs from other solutions we reviewed in that it displays updated flowcharts of source code that is being developed iteratively, providing an effective real-time graphical representation of the code under development. The flowchart produced allows for scalability, interactivity and animation. The flowchart produced by the code visualiser can easily be customised programmatically. The font size, font family, as well as the colours used for the text, shapes or backgrounds can be changed, and the team will work on bringing this new customising functionality to the users. The team will also ensure the L2C is fully accessible to all users. For example, dyslexic students can select a non-white background colour and a larger font size to facilitate ease of reading. Phase Three will involve students as co-creators to further develop the L2C Application. We have identified a number of improvements and additions for Phase Three, which can be tackled by students with a range of skills and abilities, hence students should be able to find something they can contribute to. The code visualiser can only analyse code written in JavaScript and Java, therefore we need to develop the tool to dynamically produce flowcharts for Python code. A Content Management System to facilitate the addition of new lessons is being developed. An array and object visualiser is near completion. The lessons will be expanded to add function calls, arrays, and objects and video examples will be added to all lessons. To enhance student-centred activity, a framework to incorporate selfassessing tutorial exercises will be created. A future goal is to review requests to add further languages such as PHP, ES6 and Swift to the lessons. Phase Three will enable students to contribute code to an application that they used to learn to code and is a fitting way to close the project loop.

References 1. Lahtinen, E., Ala-Mutka, K., Järvinen, H.: A study of the difficulties of novice programmers. In: Proceedings of the 10th Annual SIGCSE Conference on Innovation and Technology in Computer Science Education, Portugal, pp. 14–18 (2005) 2. Robins, A., Rountree, J., Rountree, N.: Learning and teaching programming: a review and discussion. Comput. Sci. Educ. 13(2), 137–172 (2003) 3. Sarasin, L.C.: Learning Style Perspectives, Impact in the Classroom. Atwood Publishing, Madison (1999) 4. Dwi Surjono, H.: The design of adaptive e-learning system based on student’s learning styles. Int. J. Comput. Sci. Inf. Technol. 2(5), 2350–2353 (2011) 5. Hockings, C., Cooke, S., Bowl, M.: Learning and teaching in two universities within the context of increasing student diversity: complexity, contradictions and challenges. In: David, M. (ed.) Improving Learning by Widening Participation. Routledge, London (2010) 6. Python Tutor. http://www.pythontutor.com/. Accessed 21 May 2019 7. Visustin v8. https://www.aivosto.com/visustin.html. Accessed 21 May 2019 8. Graphviz. https://www.graphviz.org/. Accessed 21 May 2019

Learn to Code, an Interactive Application to Promote

241

9. Dynagaph. https://www.dynagraph.org/. Accessed 21 May 2019 10. Ellson, J., Gansner, E.R., Koutsofios, E., North, S.C., Woodhull, G.: Graphviz and dynagraph – static and dynamic graph drawing tools. In: Jünger, M., Mutzel, P. (eds.) Graph Drawing Software. Mathematics and Visualization. Springer, Heidelberg (2003) 11. Scanlan, D.: Structured flowcharts outperform pseudocode: an experimental comparison. IEEE Softw. 6(5), 28–36 (1989) 12. Meyer, J., Land, R.: Threshold Concepts and Troublesome Knowledge: Linkages to Ways of Thinking and Practising within the Disciplines. Enhancing Teaching-Learning Environments in Undergraduate Courses, Occasional Report 4 (2003)

Emerging Technologies and Augmented Reality in the Development of Learning and Human Potential Barba Téllez María Nela1 , Pullas Tapia Paúl Santiago1 Mocha-Bonilla Julio Alfonso2(&) , and Morales Jaramillo María Belén1

,

1

Facultad de Ciencias Humanas y de la Educación, Universidad Técnica de Ambato, UTA, Carrera de Psicopedagogía, 180103 Ambato, Ecuador {mn.barba,paulspullas,mb.morales}@uta.edu.ec 2 Facultad de Ciencia Humanas y de la Educación, Universidad Técnica de Ambato, UTA, Unidad Operativa de Investigación y Desarrollo, Carrera de Pedagogía de la Actividad Física y Deporte, 180103 Ambato, Ecuador [email protected]

Abstract. This article establishes a frame of reference on the use of Augmented Reality (AR) applications in education, the significance of neurobiological maturation and enriched environments as conditioning factors for the development of human potential. A literary review of recent research on this subject was carried out, using as reference sources the bases of data: Google academic, Academic Search Complete, Lilacs and Scielo, using as keywords the terms: “Human Potential, Augmented Reality, Enriched Environments, Mental Processes”, in Spanish and English. The time period considered was the last five years. The search made it possible to identify multiple related works, but only the most significant were included in the review. The selection of papers included articles, research reports, monographs and book chapters. The analysis of the identified contents was carried out in function of a perspective of reflection, using for it the analytical-synthetic, inductive-deductive, historicallogical and dialectical methods. After the analysis of the information gathered, it was possible to conclude that the education of emotional learning capacity, sensory integration and significance in learning are neuropsychological conditions of learning mediated by Augmented Reality, considering challenges for education professionals the educational application of emerging technologies and the evaluation of their impact on the integral development of the student. Keywords: Emerging technologies
Augmented reality
Emotional learning
Significant learning
Sensory integration

1 Introduction The concept of Augmented Reality (AR) “is to combine what is not explicit with what exists imperceptibly and offer users an improved or augmented representation of the world around them” [1]. This emerging technology has many potentialities for © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 242–252, 2021. https://doi.org/10.1007/978-3-030-49932-7_24

Emerging Technologies and Augmented Reality

243

multisensory interaction, emotion activation, and training in social skills, since it is a dynamic process that allows the integration and creation of virtual learning environments, it also allows to represent the complexities of the real world, or even social interactions in a simulated way [2] and [3]. The development of learning and human potential, as well as the application of the different types of augmented reality enrichment for different educational purposes constitutes today one of the great dilemmas of Psychology and Pedagogy, which have strengthened relations with neuroscience for a greater understanding of the complexity of the development of human potential, but no answers have yet been found to the following questions: Which are considered neuropsychological conditioning factors for the development of learning and human potential in conditions of application of AR? How to modify cognitive structures, and traditional forms of learning from the application of enriched environments of augmented reality? What is the application of different types of augmented reality enrichment for various educational purposes? How to achieve greater significance in brain neuroplasticity? [4] and [5]. The previous questions show the need to continue deepening from neuropsychology in the complex interactions that occur in ontogeny where it is necessary to consider the conditions in which learning takes place and the development of human potential. In this sense, it is not intended to give finished criteria on the aspects that will be addressed; the objective of this article is to describe the trends in the approach to educational applications of emerging technologies, considering the types of enrichment of augmented reality in learning and in the development of human potential. 1.1

Enrichment of Augmented Reality: Added Motivation for Learning and Development of Human Potential

Transformations are taking place in the world because of technological development and orders that have been incorporating visual experiences and diversifying communication channels. This transformation or combination of worlds and experiences can be visualized from two sources: (1) virtual reality and (2) augmented reality. The first is digital and requires hardware and software for its operation, while the second is the combination of the virtual with the real generating interactive experiences to the user that provide relevant knowledge about the environment in real time. According to the information provided by the company AppBrain, within the top of applications, education is shown as the category that has the most app within its database [6], but how many of them are augmented reality (RA) and how many are virtual reality (RV)? What is the acceptance of RA and RV among people? Augmented reality (AR) allows us to enrich the perception of the environment through the senses with the support of digital components that are superimposed in real time from images, markers or any type of information generated virtually on the physical world [3]. On the other hand, we must remember that there is no dissonance between virtual reality and augmented reality since one is related to the other because they have characteristics of inclusion of virtual models such as 2D and 3D graphics [7]. The enhancements of augmented reality constitute an added motivation for the learning and development of human potential, by placing the student in conditions, whether confronting realities known for their essence, or unknown for their appearance,

244

B. T. M. Nela et al.

which facilitates the recreation of three-dimensional images, which are nothing more than simultaneous images sequenced, complex and integrated producing multiple sensoperceptions, which promote greater connectivity with the environment, where it is obvious that brain neuroplasticity intervenes, which is understood as the capacity to modify the number of synapses, of connections from neuron to neuron, demonstrating that the brain is not an immutable structure, and that thanks to its plasticity it responds to new learning experiences, adapts to the environment, and is capable of constantly creating new neuronal connections if it is provided with adequate stimulation [8], by processing a greater amount of information and representing reality through more complex mental models through AR, which give dynamism and variability to the forms in which learning takes place. The enrichment of augmented reality, when it responds to students’ learning styles and multiple intelligences, facilitates the integration of sensorimotor schemes, communication, language, affective and experiential codes, as well as attitudinal and entrepreneurial codes, which activates the integral functioning of the brain and elevates human potential based on hybrid cognition. The educational value of emerging AR technologies when linked to play activities is significant. In this sense, the studies carried out by [9] and [10] are attractive, demonstrating that leisure activities related to the eight intelligences pose emotional challenges for students in terms of their ways of learning, as they are endowed with multisensory components with suggestive attractions for learning, revealing the need for flexible and attractive designs, and the use of alternative procedures that offer variants in the forms of teaching, and in the forms of learning [8]. The development of emotions towards learning is the source from which new motives for learning and the need for lifelong learning constantly arise. The awakening of emotions constitutes a vital element of learning with significance depending on the intensity of stimuli and emotions. In the enriched environments of AR, it is recommended to assume the theory of Multiple Intelligences, which allows exploring learning styles (visual, auditory, linguistic, or logical) opening a range of possibilities for learning of all kinds, which leads to the promotion of new ways of learning from teaching (reflexive, intuitive, analytical, conceptual, perceptive, motor, emotional, visual, auditory, intrapersonal, and interpersonal). In recent years, there has been talk of how the brain is capable of learning in different ways, using various strategies and elements of the environment. One of the most significant contributions in this direction are those made by [11] in his research on the multiple intelligences that make up the human brain. He explains, in his theory, that the brain does not have only one type of intelligence, but several interconnected intelligences, but that at the same time they can work independently and have an individual level of development, using various alternatives, among them, music and other experiences [12, 13] and [14]. Research using robot models and applications based on augmented reality technology, Robotics and 3D modeling have yielded positive results regarding their usefulness in providing an effective response from an innovative perspective to the new learning styles of primary and secondary school students [15]. The emerging technology of AR is especially useful for children with little imagination and learning difficulties, but it is equally important for stimulating

Emerging Technologies and Augmented Reality

245

high-potential brains, helping them to live the learning situation in a more intense and fun way [16] and [17]. The studies of [18, 19] and [20], pose challenges for teachers of all levels of education, who should be prepared in the pedagogical uses of emerging technologies and the evaluation of their impact on the integral development of the student. 1.2

The Enrichment of AR in the Capacity for Emotional Learning

Emotion and cognition are a necessary binomial in learning processes mediated by emerging AR technologies. In the studies by [21] and [22]; it is demonstrated that learning processes are closely related to emotions, that the plasticity, flexibility and adaptability of the brain can only be enhanced through the implementation of favorable learning environments and climates, resulting in a counterproductive lack of comprehensiveness of the cognitive and affective in learning processes mediated by emerging technologies of the AR. Neuroscience has revealed the role of emotions in mental processes and their importance in the development of brain functions [23]. The emotional brain dynamizes the human potential from the mind-body-environment interactions. According [8], the learning process must involve the whole body and the brain, the latter acts as a receiving station of stimuli from the environment, considering the need to learn to capture emotional signals [24], and offer emotional education throughout life [25]. Learning in environments mediated by emerging AR technologies provokes intense emotions that activate physiological responses that are difficult to control, but that must be learned to enjoy positively. Scholars of Psychology and Pedagogy, including [8, 26– 28] and [29], defend the idea that without emotions there is no learning. Learning without emotions generates demotivation, stress, depression, anguish, boredom and can even place students in situations of academic and school risk [30]. The contributions of neuroscience that serve as support to the studies of Positive Psychology are considered relevant, which deepens the cerebral functioning of emotions, proving that the neural circuits involved in emotional well-being have plasticity to adapt to the changing situations of the enriched environments of the AR. This is still an unfinished task in pedagogical research, considering that the emotional learning category must acquire connotation and scientific systematization with respect to its understanding as a high capacity to be developed in the student who learns with the emerging technologies of AR. To learn, to grow and to develop in the enriched environments of AR; to elaborate emotions and anxieties, to reflect, to decide and to act in each learning situation, depends largely on the management of the emotional learning capacity. Emotional learning ability is understood, as the subject is potential to feel, express, and control their emotions in simulated or non-simulated significant situations. The affective states derived from the enriched environments of AR are highly complex and can negatively affect the neuropsychological balance of each individual, hence the need for their education and development.

246

1.3

B. T. M. Nela et al.

Enrichment of AR in the Significance of Learning

From birth itself, there are capacity differences between individuals, but many of them can be enhanced in enriched environments. For behavioral psychologists, the principles of learning are valid for all levels, because the individual always tends to repeat those behaviors that achieve their goal and eliminate those that do not. One of its representatives is Skinner, who describes how reinforcements form and maintain a certain behavior. However, research results show that learning is conditioned by the brain of the learning subject, the situations and types of environments where he learns [31]. Learning is closely linked to the development of human potential, considering that human learning is a process of knowledge construction, which occurs gradually and progressively through different internal functions of the brain that act in the form of a system, and that transcend behavior. In cognitivism, where one of its fundamental representatives was David Ausubel, it was advocated that prior knowledge constitutes a condition for learning; however, it is not clear how the significant is incorporated into the cognitive structure, and brain neuroplasticity? In the case of Operative Pedagogy, based on the work of Jean Piaget, we start from the conception that knowledge is a construction mediated by meanings made by the individual through his activity in different environments, where knowledge will be more or less comprehensible to the subject depending on the intellectual instruments he possesses. The socio-cultural approach of Lev S. Vigotsky is considered valid insofar as it allows the active and participative learner to understand that he approaches knowledge and, from there and from his previous experiences, constructs meanings, but not in an isolated way but in interaction with others. Note that learning is not based on the transmission-reception of knowledge, but on a process of “mediated construction of meanings” in which the actors of the educational process, according to Vigotski, “are part and product of the activity, the context and the culture” [32]. Vigotsky asserted that psychic phenomena are social from their very origin, and are not given for the last time now of birth, but that they will develop in dependence of the conditions of life and education to which the subject is exposed [31]. The application of AR enhances the stimulation of mental processes (operative pedagogy), as well as allowing the construction of knowledge with a more active role of the students, and in conditions of interaction with others. According to [33], the interaction generated by AR in different learning environments is one of its main advantages, showing the possibility of analyzing interactions between students in the real context of the classroom, where non-violent behavior can be rewarded, with the desire to evaluate school coexistence in relation to academic performance. 1.4

The Enrichment of AR in Sensory Integration

Throughout his early years and up to adolescence, the child builds his intellectual structures and a representation of the outer world, considering sensory integration as a neurological condition for the development of human potential, which not only depends on the organic (biological) constitution of the student, but also of the enriched environments of augmented reality, which allow us to perceive, discriminate, interpret

Emerging Technologies and Augmented Reality

247

and respond to different codes, producing new sensations that, when integrated with those already stored, generate simultaneous multi-sensations and sequences of visual images, expressed at different levels: visual, auditory, and kinesthetic. Sensory integration is a fundamental neurological process for the growth of human potential, which can be strengthened or weakened depending on the changing conditions of the environment. According to [34], when students have a deficit in sensory integration, adults must modify the environment and even serve as sensory “filters”, since the brain is blocked when there is a disorganized flow of sensory information or when sensations are not of good quality and therefore cannot be discriminated against. Neuroplasticity according to [35] is a process by which neurons manage to increase their connections with other neurons and make them stable because of experience, learning, sensory and cognitive stimulation. In this case, it is interesting to know when an area of significance in brain neuroplasticity occurs. The above reflections reveal the significant value of brain neuroplasticity, learning experiences, and environmental stimuli for the development of human potential. “The plasticity of the human genome is such that there is no single way in which the human being develops that is independent of the opportunities for fulfillment provided by the culture in which that being is born and grows” [36].

2 Methodology A bibliographic review of 60 documentary sources was conducted on academic Google, Academic Search Complete, Lilacs and Scielo; the keywords used were: Neuropsychological Determinants; Human Potential; Enriched Environments, Emerging Technologies, Augmented Reality, Emotional and Educational Consequences of AR, Significant Learning, Sensory Integration; in Spanish and English; with emphasis on publications from the last five years. The selection included articles from scientific journals, research reports and books, although 48 sources were cited in the text. The analysis of the identified contents was carried out from a perspective of reflection, using analytical-synthetic, historical-logical and dialectical methods.

3 Results The sources consulted allow us to learn the research in three groups, according to their applications to education. 1. Research Group with a Tendency to Innovate Emerging Technologies for Education It is inherent in what the application of this technology projects in the educational field to see a very strong trend in the aspect of innovation and application in the classroom; however, the experiences of its application have developed from the first signs in the educational part [37]. The Magic Book project developed by New Zealand’s active HIT group established a trend in education by interactively displaying three-dimensional objects for learning [38]; setting a baseline of AR

248

B. T. M. Nela et al.

didactic books as indicated [14]; currently getting students to generate their own booklets of AR models, one of the elements worth highlighting is the playful aspect of the learning process, teaching while playing is an element that is very well received by the student [14]. 2. Research Group that Deals with the Application of Emerging Technologies in the Classroom With respect to the application of emerging technologies in the classroom, the broad spectrum of studies in which the RA is found is considered, 6 significant trends in educational applications can be pointed out according to [17]: Learning based on discovery based on the potential of geolocation of augmented reality, the field of development of professional skills itself that allows improving understanding in training activities with environments and practical situations, educational games that enhance learning through leisure activity, modeling by means of virtual objects in three dimensions that allows to create to visualize 3D objects allowing to improve the capacity of spatial abstraction, the interactive books that facilitate the immersion towards the learning, besides the varied didactic materials designed for different types of areas of the science that thanks to the mobile technology and the use of tags (marks) and codes QR can facilitate the access to resources of great utility for the process of teaching learning. 3. Research Group on Emerging Technologies that Contribute to the Training of the Student and Notably Improve Their Confrontation with Real Scenarios In addition to the above-mentioned contributions, in fields such as Psychology, the RA has great potential with projects that allow facing realities in a more adequate way thanks to real time interaction, as indicated [39], in its study, where it enhanced resilient characteristics with the use of these technologies. In addition, the instructional process for practical learning is considered an important element in the training of the student and significantly improves their confrontation with real scenarios [40].

4 Discussion According to [18], the RA acquires a fundamental role in the field of education, from very technical components to recreational activities that generate a context of exploration and discovery by those who have used the emerging technologies of the RA. The Massachusetts Institute of Technology (MIT) as well as Harvard are prestigious institutions that have developed applications in games format to generate new learning experiences using their mobile devices [7], [41] quoted by [14] stated that: augmented reality cannot be the ideal solution for all the needs of educational applications but it is an option to consider. Children and adolescents today “have an intuitive technological conscience” [42] which has caused the traditional method of learning to be displaced more and more with the development of technology [43]. On the other hand, are parents able to generate an accompaniment and strengthen the knowledge provided by the facilitators? If the RA is in real time, will the connected society allow the development of intelligences in those who make use of it; daily life in a few years will

Emerging Technologies and Augmented Reality

249

require the use of technologies more and more [44]. Probably, AR is an ecosystem where it is not necessary to present physical aspects and in which students only require a printed RA, a webcam and an internet connection to learn [45] and [46]. The previous vision allows to visualize a divergence towards certain emerging technologies, as is the concrete case of the Augmented Reality (AR) technology, which has been shown with great potential for a different and attractive learning, with a great tendency in the proliferation of its application in the classroom, for its enrichment in the learning of students, and in the development of their human potential [47, 20] and [16]. Companies such as Gardner Research, bet on AR as one of the 10 technologies of great impact in the following years, concretely according to research 30% of users will use it, thanks to the diversification of mobile technologies whose gains lead to significant learning, collaboration and interaction, in addition to enriching the capacity for emotional learning, and sensory integration. There are many educational applications of augmented reality that have been developed over the years, being able to appreciate their enrichment in the application of learning content in three dimensions, ubiquitous learning collaborative and situated, the senses of the student body with the presence of immediacy and immersion, as well as a visualization of the invisible reducing learning from formal to informal; in addition to offering us the possibility in the educational context of enriching the information with different formats favoring the individualization and adaptation towards the types of intelligence and symbolic preferences; facilitating the comprehension of integral concepts and the decomposition of the objects in their parts, phases, stages or parts [48].

5 Conclusions RA-mediated learning is based on the contributions of neuroscience, and opposes traditional learning methodologies that turn the student into a passive observer. In the case of AR, mind-body-environment relationships are favored, producing different types of enrichment that favor learning and development of human potential, among them: sensory integration, education of emotional learning capacity, and the significance of learning. There is no doubt that the protagonism that AR is acquiring in education is becoming more and more connoted, due to the great facility that it offers to integrate and adapt to the educational context. According to [19], technical quality, usability, sensation of enjoyment and usefulness are determining variables in the perception that leads to the use of AR objects; therefore, educational environments lend themselves to this environment being forged and immersion into a world of learning possibilities shows a promising panorama in the application and development of this emerging technology in the educational context. It is evident that AR has demonstrated its potential under its fundamental characteristics in the classroom; therefore, its application has been enriched with a diversity of applications in different areas. It could be proved that the training offer related to these topics is still insufficient, considering that as education professionals become more familiar with the use of AR, its application will increase. It is a challenge for teachers at all levels of education, the educational application of emerging technologies and the evaluation of their impact on the integral development of the student.

250

B. T. M. Nela et al.

References 1. Ayres, A.J., Robbins, J.: Sensory Integration and the Child: Understanding Hidden Sensory Challenges. Western Psychological Services, Torrance (2005) 2. Barba-Téllez, M.N., Cuenca, M., Rosa, A.: Piaget y LS Vigotsky en el análisis de la relación entre educación y desarrollo. Revista Iberoamericana de Educación 42, 7–25 (2007) 3. Barroso-Osuna, J., Gallego-Pérez, Ó.: La realidad aumentada y su aplicación en la educación superior. Revista Caribeña de Investigación Educativa (RECIE) 1(2), 111–124 (2016). https://doi.org/10.32541/salome.2016.v1i2 4. Basogain, X., Olabe, M., Espinosa, K., Rouèche, C., Olabe, J.: Realidad Aumentada en la Educación: una tecnología emergente. Escuela Superior de Ingeniería de Bilbao, EHU (2007) 5. Billinghurst, M., Kato, H., Poupyrev, I.: The MagicBook: a transitional AR interface. Comput. Graph. 25(5), 745–753 (2001). https://doi.org/10.1016/s0097-8493(01)00117-0 6. Bisquerra, R.: Educación emocional y competencias básicas para la vida. Revista de investigación educativa, RIE 21(1), 7–43 (2003) 7. Bisquerra, R., Pérez-González, J.C., Navarro, E.G.: Inteligencia emocional en educación. Síntesis, Madrid (2015) 8. Bonifaz, E., Benítez, M.G., Tenelanda, D., Guerrero, D.: The use of augmented reality through dental simulator software to improve anesthesia infiltration techniques. Int. J. Adv. Sci. Eng. Technol. (IJASEAT) 6(4), 75–78 (2018) 9. Bruner, J.: Acción, pensamiento y lenguaje, vol. 19. Alianza Editorial SA, Madrid (1984) 10. Almenara, J.C.: Presentación: Aplicaciones de la Realidad Aumentada en educación. EDMETIC 6(1), 4 (2016). https://doi.org/10.21071/edmetic.v6i1.5805 11. Almenara, J.C., Osuna, J.B.: The educational possibilities of Augmented Reality 5(1), 7 (2016). https://doi.org/10.7821/naer.2016.1.140 12. Almenara, J.C., Osuna, J.B., Obrador, M.: Realidad aumentada aplicada a la enseñanza de la medicina. Educación Médica 18(3), 203–208 (2017). https://doi.org/10.1016/j.edumed.2016. 06.015 13. Almenara, J.C., Osuna, J.M.B.: Posibilidades educativas de la Realidad Aumentada. J. New Approaches Educ. Res. 5(1), 46–52 (2016) 14. Pérez, O.C.: La integración de las neurociencias en la malla curricular de la formación inicial de docentes en Cuba. Paper presented at the Conferencia pronunciada en el Congreso Internacional Neurociencias, Familia y Educación (2015) 15. Campos, A.L.: Los aportes de la neurociencia a la atención y educación de la primera infancia (2014) 16. Carracedo, J.D.P., Méndez, C.L.M.: Realidad Aumentada: Una Alternativa Metodológica en la Educación Primaria Nicaragüense. IEEE-RITA 7(2), 102–108 (2012) 17. García, M.E.C., López, A.L.: La teoría de las inteligencias múltiples en la enseñanza de las lenguas. Contextos Educativos. Revista de Educación (17), 79–89 (2014). https://doi.org/10. 18172/con.2594 18. Castrillón, J.J.C., Cala, M.L.P.: Inteligencia emocional y rendimiento académico en estudiantes universitarios. Psicología desde el Caribe 32(2), 268–285 (2014) 19. Prado, M.J.C., Latorre, S.: Efectos del videojuego Spock sobre la mejora de la inteligencia emocional en adolescentes (2015) 20. del Moral Pérez, M., García, L.F., Duque, A.G.: Videojuegos: Incentivos Multisensoriales Potenciadores de las Inteligencias Múltiples en Educación Primaria. Electron. J. Res. Educ. Psychol. 13(36), 243–279 (2015)

Emerging Technologies and Augmented Reality

251

21. Delgado, M., Arrieta, X., Riveros, V.: Uso de las TIC en educación, una propuesta para su optimización. Omnia 15(3), 58–77 (2009) 22. Campos, B.D.: Realidad Aumentada en la educación. Entorno (61), 47 (2016). https://doi. org/10.5377/entorno.v0i61.6129 23. Díaz-Barriga, F.: Cognición situada y estrategias para el aprendizaje significativo. Revista electrónica de investigación educativa 5(2), 1–13 (2003) 24. Feldman, D.E., Brecht, M.: Map plasticity in somatosensory cortex. Science 310(5749), 810–815 (2015). https://doi.org/10.1126/science.1115807 25. Fernández, B.: Factores que influyen en el uso y aceptación de objetos de aprendizaje de realidad aumentada en estudios universitarios de Educación Primaria. EDMETIC 6(1), 203– 220 (2017) 26. Ferrés, J: Las pantallas y el cerebro emocional: Editorial Gedisa (2014) 27. Fombona, J., Pascual-Sevillana, Á., González-Videgaray, M.: M-learning and augmented reality: a review of the scientific literature on the WoS repository. Comunicar 25(52), 63–72 (2017). https://doi.org/10.3916/c52-2017-06 28. Retana, J.Á.G.: La educación emocional, su importancia en el proceso de aprendizaje. Educación 36(1), 1–24 (2012) 29. Gardner, H.: La Teoría de las Inteligencias Múltiples. Basic Books, Nueva York (1983) 30. Goleman, D.: La inteligencia emocional, porque es más importante que el cociente intelectual, Buenos Aires, Vergara, vol. 13 (1996) 31. Grasset, R., Dunser, A., Billinghurst, M.: The design of a mixed-reality book: is it still a real book? Paper presented at the Proceedings of the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality (2008) 32. Hinz, A.: Mejora de la satisfacción corporal en chicas y chicos preadolescentes: efectos a corto plazo de un programa escolar (2017) 33. Kaufmann, H.: Collaborative augmented reality in education. Institute of Software Technology and Interactive Systems, Vienna University of Technology (2003) 34. Llorente, M.D.C.L., Gutiérrez, J.J., Cabero-Almenara, J.: Evaluación por y desde los usuarios: objetos de aprendizaje con Realidad aumentada. RED Revista de Educación a Distancia 4(53), 1–17 (2017) 35. Mera, J.A., Roncallo, O.A., Valencia, R.E., Castaño, F.C., Villa, J.C.V.: Realidad aumentada: una herramienta tecnológica para respaldar la resiliencia psicológica en menores de edad víctimas del conflicto armado caso Caldono Cauca. Paper presented at the Memorias de Congresos UTP (2017) 36. Mesquida Jerez, M. C., & Pérez, A: Estudio de APPs de realidad aumentada para su uso en campos de aprendizaje en un entorno natural Edutec. Revista Electrónica De Tecnología Educativa, (62), 19–33 (2018). https://doi.org/10.21556/edutec.2017.62.1017. (a370) 37. Moreno, N.M., Leiva, J., López, E.: Robótica, modelado 3D y realidad aumentada en educación para el desarrollo de las inteligencias múltiples. Aula de encuentro 18(2), 158–183 (2016) 38. Mullen, T.: Realidad aumentada: crea tus propias aplicaciones: Anaya Multimedia (2012) 39. Pasaréti, O., Hajdin, H., Matusaka, T., Jambori, A., Molnar, I., Tucsányi-Szabó, M.: Augmented Reality in education. In: INFODIDACT 2011 Informatika Szakmódszertani Konferencia (2011) 40. Pintor-García, I., Fernández-Méndez, J.C., Bello-Grela, A.A.: Estudio sobre teoría de la mente en personas con síndrome de Down. Revista de Estudios e Investigación en Psicología y Educación (09), 085–089 (2015) 41. Espinosa, C.P.: Realidad aumentada y educación: análisis de experiencias prácticas. PíxelBit, Revista de Medios y Educación (46), 187–203 (2014). https://doi.org/10.12795/pixelbit. 2015.i46.12

252

B. T. M. Nela et al.

42. Acosta, F.P., Clavero, F.H.: La inteligencia emocional como predictora del rendimiento académico en la infancia: el contexto pluricultural de Ceuta. Revista Complutense de Educación 28(4), 1251 (2017) 43. Reinoso, R.: Posibilidades de la realidad aumentada en educación. Paper presented at the Tendencias emergentes en educación con TIC (2012) 44. Silva, R.S.: ¿La educacion necesita realmente de la neurociencia? estudios pedagógicos (valdivia), pp. 155–171 (2003) 45. Saldarriaga-Zambrano, P.J., Bravo-Cedeño, G.D.R., Loor-Rivadeneira, M.R.: La teoría constructivista de Jean Piaget y su significación para la pedagogía contemporánea. Dominio de las Ciencias, 2(3 Especial), 127–137 (2016b) 46. Serrano-Puche, J.: Internet and emotions: new trends in an emerging field of research. Comunicar 24(46), 19–26 (2016). https://doi.org/10.3916/c46-2016-02 47. Burgos, B.M.V., de Cleves, N.R., Márquez, M.G.C.: El cerebro que aprende. Tabula Rasa (11) (2009). https://doi.org/10.25058/20112742.380 48. Yuen, S.C.-Y., Yaoyuneyong, G., Johnson, E.: Augmented reality: an overview and five directions for AR in education. J. Educ. Technol. Dev. Exch. 4(1) (2011). https://doi.org/10. 18785/jetde.0401.10

Open and Distance Mobile Learning

Level of Digital Literacies Among Austrian College Students Assessed with an Online Survey Anita Kloss-Brandstätter1(&), Andreas Pester1, and Gila Kurtz2 1 Carinthia University of Applied Sciences, Villach, Austria {a.kloss-brandstaetter,a.pester}@fh-kaernten.at 2 Holon Institute of Technology (HIT), Holon, Israel [email protected]

Abstract. The digital age offers opportunities for the development of online information and communications technology (ICT) systems that emphasize the important role of digital literacy, including communication between people and the values necessary for working in an online environment, which greatly influences the processes and practices of learning and teaching. Especially in the field of mobile learning, digital and data-evaluating skills are of great importance, as this type of technologically extended learning is based on a comprehensive use of cloud and internet-based services. The purpose of this study was to evaluate the German version of a validated 54-item questionnaire and to assess the level of digital literacies and digital readiness of undergraduate and graduate students at the Carinthia University of Applied Sciences. An online questionnaire was developed based on the items designed by Kurtz and Peled [1]. The questionnaire included 54 items divided into seven areas: data research and retrieval; data validation; data management; data processing; teamwork; integrity awareness; and social responsibility. The items were translated into German language and their validity was evaluated with Cronbach’s Alpha in a pre-study on 155 students. After improvement of some items, all students from the Carinthia University of Applied Sciences were asked to complete the questionnaire. The first key outcome of this study is the validation of the German version of a validated 54-item questionnaire. The second key outcome is the level of digital literacies and digital readiness of undergraduate and graduate students at the Carinthia University of Applied Sciences. Finally, we want to find out if there are differences in the digital competencies between the various degree programs of the Carinthia University of Applied Sciences. Our findings should enable faculty and educational policymakers to identify the strengths and weaknesses of digital literacies of students from the Carinthia University of Applied Sciences. Keywords: Digital learning literacies


21st century skills
Learning skills

© Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 255–265, 2021. https://doi.org/10.1007/978-3-030-49932-7_25

256

A. Kloss-Brandstätter et al.

1 Introduction 1.1

Digital Literacies

Models of online and blended learning are gaining popularity in higher education, as institutions seek to support and facilitate modern, flexible educational opportunities for students, and create interactive digital learning experiences that are engaging and effective in undergraduate and postgraduate education. In part, the challenge of providing an interactive learning experience for students in large class groups and concerns about the quality of pedagogy in this environment have been key catalysts for the reconsideration of teaching approaches in higher education, as well as to overcome issues of distance between learners and instructors, and to facilitate increased peer interaction and collaboration. For information and digital literacy skills development, in particular, the adoption of online modes is increasingly popular [2]. The rationale for formulating ‘21st century skills’ is that the 21st century is very different from the 20th century in terms of the skills required for work, citizenship, and self-accomplishment [3]. These developments call for a new form of learning, which should be receptive to this continually changing world. In fact, certain skills and ways of thinking are becoming more and more essential for college students [4]. Part of the set of skills required for the 21st century are digital literacies, which might best be defined as “those capabilities which fit an individual for living, learning, and working in a digital society” [5]. In addition, digital literacy helps an individual navigate current digital environments actively, collaboratively, and in a participative manner [1]. Digital literacy includes communication between people and the tools necessary for working in an online environment, which greatly influences the processes and practices of learning and teaching [6]. These processes influence the training and professional development of college lecturers that are aware of the requirements of the digital age and its implications for education. 1.2

Conceptual Framework

The idea of this project was to apply a validated questionnaire on digital literacy to Austrian college students. The original questionnaire was conceived in English language and needed translation into German first. Seven domains represent digital literacy [1]: Data Collection; Evaluation of Data; Data Management; Data Processing; Teamwork; Integrity awareness; and Social responsibility. These domains are the basis for one’s ability to manage in the complex, ever-evolving 21st century environment. This capability is described as digital readiness, which is an interpretation of digital literacy, and is defined as the extent to which an individual or group has the knowledge and ability to integrate cognitive skills appropriate for activity in digital environments [7, 8].

Level of Digital Literacies Among Austrian College Students

1.3

257

Aim of the Study

The aim of this study was to implement a German version of a validated questionnaire of digital literacy of college students. Precisely, we wanted to answer the following research questions: 1) Is the internal consistency of the German version comparable to the internal consistency of the original version? 2) Are there any gender-specific differences in the seven dimensions of digital literacy? 3) Are there any differences between bachelor and master students in the seven dimensions of digital literacy? 4) Are there any differences between full-time and extra-occupational students in the seven dimensions of digital literacy?

2 Methods 2.1

Questionnaire

This online questionnaire was based on a validated 54-item questionnaire by Kurtz and Peled [1] and translated into German. The following Likert-scale for answering the items was applied: 1) Strongly disagree; 2) Somewhat disagree; 3) Neither disagree nor agree; 4) Somewhat agree; 5) Strongly agree. In addition, the following demographic and study-specific variables were assessed: age, gender, field of study and whether it was a full-time study or extra-occupational education. The online-questionnaire was designed and distributed using the QuestorPro (Blubbsoft GmbH, Berlin, Germany). All students of the Carinthia University of Applied Sciences (n = 2.167) received an e-mail with the invitation to participate in this survey (Tables 1, 2, 3, 4, 5, 6 and 7).

Table 1. Translation of items for the scale “Social Responsibility” into German. Item Item 1

Item 2

Item 3

Original I adhere to the rules of discourse and proper behavior in social networks I make sure not to reveal information about organizations without consent I make sure not to hurt others – people and organizations – online

German translation Ich halte mich an die Regeln des Diskurses und des richtigen Verhaltens in sozialen Netzwerken Ich stelle sicher, dass Informationen über Organisationen nicht ohne Einwilligung veröffentlicht werden Ich stelle sicher, dass ich andere - Menschen und Organisationen - nicht online verletze

258

A. Kloss-Brandstätter et al. Table 2. Translation of items for the scale “Data Collection” into German.

Item Item 1 Item 2 Item 3 Item 4 Item 5 Item 6 Item 7 Item 8 Item 9 Item 10 Item 11 Item 12

Original I know when I need to look for information I am able to identify information for research I am able to collect information from the web I can define the objective of the search I can articulate what information I need I know how to search effectively I can define research terms I can distinguish between types of search I can retrieve information from various sources I am able to collect information from databases I am able to re-locate information I can re-locate a specific web page

German translation Ich weiß, wann ich nach Informationen suchen sollte Ich kann Information zu Forschungszwecken richtig identifizieren Ich kann Informationen im Internet sammeln. Ich kann das Suchobjekt genau definieren Ich kann formulieren, welche Informationen ich brauche Ich weiß, wie man effektiv sucht Ich kann Forschungsbegriffe definieren. Ich kann zwischen Sucharten unterscheiden Ich kann Informationen aus verschiedenen Quellen abrufen Ich kann Informationen aus Datenbanken sammeln Ich kann Informationen wiederfinden Ich kann eine bestimmte Webseite wiederfinden

Table 3. Translation of items for the scale “Evaluation of Data” into German. Item Item 1

Original I am able to judge the degree to which information is practical or satisfies the needs of the task

Item 2

I am able to determine the information required for a specific task

Item 3

I am able to assess the accuracy of information I am able to assess the credibility of information

Item 4

Item 5

I am aware of the difference in credibility of information from various sources

German translation Ich bin in der Lage zu beurteilen, inwieweit eine Informationen praktisch ist oder die Bedürfnisse meiner Aufgabe erfüllt Ich kann bestimmen, welche Information für eine bestimmte Aufgabe notwendig ist Ich bin in der Lage, die Genauigkeit einer Information zu beurteilen Ich bin in der Lage, die Glaubwürdigkeit einer Information zu beurteilen Ich bin mir der unterschiedlichen Glaubwürdigkeit von Informationen aus unterschiedlichen Quellen bewusst

Level of Digital Literacies Among Austrian College Students

259

Table 4. Translation of items for the scale “Integrity Awareness” into German. Item Item 1

Item 2 Item 3

Original I understand the ethical consequences of the use of technology I understand the social consequences of the use of technology I do not acquire digital information, files, programs, databases, etc. via illegal means

Item 4

I do not use technology for purposes that are intimidating or threatening

Item 5

I am aware of the prohibition of illegal file download I am aware of copyright issues

Item 6 Item 7

Item 9

I am aware of appropriate acknowledgement of sources I use I am aware of the danger of being online to my data I am aware of cyberbullying issues

Item 10

I am aware of identity theft issues

Item 11

I am aware of e-theft issues

Item 12

I am aware of the danger from my online activities I am aware of the influence my online data has I am able to identify/avoid online fraud or identity theft situation

Item 8

Item 13 Item 14

Item 15

I am able to protect myself from online predators

German translation Ich weiß, welche ethischen Folgen der Einsatz von Technologie hat Ich weiß, welche sozialen Folgen der Einsatz von Technologie hat Ich nutze keine illegalen Mittel, um digitale Informationen, Dateien, Programme, Datenbanken usw. zu sammeln Ich setze Technologie nicht ein, um jemanden einzuschüchtern oder ihm/ihr zu drohen Ich weiß, dass das illegale Herunterladen von Dateien verboten ist Ich weiß, dass es Probleme mit dem Urheberrecht geben kann Ich weiß, dass ich die von mir genutzten Quellen angemessen referenzieren muss Ich weiß, dass Internetnutzung meine Daten gefährden kann Ich bin mir der CybermobbingProblematik bewusst Ich weiß, dass es Probleme mit Identitätsdiebstahl geben kann Ich weiß, dass es Probleme mit Gelddiebstählen im Internet geben kann Ich weiß, dass meine Internet-Aktivitäten Gefahren mit sich bringen Ich weiß, welchen Einfluss meine Online-Daten haben Ich kann erkennen, wenn im Internet ein Betrugs- oder Identitätsdiebstahl geschieht und weiß, wie ich diese Situationen vermeiden kann Ich kann mich vor Internet-Gaunern schützen

260

A. Kloss-Brandstätter et al. Table 5. Translation of items for the scale “Teamwork” into German.

Item Item 1

Item 2

Original During the preparation of a joint task, I know how to fit in among team members During the preparation of a joint task, I share my thoughts and insights with my peers

Item 3

During the preparation of a joint task, I know that I have an influence on the work process

Item 4

During the preparation of a joint task, I know what is expected of me

Item 5

While performing a joint task I feel that my contribution to the team is meaningful

Item 6

My peers are aware of my abilities and of what I can contribute

Item 7

I have no reservation regarding joint tasks I like to work with my peers on a joint task

Item 8

German translation Bei der Vorbereitung einer gemeinsamen Aufgabe weiß ich, wie ich mich in die Gruppe einfügen kann Bei der Vorbereitung einer gemeinsamen Aufgabe teile ich meine Gedanken und Erkenntnisse mit meinen Gruppenmitgliedern Bei der Vorbereitung einer gemeinsamen Aufgabe weiß ich, dass ich den Arbeitsprozess beeinflussen kann Bei der Vorbereitung einer gemeinsamen Aufgabe weiß ich, was von mir erwartet wird Bei der Durchführung einer gemeinsamen Aufgabe habe ich das Gefühl, einen sinnvollen Beitrag für die Gruppe zu leisten Meine Gruppenmitglieder sind sich meiner Fähigkeiten und meines Beitrags bewusst Ich stehe gemeinsamen Aufgaben unbefangen gegenüber Ich arbeite gern in Gruppen an gemeinsamen Aufgaben

Table 6. Translation of items for the scale “Data Management” into German. Item Item 1 Item 2 Item 3

Original When I store a file, I give it a specific name I story my files in designated folders I tag information

German translation Wenn ich eine Datei speichere, gebe ich ihr einen bestimmten Namen Ich speichere meine Dateien in entsprechenden Ordnern Ich zeichne meine Informationen mit entsprechenden Attributen aus („Tags”)

Level of Digital Literacies Among Austrian College Students

261

Table 7. Translation of items for the scale “Data Processing” into German. Item Item 1 Item 2 Item 3 Item 4 Item 5

Item 6 Item 7

Item 8

2.2

Original I am able to interpret information from multiple sources I am able to analyze information from multiple sources I am able to synthesize information from multiple sources I am able to write an appropriate response to a post I am able to use ICT to design or create new information from information already acquired I am able to visually organize data for learning purposes I can represent knowledge in a variety of ways such as PPT, website, blogs, etc. I am aware of the difference in written, graphic or video representations

German translation Ich bin in der Lage, Informationen aus mehreren Quellen zu interpretieren Ich bin in der Lage, Informationen aus mehreren Quellen zu analysieren Ich bin in der Lage, Informationen aus mehreren Quellen zusammenzuführen Ich kann eine angemessene Antwort auf einen Beitrag („Post”) verfassen Ich bin in der Lage, IKT zu nutzen, um neue Informationen aus bereits gesammelten Informationen abzuleiten oder zu entwickeln Ich bin in der Lage, Daten zu Lernzwecken visuell aufzuarbeiten Ich kann Inhalte auf unterschiedliche Art und Weise darstellen, wie z. B. PowerPoint, Webseiten, Blogs usw Ich bin mir der Unterschiede zwischen Präsentationen im Text-, Bild- oder Videoformat bewusst

Statistical Analyses

Statistical analyses were conducted with IBM SPSS (version 25). As measures of central tendency and dispersion, mean values and standard deviations were calculated. The reliability of the different scales was assessed using Cronbach’s alpha. Independent two-sample t-tests were applied to determine if the means of two sets of data with equal variances (approved with a Levene test) were significantly different from each other. The significance level was set to 5%; therefore, p-values 150 = Dunbar’s number) open to everyone with entry qualifications making it ideal for athletes who wish to expand their knowledge, skills, competencies without the necessity to engage in formal educational practices. The dual career guidelines highlight athlete’s re-integration to education or work by attempting to provide a flexible solution to athletes to attend educational programmes, especially for those who are travelling in professional athletic events and don’t possess the time to invest in cultivating their training or education. Hence the Dual Career Association goal is to share knowledge on the importance of education of athletes and the significance in preparing for the business world after their sports career. GOAL aims to harness the potential of MOOCS to empower athletes to invest in their educational/professional development by accessing and enrolling in modules that have a dual focus: (a) entrepreneurial/business studies and (b) sports sciences. GOAL innovates by introducing educational modules for athletes’ dual careers which have not been considered or applied in any European MOOC in terms of offering a full-course online experience encompassing associated content, learning design and facilitation of interaction among peers following a structured study guide/syllabus. Educational programmes for athletes have been created statically, using a traditional web-page for accessing content or at its most advanced stage, providing coaching, mentoring and content via a Virtual Learning Environment (VLE) (see for example the dual athlete career project) not foreseeing the importance of providing a dual career educational modality widely to a massive amount of athletes around Europe who wish to acquire knowledge and skills via flexible learning opportunities provided by a MOOC. GOAL permeates innovation in terms of offering MOOC modules for dual athletes careers by contributing to the transition (static and linear teacher-led low performing learning activities to flexible, student-led activity-based learning), responding to the demands of wider dispersed athletes and societies.

404

T. Tsiatsos et al.

The innovative character of GOAL continues toward the realization of an informed approach to alleviate the dual career challenge of athletes via the MOOC and dedicated Web dual career interface that mashes up the games, courses via MOOC and knowledge exchange area as means to offer a social perspective to communal constructing experiences, content and expertise in dual careers both from the educators, coaches, parents and sports organisations to athletes, students and young professionals perpetuating sustainable choices for athlete’s future careers.

3 GOAL’s MOOC Curriculum As it was presented before [8] a state of the art was conducted about the e-learning platforms and the result was that the most appropriate is Moodle (https://moodle.org/). This platform will contain the six courses of the curriculum: • Cycle 1: • Entrepreneurship • Personal Skills Development – Teamwork • Personal Skills Development – Decision Making Skills • Cycle 2: – Sports Management – Sports Marketing – Coaching in Sports In this study the GOAL consortium will present a case study and the evaluation of one course and more specifically Coaching

4 Coaching in Sports Sport coaching is as difficult and as demanding as any other aspect of sport. Good coaching and poor coaching often have impacts on the individual athlete or a team and can become magnified out of proportion to the coaching direction itself. The complete and well-trained sports coach is seemingly a multidimensional personality, possessing a wide range of technical, communication, and interpersonal skills. There is no one source from which strong sports coaches are produced. Many successful coaches were sports players with average physical talents; others developed coaching skills through formal academic or sports institute education. All sports coaches must possess certain attributes, some in greater measures than others, to provide effective direction to their athletes or teams. An important attribute is a technical knowledge of the sport. GOAL team’s experts tried to create a syllabus to cover most of the important aspects of coaching, such as: • Basic concepts of coaching and training • Didactic Methods in Sports • Strength Training

Evaluating a Coaching MOOC Course to Support Dual Career of Athletes

405

• Endurance Training. Learners are supplied with a lot of educational material (videos, presentations and more) to complete successfully this course.

5 Evaluation This section presents the methodology followed in order to evaluate our research work. 5.1

Participants

In order to test the research questions of this article 22 athletes and former athletes from all the countries of the consortium attended and completed the course. Their previous exposure on Coaching courses was measured with four 5-point Likert scale questions with the following anchors: 1. None, 2. Mild, 3. Moderate, 4. Significant, 5. Severe and the results are presented in Fig. 1.

Previous experience on Coaching 4 3.9 3.8 3.7 3.6 3.5 3.4 3.3 3.2 3.1 3

3.8

3.8

3.9

What is the degree What exposure have What is the degree of exposure of your you had before this of your theoreƟcal profession to course to Coaching? knowledge of Coaching? Coaching?

3.7

What was the degree of your pracƟcal involvement in Coaching before this course?

Fig. 1. Participants’ previous experience on coaching

5.2

Instruments

In order to evaluate game’s usability, project’s experts created a questionnaire and made available through the MOOC. The questionnaire consists of 41 questions measuring to what degree participants react favourably to the training and to what degree targeted outcomes occur as a result of the training event and subsequent reinforcement. The questions’ type is a 5-point Likert rating scale with the following anchors: 1 strongly disagree, 2 disagree, 3 neutral, 4 agree and 5 strongly agree.

406

T. Tsiatsos et al.

6 Data Analysis The statistical analysis was performed using Excel and the results are presented below. 6.1

Evaluation of Course and Structure

As it can be seen in Table 1 participants believed that this course is very important to their development and that the knowledge, they will acquire from it will help them improve their athletic and post athletic career. Table 1. Relevance of the course to your professional development/athletic career Coaching learning objectives and topics are suitable for me in regard to my professional/athletic career level Coaching learning objectives and topics are suitable for me in regard to my prior knowledge on the topic The degree of importance of this course to my professional development/athletic career is high My professional development/athletic career is dependent on such courses With the knowledge I acquire from this course I will be more effective in my current job/athletic career With the knowledge I acquire from this course I will improve my future job/athletic career

4.20 4.20 4.50 4.10 4.30 4.30

Additionally, evaluating the content of the course, participants are very positive with it evaluating it in almost all questions with an average of 4 and above (Table 2). They believe that the subjects covered in course was well segmented and structured (Table 3).

Table 2. Participants opinion Coaching Course learning content The The The The The The The The The The The The

MOOC is suitable for me regarding my professional/athletic career level MOOC is suitable for me regarding my prior knowledge on the topic MOOC is suitable for my learning goals learning objectives and topics are clearly defined instructions on how to follow this course were comprehensive and suitable MOOC improves my knowledge regarding coaching supporting reading material facilitated the study of the topics supporting reading material was informative enough supporting reading material was easy to follow multimedia material facilitated the study of the topics multimedia material was informative enough multimedia material was easy to follow

3.90 3.90 4.00 4.20 4.30 4.20 4.40 4.40 4.10 4.10 4.30 4.30

Evaluating a Coaching MOOC Course to Support Dual Career of Athletes

407

Table 3. Participants‘opinion about segmenting, sequencing and navigation of the MOOC Is the content segmented adequately (neither too wide nor too narrow)? Is the sequencing well structured? Is the navigation bar in balance between too many and too little sublevels and are the menu items well chosen?

6.2

3.7 3.7 3.8

Evaluation of Teaching Material

Participants are very positive with the video, texts and graphics presented to the course evaluating it in almost all questions with an average of 4 and above (Table 4). They also have a very positive opinion about the delivery of the course (Table 5). Table 4. Participants opinion about videos and graphics Texts were readable Texts were understandable Graphics were understandable The quality of graphics was high Graphics and video were readable Videos were understandable The quality of videos was high Videos were readable

4.2 4.2 4.1 3.9 4.2 4.2 4.1 4.1

Table 5. Participants opinion about course delivery Content of module was understandable? The volume of activities was equally distributed during the module Activities (i.e. quizzes) were clear Support and help were accessible when it was needed The learning material was structured clearly and user-friendly There were enough tools for interaction when it was needed

6.3

4.3 4.3 4.2 4.1 4.3 4.1

Impact of the Course

Comparing this coaching course with other same courses participants believe that the whole content of the course is better than other courses they have already attended and completed (Table 6). Finally, they believe that the impact that this course will have on their professional practice will be very significant (Table 7).

408

T. Tsiatsos et al. Table 6. Participants opinion about this course in comparison other sane courses

The degree of my exposure to courses about Coaching in the past was high The course content compared to the content of other courses about injury Coaching, I have attended in the past, is more informative The degree of my exposure to MOOCs in the past was high

3.8 3.6 3.8

Table 7. Participants opinion about the impact of the course to their professional practice This course will change my professional practice 3.6 The degree of the change in my professional practice will be high due to this course 3.8 This course will improve my professional practice 4.2

7 Conclusion and Future Work This paper presents the evaluation of Coaching Course in a MOOC platform to support dual career of athletes. This course has been deployed in the context of GOAL Erasmus +Sport project. The development of this course has taken into account the needs analysis and the design conducted in previous steps of the GOAL project and presented at IMCL Conference 2017 (Tsiatsos et al., 2018). The evaluation results show that the course is complete, well-structured and segmented, covering the most important parts of coaching. It is worth mentioning that the overall results are very satisfying. Participants seem to be very satisfied with the educational content (presentations, videos, documents etc.) The most satisfying result is that participants believe that this course will change and improve their professional practice. The next step is to evaluate this course and all the other 5 courses available on http://goal.csd.auth.gr/elearning/ on a larger user base. Acknowledgement. This project has been funded with support from the European Commission. This publication reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein. The authors of this research would like to thank GOAL team who generously shared their time, experience, and materials for the purposes of this project.

References 1. Amara, M., Aquilina, D., Henry, I., PMP.: Education of young sportspersons. (IoT 1). European Commission, Brussels (2004) 2. Aquilina, D.: A study of the relationship between elite athletes’ educational development and sporting performance. Int. J. Hist. Sport 30(4), 374–392 (2013). https://doi.org/10.1080/ 09523367.2013.765723

Evaluating a Coaching MOOC Course to Support Dual Career of Athletes

409

3. Aquilina, D., Henry, I.: Elite athletes and university education in Europe: a review of policy and practice in higher education in the European Union Member States. Int. J. Sport Policy 2, 25–47 (2010) 4. Brackenridge, C.: Women and children first? child abuse and child protection in sport. Sport Soc. 7(3), 322–337 (2004) 5. Stambulova, N., Alfermann, D., Statler, T., Côté, J.: ISSP position stand: career development and transitions of athletes. Int. J. Sport Exerc. Psychol. 7, 395–412 (2009) 6. Wylleman, P., Lavallee, D.: A developmental perspective on transitions faced by athletes. In: Weiss, M.R. (ed.) Developmental Sport and Exercise Psychology: A Lifespan Perspective, pp. 507–527. Fitness Information Technology, Morgantown (2004) 7. Burnett, C.: Student versus athlete: professional socialisation influx. Afr. J. Phys. Health Educ. Recreation Dance 16, 193–203 (2010) 8. Tsiatsos, T., Douka, S., Politopoulos, N., Stylianidis, P., Ziagkas, E., Zilidou, V.: Gamified and online activities for learning to support dual career of athletes (GOAL). In: Interactive Mobile Communication, Technologies and Learning, pp. 623–634. Springer, Cham, November 2017

Wearables and Internet of Things (IoT)

LoRa Technology Benefits in Educational Institutes Apostolos Gkamas(&) University Ecclesiastical Academy of Vella, P.O. Box 1144, 45001 Ioannina, Greece [email protected]

Abstract. The Internet of things (IoT) is the extension of Internet connectivity into physical devices and everyday objects. LoRa (short for long range) is a spread spectrum modulation technique derived from Chirp Spread Spectrum (CSS) technology. This paper presents the benefits of using LoRa technology in Educational Institutes. The aim of this paper is to answer the following question: “How has the emergence of LoRa technology affected the education process and an Educational Institute operation?”. Initially the paper presents a short introduction of IoT networks and LoRa technology and after that the possible benefits of LoRa technology in Educational Institutes are presented and analyzed. In addition, we analyses the potential impacts of LoRa technology in the education process and an Educational Institute operation. Keywords: LoRa


IoT
Impact on education

1 Introduction In nowadays, education has changed from a knowledge-transfer model to a more collaborative self-directed model by the important influence of technology. This has forced many Educational Institutions to rethink teaching and learning. The influence of technology can be seen in many aspects of education from student engagement in learning, content creation, implementation of personalized content and etc. The Internet of things (IoT) is the extension of Internet connectivity into physical devices and everyday objects. Embedded with electronics, Internet connectivity, and other forms of hardware (such as sensors), these devices can communicate and interact with others over the Internet, and they can be remotely monitored and controlled. LoRa (short for long range) is a spread spectrum modulation technique derived from Chirp Spread Spectrum (CSS) technology. LoRa Technology is a long range, low power wireless platform which can be used for building IoT networks worldwide. LoRa Technology enables smart IoT applications that solve some of the biggest challenges facing our planet: energy management, natural resource reduction, pollution control, infrastructure efficiency, disaster prevention, and more. LoRa Technology has amassed over 600 known uses cases for smart cities, smart homes and buildings, smart agriculture, smart metering, smart supply chain and logistics, and more. The LoRa is a new technology designed to increase connectivity between computer systems and other devices, rapidly making its way into classrooms and generally to © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 413–424, 2021. https://doi.org/10.1007/978-3-030-49932-7_40

414

A. Gkamas

education. LoRa improves the education system and brings a lot of added value to the physical teaching environment and structured learning. Smart Educational Institutes have facilities that function in a smooth fashion with highly personalized learning methods. This paper presents the benefits of using LoRa technology in Educational Institutes. The aim of this paper is to answer the following question: “How has the emergence of LoRa technology affected the education process and an Educational Institute operation?”. Before we provide information regarding LoRa technology benefits in Educational Institutes, we will answer the question why LoRa technology is so important: The two main approaches to provide data access in IoT have been based on either multihop mesh networks using short-range communication technologies in the unlicensed spectrum, or long-range legacy cellular technologies operating in the corresponding licensed frequency bands. Recently, these reference models have been challenged by a new type of wireless connectivity, characterized by low-rate, long-range transmission technologies in the unlicensed sub-gigahertz frequency bands, used to realize access networks with star topology referred to as low-power WANs (LPWANs). LoRa technology is one of the most promising LPWANs technologies and in addition LoRa technology transmit in the unlicensed sub-gigahertz frequency bands, something very important because an Educational Institute can operate his own LoRa infrastructure and not depend on the infrastructure of a cellular operator. In addition, LoRa technology provide long range up to 30 miles covering even the biggest campus (or connecting distributed campuses on a metropolitan area) and minimal energy, with prolonged battery lifetime of up to 10 years, which minimize maintain cost and battery replacement costs The LoRa technology has impact in the various aspects of an Educational Institute operation including: • Organization operation benefits, which includes – Campus energy management and eco-system monitoring: LoRa technology can be used in energy management and eco-system monitoring to provide energy efficiency for a much more sustainable development. – Secure campus and classroom access control: LoRa technology can be used for managing students and teachers’ access to classrooms, laboratories and other places in the universities. – Structure health monitoring system which enables remote and real-time monitoring of building structure safety and integrity. – Dynamic evacuation signs to help people navigate complex hazardous zones in real-time by indicating the quickest and safest direction out of the premises. • Student’s health monitoring: The LoRa technology can play an important role in a wide range of healthcare applications, from monitoring people health to preventing disease. Moreover, such technology can also reduce the cost of care. LoRa technology can be used for implementing healthcare monitoring solution based on the use of wearable devices. • LoRa technology provides many education benefits on the operation of an educational institute including the following:

LoRa Technology Benefits in Educational Institutes

415

– Interactive learning: Most of the textbooks that we use in nowadays are online and teachers can engage students in the classroom environment by providing a series of additional materials, assessments, videos, and various interactive learning experiences through IoT software. – Different kinds of learning sources: There are many ways that students can learn things using advanced technology. – Learners with disabilities: The LoRa technology can be very helpful in teaching and enabling the special needs people with disabilities to work and be dependent. – Personalized learning: Lot of research has been conducted in making grounds for the personalized learning providing learners with freedom of pace, time, space and still making them achieve the best results. – Attendance monitoring system: An attendance monitoring system assist Educational Institutes in many methods. • E-Learning benefits: The main benefit that LoRa technology brings to e-learning services is the expansion of the learning ecosystem combining the physicality and virtuality. • Research benefits: The growth of IoT applications and services involve the most important topics of the Research and Developing science groups around the world including almost every scientific area. Initially, the paper presents a short introduction of IoT networks and LoRa technology and after that the possible benefits of LoRa technology in Educational Institutes are presented and analyzed. In addition, we analyses the potential impacts of LoRa technology in the education process and an Educational Institute operation. This paper has the following structure: Next section presents the IoT evolution which takes place nowadays. In Sect. 3, we present the related work available in the literature and Sect. 4 provides information about the LoRa technology. After that we present the benefits of LoRa technology in Educational Institutes in Sect. 5. We conclude our paper in Sect. 6 and Sect. 7 presents our future work.

2 IoT Evolution IoT started as a small market for hobbyists and today has become an important industry. IoT can be considered any device that is able to connect to the Internet and transmit data like smart lights, online cameras, access-control devices, biometric readers, thermostats, voice-activated assistants like Amazon Alexa, etc. The difference between an IoT device and a traditional device like a computer or a smartphone is that the IoT device in the most cases has a very limited user interface and are considered as a device which is used to measure and transmit (small amount in most cases) of data. The total number of IoT devices is very big and it continues to grow at a rapid rate, with some estimates pointing to some serious future growth in the short- to mid-term. According to IoT analytics report [13] in 2018 there was 7 billion IoT devices with a trend to become 21.5 billion devices by 2025. This important growth, of IoT devices needs support of the current Internet infrastructure and leads to some

416

A. Gkamas

demanding problems that we need to deal with especially in the domain of Internet security. IoT provides new capabilities both on the industry side and end user side. Industry has seen massive benefits from adopting IoT technologies. For example, expensive PLC (Programmable Logic Controller) modules that use old technologies and require proprietary software to read can be replaced by cheaper IoT devices that can perform similar functions with far greater connectivity options. Consumer IoT devices can also be used to create a smart home that can monitor things like lighting, temperature, surveillance systems, power consumption etc. A lack of security standards, coupled with little to no update features on some of early IoT devices, means that some IoT devices may suffer by security flaws. This is an important concern for the IT professionals responsible for the safety and security of an organization’s data systems. The last year all major IoT security areas [14], including previously underrepresented ones, are being explored, the amount of research keeps growing, and both existing and novel mechanisms are being implemented and deployed. IoT infrastructure generates massive amounts of data that needs to be processed and technologies like big data manipulation, machine learning, and artificial intelligence networks must be used.

3 Related Work As we present in the last section, IoT is a real evolution of our lives and as results IoT influence also the education process and especially the operation of Educational Institutes. Authors in [1] present the recent IoT projects in education and investigate and analyse how IoT has changed the Education Business Model and added new value propositions in such organizations. IoT has an important impact in higher education, authors in [4] present a study about the impact of IoT on higher education especially universities. IoT stands to change dramatically the way universities work and enhance student learning in many disciplines and at any level. It has huge potential for universities or any other educational institutions; if well prepared to ensure widespread and successful implementation by leadership, staff, and students. IoT needs development where universities can lead. Academics, researchers, and students are in a unique place to lead the discovery and development of IoT systems, devices, applications, and services. Paper [6] focuses on a research associated with the predictable impact of IoT in the superior education. A theoretical analysis along with a statistical study is explored in this paper. The theoretical and statistical results showed non-ignorable influence of the IoT in the education ecosystem in terms of the learning and managing factors. In some cases, such as the hyper- connectivity, collaboration and research opportunities the effect is significant. Emergent technologies such as the IoT at present are rapidly developing in the digital world and transforming traditional education system into a scalable, adaptable with rapid dynamic changes, flexible and more efficient e-learning with a topology where the huge number of physical and virtual interacting objects are involved in the process of learning.

LoRa Technology Benefits in Educational Institutes

417

In addition, authors in [8] demonstrate the importance of introducing IoT in higher education. The paper identifies several practical methods identified for integrating IoT features in academia, especially in the areas of teaching and learning enhancements. Moreover paper [9] studies how higher education and the Internet of Things should interact. IoT are promoting changes in higher education, such as, education and teaching changes, learning changes, management changes, experiment and training changes, school change, and so on. In addition, paper [10] presents the five stages to solve one of the higher education environments using intelligent data analysis (IDA) and IoT. The main points of this system are (i) it reduces the time rate and effort wasted in taking daily attendance of students in each lecture. (ii) Increased accuracy in the administration reduces the need for human effort. (iii) Increase the efficiency of teachers themselves because the lecture time now is set automatically from the moment he/she enters the lecture hall to the time he/she exits from it. (v) This integrated system is considered part of the e-government desired application in Iraq. (vi) Apply the decisions automatically against violator’s students, where the name of the offending student (his/her) could pass own university system and thus prevent from entering the lecture for the period of punishment. IoT affects not only higher education but also undergraduate education, authors in [5] propose to transform Science, Technology, Engineering, and Mathematics core courses by integrating IoT-based learning framework into their corresponding lab projects. The design challenges of the new learning framework are summarized in the paper. In addition, the paper presents a case study by incorporating IoT-based learning framework into a Software Engineering (SWE) embedded system analysis & design course. Influence of IoT is not only important for the operation of Educational Institutes but also affects the operation of educational platforms. Authors in [2] present an integrated platform by utilizing the advanced IoT devices to improve the quality of education. Several IoT controller boards capabilities and features are described and compared for realizing the IoT solution in educational platform. Target of the proposed platform is to connect educational process with a real-life environmental condition. In addition [3] presents a new model for integrating objects to Virtual Academic Communities (VAC). Tests of the proposed model were performed by the implementation of a case study, and the outcomes show that using IoT provides a more engaging learning environment for students and more data about the learning process to help teachers to enhance their knowledge about the learning pace of their students and their learning difficulties. With the evolution of the IoT, the vision of completely smart classroom has never been closer to reality. However, despite diverse benefits of novel technology solutions, manufacturing, distribution, and utilization of IoT products and systems are energy and resource intensive and accompanied by escalating volumes of waste and toxic pollutions. Hence, in order to maximize benefits and minimize harm to people and the environment, Green IoT (G-IoT) appears as the adequate solution. Paper [7] analyzes the possibilities of the G-IoT utilization in the engineering education. Therefore, the main benefits, as well as challenges for the appliance of G-IoT vision in a smart classroom, have been discussed. As we have already presented, IoT is a real evolution of our lives and connectivity is probably the most basic building block of the IoT paradigm. The two main

418

A. Gkamas

approaches to provide data access in IoT have been based on either multihop mesh networks using short-range communication technologies in the unlicensed spectrum, or long-range legacy cellular technologies operating in the corresponding licensed frequency bands. Recently, these reference models have been challenged by a new type of wireless connectivity, characterized by low-rate, long-range transmission technologies in the unlicensed sub-gigahertz frequency bands, used to realize access networks with star topology referred to as low-power WANs (LPWANs). Authors in [11] introduce this new approach to provide connectivity in the IoT scenario, discussing its advantages over the established paradigms in terms of efficiency, effectiveness, and architectural design, particularly for typical smart city applications. LoRa in one of the above mentioned LPWAN technologies. Authors in [12] present a review of the challenges and the obstacles of IoT concept with emphasis on the LoRa technology. The main contribution of this paper is the evaluation of the LoRa technology considering the requirements of IoT. In the next section we provide a summary of the LoRa technology.

4 LoRa Technology LoRa [15] is a spread spectrum modulation technique derived from Chirp Spread Spectrum (CSS) technology and is the first low-cost implementation of CSS for commercial usage. It was developed by Cycleo of Grenoble, France, and acquired by Semtech in 2012, a founding member of the LoRa Alliance. LoRa devices and wireless radio frequency technology (LoRa Technology) is a long range, low power wireless platform that can be used for the implementation IoT networks worldwide. LoRa Technology enables smart IoT applications that solve some of the biggest challenges facing our planet: energy management, natural resource reduction, pollution control, infrastructure efficiency, disaster prevention, and more. LoRa Technology has amassed over 600 known uses cases for smart cities, smart homes and buildings, smart agriculture, smart metering, smart supply chain and logistics, and more. The usage of LoRa Technology is very important with 97 million devices connected to networks in 100 countries and growing. Industry analyst IHS Market projects that 40% of all Low Power WAN connections will be based on LoRa Technology by the end of 2019 [15]. LoRa Technology enables data communication over a long range while using very little power. When connected to a non-cellular LoRaWAN network, LoRa devices accommodate a vast range of IoT applications by transmitting packets with important information. LoRaWAN fills the technology gap of cellular and Wi-Fi/BLE (Bluetooth Low Energy) based networks that require either high bandwidth or high power or have a limited range or inability to penetrate deep indoor environments. In effect, LoRa Technology is flexible for rural or indoor use cases in smart cities, smart homes and buildings, smart agriculture, smart metering, and smart supply chain and logistics. The key features of LoRa technology include the following: • Long Range: Connects devices up to 30 miles apart in rural areas and penetrates dense urban or deep indoor environments

LoRa Technology Benefits in Educational Institutes

419

• Low Power: Requires minimal energy, with prolonged battery lifetime of up to 10 years, minimizing battery replacement costs • Secure: Features end-to-end AES (Advanced Encryption Standard) 128 encryption, mutual authentication, integrity protection, and confidentiality • Standardized: Offers device interoperability and global availability of LoRaWAN networks for speedy deployment of IoT applications anywhere • Geolocation: Enables GPS (Global Positioning System) free tracking applications, offering unique low power benefits untouched by other technologies • Mobile: Maintains communication with devices in motion without strain on power consumption • High Capacity: Supports millions of messages per base station, meeting the needs of public network operators serving large markets • Low Cost: Reduces infrastructure investment, battery replacement expense, and ultimately operating expenses.

5 Benefits in Educational Institutes LoRa technology, and IoT technologies in generally, have affect or will affect every accepts of human life and the operation of any organization. In the following paragraphs we present the benefits of LoRa technology in the operation of Educational Institutes. We present the benefits in the following five categories: • • • • •

Organization operation benefits Students and Personnel health monitoring Education benefits E-Learning benefits Research benefits

Finally, we present the Green IoT paradigm which is very important for a sustainable development of IoT services. 5.1

Organization Operation Benefits

LoRa technology provides many benefits on the operation of an Educational Institute including the following: • Campus energy management and eco-system monitoring: LoRa technology can be used in energy management and eco-system monitoring to provide energy efficiency for a much more sustainable future. The above allows an Educational Institute to create a green campus environment by reducing CO2 fingerprint, monitoring and controlling energy and water, etc. The Educational Institute can effectively balance power generation and energy usage to provide more efficient operations by adding intelligence to the existing infrastructure. With the use of specialized sensors and actuator systems, energy consumption information will be gathered automatically to improve economy, efficiency and reliability of the systems.

420

A. Gkamas

• Secure campus and classroom access control: LoRa technology can be used for managing students and teachers’ access to classrooms, laboratories and other places in the universities. As result LoRa technology can be used to simplify access control and improve university security. For example, a classroom access control method can be implemented using LoRa technology. The classroom registration system is based on a network of connected sensors that collect classroom access information and display the status of the classroom on both a web-based application and university TV panels. • Smart Fire Evacuation System: Emergency data transmission is extremely important in a fire scene. LoRa Technology creates an innovative IoT application use case that better serves cities and communities. This technology to make public space and large-scale buildings safer and more comfortable. This technology uses dynamic exit signs integrated with LoRa Technology which helps people navigate complex hazardous zones in real-time by indicating the quickest and safest direction out of the premises. The conventional exit signs only signal where the closest exit is, which may mistakenly lead people into hazard zones. On the other side dynamic evacuation system communicates with the building’s existing fire alarm system and sends the safest direction to every dynamic exit sign in the building. • Smart Buildings: LoRa Technology provides IoT connectivity in large, dense environments where hundreds of connected devices need to be managed independently. This allows managers to make better choices, leading to more efficient energy use and lower energy costs. By incorporating together LoRa-enabled products include thermostats, sprinkler controllers, door locks, leakage monitors, and smoke alarms. These devices connect to a building’s network and allow consistent, remote monitoring to better conserve energy and predict when maintenance is necessary, saving property managers money. LoRa Technology’s scalability and capability to penetrate dense building materials make it an ideal platform for IoT connected smart building applications, no matter how large. • Structure Health Monitoring system with LoRa Technology monitors buildings for a variety of safety indicators in real-time: LoRa-based wireless sensors and gateways, we can get a building connected within a couple of hours by eliminating the need for power wiring installation. In comparison, a wired solution could take around a week to install. Such system consists of various LoRa-based IoT sensors, a LoRa-based gateway, and a proprietary Cloud-based data management platform. A building is equipped with an end-to-end solution consisting of several sensors on the walls and roof to measure building surface wall crack growth, tilt and vibration, and often a soil moisture sensor to monitor water saturation at the building’s foundation, at a fractional cost of the traditional wired solution or human inspection. 5.2

Students and Personnel Health Monitoring

The LoRa technology can play an important role in a wide range of healthcare applications, from monitoring people health to preventing disease. Moreover, such technology can also reduce the cost of care. LoRa technology can be used for implementing healthcare monitoring solution based on the use of wearable devices. A wearable device monitors physiological signals over long periods of time in a

LoRa Technology Benefits in Educational Institutes

421

non-invasive and non-obtrusive manner and can help to identify possible medical situations. This wearable devices except of indicating user vital signs/condition can also used for locating the person carrying the device. LoRa is a very applying technology for such services because offers long range and minimal energy requirements, with prolonged battery lifetime. Such service is very important especially for persons with impaired mental state or inability to react are in jeopardy, while they are away from their familiar environment. Indicatively: (a) People in the autism spectrum disorders for whom, it has been reported that, at a percentage of 49%, they have disappeared or have been at risk due to a tendency to flee, at least once since the age of 4. (b) People suffering from some form of dementia who statistically have at least 60% chance of being lost outdoors. 5.3

Education Benefits

LoRa Technology can support Educational Institutions to improve the quality of teaching and learning by providing a richer learning experience and provides a great potential to enhance the teaching-learning process, providing support to make real the concept “anytime anywhere”. More specifically LoRa Technology can improve teaching and learning by the following ways: • Interactive learning: Most of the textbooks that we use in nowadays are online and teachers can engage students in the classroom environment by providing a series of additional materials, assessments, videos, and various interactive learning experiences through IoT software. • Different kinds of learning sources: There are many different ways that students can learn things using advanced technology. There is a number of different management tools that teachers can use in order to create new sources for student learning. • Learners with disabilities: The LoRa technology can be very helpful in teaching and enabling the special needs people with disabilities to work and be dependent. LoRa technology can offer these people a respectable way of life, where they can learn skills and then get jobs to support themselves regardless of their disabilities. • Personalized learning: Lot of research has been conducted in making grounds for the personalized learning providing learners with freedom of pace, time, space and still making them achieve the best results. With the advancements in technological enhancements and its integration into teaching, educators are able to place personalized education at the center of a student-centered teaching approach. LoRa technology provides an effective solution to such issues through integration of technology after careful planning and studies • Attendance monitoring system: An Attendance Monitoring System assist Educational Institutes in many methods. For example, it allows the academics to input the vital records immediately. This could help the Educational Institutes to reduce the time it takes to publish attendance facts, etc.

422

5.4

A. Gkamas

E-Learning Benefits

The main benefit that LoRa technology brings to e-learning services is the expansion of the learning ecosystem combining the physicality and virtuality and will come close to the susceptibility of the learning process [6]. LoRa technology allows the implementation of Intelligent Agents which can be used for the interact of the user with a computer application. The E-learning limitations in the communication, collaboration and “face-to-face” interactions between the participants can be minimized by using Intelligent Agents based on LoRa technology sensors in the learning environment which allows the interaction among the virtual e-learning space and the different physical location of the participants. This allows a novel e-learning platform with a wide variety of distance learning objects. The virtual object can interact with another object either virtual either physical which generate a different means for collaboration among the participants. 5.5

Research Benefits

The LoRa based networks and in general IoT networks connect billions of everyday devices to the Internet and collect very big volumes of information that could be used in every aspect of today life. The above provides many new opportunities and challenges for industry, education, production and business sections. The growth of IoT applications and services involve the most important topics of the Research and Developing science groups around the world including almost every scientific area that are targeted by enormous numbers of the research and development departments of companies and Universities around the world. IoT, as mentioned, is expected to promote significant improvements in the production, education, manufacturing, healthcare, energy, transportation, security, communication, government, and economic growth which means generating innovative modern challenges finding solutions. Moreover, IoT provides connecting the massive number of the institutes, universities and research centers in the real time. This enables the researchers to have access to the big data in order to seek the necessary information for the subjects that are investigating, as well as identifying the future projected topics. 5.6

Green Internet of Things

With the evolution of the IoT, the vision of completely smart classroom has never been closer to reality. However, despite the various benefits of IoT evolution the IoT infrastructure is energy and resource intensive and accompanied and contributes on waste and toxic pollutions. As result, in order to maximize benefits and minimize harm to people and the environment, Green IoT (G-IoT) appears as the adequate solution [7]. The G-IoT process includes the economically and environmentally responsible ways of design, fabrication, and use of IoT devices and infrastructures. From the technical perspective, green G-IoT is a broad concept, which includes numerous technologies and based on:

LoRa Technology Benefits in Educational Institutes

423

• Green communication technology: Green communications and networking are the practice of choosing energy-efficient, environmentally friendly and sustainable communications and networking technologies and products. • Green computing technology: Usage of energy-efficient ICT equipment (computers, servers, printers and other peripherals) alongside reduced resource consumption and proper disposal of e-waste. • Smart grid and applications: Smart grids are sets of hardware and software tools that manage energy flows, supply and demand in a sustainable, reliable and economic manner. The G-IoT paradigm must be used on educational institutions in order to take care of energy and resource consumption, e-waste, hazardous emissions and toxic pollutions, holds the potential to create affordable, flexible and engaging educational practices.

6 Conclusion The IoT is the extension of Internet connectivity into physical devices and everyday objects. LoRa is a spread spectrum modulation technique derived from CSS technology. In this paper we present the benefits of using LoRa technology in Educational Institutes. LoRa technology have tremendous potential to bring significant values to educational institutes by engaging and motivating the students and staff, and to increase speed of learning. The LoRa technology brings benefits in various aspects of Educational Institutes including organization operation benefits, improvement of the learning process, improvement of E-Learning services and research activities. It is important to mention that that Educational Institutes in order to provide a sustainable development must incorporate the G-IoT paradigm during the incorporation of IoT and LoRa technology in their operation.

7 Future Work Our future work includes the study of the impact on Educational Institute operation of other LPWANs technologies like Narrow Band IoT (NB-IoT) [16]. NB-IoT is a LPWAN radio technology standard developed by 3GPP (3rd Generation Partnership Project) to enable a wide range of cellular devices and services. NB-IoT is considered as part of 5G mobile networks and focuses specifically on indoor coverage, low cost, long battery life, and high connection density. NB-IoT uses a subset of the LTE (Long Term Evolution) standard but limits the bandwidth to a single narrow-band of 200 kHz. It uses OFDM (Orthogonal Frequency Division Modulation) modulation for downlink communication and SC-FDMA (Single-Carrier Frequency Division Multiple Access) for uplink communications. NB-IoT has similar characteristics with LoRa and one of the most important differences is the fact that NB-IoT uses licensed frequency bands and LoRa uses unlicensed frequency bands.

424

A. Gkamas

References 1. Bagheri, M., Movahed, S.H.: The effect of the Internet of Things (IoT) on educational business model. In: 12th International Conference on Signal-Image Technology & InternetBased Systems (SITIS), pp. 435–441, December 2016 2. Tew, Y., Tang, T.Y., Lee, Y.K.: A study on enhanced educational platform with adaptive sensing devices using IoT features. In: 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). IEEE (2017) 3. Marquez, J., et al.: IoT in education: integration of objects with virtual academic communities. In: New Advances in Information Systems and Technologies, pp. 201–212. Springer, Cham (2016) 4. Aldowah, H., et al.: Internet of Things in higher education: a study on future learning. J. Phys: Conf. Ser. 892(1), 012017 (2017) 5. He, J., et al.: Integrating Internet of Things (IoT) into STEM undergraduate education: case study of a modern technology infused courseware for embedded system course. In: 2016 IEEE Frontiers in Education Conference (FIE). IEEE (2016) 6. Abbasy, M.B., Quesada, E.V.: Predictable influence of IoT (Internet of Things) in the higher education. Int. J. Inf. Educ. Technol. 7(12), 914–920 (2017) 7. Maksimovic, M.: Green Internet of Things (G-IoT) at engineering educational institution: the classroom of tomorrow. Green Internet of Things 16, 270–273 (2017) 8. Banica, Logica., Burtescu, Emil, Enescu, Florentina: The impact of internet-of-things in higher education. Sci. Bull. Econ. Sci. 16(1), 53–59 (2017) 9. Tianbo, Z.: The Internet of Things promoting higher education revolution. In: 2012 Fourth International Conference on Multimedia Information Networking and Security. IEEE (2012) 10. Al_Janabi, S.: Smart system to create an optimal higher education environment using IDA and IOTs. Int. J. Comput. Appl. 40(3), 1–16 (2018) 11. Centenaro, M., et al.: Long-range communications in unlicensed bands: the rising stars in the IoT and smart city scenarios. IEEE Wirel. Commun. 23(5), 60–67 (2016) 12. Lavric, A., Popa, V.: Internet of Things and LoRa™ low-power wide-area networks: a survey. In: 2017 International Symposium on Signals, Circuits and Systems (ISSCS). IEEE (2017) 13. State of the IoT 2018: Number of IoT devices now at 7B – Market accelerating. https://iotanalytics.com/state-of-the-iot-update-q1-q2-2018-number-of-iot-devices-now-7b/. Accessed 11 Jul 2019 14. Román-Castro, Rodrigo., López, Javier, Gritzalis, Stefanos: Evolution and trends in IoT security. Computer 51(7), 16–25 (2018) 15. Semtech LoRa Technology Overview|Semtech. https://www.semtech.com/lora. Accessed 11 Jul 2019 16. Ratasuk, R., et al.: NB-IoT system for M2M communication. In: 2016 IEEE Wireless Communications and Networking Conference. IEEE (2016)

Three IoT Wearables in Six European Cities! Reality and Perception Rasha Ibrahim(&), Holly Towndrow, and Dorothy Monekosso Leeds Beckett University, Leeds, UK {rasha.ibrahim,D.N.Monekosso}@leeedsbeckett.ac.uk, [email protected]

Abstract. In the last decade, the Internet of Things (IoT) technology has attracted lots of attention. This paper evaluates the impact of three different IoT technologies represented in three types of wearables; Smart Glasses, Tracking Devices, and Crowd and Staff Wristbands. The deployment of these devices took place in a number of cultural and sports events in six European counties, as part of the European Commission funded project; MONICA. The analysis focused on the usability of the wearables and their impact on safety and security. Keywords: IoT
Wearables
Mobiles
Usability
User-experience
Safety
Security
Privacy
HCI

1 Introduction and Scope The Internet of Things (IoT) is regarded as one of the prominent technologies of this century which has attracted lots of attention in society, industry and academia. Examples of IoT devices include IoT enabled sensors with different data capabilities (video, audio and other data types), resource constraints (wearables, Smartphones and Smartwatches), bandwidth (UWB and M2M), costs and deployment (wearable, mobile, fixed, and airborne) in addition to actuators (lights, LED, cameras, alarms, drones, and loudspeakers). Wearable devices with sensing, actuating, localisation, and communication capabilities can support several applications by being deployed as actuators (LEDs) through automated closed-loop solutions and can be integrated with more powerful Smartphones and/or Smartwatch apps. IoT can be used in a wide range of industries. In this paper we are interested in IoT technologies used for smart living, in particular, tourism industry for the purpose of safety and security in large festivals and sports events. This paper investigates the deployment of such technologies in 6 European cities; Leeds (UK), Copenhagen (Denmark), Hamburg (Germany), Turin (Italy) Lyon (France), and Bonn (Germany). This deployment took place as part of the European Commission funded project; MONICA (Management of Networked IoT Wearables- Very Large Scale demonstration of Cultural Societal Applications). The project is funded through HORIZON 2020 Framework Program for Research and Innovation; grant agreement No 732350. In the MONICA project, IoT technologies are used for a number of applications including sound monitoring and control, crowd safety and security, and citizen © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 425–436, 2021. https://doi.org/10.1007/978-3-030-49932-7_41

426

R. Ibrahim et al.

management and innovation. This is performed by exchanging information via IoTenabled devices deployed at cultural events. These can be worn (smart wristbands, glasses), mobile (smartphones, airships) or fixed (video cameras). IoT-enabled features mean that such devices have sensors that can wirelessly connect to the internet, communicate and instigate actions. This paper will focus on three IoT technologies; smart glasses, smart wristbands, and tracking technologies. The evaluation aspects covered include; usability of such devices, fit for purpose, and users’ perception regarding privacy and data protection aspects accompanied by such technologies. In the following sections, we will briefly describe the events taking place in the six European cities where the MONICA IoT technologies are being deployed. We will refer to these events as ‘pilots’. Then we will briefly describe the three mobile technologies assessed, and explain the aspects covered in the assessment. Afterwards, the actual assessment is reported followed by our conclusion and lessons learned.

2 Pilots Context This section provides a brief description of the events in the six European cities where the IoT wearable technologies are deployed. 2.1

Leeds Rugby Varsity Game

Leeds Varsity is an inter-university sports competition between arch rivals University of Leeds and Leeds Beckett University. It begins in January with events throughout the year and finishes with one final day in October hosting over 60 fixtures across 25 different sports. The event culminates with the Men’s Rugby Union Grand Finale at Headingley Carnegie Stadium in the evening, which attracts an audience of over 15500 students each year (LeedsVarsity 2019). 2.2

Movida

The nightlife in Turin, which is known as “Movida” happens every day, particularly at the weekend, where large crowds group together in the San Salvario District to have fun and socialise. This movement began in the 1990’s and has grown ever since. As a consequence of Movida, the 7300 residents living in the area are being affected by the disturbance and noise from the crowd (MONICA-Project, 2019). To reduce these consequences, Turin have produced restrictions such as opening hours, the sale of takeaway and alcoholic drinks, consumption of alcoholic drinks in glass/cans around a public area. The main goal for MONICA at Movida is to correlate sound with crowd data and handle security incidents, supporting the municipality to find an adequate balance between safety and security, amusement and quality of public spaces. 2.3

Winter DOM

The Hamburger DOM is Northern Germany’s biggest funfair with 7–10 million annual visitors during the 91 DOM days (MONICA-Project, 2019). The funfair takes place in

Three IoT Wearables in Six European Cities! Reality and Perception

427

the premises of the Heiligengeistfeld with a total of around 251 attractions and 60 food stalls. The area is 160.000 m2, fenced, with nine free entrances. In the past three Winter editions, there have been an average of 2–3 million visitors. For Winter DOM, MONICA was used for Crowd and Capacity Monitoring, locating staff, and managing safety incidents. 2.4

Fête des Lumières (Festival of Lights)

Fête des Lumières is a free cultural event hosted by the city of Lyon. It is a festival composed of light installations with, for some of them, sound animations. The installations are located in well-known public areas of Lyon (places, streets, squares, parks). Around 50 light performances take place sometimes with sound playing. Depending on the year, the event can last 1 to 4 days and people can use free public transportation to admire the shows. The timeslots are usually from 6 p.m. to midnight. This site welcomed approximately 2 000 000 visitors in 2016, this low attendance was caused by terror risk (Jahn et al. 2018). The 2015 edition was cancelled for this reason too. In the previous editions 3–4 million visitors were present during the days of the event. MONICA was used for sound monitoring, crowd and capacity monitoring, locating staff, and managing security incidents. 2.5

Friday Rock Tivoli Gardens

Friday Rock (Tivoli) is Copenhagen’s famous amusement park located in the heart of the city. The park opened in 1843 and is the second-oldest amusement park in the world. The park offers a wide variety of rides, shows and attractions with around 4.5 million visitors annually. From April through September, Tivoli organises Friday Rock, a recurrent annual open-air concert-series with national and international performers. The capacity of the whole concert area is 25.000 people and the main stage area for Friday Rock is around 2.700 m2. MONICA was deployed in Tivoli for sound monitoring and control, crowd and capacity monitoring, missing person, and managing security incidents. 2.6

Pützchens Markt

Pützchens Markt (PM) is a street festival that takes place in Bonn every year in the second week of September. For five days, the event offers all kinds of attractions such as traditional merry-go-rounds, more than 550 commercial stalls and two stages. This massive offer extends over the length of 4,5 km in an area of 80.000 m2. Pützchens Markt attracts around 1,35 Million visitors (Jahn et al. 2018). MONICA was deployed in PM for crowd and capacity monitoring, and for locating staff.

428

R. Ibrahim et al.

3 IoT Wearable Technologies 3.1

Smart Glasses

The smart glasses deployed in the MONICA project were produced by Optinvent. They are ORA-2 products which allow a full see-through feature. The ORA-2 product feature includes: • Standalone wireless product with embedded battery, weight −90 g. • Runs generic Android Kitkat (4.4.2) and has open platform to create and execute any Android applications. • Use Arm 9 Dual Core 1.2 GHz processor with GPU & 5.3 Gb Storage Memory and 1 Gb Flash Memory D3.1 IoT Enabled Devices and Wearables 1 Document version: 1.0 Page 39 of 60 Submission date: 2017-12-27. • Track pad on rigid side of the frame (True Mouse not only swipe). • Embedded 1200 mAh rechargeable battery. • Can be worn over most of the user’s glasses. • The glasses have the following sensors • 5 megapixel Auto-Focus camera. • Light sensor to adjust automatically display brightness to environment brightness. • Active GPS. • 9 axis sensors (Gyro/Accelerometer/Compass; MPU9250 from Invensens). • Low noise microphone and mono audio out through µUSB provided accessory. • Connectivity: • WiFi b, g, n; 2.4 GHz. • BT 4.0 Low Energy. • µUSBS 2.0 for charging, data exchange and transporting mono audio to an audio headset accessory. • Display: • See-through Display Feature with 50% photopic transparency. • 800  480 pixels native resolution RGB colour display with 42 pixel/deg. • Field Of View of 22° with Flip-Vu feature to move the image location 5° up, centred to eye sight and 20° down. • High brightness (Schoneveld et al. 2017). 3.2

Smart Wristbands

The wristbands used in MONICA are produced by DEXELS. They monitor both crowd and staff. A. Crowd In order to deploy, the Crowd Wristband requires – • • • • •

Base stations Ethernet (PoE) network between base stations Sufficient spots to mount base stations, which are at least 4 m high Professional installation Production office space

Three IoT Wearables in Six European Cities! Reality and Perception

429

The features of the Crowd Wristband include brand activations, access control and cashless payments, which is captured using RFID. Other features include profile exchange, matching, friend finder, child finder, voting and alerts. Crowd monitoring also takes place, so there is a display of heat maps to show busy areas, and it also helps with the visitors’ journey. LED is another feature of the Crowd Wristband. This allows a group notification or an individual notification, and also enables for a light show. B. Staff There are several features regarding the Staff Wristband. One feature is the localisation of staff, which adopts a very high accuracy. Staff are also able to send short messages amongst each other, which the recipient can acknowledge they have received the message. The Wristband also includes a buzzer and sound. In order to deploy, the Staff Wristband requires – • • • • 3.3

Anchors Ethernet (PoE) network for at least 1 anchor Sufficient spots to mount anchors are least 6 m high Professional installation Tracking Devices

The tracking device assessed in this paper is LoRa Tracking device. The functionality of this device is to track individuals such as event staff. It has features such as communication buttons, which enables the user to send and receive a text.

4 Assessment Aspects This section discusses the aspects according to which the evaluation of the IoT wearable technologies took place. 4.1

Wearables Usability

A wearable is a technology which is worn on the body, and is one of the devices at the forefront of new technologies. The technology for wearable gadgets has vastly grown over the years, particularly in healthcare. Smart watches, for example, are one of the major wearables that can be used today. Razvan (2015) stated in “Designing a User Experience for Wearable Devices”, that the early process of wearables included two types – eyewear (smart glasses) and smart watches. Key features of eyewear is the ability to navigate, track location and provide route information, take pictures and record videos, as well as performing online searches. Smart watches are more focused on tracking physical actions, such as running, walking etc. Most smart watches now provide navigation, music listening abilities, the ability to make payments and calls and send texts.

430

R. Ibrahim et al.

Comparing the two, smart watches have a higher amount of apps that are available, compared to smart glasses, and they’re often cheaper (Razvan 2015). Razvan (2015) also suggests that to make the device useable, you must keep the device small, easy to attach and comfortable. Sticking to the core features also makes the devices usable, not overcomplicating things and minimal elements on the screen enables the user to interact easily. It’s important that the wearables are minimal in weight and are comfortable. In a report by Knecht et al. (2016), they also elude to the idea that wearables should be comfortable and lightweight. The study undertaken showed that participants were affected by different types of material and this had an implication on whether they would use the devices in the future. For example, parts of the wearable were described as an obstacle and burden. On the other hand, a small device isn’t able to produce a long battery life and distinctive apps. Liang et al. (2018) produced a study on system usability scale evaluation of mainstream wearable fitness devices. The study evaluated that the usability of mainstream wearables has a negative perception from users and their need to buy wearables is low. In addition, Liang et al. (2018) suggests that manufacturers of these wearable devices should further their effort to develop innovative and unique functions, and improve the overall usability by adding cognitive behaviour change techniques into the system. 4.2

Fitting for Purpose

The IoT mobile devices reported in this paper were mainly used to improve conditions in large-scale events attended by thousands, and in some cases millions of people. Accordingly, the wearables were assessed according to their fit for purpose. This was done in relation to the project use-cases in terms of crowd monitoring, sound monitoring, managing security incidents, and health incidents, and locating staff. 4.3

Privacy and Data Protection Issues

An important aspect that was evaluated in some of the wearable devices reported here are the privacy and data protection aspect. Some of the wearables collect location data or other personal information of either the staff or audience using them. We needed to assess how users feel about capturing their personal information for the purpose of improving the service provided by the event organisers in such large-scale cultural events.

5 IoT Technologies Evaluations This section reports the assessment of three IoT wearables; Smart glasses, smart wristbands, and tracking devices in six European cities deploying IoT technologies for the MONICA project.

Three IoT Wearables in Six European Cities! Reality and Perception

5.1

431

Smart Glasses

Thirteen users participated in the deployment of the smart glasses in Rugby Varsity Game in Leeds (UK), Tivoli in Copenhagen (Denmark), Movida festival in Turin (Italy), and Festival of Lights Lyon (France). In Leeds, OPTINVENT smart glasses were used by a professional steward with an extensive experience as a security staff. To evaluate the smart glasses, we conducted a semi-structured interview with the steward. This method allowed us to have a one-oone conversation with the steward and get his thorough feedback about the smart glasses and its effectiveness as a mobile technology to ensure safety and security in the stadium. Normally, the glasses have a joystick that the user can use. However, we decided that the steward use the glasses touch pad on the frame instead so his hands were free. The steward felt the smart glasses were easy to use and comfortable. He felt they helped him to quickly and effectively complete his work. The clarity of images of the IoT smart glasses were much better than the use of normal body-cam in terms of targeting and focus. The steward felt the glasses could definitely help to improve the security operations in the stadium especially in relation to: • Bags search: The glasses gave a closer and a more accurate view, better than just the eyes. “I felt by just looking through the bags via the smart glasses, as if my hands were already inside the bags”. • Security and health incidents: Specially using the video mode, the smart glasses helped the steward to feel both match spectators and the objects much closer. Accordingly, the abnormalities were easier to detect. The steward commented that the memory card finished after 4 h because he used the video mode a lot. All the images and videos captured by the glasses were remotely stored at a laptop in the MONICA control room where Leeds Beckett University staff were located. The steward suggested some adjustments to the glasses bridge over the nose for more comfortability. He recommended to integrate the headset with the glasses so that the stewards won’t have two pieces on the head to manage. Regarding stability of connectivity, because of some construction work at the Stadium on the day, the internet connection wasn’t working everywhere in the stadium, so the steward couldn’t send pictures and videos when he was far from activated access points. But the glasses worked well for him everywhere. Near the access points where the internet connection was working well, the steward was able to send all videos and photos taken to the control room remotely and instantly. In Turin, at the nightlife of San Salvario District, Movida, three smart glasses were used by Turin local police officers. A questionnaire was distributed among them to get their evaluation of the smart glasses, and if they believe the glasses can be an effective tool for local police patrolling during Movida. The MONICA server sent a message to the police officers regarding a suspicious individual. The officers checked an Identity Document (ID) of the suspect by exchanging photos, both from the officers to the MonicOra server and vice versa. The ID was then verified by the station and then the intervention was closed. The first attempt resulted in the officers receiving the messages from the MonicOra server,

432

R. Ibrahim et al.

however the MonicOra server did not receive the messages from the officers. This is down to a connectivity issue between the server and the glasses. During the second attempt, all messages between the glasses and MonicOra server and vice versa were exchanged and received. Nine participants in total filled the questionnaire which was focused on the usability of the smart glasses and how they aided the end users during their work. This questionnaire helped evaluate the impact that the smart glasses have on both the end user and the work environment. About the end users were made up of both male and females, from different professional backgrounds within the police sector. The table below summaries the characteristics of the participants. Total participants 9

Gender Male (7) Female (2)

Job position Police officer (3) Police officer and analyst (1) Superior police officer (1) N/A (4)

Smart Glasses Efficiency The objective of the smart glasses is to improve the efficiency of the user’s work in terms of securing the event. 56% of participants either agreed/strongly agreed that using the smart glasses ensured them that they could effectively complete their work. Three participants disagreed with this. Regarding the easiness via which the participants learnt how to use the smart glasses, 78% of all participants agreed that it was easy to learn how to use the smart glasses, and only one participant found difficult to understand how it worked. In terms of the usability of the glasses, 56% agreed that they were satisfied with how easy it was to use the smart glasses, only one participant disagreed. 33% of participants were neutral on this. Regarding the speed of performing tasks, participants were also asked whether they could complete tasks quickly using the smart glasses. Three out of nine participants believed they could complete tasks quickly, whilst another three participants were neutral. The remaining three participants believed that it took them extra time to complete their tasks using the smart glasses. When the smart glasses comfortability was investigated, 67% of all participants felt they were comfortable with using the smart glasses. Only two participants felt they weren’t comfortable using them. One justified that by the weight of the glasses, and the too high overall dimensions. The final participant was neutral. When making a mistake, 2/3 of the participants felt by using the smart glasses they could recover easily and quickly. One participant disagreed with this statement and two stayed neutral. Distinct Features The participants were asked to determine their most important feature of the smart glasses. Only five participants answered this question, and they praised the innovation. With quotes such as “undoubtedly innovative” and with some commenting on the freedom of hands they experienced wearing the smart glasses and how helpful it was to have their hands almost completely free. The most appreciated feature of the smart glasses was the ability to share data simultaneously with colleagues remotely. Participants thought that the idea of sharing real-time images, as well as also recording, was

Three IoT Wearables in Six European Cities! Reality and Perception

433

an important feature of the smart glasses. So, they stressed on the importance of both the video and the camera. Regarding safety and security incidents, 67% of the participants confirmed that the smart glasses were useful for detecting such incidents. Regarding reporting, 44% of the participants believed that the glasses would help report an incident. The smart glasses were regarded as a “new communication tool”. Additional Functions The participants were also asked what capabilities they would have expected the smart glasses to have. Out of the six participants who answered this question, one participant stated that the smart glasses in their current status covered all functionalities they needed and that they “would not expect to see anything else”. One participant suggested to add an emergency button, and another participant would have liked to see voice commands as an option. Four participants stated that they would have liked to be able to film in real-time. It’s worth mentioning that such functionality is already included in the smart glasses, and was deployed in other pilots (such as Leeds), but was not used in Movida due to unstable connection the deployment suffered in November 2018. Connectivity was an issue with two participants. One participant stated that there was a limited possibility of writing messages without a remote controller. Whilst another stated the modem/joystick and the smart glasses were not something they liked. Regarding the physicality’s of the glasses, two participants stated the weight was an issue, as well as too high overall dimensions. Overall, out of nine participants, five were satisfied with the MONICA smart glasses. In Lyon, at the festival of lights (Fete des Lumieres), three smart glasses were deployed and a questionnaire was conducted to evaluate the participant’s opinions on this technology. For the impact analysis, several aspects have been considered: 1) Smart Glasses Aiding Participants: The participants were asked what was the most important thing they felt the smart glasses helped them in. One participant liked the information collection in real time, as well as the ease of transmission; another participant found that the keypad was nice to use, though takes up use of one hand; finally, the last participant appreciated the fast GPS tracked messages. 2) Additional Functionalities: The participants were asked, in addition to the functions the smart glasses currently provide, what else they expected the glasses to feature. The following feedback was received: • Further upgrades of the product, such as “send pictures to nearby glasses” and direct contact between users (skipping the COP) • Improve physically: the glasses are rigid and cannot be folded or stored easily when not being used. • Inboard display of the COP, a map which displays the users position. 3) Cons of Smart Glasses: To gain further feedback, the participants were also asked to mention two things they did not like about the MONICA smart glasses. The following feedback was received: • Important to be able to focus on a single point because it was hard to balance viewing with environment. • The side view is limited by the battery icon.

434

R. Ibrahim et al.

• Storing was an issue. • Smart glasses are only for people with right eye as the main eye. Those with left guiding eye couldn’t use. 4) General Feedback: Out of three participants, only one participant gave general feedback. The general comments were that the smart glasses are weather dependent; when raining or snowing, they can’t be used because they aren’t 100% waterproof and feature no infra-red camera. Furthermore, the smart glasses are unable to be stored in a pocket or fit around the end user’s neck. The user also referred to the messages received, stating that there were too many to scroll. Finally, the WIFI limited the covered area – so for festivals with more than one venue, this can be an issue. 5.2

Smart Wristbands

Six crowd wristbands were deployed in Tivoli Gardens in Copenhagen for the purpose of locating a missing person. A questionnaire has been distributed to the end-users for evaluating the wristbands. Regarding the usability of the wristbands, 100% of the participants were happy with the wristband weight. Also, 67% of the participants agreed on the wristbands suitability for a child in terms of their weights but two participants were neutral. All the participants except one (who was neutral) agreed that it was very easy to operate the wristband. Regarding the comfortability of the wristbands, 100% of the participants felt they were comfortable to wear. 50% of the participants felt the wristband helped improve the communication with their colleagues. The other half were neutral. In regard to the issue of privacy and data protection, the participants stated that they do not mind being tracked as they understood that the purpose of the tracking is for their own interest, as shown from some of their quotes below: “It’s totally fine, it’s for my safety”. “I’m okay with it”.

5.3

Tracking Devices

Fifteen LoRa tracking device were deployed in Pützchens Markt in Bonn (Germany) for the purpose of monitoring the location of staff. Users were asked to evaluate the feasibility of the LoRa Tracking devices based on stability, accuracy, uptime, reliability and range. Feedback included improvements for the app. For example, it was suggested that it would be nice to have the map on one screen and the information on a second screen. It was also suggested that “early warnings” before an area gets too full would be a good feature to add. The tracking marker should also be blurred or indicate to show when a device didn’t transmit for a prolonged period of time. Furthermore, the locations often jumped, which provided inaccurate results. The tracking device as a whole could have improvements. For instance, the device should be more robust and smaller in size. As well as this, the on/off button is easily switched which could hinder the user. Extensive communication options would also be a preferred feature to add to the

Three IoT Wearables in Six European Cities! Reality and Perception

435

device. The ability to send an emergency signal by one push is highly desired. Finally, the user would require faster updates and an extensive battery life. In Winter-DOM in the city of Hamburg (Germany), the initial plan was to deploy several trackers for this pilot demonstration, for use by police, medics and security staff. Privacy and security concerns meant this was not possible at that time as this needs better preliminary planning ahead of the following DOM in 2019 to allow for larger deployment of devices. Accordingly, only six trackers were deployed in 2018. They were successfully tested on the first pilot at the DOM. For the evaluation of the tracking devices, several aspects have been considered: 1) Privacy: in terms of privacy, the participants were asked about the way they felt about other staff/people tracking their location. 100% of participants who answered the question believed that for work purposes they had no concerns and would continue to wear the device. Whilst the device in the participant’s opinions did not invade privacy in the work place, 2/3 of the participants that answered believed that it would be more concerning should the user not be able to switch the device off during home time. 2) Usability and ease of use: 75% of the participants believed that the device was easy to use, with one participant being neutral on this question. 50% believed that the device was of sufficient weight. This is further echoed by one participant suggesting she forgot she was wearing the device during their day. 75% of the participants was neutral when commenting whether the tracking device was comfortable to wear. One participant disagreed and believed that the tracking device was “too big” and had “no possibilities to attach” to the user. Whilst 75% of participants were neutral regarding the comfort of the tracking device, all the participants agreed that the device did not hinder or obstruct them from doing their normal daily job. The participant who found the device “too big” also agreed the device did not obstruct or hinder her duties. 3) Overall performance: when asked if the tracking device met their expectations, 75% of participants stated that their expectations were met with some suggesting some improvements as reported in the section below. Overall, the participants believed the device worked as expected and for most parts did what was anticipated. Only one participant stated the device did not do as expected. 4) Improvements: In regards to battery life, whilst 50% of participants clarified that the device had a substantial battery life, 100% of participants agreed that showing the battery life/status of the device would have been incredibly important. A visual battery percentage was the participant’s suggestions. Device on/off: 75% of participants stated that not being able to switch on/off the device was an issue. To further improve the device, it has been suggested to add an on/off button, enabling the user to manage the devices status. They would like to be able to switch the device off even for a small period of time. 50% of participants stated that the on/off switch would be useful when off duty.

436

R. Ibrahim et al.

6 Summary The paper presents assessment of the IoT wearables used for smart living in the tourism industry, in specific, in festivals and sports events. The technologies assessed were smart glasses, wristbands, and tracking devices. The technologies were evaluated in terms of their usability, and impact on safety and security. The results showed that in general, most of the users of the wearables found them easy to wear and operate with the exception of situations when there were issues with connectivity. In regards to the users of the wearables, all the devices were assessed by the real users with the exception of child visitors wristbands, which were evaluated by adults. We believe, it is essential that children of different age groups conduct the assessment of the devices putting into consideration all ethical consideration necessary. In terms of privacy, most of the users were happy that their location is tracked for the purpose of improving security with a recommendation of making sure the devices are switched off after working hours. The results showed high potential of the wearables to improve safety and security in cultural events by empowering key personnel such as stewards and police officers with devices connected to the IoT network platform. Acknowledgement. The authors would like to thank Veronica Chesi (In-JeT ApS) for her efforts in the development of the impact assessment plan of the MONICA project and her effective engaging of all stakeholders for a constructive user experience management.

References Knecht, K., N, B. and K, S.: A study on user experience and usability of apple-watch as wearable devices (2016) Liang, J., Xian, D., Liu, X., Fu, J., Zhang, X., Tang, B., Lei, J.: Usability study of mainstream wearable fitness devices: feature analysis and system usability scale evaluation. JMIR mHealth uHealth 6(11), e11066 (2018) Razvan, G.: Designing a user experience for wearable devices - usability geek. https:// usabilitygeek.com/wearable-devices-user-experience. Accessed 14 Feb 2019 Schoneveld, A., Frey, N.: IoT Enables Devices and Wearables – MONICA Project. European Commission HORIZON 2020 (2017) Jahn, M., Michel, O., Suleri, S.: Site Surveys and Pilot Plans for MONICA IoT Platform Pilots MONICA Project. European Commission HORIZON 2020 (2018) Leeds Varsity, 21 August 2019. http://LeedsVarsity.com/

Work-in-Progress: Designing an e-Coaching System for Chronic Heart Failure Patients Evdokimos Konstantinidis1, Niki Pandria1, Antonis Billis1, Sophia-Anastasia Mouratoglou2, and Panagiotis D. Bamidis1(&) 1

2

Faculty of Health Sciences, Lab of Medical Physics/Aristotle University of Thessaloniki, Thessaloniki, Greece [email protected], {npandria,ampillis,bamidis}@auth.gr First Cardiology Department, AHEPA University Hospital/Aristotle University of Thessaloniki, Thessaloniki, Greece [email protected]

Abstract. Current guidelines for heart failure (HF) patients’ treatment are focused on improving their clinical status, functional capacity and quality of life. However, these guidelines are more difficult to be followed when the patients are older adults, because of the comorbidities and the potential cognitive decline. Many studies have proven so far the effectiveness of various rehabilitation programs and the physical exercises, which however requires lifestyle changes for a long-term effect. The main questions raised from these studies remain the: “Do the participants follow the recommended systems after the end of the research pilots, even though they know the effectiveness?” The challenge of the current project is to measure real life adherence to a regular physical exercise program, and to attempt to influence the older adult patients with HF towards being more active. To do so, BioTechCOACH-ForALL uses wearable devices to measure the activity levels, exergames (serious games for physical exercise) to deliver a physical exercise plan as a joyful activity and e-coaching techniques to influence the patients attitude towards a more healthy and recommended lifestyle. Keywords: Chronic heart failure
Self-care serious games
E-coaching
Virtual agents
Digital biomarkers
Electroencephalogram (EEG)
Affective neuroscience

1 Introduction Heart failure (HF) is a clinical syndrome characterized by typical symptoms (e.g. shortness of breath, edema and fatigue) accompanied by signs (e.g. elevated jugular venous pressure, pulmonary crackles, etc.) caused by a structural and/or functional cardiac abnormality [1]. Current guidelines for HF patients treatment are focused on improving their clinical status, functional capacity and quality of life. However, these guidelines are more difficult to be followed when the patients are older adults, because of the comorbidities and the potential cognitive decline. © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 437–444, 2021. https://doi.org/10.1007/978-3-030-49932-7_42

438

E. Konstantinidis et al.

Many studies have proven so far the effectiveness of various rehabilitation programs and the physical exercises. Among them, regular aerobic exercise is recommended in HF patients in order to improve their functional capacity and symptoms [1]. However, changing lifestyle, especially someone being an older adult, is very difficult. And this is one of the barriers in engaging older adults with HF in regular physical exercise. Many projects conduct pilot studies with strict schedule of physical exercise to which the participants should adhere. But is this always the case? Do the participants follow the recommended system after the end of the research pilot, even though they know the effectiveness? The need for more innovative and cost-effective treatment strategies led to studies of e-health programs showing promising results in patients with HF [2, 3]. These studies increased political and clinical attention to e-health strategies as a mean of improving outcomes in patients with HF. [4–6] present the role of e-coach system in the management of heart failure. The challenge of the current project is to measure real life adherence to a regular physical exercise program, and to attempt to influence the older adult patients with HF towards being more active. To do so, BioTechCOACH-ForALL uses wearable devices to measure the activity levels, exergames (serious games for physical exercise) to deliver a physical exercise plan as a joyful activity and e-coaching techniques to influence the patients attitude change towards a more healthy and appropriate lifestyle. The remainder of this paper is structured as follows. A background on the Chronic Heart Failure patients and their specific needs in terms of physical activity is presented in the following section. The concept and rationale of the BioTechCOACH-ForALL project, along with the procedures and technologies to be used are presented in the BioTechCOACH-ForALL section. The concept of the Virtual agents as coaches concludes the methodology part. The next section presents the pilots’ planned approach and the future perspective section sets the next steps by highlighting the challenges that will be tackles during the lifecycle of the project.

2 BioTechCOACH-ForALL Project 2.1

Concept and Rationale

High rates of non-compliance to treatment plan urge the need for developing sustainable solutions in order to support and enhance the self-care of HF patients. BioTechCOACH-ForALL, implemented within the framework of the operational program “Human Resource Development, Education and Lifelong Learning” cofunded by the European Social Fund and national resources, investigates and researches a potential response to this challenge. The main goal of the project is the development of an e-coaching system based on neuroscience evidence, incorporating exergaming [7] and decision support system [8] techniques. The patients’ engagement to different User Interface (UI) interaction means, such as virtual projected coaches with different characteristics (presence/absence of medical uniform, gender, age) will be explored by means of electroencephalography (EEG) and analyzing various biosignals such as heart rate (HR), electrodermal activity

Work-in-Progress: Designing an e-Coaching System

439

(EDA), external body temperature and eye gaze). Furthermore, we will extend our previous experience in developing and applying innovative systems for physical training of elderly (webFitForAll [2]), in living labs or even at their home [9], encouraging physical exercise and promoting independent living. In addition, the e-coach platform will be enriched by a decision support system that is based on analysis and collection of interaction data. Commercial non-intrusive sensors will collect activity data in order to capture daily activity patterns [8] as well as activity volume. Daily activity patterns will be used to track their daily activity level regarding the doctor’s recommendation as well as to readjust e-coaching system parameters. 2.2

Procedures and Technologies

BioTechCOACH-ForALL will be delivered as three different technologies in three phases, each of them fulfilling a different scope. The technologies along with their scope are presented herein (Fig. 1).

Fig. 1. Technology used in the BioTechCOACH-ForALL

Wearable Monitoring The first technology that will be introduced (Phase 1), is the Wavelet health monitoring device, which is selected in many similar studies as one of the best options for empowering investigators and patients. Its clinical-grade PPG sensor along with the accelerometer and gyroscope can collect continuous physiological and activity data that are dissected and processed using robust algorithms. Steps, calories, heart rate variability, sleep quality and total sleep time are among the measurements that will be used for the patients daily monitoring as far as activity is concerned. The outputs of the wearable monitoring will enable the system to collect and analyze activity levels, comparing them with the physical exercise performed (see Exergames for physical exercise.) or the e-coaching behavior (see Projected ecoaching.). The data will be analyzed on daily basis and an activity score (taking into account all the individual measurements) will be calculated. The patient will have to wear this wristband daily, charging it when needed or during predefined intervals. The wristband sends data only when it is connected to a smart phone gateway. Consequently, the corresponding app will be installed in the patients’ smart phones.

440

E. Konstantinidis et al.

Exergames for Physical Exercise The exergames application that will be used for delivering appropriate physical exercises to the patients is the webFitForAll platform [8]. Apart from incorporating physical exercises recommended by the American College of Sports Medicine and the American Heart Association [8], the webFitForAll platform provides a very detailed performance capturing mechanism. The webFitForAll platform guides the patient through a protocol of different exercises focusing on upper and lower strength, stretching and flexibility as well as aerobic exercise. In game performance metrics, including but not limited to reaction times, score, body extremities moving range, etc., will provide us with a noteworthy information on the patients’ adherence to the training schedule as well as on the patients’ adherence to the proper physical exercise instructions. The patients will be assigned with a recommended schedule (3 times/week), starting from a light training protocol (familiarizing also with the technology) towards a moderate intensity one. The webFitForAll technology will be installed either in the patients’ homes or in a common room where two or more patients can go through the training at the same time. Projected e-Coaching The last technology that will be introduced in the patients is the projected e-coaching functionality. A small projector, installed in the patients’ homes, will be running a decision support system which will be delivering messages and recommendations to the patient by projecting the selected virtual coach, on a predefined surface in the home (ceiling, wall, etc.). Just after delivering the message, the projector will turn off, to avoid distracting further the patient’s environment. The date and time the message delivered along with the type of message will be collected in order to be further compared with the potential effects (wearable monitoring outputs and physical exercise adherence). In addition, since it is foreseen to research the effectiveness of the representation of various virtual coaches (based on the results of the 3.2), the coach used every time will be also collected. The user interaction with the projected virtual coach will be designed to be kept to the minimum, being one-direction messages and recommendations delivery (from the coach to the patient). The goal is to attempt to change the patient’s attitude towards physical exercise without being too intrusive. More information on the theory behind the virtual agents as coaches is provided in the following sub section. Virtual Agent Based on scientific evidence social models are getting more effective as they resemble to the observer or an ideal virtual self of the observer [11]. Appearance is considered to be an important attribute while designing a virtual agent and therefore the agent should be visibly present [12]. However, the potential of characteristics’ customization remains unlimited. Existing evidence on learning showed that agents who have similar characteristics, with respect to appearance-related traits such as age and race/ethnicity, to trainees are more influential [11]. In line with this suggestion, Rosenberg-Kima et al. (2008) [12] showed that a young, attractive agent and cool female agent was the most effective for undergraduate females affecting their stereotypes and their self-efficacy

Work-in-Progress: Designing an e-Coaching System

441

toward engineering. However, prior expectations and stereotypes could influence the desired outcomes [13]. Another study showed that a physically similar avatar to the observer could affect more the emotional valence and arousal compared to a neutral one. Additionally, the induced emotional states were more intense than those of the neutral avatar [14]. Apart from the key components in virtual agent design, when embedding a virtual agent into a system, user engagement constitutes a very important factor [15]. Fairclough et al. (2013) [16] defined user engagement in a task with regards to cognitive activity (mental effort), motivation (approach or avoidance) and affective state (positive, negative) and then, linked the user engagement’s components to psychophysiological measurements. Revisiting the literature they found that increased theta activity in frontocentral sites along with decreased alpha activity in occipital sites were associated with higher mental effort due to working mental load [17]. Pupil dilation is greater when complex cognitive processing is performed [18]. On the other hand, motivation and emotional experience (affect) were correlated to frontal asymmetry. More precisely, greater levels of left frontal activity were associated to positive emotions and motivational approach respectively whereas higher right frontal activity was linked to negative emotions and motivational avoidance [19]. Other biomarkers of motivation are sympathetic nervous system indexes, mainly the systolic blood pressure [20]. In another study, users’ approval of an online avatar was explored by means of skin conductance, heart rate and respiration. Their results indicated that higher respirations were positively correlated to the degree of agent approval [21]. Peters et al. (2009) [22] proposed another component of human-agent interaction that is the user’s attention. They modeled user’s attention using three components, gaze detection, neurophysiological analysis as well as an attention representation module for storage, integration and interpretation of attention information.

3 Pilots 3.1

Pilot Deployment Plan

BioTechCOACH-ForALL will be delivered in three phases. Each of the three different phases will allow the participants to familiarize with the delivered technology of each phase, so that the effects of each phase are as isolated to the previous phase’s as possible, allowing for effect comparison among them. Phase 1 sets the objective measurement technology, the wearable monitoring device. This technology will be running throughout the project’s lifetime and will provide objective information on the patient’s activity. This information, along with the doctor’s baseline and intermediate assessment, will be used as an indicator of the effectiveness of the phases 2 and 3. The Phase 2 introduces a joyful way of exercising, allowing the patients to select between the comfort of their house or the socialization with other patients by playing together in groups. Finally, Phase 3, building on top of the Phase 2, introduces the coaching aspects, where, apart from the personalized recommendations and suggestions (designed by the doctor), the exergames are consider as an option to be recommended (Fig. 2).

442

E. Konstantinidis et al.

Fig. 2. The 3 phases along with the technology introduced in each phase

3.2

EEG Study

The main objective of the study is to explore neurophysiologically different appearancerelated characteristics of virtual coaches such as the age, gender and presence/absence of medical uniform. In more detail, participants will undergo an experimental procedure that consists of two parts. In part I, the HF patients will undergo electroencephalographic (EEG) recordings in resting-state with eyes-closed (5 min) and while passively viewing thirty (30) agents which are grouped with respect to their appearance characteristics. In the second part (part II), participants will passively view combinations of virtual agents who will differ regarding the presence/absence of medical uniform but preserving the other characteristics (age, gender). During the EEG recordings, we will also collect other biosignals such as electrodermal activity (EDA), blood volume pulse (BVP), external body temperature and eye gaze. The aforementioned biosignals will be collected by means of smartwatch (Empatica E4) while eye gaze detection will be performed using Gazepoint (GP3 eye tracker). Regarding the EEG analysis, event-related potentials (ERPs) will be computed for each condition (e.g. presence versus absence of medical uniform). Equal attention will be given to the spatial distribution and the temporal sequencing of neural activity. Time segments of significant differences between responses to each condition can be identified via either a sliding window analysis [23], selecting a predefined window based on literature evidence [24] or a paired topographic analysis of variance tests of the ERP maps [25]. Moreover, other biosignals will be compared between conditions (young vs. old/female vs. male/doctors vs. peers) after extracting the grand average values for each condition. The main outcomes of this study will provide inputs to the development of the virtual coach that will be used in Phase 3 of the pilot. 3.3

Study Endpoints

At all stages of technology piloting, the patients’ clinical condition, quality of life and health related costs will be considered. More specifically, the blood pressure, heart rate, blood oxygen saturation and body weight will be measured for the clinical assessment. On the quality of life front, the six minute walk test and the WHO functional class will be used for the exercise intolerance domain of the quality of life while on the health related domain the Beck Depression Scale, the Dukes questionnaire and the SF-36

Work-in-Progress: Designing an e-Coaching System

443

questionnaire. Finally, for the health related costs, the effect on number of hospital admissions along with the effect on healthcare usage will be calculated.

4 Conclusion The main challenge of the BioTechCOACH-ForALL project is the change of attitude of older heart failure patients through an e-coaching system. Although the pilot Phase 1 has already started, Phase 3 is expected to be the most challenging and innovative one. Given the adoption of co-creation and living lab approach and the main objective for real life evaluation, the project is ready to react to any collected feedback, even during the implementation of the plan. The clinical assessment and the objective evaluation are expected to provide all the required information for secure and important findings. An innovative point that has not been addressed yet by other studies is the design of the ecoaching system based on neurophysiological findings. Unobtrusive human-computer interaction episodes is expected to increase the acceptance of such systems at patients’ own homes. Real-world digital endpoints will allow for a more detailed evaluation of such interventions when take place in the wild. Acknowledgements. This research is implemented through the Operational Program “Human Resources Development, Education and Lifelong Learning” and is co-financed by the European Union and Greek national funds.

References 1. Ponikowski, P., et al.: 2016 ESC guidelines for the diagnosis and treatment of acute and chronic heart failure. Eur. Heart J. 37(27), 2129–2200 (2016) 2. Louis, A.A., Turner, T., Gretton, M., Baksh, A., Cleland, J.G.F.: A systematic review of telemonitoring for the management of heart failure. Eur. J. Heart Fail. 5(5), 583–590 (2003) 3. Anker, S.D., Koehler, F., Abraham, W.T.: Telemedicine and remote management of patients with heart failure. Lancet 378(9792), 731–739 (2011) 4. Stut, W., Deighan, C., Cleland, J.G., Jaarsma, T.: Adherence to self-care in patients with heart failure in the HeartCycle study. Patient Prefer. Adherence 9, 1195–1206 (2015). https:// doi.org/10.2147/PPA.S88482 5. Stut, W., Deighan, C., Armitage, W., Clark, M., Cleland, J.G., Jaarsma, T.: Design and usage of the heartcycle education and coaching program for patients with heart failure. JMIR Res. Protoc. 3(4), e72 (2014). https://doi.org/10.2196/resprot.3411 6. Sharma, A., Mentz, R.J., Granger, B.B., Heitner, J.F., Cooper, L.B., Banerjee, D., Felker, G. M.: Utilizing mobile technologies to improve physical activity and medication adherence in patients with heart failure and diabetes mellitus: rationale and design of the TARGET-HFDM Trial. Am. Heart J. 211, 22–33 (2019). https://doi.org/10.1016/j.ahj.2019.01.007 7. Konstantinidis, E.I., Bamparopoulos, G., Bamidis, P.D.: Moving real exergaming engines on the web: the webFitForAll case study in an active and healthy ageing living lab environment. IEEE J. Biomed. Heal. Informatics 21(3), 859–866 (2017) 8. Billis, A.S., Papageorgiou, E.I., Frantzidis, C.A., Tsatali, M.S., Tsolaki, A.C., Bamidis, P.D.: A decision-support framework for promoting independent living and ageing well. IEEE J. Biomed. Heal. Informatics 19(99), 199–209 (2015)

444

E. Konstantinidis et al.

9. Bamidis, P.D., Karagianni, M., Batziakas, A., Billis, A.S., Tsatali, M.S., Bratsas, C.: Enabling active and healthy ageing decision support systems with the smart collection of TV usage patterns. Healthc. Technol. Lett. 3(1), 46–50 (2016) 10. Nelson, M.E., et al.: Physical activity and public health in older adults: recommendation from the American college of sports medicine and the American Heart Association. Circulation 116(9), 1094–1105 (2007) 11. Baylor, A.L.: Promoting motivation with virtual agents and avatars: role of visual presence and appearance. Philos. Trans. R. Soc. B Biol. Sci. 364(1535), 3559–3565 (2009) 12. Rosenberg-Kima, R.B., Baylor, A.L., Plant, E.A., Doerr, C.E.: Interface agents as social models for female students: the effects of agent visual presence and appearance on female students’ attitudes and beliefs. Comput. Human Behav. 24(6), 2741–2756 (2008) 13. Baylor, A.L.: The design of motivational agents and avatars. Educ. Technol. Res. Dev. 59 (2), 291–300 (2011) 14. Wrzesien, M., Rodríguez, A., Rey, B., Alcañiz, M., Baños, R.M., Vara, M.D.: How the physical similarity of avatars can influence the learning of emotion regulation strategies in teenagers. Comput. Hum. Behav. 43, 101–111 (2015) 15. O’Brien, H.L., Toms, E.G.: What is user engagement? a conceptual framework for defining user engagement with technology. J. Am. Soc. Inf. Sci. Technol. 59(6), 938–955 (2008) 16. Baylor, A.L., Kim, S.: Designing nonverbal communication for pedagogical agents: when less is more. Comput. Hum. Behav. 25(2), 450–457 (2009) 17. Gevins, A., Smith, M.E.: Neurophysiological measures of cognitive workload during human-computer interaction. Theor. Issues Ergon. Sci. 4(1–2), 113–131 (2003) 18. Fairclough, S.H., Gilleade, K., Ewing, K.C., Roberts, J.: Capturing user engagement via psychophysiology: measures and mechanisms for biocybernetic adaptation. Int. J. Auton. Adapt. Commun. Syst. 6(1), 63 (2013) 19. Davidson, R.J.: What does the prefrontal cortex ‘do’ in affect: perspectives on frontal EEG asymmetry research. Biol. Psychol. 67(1–2), 219–234 (2004) 20. Richter, M., Gendolla, G.H.E.: Incentive value, unclear task difficulty, and cardiovascular reactivity in active coping. Int. J. Psychophysiol. 63(3), 294–301 (2007) 21. Foglia, P., Zanda, M.: Towards relating physiological signals to usability metrics: a case study with a web avatar. WSEAS Trans. Comput. 13, 624–634 (2014) 22. Peters, C., Asteriadis, S., Rebolledo-Mendez, G.: Modelling user attention for human-agent interaction. In: 2009 10th Workshop on Image Analysis for Multimedia Interactive Services, pp. 266–269 (2009) 23. Styliadis, C., Ioannides, A.A., Bamidis, P.D., Papadelis, C.: Mapping the spatiotemporal evolution of emotional processing: an MEG study across arousal and valence dimensions. Front. Hum. Neurosci. 12, 322 (2018) 24. Paraskevopoulos, E., Kuchenbuch, A., Herholz, S.C., Pantev, C.: Musical expertise induces audiovisual integration of abstract congruency rules. J. Neurosci. 32(50), 18196–18203 (2012) 25. Esslen, M., Pascual-Marqui, R., Hell, D., Kochi, K., Lehmann, D.: Brain areas and time course of emotional processing. Neuroimage 21(4), 1189–1203 (2004)

A Comparative Examination of AR and Video in Delivering Assembly Instructions Kaija Petrone(&), Richard Hanna(&), and G. Shankaranarayanan(&) Babson College, Boston, MA, USA {kpetrone,rhanna,gshankar}@babson.edu

Abstract. Recent developments in technology have made augmented reality (AR) accessible through smartphones, thus allowing millions of users to experience AR. With such unlimited market availability, every industry is attempting to leverage the potential of AR. In this paper, we examine the effectiveness of AR in providing assembly instructions to lay (or novice) users. We compare video and AR instructions in assisting lay users to complete an assembly process; we manipulate the complexity of the assembly task to help validate results. Using task accuracy, task efficiency and user engagement proxies to measure effectiveness, we find that as the task complexity increases, AR instructions begin to yield higher objective accuracy and efficiency than video instructions do. We observe a learning curve associated with using the headset, the device through which the AR instructions were delivered; consequently, less complex tasks were better supported by video instructions than AR ones. We believe this study provides important insights for companies considering the use of AR to provide assembly instructions to their user base. Keywords: Assembly instructions
Augmented Reality Effectiveness
Accuracy
Efficiency
Engagement


Video


1 Introduction Many of us have sat on the floor surrounded by pieces of wood and metal as we unsuccessfully attempt to decrypt the assembly instructions written in micro-font and in eight different languages. Worse still is when, as you try to add that last piece, you realize that in your initial step you switched the right and left-side pieces because they looked identical. Now, imagine that scanning a code on the box with your smartphone would let you access an augmented reality application to walk you through the assembly process and even tell you if you have the correct piece in the appropriate place. Augmented Reality (AR) is a technique of overlaying virtual images over real world (physical) objects. Unlike virtual reality (VR), which creates a completely artificial environment, AR takes an existing real environment and overlays (adds) new information (such as text, images or three-dimensional objects) onto it. Krevelen and Poelman (2010) define AR as a system that has (a) the ability to combine virtual and real objects in a real environment, (b) the ability to align real and virtual objects with each other and (c) the ability to run in real time, interactively and in three dimensions. © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 445–456, 2021. https://doi.org/10.1007/978-3-030-49932-7_43

446

K. Petrone et al.

To experience AR, users need special visual support such as smart-glasses and smartlenses. Recent developments in technology have made AR accessible through smartphones, thus allowing millions of users to experience AR. With unlimited market availability, numerous industries are attempting to leverage the potential of AR. The gaming industry is one that has developed and successfully monetized AR applications. Tech savvy industries, such as tourism, hospitality, advertising and retail, have also developed AR applications but are yet to successfully monetize them. From an assembly perspective, AR should make it significantly easier to assemble most anything, including furniture. Adam Pickard’s AssembleAR app turns paper IKEA manuals into an AR app built using 3D modeling and post-production editing (Morby 2018). The furniture parts in this app are modeled using Cinema 4D software and made to look identical to the IKEA original. Manufacturing industries have used AR for assembly for over two decades, initially using projectors and recently switching to head-mounted or heads-up displays (HMDs or HUDs). These applications of AR in assembly enable trained engineers and technicians to assemble/inspect manufactured components/goods more easily, quickly and precisely. AR and related technologies have advanced significantly and access to HMD/HUD has become relatively more prevalent in the recent past. With these advances in AR and the relative ease of access to view and experience it, AR should make it easier for untrained (lay) users to fix simple issues with their copier, their printer, their washer/dryer and even their automobiles, eliminating the need to call the specialist each time something isn’t quite right. However, AR’s effectiveness in supporting lay users in the assembly process has not been well-examined. Given the proliferation of do-it-yourself and other instructional videos available on the web, a key, yet-to-be-thoroughly-explored question becomes: would AR be more effective than such videos? To this end, in this paper, we compare video and AR instructions in assisting lay users to complete an assembly process. The research question we raise in this paper is “how effective is AR in providing assembly instructions to lay users compared to video instructions”? Adapting from literature, we posit that effectiveness is a composite measure made up of accuracy, efficiency and engagement. We believe that understanding effectiveness will allow content creators to design more effective AR experiences. Our key contributions are: (1) we believe we are one of the first to compare video and AR in the context of assembly; (2) we define and use the construct of effectiveness, which consists of accuracy, efficiency and engagement; (3) we reveal important insights into not only when AR is more effective but also on how to design AR applications that can deliver effective instructions for assembly tasks. In the remainder of this paper, we first present relevant research to differentiate our work. We then present the theoretical foundation of our model and present our methodology. The results and observations are presented next, followed by our discussions and a brief conclusion.

2 Relevant Literature Although there has been a considerable body of research in AR, for brevity, we expand on such research that examines a comparative evaluation of AR in the context of manufacturing assembly. Baird and Barfield (1999) evaluated the effectiveness of AR

A Comparative Examination of AR and Video in Delivering Assembly

447

when assembling computer motherboards. They concluded that, when assisted with AR, users took less time to complete the task than when assisted with displays or paper-based instructions. Boud et al. (1999) investigated the use of an AR application that uses a HUD with a see-through display for water pump assembly. They, too, concluded that the task completion time was shorter when users used their application than when they used engineering (two-dimensional) drawings. Tang et al. (2003) evaluated the effectiveness of using AR for assembling Duplo blocks. In this research, the authors compared an AR application on a HMD to static instructions provided via a paper or computer display. They observed that AR reduced the errors by 82% compared to static instructions. Wiedenmaier et al. (2003) developed an AR application to provide guidance for a typical industry assembly task. They compared the performance of users who were provided either AR guidance, paper-based instructions or expert guidance. They observed that providing expert guidance resulted in the quickest completion time, followed by AR guidance. Paper-based instructions resulted in the slowest completion time. Hou et al. (2013) developed a prototype AR system for Lego assembly and compared the use of this system with paper-based instructions. The authors examined cognitive task load as well as the learning curve for novice users. They concluded that the AR system resulted in shorter task completion times with fewer assembly errors. They also concluded that the cognitive load was lower for AR. Similar to the research summarized above, our paper also performs a comparative evaluation of an AR application for assembly. Also like the above, we too, construct an AR application for assembling a 3D puzzle. Unlike the above, however, which compares AR against paper-based instructions, in our research, we compare AR with videobased instructions. The motivation behind such a decision is that today, with the ready availability of online DIY videos, lay users resort more to video instructions than to paper-based instructions. Syberfeldt et al. (2015) examine how user acceptance of AR applications may be improved. They observe building a prototype system for an assembly task involving a 3D puzzle. The assembly task requires users to put the pieces together in a specific sequence. The authors compare the AR system with paper-based instructions. Acceptance was examined using ease-of-use, satisfaction and approval as subjective proxies. In addition, objective measures such as error-rate and completion time were also measured. The analyses of the subjective measures revealed that paper-based instructions were easier to use, more efficient and less physically demanding. In terms of ease of understanding and mental strain, both AR and paper-based instructions were the same. Analyzing the objective measures, paper-based instructions were superior to AR in terms of errors and in terms of completion time. The authors concluded that, while AR applications are acceptable to users, AR technology should become stronger for it to provide superior support. Similar to Syberfeldt et al. (2015), we also built an AR application for assisting with the assembly of a 3D puzzle. Like Syberfeldt et al. (2015) we adopt both perceptual and objective measures. However, we do not examine user acceptance. Instead, we look at effectiveness of AR and compare it to video instructions. In this paper, we hypothesize that effectiveness is a complex construct that is made up of accuracy, efficiency and engagement.

448

K. Petrone et al.

3 Theoretical Model and Hypotheses The purpose of this study is to determine how to measure the effectiveness of augmented reality in providing assembly instructions. It is important to understand how to measure effectiveness because it can predict the likelihood of sustained usage overtime and improve the ability to create AR experiences that enhance user productivity, better meet users’ targeted needs and provide greater value. Ultimately, understanding how to measure effectiveness allows content creators to design better experiences that yield greater productivity. To investigate the effectiveness of AR, it is important to define “effectiveness” and establish a way to measure it. In AR research, there is a paucity of research that addresses effectiveness and there is no clear definition of how to measure it. Chung et al. (2016) examine the effectiveness of an AR application that guides users through a building to locate specific landmarks. The authors define effectiveness as how well the users remember the nuances of the building’s layout and navigational aspects. This definition does not offer much value in the context of assembly instructions. However, usability and engagement are metrics that have been extensively researched in the areas of human-computer interaction. Usability, as defined by Iordache and Pribeanu (2009), is the extent to which a product may be used by specific users to achieve specific goals effectively and efficiently as well as with satisfaction in the context of use. They state that effectiveness can be measured using completion rate and errors while efficiency can be measured using mean execution time. Alshamari and Mahyew (2009) state that effectiveness can be measured by examining aspects like binary task completion, accuracy, completeness and quality of outcome. Efficiency may be measured, according to the same authors, examining aspects like task completion time, mental effort and learning time. In the context of AR and our research, we posit that effectiveness is a construct that is made up of more than accuracy and efficiency. The extent to which the AR application engages the user is a key component of what we define as effectiveness. Wiebe et al. (2013) emphasize the importance of understanding engagement by arguing that “engagement has been recognized as a key factor in understanding general user behaviour and overall efficacy of goal or task-oriented behaviour within computerbased environments”. Kim et al. (2013) provide a framework for measuring engagement in a digital environment. Both argue that engagement consists of hedonic (e.g., fun, pleasant) and functional (e.g., ease of use, saving time, perceived value, satisfaction with the outcome). They argue that when users are enjoying what they do, find it interesting and useful, they are more likely to be engaged. Wiebe et al. (2013) also argue that higher engagement can lead to the user reaching a higher level in a video game. Based on the above theoretical foundation, we posit that effectiveness of an AR application is affected by task accuracy, efficiency and engagement. We define accuracy the percentage of a task completed correctly relative to the final completed end product. We define efficiency as the rate at which a user completes a task. Efficiency can be calculated in an assembly task by dividing the number of pieces assembled in a given time by the time taken to assemble them. We define engagement as the level of personal

A Comparative Examination of AR and Video in Delivering Assembly

449

investment or motivation a user has in completing a given task. The more personally invested a person is in completing a task, the deeper their engagement is (Bandura 1986). Factors that influence engagement include motivation, happiness, mental challenge and frustration. Figure 1 depicts our causal model validated in this study.

Fig. 1. Research model

To validate the research model, we formulate the following three hypotheses. H1: Accuracy in assembling the puzzle will be higher for users with AR instructions than for users with video instructions. H2: Efficiency in assembling the puzzle will be higher for users with AR instructions than for users with video instructions. H3: Engagement in assembling the puzzle will be higher for users with AR instructions than for users with video instructions.

4 Methodology and Experimental Setup To provide a challenging assembly process, we opted to use a 3D puzzle ball made up of 12 pieces. Some of the pieces look almost identical to each other, as can be seen in Fig. 2 below. The augmented reality experience was created by sourcing a CAD model of the puzzle from GrabCAD and importing it into PTC’s Creo Illustrate software to color-code the pieces to match the actual (physical) puzzle and create the sequences (steps) necessary to solve the puzzle. The Creo Illustrate file was then published and uploaded to PTC’s Vuforia Studio, an augmented reality authoring tool, to create the augmented reality experience. We initially tested the AR application on a smartphone. A major hurdle to solving the puzzle with the smartphone was that the users needed both hands to hold and assemble the pieces. We then decided to use the HMD to deliver the AR instructions. We created a video of the assembly instructions that followed the same sequence that the AR application did. We pilot-tested the video and improved it based on feedback to make the video more user-friendly and to improve clarity of instructions. We further tested the instructions with subjects of different age, gender and attitude for puzzles to determine an appropriate time limit to complete the puzzle. The option to impose a time constraint was debated due to the sense of urgency it would create. Pretests were conducted to justify the use of a time constraint by providing preliminary evidence for an appropriate length of time.

450

K. Petrone et al.

Fig. 2. The assembly task puzzle

We varied complexity by creating a simpler version of the puzzle. We achieved this by color-coding the pieces that were identical so that the users could easily distinguish the pieces. We created the corresponding AR application and video for this color-coded puzzle. The subjects were primarily college students who volunteered for the task. Each subject came into a quiet space where the puzzle and instructions were provided to the subject. The entire process was observed and recorded. At the end, each subject was administered a survey to collect demographic data as well as perceptual data on the instructions and experience. Subjects were randomly assigned to one of four groups: simple puzzle with video, simple puzzle with AR, complex puzzle with video or complex puzzle with AR. Our analyses of the data indicated that there was no significant difference in age or gender across these four groups. To gather data in this experiment, a questionnaire, supplemented by direct observation, was used. The questionnaire collected information on perceived accuracy, perceived efficiency and engagement; direct observation was used to gather objective measures like number of pieces assembled correctly (objective accuracy) and time taken (objective efficiency). Perceptual data on accuracy and efficiency was collected using a 5-point Likert scale where 1 indicated an extreme low end and 5 an extreme high end. Perceptual data on engagement was collected using a 7-point Likert scale where 1 indicated an extreme low (e.g., extreme dissatisfaction) and 7 indicated an extreme high. To test accuracy, we collected both perceptual (how accurate did the user think they were) and objective data (how many pieces were completed within the time and how many were correct). Similarly, for efficiency we collected both perceptual (how much faster did the user think they were compared to having no instructions and how close to completing the puzzle were they) and objective data (binary completion as well as how long it took to complete the puzzle). For engagement, we collected only perceptual data. The questions were adopted from established instruments (Wiebe et al. (2013) and Alshamari and Mahyew (2009)). The survey was pre-tested and improved for clarity.

A Comparative Examination of AR and Video in Delivering Assembly

451

5 Results and Observations The collected data was analyzed using SPSS. Crosstab tests were run for yes/no questions (Boolean responses), and each question that required a crosstab test had two crosstab tests done: one test against level of complexity and one test against type of instruction. UNIANOVA tests were run for questions that utilized scales when measuring the dependent variable against level of complexity and type of instruction. No significant difference in completion rate (i.e., whether a user completed the task) was observed between AR subjects and video subjects or between high-complexity and low-complexity subjects. Of the 25 subjects using video instructions, 18 (or 72.00%) completed the puzzle; of the 24 subjects using AR instructions, 15 (or 62.50%) completed the puzzle. Of the 27 subjects exposed to the low complexity, 19 (or 70.37%) completed the puzzle; of the 22 subjects exposed to high complexity, 14 (or 63.64%) completed the puzzle. The data is shown in Table 1. Table 1. Completion (instruction type) and completion (complexity)

Puzzle completed Puzzle not completed

Instruction Video AR 18 15 7 9

Puzzle completed Puzzle not completed

Complexity Low High 19 14 8 8

The data gathered and analyzed for perceived and objective accuracy is shown in Table 2. The data suggests that video users had greater perceived accuracy when solving the low-complexity puzzle than when solving the high-complexity puzzle. Alternatively, AR users had higher perceived accuracy when solving the highcomplexity puzzle. Video users’ perceived accuracy declined with level of complexity, whereas AR users’ perceived accuracy increased with level of complexity. This suggests that, when users are exposed to a high-complexity task, AR users feel that their work is more accurate than video users. To measure objective accuracy, the moderator counted the number of pieces completed at the end of the allotted thirty minutes and determined the number of them that were correct. For participants that completed the puzzle correctly, objective accuracy was recorded as 100% (or 5 on the 5-point scale). For both low- and high-complexity tasks, objective accuracy was higher for video instructions than for AR instructions. Objective accuracy declined from low- to highcomplexity for both video and AR users, suggesting that, as complexity increased, the number of pieces the participant assembled correctly decreased regardless of instruction type. Although it appears that this decline is smaller for participants with AR compared with video, the difference is not statistically significant. Hypothesis H1 is hence not supported. Looking at efficiency, for high-complexity tasks, AR users reported greater perceived efficiency than video users did. The reason for this may be that, for simple tasks, the AR itself adds to the complexity (making the simple task relatively more complex) because users are unfamiliar with AR and find the instructions more difficult to follow. For low-complexity tasks, however, video users reported greater perceived efficiency

452

K. Petrone et al.

than AR users. Thus, AR seems to be more of a hindrance than a help for lowcomplexity tasks. For high-complexity tasks, however, AR yielded greater perceived efficiency than video did. For low-complexity tasks, video users were more efficient than AR users. The results are summarized in Table 3 below. Table 2. Perceived and objective accuracy Complexity Low High How accurately did you assemble the puzzle? - Video 4.857 4.455 How accurately did you assemble the puzzle? - AR 4.462 4.545 How many pieces were put together? - Video 11.571 10.364 How many pieces were put together? - AR 10.385 9.818 What percentage of the pieces assembled were correct? - Video 4.857 4.636 What percentage of the pieces assembled were correct? - AR 4.308 4.182

Table 3. Perceived and objective efficiency

How much faster did the instructions help you solve the puzzle? Video How much faster did the instructions help you solve the puzzle? AR How close to completion were you (number of pieces)? - Video How close to completion were you (number of pieces)? - AR Objective time (seconds) to complete - Video Objective time (seconds) to complete - AR Objective efficiency (pieces/time) - Video Objective efficiency (pieces/time) - AR

Complexity Low High 7.929 7.364 7.000

7.636

9.929 9.538 744.545 935.625 1.779 1.139

9.727 9.636 1000.000 942.714 1.249 1.109

We further computed objective efficiency as the ratio of pieces assembled to time (either the actual time taken to complete the task or 30 min for the time limit on incomplete tasks). Objective efficiency substantially declined for video users from low to high complexity. It also declined for AR users, but at a much slower rate. The substantial decline in efficiency for video users suggests that, as a task becomes more complex, video users quickly become less effective; while a decline is also present for AR users, it is much flatter, suggesting that the effectiveness of AR may be less impacted by level of complexity. Moreover, although efficiency of the participants with the high-complexity task is higher for video users than for AR users, the steep slope of video users suggests that, once a task becomes significantly more complex, AR will yield greater efficiency than video. The overall lower efficiency for AR users than for video users (in both low- and high-complexity) may be explained partly by the fact that a learning curve likely inhibited the effectiveness of the AR instructions. Although we

A Comparative Examination of AR and Video in Delivering Assembly

453

perceive differences in the values observed, none are statistically significant. Hence hypothesis H2 is not supported by our results. Users were also asked how useful they found the instructions to solving the puzzle. We define “usefulness” of instructions as the importance of the presence of instructions (i.e., having instructions or not). We do not observe a difference between AR and video, but the usefulness declined with complexity. Users were also asked to report how challenging they found the instructions to follow. AR was more challenging to follow than video for low-complexity tasks (likely due to the learning curve argument associated with using AR). However, AR was easier to follow than video for highcomplexity tasks. Again, this supports that high-complexity tasks are better suited for AR instructions than for video instructions. Additionally, users were asked to evaluate how satisfied they were with the instructions. The perceived satisfaction was higher for video instructions than for AR instructions for the low-complexity task. The satisfaction of AR users remained relatively equal from low to high complexity. For video users, however, satisfaction sharply declined from low to high complexity. This suggests that the more complex a task is, the less effective video instructions may be. The results are summarized in Table 4. Table 4. Engagement Complexity Low High How useful were the instructions in solving the puzzle? - Video 7.500 7.182 How useful were the instructions in solving the puzzle? - AR 7.077 6.636 How challenging were the instructions to follow? - Video 4.714 5.000 How challenging were the instructions to follow? - AR 5.385 4.909 How satisfied were you with the instructions? - Video 7.000 6.455 How satisfied were you with the instructions? - AR 6.462 6.455 Were the instructions too fast? - Video 3.571 4.100 Were the instructions too fast? - AR 2.333 1.818 The instruction delivery was difficult to follow. - Video 2.429 3.900 The instruction delivery was difficult to follow. - AR 3.000 3.182 How motivated were you to complete the puzzle? - Video 7.429 7.455 How motivated were you to complete the puzzle? - AR 7.462 7.727 How happy were you working on the puzzle? - Video 6.643 5.636 How happy were you working on the puzzle? - AR 6.231 6.818

To confirm the engagement in instructions, we asked users to report on how they perceived the difficulty in following the instructions. For subjects solving the lowcomplexity puzzle, more AR users than video users found the instruction delivery method (i.e., video or AR) difficult to follow, whereas, for the high-complexity puzzle, video appeared more difficult to follow than AR. This further suggests AR is more engaging for high-complexity tasks. As another proxy to measure engagement, we asked participants about their motivation to complete the puzzle. AR and video users

454

K. Petrone et al.

reported very similar motivation for low complexity, but AR users reported much higher motivation for high complexity than video users did. We can, with some confidence, state that the type of instruction did have an impact on motivation, especially for complex tasks. Thus, for greater engagement (as measured by motivation) in highcomplexity tasks, AR instructions should be used rather than video instructions. Users were also asked about their level of happiness in completing the task. For the low-complexity puzzle, video users were happier than AR users, whereas, for the highcomplexity puzzle, the opposite is true. For high-complexity, AR users reported a higher level of happiness than video users did. Like motivation, happiness in completing the task appears to be affected by instruction type, especially with the increasing complexity of tasks. The fact that happiness increased for AR users and decreased for video users for the more complex task supports our argument that highly complex tasks are better suited for AR instructions than for video instructions. As in the previous two metrics, the results were not statistically significant for it to support H3.

6 Discussion and Conclusion There was no statistically significant difference in completion time when manipulating complexity for AR and video. This leads us to believe that the puzzle was not sufficiently complex (i.e., color-coding, number of pieces, difficulty of puzzle, etc.). This is particularly surprising given the numerous elements of the puzzle (e.g., similarly shaped pieces with subtle differences and complex shapes) that made it challenging. The color-coding was intended to highlight these differences and aid users in selecting the correct piece for a given step. It was expected that participants with the non-colorcoded puzzle (i.e., the single-color puzzle) would confuse the near-identical pieces and have more difficulty. This was not the case. Users were asked to provide recommendations on how to improve the instructions. Feedback on the AR instructions can be divided into three categories: technological improvements, task-specific AR improvements and broadly applicable AR improvements. From the technology perspective, many users complained of blurry images. One user experienced a headache and nausea while using the device, which inhibited their ability to move forward with a clear head. While the technology is continually being enhanced, blurry vision and headaches are still common problems reported with AR via HMD. Furthermore, one user reported that the “AR disappeared at certain angles” and they “had to crouch to see the puzzle.” This was likely due to the AR headset being improperly fitted at the beginning of the experiment. Better explanations and more practice with the AR headset can prevent or mitigate this issue in future studies. From a task-specific perspective, the puzzle used in this study contained two sets of pieces that looked nearly identical but had subtle differences. The subtlety of these differences often resulted in users incorrectly believing they had the right piece for a step. Participants recommended that the subtleties be highlighted so that they could be more easily identified. Additional recommendations for the AR instructions include the ability to rotate the puzzle—not only the individual pieces, but the entire puzzle itself. One user, for example, reported that “movement around the puzzle to see different angles of the

A Comparative Examination of AR and Video in Delivering Assembly

455

pieces was a little awkward,” and a second user noted, “I wish I could have moved the puzzle around in the AR.” Thus, it is important to have the ability to rotate both the individual pieces as well as the entire puzzle itself. Another user reported, “[I] can’t move my head around to see [the AR] from below or the back.” Users were given a table and chair at which to sit to solve the puzzle; some users were given larger tables than others, which, retrospectively, appeared to be a potential drawback or inconsistency of the study. Participants that were provided a narrow table (approximately two feet in depth) were more easily able (and perhaps more likely) to walk around it to see the AR at different angles, whereas participants that were provided a wider table were less easily able to do so because the table acted as an obstacle. This is an important consideration when designing AR instructions because it is highly probable that users of AR in real-world settings will have limited space around which to move. For example, if a user is on a ladder twenty feet in the air using AR to understand how to repair equipment at the top of a telephone pole, their mobility is highly limited. Similarly, if an electrician needs to access equipment below the deck of a house, their ability to rotate their body around to see different angles of the instructions in a tight space is severely restricted. Perhaps the strongest recommendation received is to incorporate a method of immediate feedback into the AR so that users can instantly know whether the piece they just inserted was done so correctly. Providing immediate feedback on whether a piece is assembled correctly would not only improve accuracy but also efficiency. While the analysis show that the results are not statistically significant, the results nonetheless provide directional guidance on when AR versus video should be used in providing assembly instructions. The sample size of approximately 12 subjects to each group (51 overall, 2 discarded, and a 2  2 experimental design) is too small to make the results statistically significant. This should be addressed in future iterations. In this study we examined the effectiveness of AR for providing assembly instructions. We compared it to video instructions. We defined effectiveness in a novel way to include not only accuracy and efficiency but also engagement, which we believe is critical to evaluating AR applications. The results of this study suggest that highcomplexity tasks are better suited for AR instructions than for video instructions and low-complexity tasks are better suited for video instructions than for AR instructions. This can likely be attributed to the fact that using AR entails a learning curve that hinders one’s ability to complete a basic task, but, once overcome, serves to improve one’s ability to complete a more complex task. Ultimately, when deciding which instruction type to use (video or AR), creators of instructions should consider the complexity of the task for which they are creating the instructions. Although AR has a great deal to offer from the perspective of helping businesses increase productivity of their workers, there is a lot to be understood in terms of how AR instructions should be created and used. This study offers some interesting insights into aspects of AR that need closer attention in order for AR to deliver most strongly on its promise.

456

K. Petrone et al.

References Alshamari, M., Mayhew, P.: Technical review: current issues in usability testing. IETE Tech. Rev. 26(6), 402–406 (2009). https://doi.org/10.4103/0256-4602.57825 Baird, K.M., Barfield, W.: Evaluating the effectiveness of augmented reality displays for a manual assembly task. Virt. Real. 4, 250–259 (1999) Bandura, A.: Social Foundations of Thought and Action: A Social Cognitive Theory. Prentice Hall, Englewood Cliffs (1986) Boud, A.C., Haniff, D.J., Baber, C., Steiner, S.J.: Virtual reality and augmented reality as a training tool for assembly tasks. In: Proceedings of 1999 IEEE International Conference on Information Visualization, pp. 32–36 (1999) Chung, J., et al.: Mindful navigation for pedestrians: improving engagement with augmented reality. Technol. Soc. 45, 30–33 (2016) Hou, L., Wang, X., Bernold, L., Love, P.: Using animated augmented reality to cognitively guide assembly. J. Comput. Civil Eng. 27, 439–451 (2013) Iordache, D.D., Pribeanu, C.: A comparison of quantitative and qualitative data from a formative usability evaluation of an augmented reality learning scenario. Informatica Economică 13(3), 67–74 (2009) Kim, Y.H., Kim, D.J., Wachter, K.: A study of mobile user engagement (MoEN): engagement motivations, perceived value, satisfaction, and continued engagement intention. Decis. Support Syst. 56(1), 361–370 (2013) Krevelen, D., Poelman, R.: A survey of augmented reality technologies, applications and limitations. Int. J. Virt. Reality 9(2), 1–20 (2010) Morby, A.: IKEA assembly made easier through Augmented Reality App (2018). https://www. dezeen.com/2018/03/23/ikea-assembly-made-easier-through-augmented-reality-app/, Accessed Jan 2019 Syberfeldt, A., Danielsson, O., Holm, M., Wang, L.: Visual assembling guidance using augmented reality. Procedia Manuf. 1 (2015). https://doi.org/10.1016/j.promfg.2015.09.068 Tang, A., Owen, C., Biocca, F., Mou, W.: Comparative effectiveness of augmented reality in object assembly. In: Proceedings of the SIGCHI Conference on Human factors in Computing Systems, 73–80 (2003) Wiebe, E.N., Lamb, A., Hardy, M., Sharek, D.: Measuring engagement in video game-based environments: investigation of user engagement scale. Comput. Hum. Behav. 2013, 123–132 (2013) Wiedenmaier, S., Oehme, O., Schmidt, L., Luczak, H.: Augmented reality (AR) for assembly processes design and experimental evaluation. Int. J. Hum.-Comput. Interact. 16(3), 497–514 (2003)

Wearable E-Textile as a Narrative Mediator for Enhancing Empathy in Moral Development George Palaigeorgiou(&), Grigoria Vroikou, Charoumenou Nikoleta, and Tharrenos Bratitsis University of Western Macedonia, Florina, Greece {gpalegeo,bratitsis}@uowm.gr, [email protected], [email protected]

Abstract. Several studies have been conducted which focus on the creation of new storytelling media that enhance empathic skills. In this study we introduce wearables as digital storytelling mediators to promote empathy in the context of bullying prevention interventions. In the proposed approach, the main “reader” of the narrative wears the e-textile in a dark room and stands opposite of a mirror in order to watch his own body as it is transformed by the illuminations. The narrative screen is the “reader’s” own body which mediate the oral narrations. In order to evaluate the proposed “empathy machine”, 21 students of 6th grade of an Elementary school interacted with the e-textile and a digital story about a young student named Orpheus, who was being bullied by a group of children at school. The participants stated that they enjoyed their interaction with the e-textile. The content of the mediated world (wearable e-textile, narrative & sound, atmosphere in the room) influenced the users’ feeling of being in the story. The participants identified with the characters, took their perspective and when they used their body, the narrative evoked even more their emotions. Keywords: E-textiles


Wearables
Digital storytelling
Empathy

1 Introduction Moral development is as vital as physical and intellectual development. Building moral lives and having moral values integrated into daily thoughts, feelings and actions is essential to any democratic society. The role of education in preparing young people to fit into society has been widely acknowledged (Althof and Berkowitz 2006) and its aim is not merely the production of good learners, but also of good people (Kohn 1991). Schools are in charge of educating not only children’s mind, but also their heart. Modern moral problems, like bullying, racism and violation can be encountered and prevented by transferring and cultivating moral values in children. Bullying is a serious moral problem affecting children and youth all across the world. It is wellknown that aggression and bullying are tied to a lack of empathy (Birch 2016). Empathy is vital for creating meaningful relationships in the social world, since the perceptual and behavioral processes that entails facilitate group living, are beneficial for social interaction (Batchelder et al. 2017) and effective education (Gottman 2011). A key part of creating an empathic connection is being able to see from another © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 457–467, 2021. https://doi.org/10.1007/978-3-030-49932-7_44

458

G. Palaigeorgiou et al.

person’s perspective. For that reason, several Prevention Bullying Programs aim to prevent or reduce bullying, as well as to achieve better peer relations at school by building empathy skills (Malti et al. 2016). Narratives and storytelling have been used by many prevention bullying programs as an effective method to promote empathy (Skaraas 2018; Manney 2008). Previous studies have pinpointed that, storytelling, in its many forms, can be a powerful tool for gaining new perspectives and creating shared understanding. Stories can have both a cognitive or emotional impact, since they evoke empathy and the sense of identification with the characters. Several studies have been conducted which focus on the creation of new storytelling media that enhance empathic skills. In this study we will introduce wearables as digital storytelling mediators to promote empathy in the context of bullying prevention interventions. We propose a storytelling scene in which a student puts on an e-textile and actively engages with a digital narration by watching and acting herself in a mirror. Our approach tries to exploit theories about empathy, storytelling, wearables and epresence.

2 Literature Review 2.1

Empathy and Storytelling

Empathy in the broadest sense refers to the reaction of one individual to the observed experiences of another (Davis 1983). Daniel Goleman claims that Empathy is the ability to understand others’ feelings and emotions (Goleman 2011). He also, notes that it is about defining, understanding and reacting to the concerns and needs of others. Despite the fact that empathy has been the subject of much research and theory, there is still a lack of consensus in the field about its definition. Nonetheless, empathy is often considered from two main perspectives (Davis 1983). One is cognitive empathy, which is the drive for a person to understand and identify with another person’s situation (Shamay–Tsoory et al. 2009; Gottman 2011). The other, emotional or affective empathy, is the drive to experience another person’s feelings and respond to their situation with an appropriate emotional response (Shamay–Tsoory et al. 2009; Batchelder et al. 2017). Bullying is negatively associated with both cognitive and affective empathy (Noorden et al. 2015). Noorden et al. (2015) found that bullies do not experience what others feel whether or not they understand what others feel. As a result, they suggest that training is necessary to enhance empathic skills, since it is possible not only to reduce bullying, but also to increase defending, which is associated with high levels of cognitive and affective empathy. A key part of creating an empathic connection is being able to see from another person’s perspective. Storytelling can have both a cognitive (relating to the structure of the narrative) and emotional impact to us (relating to the content of the narrative which can evoke empathy and the sense of identification with the characters) (Oatley 1994). Digital storytelling has emerged as a useful learning tool that combines narration, images, sounds, videos, computer generated texts and graphics in order to create emotional

Wearable E-Textile as a Narrative Mediator for Enhancing Empathy

459

representations to the audience (Matthews 2014; Robin 2006; Robin Robin 2016). Digital stories can operate as a situational, emotional and intellectual carrier by communicating someone else’s thoughts and feelings (Fokides 2017). Storytelling through Virtual Reality has been already examined as an empathy machine, since virtual reality allows the mind to believe that it is embodied somewhere else. Virtual reality does not only make you feel present, but user senses become embodied into headset (Archer and Finger 2018; Shin 2018). The term Empathy Machines ‘refers to any attempt to make sensible to oneself the emotional experience of another via technology, with the goal of inhabiting another body’ (Bollmer 2017). VRbased storytelling creates the sensation of personally having the experience of the narrative and operates as a device that enables individuals to build the bridge of understanding between one another (Shin 2018; Schutte and Stilinović 2017). 2.2

Wearables

There is a growing body of literature that recognizes that the rise of wearable technology has given a boost to educational technology and, essentially, to improving of classroom experiences (Sapargaliyev 2015). Studies have revealed that wearable technologies have the potential to engage students in learning, enable teachers to monitor the activity and behavior in the classroom, as well as to enhance the learning experiences (Sapargaliyev 2015; Borthwick 2015; Norooz et al. 2015). E-textiles are “fabric artifacts that include embedded computers and other electronics” (Peppler 2016). Research suggests that the use of e-textiles in classroom can concretize abstract concepts (such as human anatomy, circuits), create new types of learning activities (Peppler and Glosson 2013; Norooz et al. 2015; Kafai et al. 2014), promote computational thinking (Kafai et al. 2014), and improve learning outcomes. E-textiles, for example, have been used in Science lessons as an effective resource in teaching circuits. The research pinpointed that the use of e-textiles, had not only promoted participation in the classroom, but also had made the concept of circuitry more transparent and consequently understandable to students (Kafai and Peppler, 2014). In education, e-textiles have been used mainly with the form of toolkits that enable students to use their creativity in order for them to design their own artifacts (Kazemitabaar et al. 2016; Kafai et al. 2014; Peppler and Glosson 2013; Norooz et al. 2015). Wearables can function also as costumes. Costume design has always been an essential ingredient of theatre performance, because it makes an immediate and powerful statement to the audience. It communicates messages to the audience and enables them to develop a better understanding of the characters and their situation (Maclaurin and Monks 2015). Costume design plays an important role in the performance as it can set the period, the social structure and create the desirable atmosphere (Bicât 2006). Also, it creates the first impression to the audience about characters and their behavior, for that reason, sometimes wordy introductions are not necessary (Ingham and Covey 1992).

460

G. Palaigeorgiou et al.

2.3

Presence and Empathy

Sense of presence is an important and necessary element in the design and usage of the recent technologies. As Riva et al. (2014) supports “the illusion of being present is the key ingredient that gives interactive media the power to affect people profoundly, to change them” (Riva et al. 2014). Presence is defined as the psychological, more subjective, sense of being in the environment and mainly be influenced by the content of the mediated world (Vosmeer et al. 2017). The sense of presence creates an environment where reality and imagination become one (Shin 2018). According to Riva et al. (2014) three are the basic components that contribute to the creation of the sense of presence in the mediated world. These are perception, action and the body (Riva et al. 2014). In order the users to feel present in the mediated world, it is necessary for them to take the perspective of the mediated world characters, to be able to interact and use their body. Embodied experiences make the user feel that they are personally having the experience (Shin 2018). When the user feels strong mediated presence, they react not only bodily, but also emotionally (Riva et al. 2014). That implies that the degree of presence is associated with the general trait of empathy (Nicovich et al. 2005). Empathy and sense of presence have common features. These features consist of thoughts and feelings that are being evoked by the imagined experience and projection of the self into an environment of the experience of another person. 2.4

Aims

The current study is focused on the creation of a new type of empathy machine that can both activate the user’s embodied experience and promote the user’s feeling of presence in the mediated narrative world with the exploitation of an e-textile. We wanted to be able to put the user in someone else’s shoes, by presenting visual stimuli on their body, while they listen to a narrative.

3 The Proposed Environment In the proposed approach, the main “reader” of the narrative wears the e-textile and stands opposite of a mirror in order to watch his own body as it is transformed by the illuminations. The narrative screen is the “reader’s” own body which mediate the oral narrations. Next to this mirror, there was a wooden tree and a bench. Two additional participants sat on the bench and watched the interactive narration. The e-textile was almost entirely covered by el-wires controlled by an Arduino board (see Fig. 1). It could represent a variety of shapes independently or in combination, while important elements of its narrative function were the way, the sequence, and the frequency of activating the 5-color el-wires. Before the narrative began, the lights in the room were switched off and there was full darkness which provoked a more dramatic impact of both the activation of the el-wires and their non-activation.

Wearable E-Textile as a Narrative Mediator for Enhancing Empathy

461

Fig. 1. The e-textile

The narrative exploited was based on European Dafni III campaign “Awareness raising and prevention on bullying among students”. The specific program aimed to prevent bullying “by developing and implementing a preventive tool that increases the awareness of primary school teachers”. The story “Orpheus runs-Thoughts of an Old Tree” has been tailored to the goals and needs of the particular study. The story takes place at the school playground. The main character is a young student named Orpheus, who is being bullied by a group of children at school. The second character of the story is that of an old wise tree. The old wise tree plays the role of the observer. The third character of the story is Orpheus’ classmate, named Melina, who even though is aware of the situation, she does not take action to help him. The audible (storytelling) and the visual stimuli on their body (el wires) aim to make the users feel that they are role-playing the experience. With tangible technologies, the body becomes an emotional object that focuses on human experience beyond the conventional styles of expression. The narrative guided the participant towards to story’s succession, either visually (a tree flashes on the e-textile, so they expect action from the tree) or audibly (“I feel stress around my neck”, while a chain flashes on the e-textile close to the user’s neck). As a result, the user experience, interpret and interact with the events of the story through an additional modality. There were also second person instructions which guided the participant to act (e.g. “Put your hands…”, “Shout with all your strength”), to experience the story, to acknowledge the problem and its solution. An interaction was not only created between the user and the e-textile, but also between the main participant and the other two on the bench. Visual stimuli try to signify what will happen, what happened, convey the feelings of the characters, the intensity of their emotions or their absence, the transformation of emotions, reveal the narrator, etc. Narrative Techniques As regards to the narrative discourse, the narrator does not take part in the story. He is an outside observer, the third person omniscient, since he knows everything about the events and thoughts of the characters. At the beginning of the story the narration is

462

G. Palaigeorgiou et al.

unfolded in a descriptive way, with a third person point of view. Afterwards, the narrator continues by talking in a second person. Particularly, the second person narration is being accompanied by dramatic questions (e.g. “What do you say?”, “You! The one that wears the t-shirt!”) with emotional content (e.g. “Do you feel the stress around your neck?”) by bringing the listener into action and trying to transform them into the protagonist (according to the technique “Choose your own adventure”). Of vital importance is the narration of the wise old tree, which acts as a self-aware narrator, either as an observer or as an eyewitness of the Orpheus’ experience. He is an omniscient narrator, knowing and conveying past and present events to the listener, as well as the feelings of the characters. Both the use of present tenses and the tree’s inability to help the protagonist, contribute to the creation of the dramatic aspect of the story and have a profound emotional effect on the listener. The same goal is served by the inner monologue of the classmate Melina, who although is aware of the situation, she does not take action to help the protagonist. The wearable e-textile aims additionally to create an interaction between the duration of the story and the duration of the discourse, by visualizing narrative acceleration, deceleration and ellipsis. The wearable e-textile operates both as a complementary mediator that enhances the expressiveness and the graphic language of the narrative and as a meaningful, situational and emotional carrier. Visual Representations on the E-Textile We assume that the e-textile will make the user to feel a strong mediated presence, when they use their body and interact by taking the characters’ perspective. The light, the “el wire” colors and the sounds take the user to the story’s space and time. The combination of visual and oral narrative elements contributes to the creation of mental images and enhance the user’s mental representations of the story. Previous research has revealed that colors affect parts of human nervous system which stimulate emotions (Humphrey et al. 1976). This association between colors and emotions when the person looks at them is called Color Emotion (Zammitto 2005). For example, colors as red and yellow are capable of stimulating the nervous system (Zammitto 2005). The exposition in red light increases the heart rate, brain activity and skin responses. The visual representations on the t-shirt that the user can see in the mirror, will give emotional content and interaction to the narrative and they will increase the user’s involvement (Bower and Sturman 2015).

4 Methodology In order to evaluate the proposed “empathy machine”, 21 students of 6th grade of an elementary school interacted with the narration. Immediately after the narration, a short interview was conducted with each participant in order to identify the various perspectives of their experience. The interview lasted for about 15 min and was focused on the emotions evoked, the role of the e-textile and the narrative experience. The recorded interviews where transcribed, encoded and compared within and between cases. In the end, the research team collaborated in order to consensually identify the salient arising issues.

Wearable E-Textile as a Narrative Mediator for Enhancing Empathy

463

5 Findings The participants were very positive towards the digital storytelling experience. The interviews indicated that the proposed setting together with the story and e-textile managed to function effectively as an empathy machine. The participants noted that the wearable e-textile cultivated a sense of identification with the characters and their feelings. It was interesting that that they identified themselves not only with the main actor but with all speaking characters. “..the t-shirt made me feel how Orpheus felt. If I did not wear the t-shirt, I would not have put my hands on my heart… The t-shirt showed me that I can put my hands on the flashing heart in order to experience his sadness and his fear” “When the wires were flashing, I felt like I was in his place, and I could feel the same things with him somehow” “..in the past I supported my friend named L., for that reason I felt like I was the old wise tree in a lively form” “I felt the thing that we call empathy, I put myself in Orpheus place”.

Students noted that the e-textile helped them to feel various emotions for the protagonists of the story. The dominant emotions were those of anxiety and regret for the protagonist, and that also demonstrates the high level of empathy they felt. “It caused me a lot of anxiety when he started to talk about intimidation and I felt he sadness” “Yes, it caused me a lot of anxiety when he started to talk about bullying and…I felt sorry for the child” “[I felt] the child’s sadness, it’s very hard to live in such conditions”

The e-textile was also considered as an emotional screen since several students underlined that it illustrated the Orpheus’ emotions and feelings “I think that the light and the colors depicted emotions” “When the blue lights turned on, Orpheus was embarrassed and was feeling upset…” “We noticed emotions on the t-shirt”

All participants were immersed into the storytelling experience. They were impressed by the atmosphere in the room and characterized it as “unexpected”, “innovative”, “authentic”, “experiential”. The combination of the darkness, the lighting on the t-shirt and the mirror made them feel that the t-shirt “was talking” either on behalf of the heroes, the narrator or even themselves. “…the climate in the room, the darkness, made you feel like you are entering another world, a world full of emotions…you become part of the story and experience the Orpheus’ situation.”

464

G. Palaigeorgiou et al.

“The narrative created many representations, images that made you feel the drama and fear that Orpheus experienced…at some point I felt like the t-shirt was talking to me.” “The wearable e-textile was a special experience for us, since we had not experienced something similar before”,

The setting and the narration devices also helped the participants to be focused and follow the story till the end. “The t-shirt drew our attention… the result was amazing, because we could watch the narrative unfolding on the t-shirt” “The e-textile has helped us become more focused on the story plot, if we hadn’t used it, we would have been focused, but not as much as now”

Almost all participants claimed that the t-shirt brought the story to life, made the narration more authentic, more dramatic. They underlined that the e-textile made the storytelling more interactive and they felt that they were actively participating in the story. “The t-shirt made the difference! It brought mental images to life, otherwise it would be a simple narrative”, “The lights and the shapes on the t-shirt make the story to come alive” “I think that the t-shirt made the narrative more representational, because it depicted the Orpheus feelings and emotions, like we were Orpheus” “I liked the t-shirt! The one who wore the t-shirt, looked like she is the main character of the story. I think we put ourselves in the classmates’ place” “The t-shirt activated the story. Actually, it made the storytelling more interactive. As a result, we became part of the story”

The semiotics of the e-textile and the elevated levels of empathy made some participants to claim that it helped them to get a deeper understanding of the story. “We understood the story better” “I would prefer wearing a t-shirt like this everytime I listen to a story, rather than reading the story, because this is a more effective and entertaining way to understand the story.”

The most intense moments of the interaction between the participants and the etextile appeared to be the flashing heart beats on the t-shirt accompanied by the sound effects as well as the flashing chain around their neck. Several participants mentioned that they vividly experienced both the heart beat and the chain around their neck. They also stated that they liked those types of interactions, and would love to experience more of them. “The heart was beating like my heart”. “I wondered if I really felt pain in the neck”

Wearable E-Textile as a Narrative Mediator for Enhancing Empathy

465

The positive view of the experience was also validated by the participants willingness to experiment again the whole process. “I want to go back there… to listen again the story about Orpheus …”, “I would love to go back… it evokes emotions”.

6 Discussion This study was a first attempt to create an effective, easy to make narrative mediation tool to enhance empathy, by using the wearable e-textile technology. This tool was designed in order to be utilized in school and contribute to the “education of character” (Kohn 1991), because the aim of education is not merely the production of good learners, but of good people (Kohn 1991). By encouraging children to empathize with fictional characters, we can develop their empathic skills and their ability to empathize with people in real life. The current study was carried out to evaluate the effectiveness of the use of the wearable e-textile as a narrative mediator, aimed at increasing the user’s empathy by enriching the user’s sense of presence. Visual stimuli on their body were accompanied with narrations, sounds or their absence and all of these contributed to the creation of a mediated environment that triggered students to feel and share the emotions of the story characters. Our study suggests that wearable e-textile technology, in combination with the narrative can be used in prevention bullying and empathy training programs. The participants enjoyed their interaction with the wearable e-textile. The content of the mediated world (wearable e-textile, narrative & sound, atmosphere in the room) influenced the users’ feeling of being in the story. The participants identified with the characters, took their perspective and when they used their body, the narrative evoked even more their emotions. Their interaction with the wearable e-textile was the one that evoked their feelings and it contributed on making associations between the narrative and the users’ previous experience in their life. Pantouvaki (2014) emphasizes how admirable is the way in which tangible technology interacts between the participant and the whole environment, with the other participants, with the public. The participant bearing the tangible technology becomes a carrier of feelings, experiences, and important meanings in a way that is quite innovative and attractive. The body is an emotional object that focuses on human experience beyond the conventional style of expression. Interaction is made possible through storytelling, wearable represents the body, and technology embody this body. Such empathy machine could prove invaluable tools for helping the moral development of students and undergraduates. There are some limitations of this study to be noted. The sample of the study was comprised of primary school children, which may potentially limit the generalization of the results to the wider population. Also, the findings of the current study were based on semi-formal interviews. In order to further validate the results future quantitative research should study the role of the wearable e-textile as an empathy mediator, by using questionnaires for assessing empathy and sense of presence.

466

G. Palaigeorgiou et al.

References Althof, W., Berkowitz, M.W.: Moral education and character education: their relationship and roles in citizenship education. J. Moral Educ. 35(4), 495–518 (2006) Batchelder, L., Brosnan, M., Ashwin, C.: The development and validation of the empathy components questionnaire (ECQ). PloS One 12(1), e0169185 (2017) Bollmer, G.: Empathy machines. Media Int. Aust. 165(1), 63–76 (2017) Borthwick, A.C., Anderson, C.L., Finsness, E.S., Foulger, T.S.: Special article personal wearable technologies in education: value or villain? J. Dig. Learn. Teach. Educ. 31(3), 85–92 (2015) Bower, M., Sturman, D.: What are the educational affordances of wearable technologies? Comput. Educ. 88, 343–353 (2015) Bicât, T.: The Handbook of Stage Costume. Crowood Press, Marlborough (2006) Birch, N.: Development of Empathy (2016) Da Pos, O., Green-Armytage, P.: Facial expressions, colours and basic emotions. JAIC-J. Int. Colour Assoc. 1, 2 (2007) Fokides, E.: Informing students about bullying through the development of digital stories. Results of a project in Greece. Online J. New Horiz. Educ. 7(1), 113–122 (2017) Gottman, J.: Raising an Emotionally Intelligent Child. Simon and Schuster, New York (2011) Humphrey, N.K., Porter, T., Mikellides, B. (eds.) Colour for Architecture. Studio Vista, London (1976) Ingham, R., Covey, L.: The Costume Designer’s Handbook: A Complete Guide for Amateur and Professional Costume Designers, 2nd edn. Heinemann Educational Books, Portsmouth (1992) Kafai, Y., Fields, D., Searle, K.: Electronic textiles as disruptive designs: supporting and challenging maker activities in schools. Harvard Educ. Rev. 84(4), 532–556 (2014) Kafai, Y.B., Peppler, K.A.: 12 transparency reconsidered: creative, critical, and connected making with E-textiles. DIY Citizensh. Crit. Mak. Soc. Media 179 (2014) Kazemitabaar, M., He, L., Wang, K., Aloimonos, C., Cheng, T., Froehlich, J.E.: ReWear: early explorations of a modular wearable construction kit for young children. In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, pp. 2072–2080. ACM (2016 Kohn, A.: Caring kids, the role of the schools. Phi Delta Kappan 72(7), 496–506 (1991) Manney, P.J.: Empathy in the time of technology: how storytelling is the key to empathy. J. Evol. Technol. 19(1), 51–61 (2008) Maclaurin, A., Monks, A.: Costume: Readings in Theatre Practice. Palgrave, London (2015) Malti, T., Chaparro, M.P., Zuffianò, A., Colasante, T.: School-based interventions to promote empathy-related responding in children and adolescents: a developmental analysis. J. Clin. Child Adolesc. Psychol. 45(6), 718–731 (2016) Matthews, J., et al.: Voices from the heart: the use of digital storytelling in education. Community Pract. 87(1), 28–30 (2014). https://search.proquest.com/docview/1474889132? accountid=17256 Nicovich, S.G., Boller, G.W., Cornwell, T.B.: Experienced presence within computer-mediated communications: initial explorations on the effects of gender with respect to empathy and immersion. J. Comput.-Mediated Commun. 10(2), JCMC1023 (2005) Noorden, T.H., Haselager, G.J., Cillessen, A.H., Bukowski, W.M.: Empathy and involvement in bullying in children and adolescents: a systematic review. J. Youth Adolesc. 44, 637–657 (2015)

Wearable E-Textile as a Narrative Mediator for Enhancing Empathy

467

Norooz, L., Mauriello, M.L., Jorgensen, A., McNally, B., Froehlich, J.E.: BodyVis: a new approach to body learning through wearable sensing and visualization. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 1025–1034. ACM (2015) Oatley, K.: A taxonomy of the emotions of literary response and a theory of identification in fictional narrative. Poetics 23(1), 53–74 (1994) Pantouvaki, S.: Embodied interactions: towards an exploration of the expressive and narrative potential of performance costume through wearable technologies. Scene 2(1–2), 179–196 (2014) Peppler, K.: A review of e-textiles in education and society. In: Handbook of Research on the Societal Impact of Digital Media, pp. 268–290. IGI Global (2016) Peppler, K., Danish, J.: E-textiles for educators: Participatory simulations with e-puppetry. Text. Messages Dispatches World E-Textiles Educ., 133–141 (2013) Peppler, K., Glosson, D.: Stitching circuits: learning about circuitry through e-textile materials. J. Sci. Educ. Technol. 22(5), 751–763 (2013) Riva, G., Waterworth, J., Murray, D. (eds.): Interacting with Presence: HCI and the Sense of Presence in Computer-mediated Environments. Walter de Gruyter GmbH & Co KG (2014) Robin, B.: The educational uses of digital storytelling. In: Society for Information Technology & Teacher Education International Conference, pp. 709–716. Association for the Advancement of Computing in Education (AACE) (2006 Robin, B.R.: The power of digital storytelling to support teaching and learning. Dig. Educ. Rev. 30, 17–29 (2016) Sapargaliyev, D.: Wearables in education: expectations and disappointments. In: International Conference on Technology in Education, pp. 73–78. Springer, Heidelberg (2015) Schutte, N.S., Stilinović, E.J.: Facilitating empathy through virtual reality. Motiv. Emotion 41(6), 708–712 (2017) Shamay-Tsoory, S.G., Aharon-Peretz, J., Perry, D.: Two systems for empathy: a double dissociation between emotional and cognitive empathy in inferior frontal gyrus versus ventromedial prefrontal lesions. Brain 132(3), 617–627 (2009) Shin, D.: Empathy and embodied experience in virtual environment: to what extent can virtual reality stimulate empathy and embodied experience? Comput. Hum. Behav. 78, 64–73 (2018) Skaraas, S.B., Gomez, J., Jaccheri, L.: Tappetina’s empathy game: a playground of storytelling and emotional understanding. In: Proceedings of the 17th ACM Conference on Interaction Design and Children, pp. 509–512. ACM (2018 Vosmeer, M., Roth, C., Koenitz, H.: Who are you? voice-over perspective in surround video. In: International Conference on Interactive Digital Storytelling, pp. 221–232. Springer, Cham (2017) Zammitto, V.: The expressions of colors (2005)

RSSI Fingerprinting Techniques for Indoor Localization Datasets Angelos Chatzimichail(&), Athina Tsanousa, Georgios Meditskos, Stefanos Vrochidis, and Ioannis Kompatsiaris Information Technologies Institute, Center for Research and Technology Hellas, 6th km Charilaou-Thermi Road, Thessaloniki, Greece {angechat,atsan,gmeditsk,stefanos,ikom}@iti.gr

Abstract. Indoor localization techniques using Received Signal Strength Indicator (RSSI) is attractive in the Internet of Things domain due to its simplicity and cost-effectiveness. However, there are many different approaches proposed in and there is not a common, widely acceptable solution in the research community. This is mainly due to the limited number of publicly available datasets and that the multi-effect signal phenomenon limits each dataset to its gathering testbed. In this paper, we tested several fingerprinting methods in a publicly available dataset and we compared them against the RSSI regression approach, which is considered as the most prominent one in certain domains, such as indoor and outdoor localization. Keywords: RSSI


Fingerprinting
Localization
Internet of Things

1 Introduction With the great advancements in the wireless technologies, portable devices and wearables, there is a need for smart algorithms to personalize services to the users. Location-based Services, such as advertising and navigation, is one of the most popular type of services today that make use of location, as well as user context, to provide endusers with useful personalized information. One of the most important location tracking method is based on the Global Positioning System (GPS). However, GPS-based approaches are not appropriate in indoor circumstances because the position is determined from GPS satellites [1], which is not so accurate in indoor situations. Most of the research on the localization techniques has been based on the use of short-range signals, such as Wi-Fi [2, 3], Bluetooth [4], ultrasound [5], infrared [6], or RFID [7]. All of the aforementioned technologies are using the RSSI (Received Signal Strength indicator) parameter in order to estimate the position of the user. The RSSIbased approach estimates the signal strength of the received electromagnetic wave, being dependent on the distance of the signal source. Many RSSI measurements are being collected from many distances and places in databases in order to gather a larger number of measurements. Generally, there are two popular categories of indoor localization methods based on RSSI measurements, namely, triangulation and fingerprinting. © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 468–479, 2021. https://doi.org/10.1007/978-3-030-49932-7_45

RSSI Fingerprinting Techniques for Indoor Localization Datasets

469

The main idea of triangulation is to construct the function of RSSI and distance. First, the method starts with collection of enough data in different environments to describe the relationship between the measurements and the distance. Most of the researchers focuses on trying use fitting method to map the function [7, 8]. Then the position is estimated by using triangulation algorithm. For the fingerprinting method, creating an offline fingerprint database is crucial for location estimation. Fingerprinting matches the given positions with RSSI measurements gathered from access points. After the creation of an offline database, online RSSI gathering measurements are compared with fingerprints in the offline database to get the match position. The fingerprinting approach falls into the classification paradigm and there are many different algorithms such as KNN (K-nearest Neighbors) [9], SVM (Support Vector Machines) [10], RF (Random Forest) [11]. The main motivation of our work is to use the Bluetooth Low Energy (BLE) technology for indoor and outdoor position estimation. BLE is a technology with great adoption in embedded systems and electronics in general. This is because it consumes very low power and it is easy to integrate in electronics. The need to compare different localization techniques derives from the research challenges we face in estimating the location of children in one of the most crowded Christmas events in Greece that takes place in December in Trikala (Mills of the Elves). Children wear a smart bracelet with BLE technology which is wirelessly connected with the parent’s phone. In our case the child’s bracelet is the RSSI transmitter, while the parent’s phone is the RSSI receiver. In order to study the problem with the RSSI and distance we needed datasets, however there is a lack of relevant datasets to train our machine learning techniques in our case. Therefore, we gathered many RSSI measurements from smart bracelets and we created our dataset. After gathering the measurements, we performed curve fitting in order to correlate the RSSI values with the distance. We used the first approach from the two RSSI approaches in the localization services. In this case, the RSSI is the only value available to the receiver to correlate it with the distance. In this paper, we compare different fingerprinting techniques in our dataset that is publicly available1. In addition, we compare the fingerprinting results with the curve fitting results that we obtained in the same dataset in a previous work in order to compare the two different approaches. The rest of the paper is organized as follows: Sect. 2 reviews the related work. Section 3 explains the RSSI sensing infrastructure, while Sects. 4 and 5 describe the technical details of the fingerprinting and different machine learning techniques. Section 6 mentions the experimental results and the paper is concluded with Sect. 7.

2 Related Work The indoor positioning algorithms that are usually applied on indoor datasets are either based on the RSSI distance method or on fingerprinting technology [8]. Examples of RSSI fingerprinting solutions [12, 13] include probabilistic approaches [14], neural

1

http://desmos-project.gr/en/datasets-2.

470

A. Chatzimichail et al.

network-based approaches [8] and K-means algorithms [15]. The general theory for fingerprinting techniques is presented in [16] where the correlation of many parameters is presented such as the number of measurements, interference etc. Also, there are works that apply deep learning in the fingerprinting indoor localization, such as in [17] and [18]. Currently, the localization error reported in fingerprinting solutions ranges from 3 m to 10 m using WiFi and BLE RSSI [19]. In order to construct the offline database many researchers try to manually collect fingerprints at multiple known locations in a building. This is a labor-intensive and highly time-consuming procedure, even for small areas. To reduce that effort, selfguided robots equipped with sensors deployed in the building and collect data to build the dataset in [20]. Also, another technique that is used to easily collect measurements is crowdsourcing [21]. In this context, people gather random traces of measurements through smartphone applications as they walk around the investigation area. Another approach is to collect the RSSI measurements from a gateway device that is on a fixed place and listens for other devices. This method requires a prior knowledge about the position of the other devices in order to label them in the training dataset. In [13] researchers discussed indoor Wi-Fi positioning technology, including the various phases and processes of Wi-Fi fingerprinting technology and classified the methods used across. In [10] a zoning indoor localization solution, using the theory of SVM, applied to a WIFI RSSI technique. Two real world environments with different architectures (flat/multi-floors) have been used to create multiclass SVM models and to test their performance. Experimental results show that the proposed solution can greatly determine the location of the new signals, with a confidence of 97.31% for the flat topology and 88.38% for the multi-floors topology. A novel BLE RSSI ranking based fingerprinting method that uses Kendall Tau Correlation Coefficient (KTCC) is presented in [22]. The aim is to correlate a new signal position with the signal strength ranking of multiple low-power iBeacon devices situated in a retail space. This offers a higher positioning accuracy and is supported in recent smartphones. Other research studies propose using data from integrated smartphone’s sensors together with RSSI values to filter the location data and to provide more features in the dataset [23]. These applications need extra hardware and consume more battery. In our study, we used only the RSSI values of BLE smart-bracelet devices in order to build our dataset, as we explain in the following sections.

3 RSSI Measurement Procedure As a first step we consider the localization space, as a two-dimensional space. The task of localization is a problem learning the mapping of the observations to the respective locations. In fingerprinting, the localization task utilizes data collected from the

RSSI Fingerprinting Techniques for Indoor Localization Datasets

471

environment. Essentially, the fingerprinting database is filled by true locationobservation pairs and the location function is learned through the mapping of new observations to a specific position. The fingerprinting algorithm needs data in order to be trained. For the RSSI values, a RF signal feature is collected multiple times in order to compensate any signal noise effects. These RF signals have to be from the same signal bandwidth. In this work, it is assumed that the entire area is served by BLE coverage. In particular, the area is covered by multiple access points (AP). Some references suggest using multiple access points in order to cover the dead zones. An AP advertises its availability by broadcasting a message (in JSON or another format) with its MAC address. At the receivers, which are usually phones, the power of the received RF signal from all APs is measured as RSSI. The receiver will not detect the APs that are too far away from it. During the offline procedure it is assumed that the area is discretized into a set of A known locations A = {xa | a = 1…A}, where xa represents the 2-dimensional Cartesian coordinate of location a. The RSSI is scanned for a certain period to receive multiple records from every AP. Commonly, if there are N different APs, T different samples are collected for all the locations A. All these RSSI values are collected in threedimensional matrix with dimension A  N  T. During the online phase, a receiver at a location x listens to all the APs in the area and then collects the RSSI measurements in a database with dimensions of N  R. The function should be learned by mapping these new measurements to positions in the space. In most common fingerprinting algorithms, the function of pattern matching is to compare the similarity between the fingerprints of offline database and the fingerprints of the online procedure. One of the most well-known algorithms for fingerprinting and generally for pattern matching employs the Euclidean Distance to measure the similarity between fingerprints in the training and the testing procedure. This distance can be replaced by any kernel function. The whole process is completed with postprocessing methods such as k-nearest neighbors (kNN) or Support Vector Machines (SVM).

4 Data Collection System In this section, we describe the process that we followed to collect the RSSI values from the BLE devices. Standard, widely adopted techniques and common practices have been followed. To ensure diversity, 2 different BLE devices have been used to compare the RSSI values in different environments. We took the records from an office, a hall and outside of the building, which top view’s picture is shown in the Fig. 1. The first indoor environment was a 10  4 m office. It was selected due to its large number of wireless devices, resulting in an

472

A. Chatzimichail et al.

environment with a lot of noise interference. In the office there are 8 office with 1.5 to 2 m distance to each other. Each office is assigned a different label in the fingerprint dataset.

Fig. 1. Building map of RSSI measurements

The second place used was a 20  2 m. hall outside the office. This place was selected in order to correlate the distance with the RSSI values in a cleaner environment without a lot of interference. In this environment, we set as a label different distances. In order to build our offline dataset, the set of BLE modules has to be fixed at specific locations. For this purpose, we used two BLE smart sport-bracelets. The first one was Xiaomi MI Band 2 [24] and the second one was the Xiaomi MI Band 3 [25]. The RSSI tags were sent in JSON format with other information through an android application in a smartphone that was considered as a fixed anchor station. The smartphone device scans for the BLE bracelets and sends the JSON tags in a MQTT server. Through the MQTT server the JSON messages are sent to a MQTT client application in JAVA. After receiving the JSON messages, they were stored in a MongoDB database for analysis. All the measurements were performed in the same height, so as to have the same interference from the surrounding environment. The device knows from which device the messages are coming, inserting a label number for each office. To perform a fair set of tests between all the experiments, a similar transmit power and time measurement interval was required to be used in all the components. The time interval was 5 s. The procedure is showing in Fig. 2.

RSSI Fingerprinting Techniques for Indoor Localization Datasets

:UserApp

:MQTT Broker

:MQTT Client

473

Mongo DB

BLE info Set Smartphone AcƟve Subscribe to topic BLE Set BLE acƟve

Get BLE json

Store BLE info

Store locaƟon info

Get BLE info Get GPS signal msg: Search in the nearest personnel locaƟon AcƟve Smartphones AcƟve BLEs

:Child Smart Bracelet :Parent Smartphone

Fig. 2. Data collection procedure

5 RSSI Offline Dataset Description We reused a dataset collected in Project2, with some modifications in order to use the fingerprinting technique as a classification problem with multiple classes. First, the dataset has over 100 different RSSI values for each position. The measurements were related to two different environments. One office with many wireless devices and one building hall. Figure 2 shows the scatter and plot for all the RSSI measurements for both Band 2 and Band 3 in the office. Figure 3 shows the same data for the hall. The figures show that there is a better distribution of the RSSI values in the office than in the hall. This is because the different positions in the office are better distinguishable. There are 8 different indoor office positions to observe. All the measurements were carried out in the same day. The distance range varied from 0.2 to 6.0 m. In each position the measurements were from 100 to 250 recordings. As we can see in Fig. 3 there is a big fluctuation in each position changing over time. Figure 4 depicts the scatter plots of the RSSI values for Band 2 and Band 3 in the hall. In this case, we took measurements from seven different positions (1 to 7 m, with one meter increment). The number of measurements of each position were over 190 values. Here we can see that there is not so much difference between band 2 and band 3 devices. Each label here also corresponds to the real distance in meters. We can observe that there is a big fluctuation in each position.

2

http://desmos-project.gr/en/homepage-en.

474

A. Chatzimichail et al.

Fig. 3. Scatter diagram of the RSSI measurements in off

Fig. 4. Scatter diagram of the RSSI measurements in hall

From the data above, we can conclude that the raw RSSI values are not reliable enough for the localization as it is observed in most of the bibliography [8, 9]. In the following, we use Machine Learning methods to observe the performance of the techniques on our fingerprinting dataset. In addition, we compare these fingerprinting techniques with our previous work with the regression method approach.

RSSI Fingerprinting Techniques for Indoor Localization Datasets

475

6 Results and Findings In this section we analyze the techniques we selected to perform fingerprinting. The raw RSSI readings were split into train and test sets. All algorithms were developed in python. We used new measurements (around 40 measurements) as a test set for the machine learning techniques. 6.1

K-Nearest Neighbors Algorithm

The principle behind nearest neighbor methods is to find a predefined number of training samples closest in distance to the new point, and predict the label from these. The number of samples can be a user-defined constant (k-nearest neighbor learning), or vary based on the local density of points (radius-based neighbor learning). The algorithm implements learning based on the nearest neighbors of each query point, where k is an integer value specified by the user. Figure 4 shows the computed mean error for the office for the Band 3 bracelet using k = 1 to 40 nearest neighbors. As we can see the best results are obtained for k = 35. We did the same procedure for the other measurements as well (with band 2 and in the hall environment). For Band 2 in the office the best k value is 21, while in the hall is 15. For Band 3 in the hall the k value for the best results is 3 (Fig. 5).

Fig. 5. Mean error diagram for Band 3 in office

The Table 1 shows the results the accuracy for the K nearest Neighbors. Table 1. Accuracy results KNN Environment Band2 Band3 Office 0.45 0.44 Hall 0.28 0.30

476

A. Chatzimichail et al.

The best results are for Band 2 and Band 3 in the office environment. This is because each position is distributed better in the office than in the hall. 6.2

Decision Tree

Decision Trees (DTs) is a non-parametric supervised learning method used for classification and regression. The goal is to create a model that predicts the value of a target variable by learning simple decision rules inferred from the data features. For the decision tree method, we tried to find the best function to measure the quality of a tree split. We used two different function “gini” and “entropy” to see which one best fit the models. For Band 3 in the office the best criterion is entropy with maximum depth of tree 5. For Band 2 in the office the best criterion is gini with maximum depth of tree 9. In the hall environment Band 2 results were better with gini criterion and maximum depth of tree 6. For Band 3 the best results were with gini criterion and 3 maximum depth size. In Table 2 we present the results (average/total) of the Decision Tree techniques for the 4 different use cases. Support column is the total number of samples of true response lying in all eight classes. Table 2. Decision tree results (average/total) Environment/Band Office/Band2 Office/Band3 Hall/Band2 Hall/Band3

Precision 0.478 0.421 0.219 0.226

Recall 0.428 0.416 0.224 0.313

F1-score 0.417 0.382 0.213 0.260

Support 332 332 286 316

Again here, the results are much better in the office that in the hall with no difference between the two smart bracelets. 6.3

Random Forest

A random forest is a meta estimator that fits several decision tree classifiers on various sub-samples of the dataset and uses averaging to improve the predictive accuracy and control over-fitting. Again, here we performed the Random Forest technique to find the best results Table 3. Random forest results (average/total) Environment/Band Office/Band2 Office/Band3 Hall/Band2 Hall/Band3

Precision 0.407 0.421 0.271 0.276

Recall 0.407 0.431 0.273 0.282

F1-score 0.393 0.403 0.248 0.260

Support 332 332 286 316

RSSI Fingerprinting Techniques for Indoor Localization Datasets

477

The results (average/total) in Table 3 are the same as in the previous techniques and there are no big differences between them. The best results here were observed for the office. 6.4

Support Vector Machines (SVM)

SVM is a binary classifier introduced by Vapnik and Chervonenkis in 1963. SVM aims at finding a linear classifier, i.e., a hyperplane which maximizes the margin between two classes. It has extensions to non-linear classifiers and non-separable data too. We used the kernel Radial Basis Function (RBF) in python for the SVM technique. We used the RBF kernel, because this kernel gave definitely better results that the other kernels. The results are showing in the Table 4. Table 4. SVM results (accuracy-%) Environment/Band Office/Band2 Office/Band3 Hall/Band2 Hall/Band3

Precision (%) 41.56 43.97 22.72 29.74

The SVMs results show that there is a better separation of the classes in the office environment than in the hall environment, even if there is not so much interference from the surrounding environment. From the whole procedure, the best results were observed with decision trees in the office environment with Band 2. Between the smart bracelets, there is no big difference in the fingerprinting technique. As we can observe from the results of fingerprinting, it is not an appropriate approach for our dataset. Compared to our previous work, in which we tested the distance regression method for RSSI measurements, here the localization performance is lower. In general, we can observe that we need to define more discriminative labels in order to classify better the labels. This means that we have to increase the distance between the labels due to the fluctuation in the RSSI. However, this will decrease the indoor localization accuracy.

7 Conclusion and Future Work The main contribution of this paper is to examine the fingerprinting approach with RSSI recordings and their respective classes, suitable for positioning applications. The recordings were collected in two different indoor environments. The variability of the RSSI values in all the environments, creates many difficulties in the fingerprinting approach. Our experiments confirm the findings of the literature, that the raw RSSI data is an inappropriate form to use for indoor localization. In such cases, we need either to change the dataset and take more measurements, or to use the regression technique with

478

A. Chatzimichail et al.

filtering methods. The results were much better in the office than in the hall environment due to its small diversity (1 m) and the algorithms cannot separate each label. For future work, we will enrich the datasets with additional measurements and settings. We will also investigate additional techniques, like Kalman Filter, in order to find the exact position of a walking human in a specific area. Acknowledgement. This research has been co-financed by the European Union and Greek national funds through the Operational Program Competitiveness, Entrepreneurship and Innovation, under the call RESEARCH-CREATE-INNOVATE (project code: T1EDK-03487).

References 1. Gu, Y., Lo, A., Niemegeers, I.: A survey of indoor positioning systems for wireless personal networks. IEEE Commun. Surv. Tutor. 11(1), 2009 (2009) 2. Liu, H., Darabi, H., Banerjee, P., Liu, J.: Survey of wireless indoor positioning techniques and systems. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 37(6), 1067–1080 (2007) 3. Del Mundo, L.B., Ansay, R.L.D., Festin, C.A.M., Ocampo, R.M.: A comparison of Wireless Fidelity (Wi-Fi) fingerprinting techniques. In: ICTC 2011, Seoul, pp. 20–25 (2011) 4. Mendoza-Silva, G.M., Matey-Sanz, M., Torres-Sospedra, J., Huerta, J.: BLE RSS measurements dataset for research on accurate indoor positioning. Data 4(1), 12 (2019) 5. Hauschildt, D., Kirchhof, N.: Improving indoor position estimation by combining active TDOA ultrasound and passive thermal infrared localization. In: 2011 8th Workshop on Positioning, Navigation and Communication, Dresden, pp. 94–99 (2011) 6. Wang, K., Nirmalathas, A., Lim, C., Alameh, K., Li, H., Skafidas, E.: Indoor infrared optical wireless localization system with background light power estimation capability. Opt. Express 25, 22923–22931 (2017) 7. Zhu, L., Yang, A., Wu, D., Liu, L.: Survey of indoor positioning technologies and systems. In: Life System Modeling and Simulation, pp. 400–409. Springer, Heidelberg (2014) 8. Li, G., Geng, E., Ye, Z., Xu, Y., Lin, J., Pang, Y.: Indoor positioning algorithm based on the improved RSSI distance model. Sensors 18(9), 2820 (2018) 9. Spachos, P., Papapanagiotou, I., Plataniotis, K.N.: Microlocation for smart buildings in the era of the internet of things: a survey of technologies, techniques, and approaches. IEEE Sign. Process. Mag. 35(5), 140–152 (2018) 10. Farjow, W.., Chehri, A., Hussein, M., Fernando, X.: Support vector machines for indoor sensor localization. In: 2011 IEEE Wireless Communications and Networking Conference, Cancun, Quintana Roo, pp. 779–783 (2011) 11. Guo, X., et al.: Indoor localization by fusing a group of fingerprints based on random forests. IEEE Internet Things J. 5(6), 4686–4698 (2018) 12. Honkavirta, V., Perala, T., Ali-Loytty, S., Piche, R.: A comparative survey of WLAN location fingerprinting methods. In: 2009 6th Workshop on Positioning, Navigation and Communication, Hannover, pp. 243–251 (2009) 13. Xia, S., et al.: Indoor fingerprint positioning based on Wi-Fi: an overview. ISPRS Int. J. Geo-Inf. 6(5), 135 (2017) 14. Dimitris, M., et al.: Low-dimensional signal-strength fingerprint-based positioning in wireless LANs. Ad hoc Netw. 12, 100–114 (2014) 15. Bai, S., Wu, T.: Analysis of k-means algorithm on fingerprint based indoor localization system. In: 2013 5th IEEE International Symposium on Microwave, Antenna, Propagation and EMC Technologies for Wireless Communications. IEEE (2013)

RSSI Fingerprinting Techniques for Indoor Localization Datasets

479

16. Tian, X., Shen, R., Liu, D., Wen, Y., Wang, X.: Performance analysis of RSS fingerprinting based indoor localization. IEEE Trans. Mob. Comput. 16, 2847–2861 (2017) 17. Nowicki, M.R., Wietrzykowski, J.: Low-effort place recognition with WiFi fingerprints using deep learning. Automation (2017) 18. Xiao, L., Behboodi, A., Mathar, R.: A deep learning approach to fingerprinting indoor localization solutions. In: 2017 27th International Telecommunication Networks and Applications Conference (ITNAC), Melbourne, VIC, pp. 1–7 (2017) 19. Yiu, S., et al.: Wireless RSSI fingerprinting localization. Signal Process. 131, 235–244 (2017) 20. Yeh, L.-W., Hsu, M.-H., Huang, H.-Y., Tseng, Y.-C.: Design and implementation of a selfguided indoor robot based on a two-tier localization architecture. Perv. Mob. Comput. 8(2), 271–281 (2012) 21. Wu, C., Yang, Z., Liu, Y., Xi, W.: Will: wireless indoor localization without site survey. In: Proceedings of IEEE INFOCOM, pp. 64–72. IEEE (2012) 22. Ma, Z., Poslad, S., Bigham, J., Zhang, X., Men, L.: A BLE RSSI ranking based indoor positioning system for generic smartphones. In: 2017 Wireless Telecommunications Symposium (WTS), Chicago, IL, pp. 1–8 (2017) 23. Nurminen, H., Ristimaki, A., Ali-Loytty, S., Piché, R.: Particle filter and smoother for indoor localization. In: Proceedings of International Conference on Indoor Positioning and Indoor Navigation (IPIN), pp. 1–10 (2013) 24. Mi Band 2. 2019. Specifications. https://www.mi.com/global/miband2/. Accessed 5 July 2019 25. Mi Band 3. 2019. Specifications. https://www.mi.com/global/mi-band-3/. Accessed 5 July 2019

Mulsemedia Data Representation Based on Multi-image Concept Yevgeniya Sulema1(&), Abhishek Bhattacharya2, and Niall Murray2 1

Igor Sikorsky Kyiv Polytechnic Institute, Kiev, Ukraine [email protected] 2 Athlone Institute of Technology, Athlone, Ireland {a.bhattacharya,nmurray}@research.ait.ie

Abstract. MulseMedia technology enables operation with multimodal data sets that requires their complex representation. In this paper, we propose a mathematical background of MulseMedia data representation. The application of both Algebraic System of Aggregates and Multi-Image Concept allows efficient representation and processing of MulseMedia data in computer systems as well as encourages development of new algorithms for data processing, including data compression and data modelling. Keywords: Mulsemedia of aggregates


Multimodal data
Multi-image
Algebraic system

1 Introduction The recent rapid development of new hardware technologies is enabling recording, processing, and reproduction of multiple sensorial media (MulseMedia) experiences. This requires development of new data representation models and new algorithms for multimodal data processing. Such new approaches can allow more efficient representation and processing of human-perceived types of information and, in this way, create new opportunities for widening the scope of MulseMedia application. In spite of the general technological readiness in certain fields of MulseMedia and immersive technologies being relatively high [1], there is a lack of common approaches to MulseMedia data representation. Although, a systematic approach for MulseMedia content representation was defined in MPEG-V standard [2, 3], most existing standards are focused on individual media components and they do not consider the combination of data modalities. There is a gap between physical nature of data (modality) and the algorithms and methods of data processing. In our opinion, this gap should be addressed upon mathematical principles for MulseMedia and multimodal data representation and processing.

© Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 480–491, 2021. https://doi.org/10.1007/978-3-030-49932-7_46

Mulsemedia Data Representation Based on Multi-image Concept

481

2 Related Work Since MulseMedia is a promising advancement of multimedia, related works concerning different aspects of MulseMedia concept, MulseMedia applications, etc. are continuously growing. The overview of MulseMedia, its applications, and hardware is presented in [4–9]. In particular, [6] gives the detailed information on human sensorial system, sensory data representation, discusses quality of service and quality of sensory experience. In [7] an overview of different available mulsemedia hardware technologies can support the potential of mulsemedia based applications, particularly in education. Whilst [8] and [9] consider available hardware approaches as well as an overview of research methodologies and proposals of same to understanding MulseMedia Quality of Experience (QoE). Similarly in [10], sensory effects and their influence on the quality of user’s perception are discussed. Human Factors influence on user experience of MulseMedia is presented in [11] and [12]. Haptic technology and haptic devices are investigated in [13, 14, 15], particularly, the classification of haptic devices is given in [15]; with advantages and disadvantages of different types of haptic devices are pointed out as well. Considering some communication network related conditions [12, 16–18] analysed multimodal data synchronization issues. In particular, the results of a study aimed at clarification of the temporal boundaries within which video, haptic, and air-flow components can be successfully synchronized are presented in [9]. In a similar context, Quality of Service (QoS) and Quality of Experience (QoE) have been investigated in [19–21]. Particularly, in [21] the authors propose the adaptive MulseMedia framework for delivering scalable video and sensorial data to users over constrained networks. The readiness of the World Wide Web to sensory effects representation is addressed in [22]; in particular, the ways of embedding sensory effect metadata within Web content are proposed. The analysis of these and other related research allows us to conclude that available approaches to MulseMedia data representation do not use a strong mathematical background. Thus, in this paper we give an overview of a mathematical apparatus based on both the Algebraic System of Aggregates and the Multi-Image Concept [23–25] and we demonstrate how this mathematical approach can be used for representation of different data modalities in MulseMedia applications.

3 Theoretical Background 3.1

Algebraic System of Aggregates

Since observation is a time domain based process and typically involves multimodal data sets, the apparatus of the Algebraic System of Aggregates [23–25] can be used to obtain a multi-image [23–25] which can be considered as a mathematical background for a digital twin of the object (subject, process, event, etc.). Algebraic System of Aggregates (ASA) is an algebraic system, a carrier of which is an arbitrary set of specific structures called aggregates [23–25]. An aggregate A is a tuple of arbitrary tuples, elements of which belong to certain sets:

482

Y. Sulema et al.

A¼

hh
n iiN j Mj j aij i¼1 ¼ ½½f Agjh Ai j¼1

ð1Þ


 nj where f Ag is a tuple of sets Mj ; h Ai is a tuple of elements tuples aij i¼1 corresponding   to the tuple of sets aij 2 Mj . This means that for defining the aggregate we need to indicate the sets, elements of which belong to the aggregate within appropriate tuples, and next we need to place the tuples of the elements belonging to these sets in the same order because there is a strict relation between them: the elements of the first tuple belong to the first set, the elements of the second tuple belong to the second set, etc. Since the ASA is an algebraic system [26–28], it consists of sets (ℳ, ℱ, ℛ), where ℳ is a non-empty set (carrier), elements of which are elements of the system; ℱ is a set of operations; ℛ is a set of relations. The carrier of ASA is an arbitrary set of specific structures called aggregates. Tuple elements in an aggregate can be both strict and fuzzy values. Aggregates can be compatible, quasi-compatible or incompatible. Aggregates A1 and A2 are compatible ðA1 + A2 Þ if they have equal lengths and both the type and sequence order of these aggregates are the same. Aggregates A1 and A2 are quasi: compatible ðA1 ¼ A2 Þ if the type and sequence order of these aggregates coincide partly. There is no requirement of the equality of aggregates lengths in this case. Otherwise, aggregates A1 and A2 are incompatible ðA1 $ A2 Þ. Besides, aggregates A1 and A2 can be : hiddenly compatible: A1 ð+Þ A2 if A1 ¼ A2 or A1 $ A2 but at the same time fA1 g  fA2 g which means that both aggregates have the same set of sets but the order of these sets differs. The basic relations in ASA are: Is Equal (=), Is Less (), Is Equivalent (), Includes (), Is Included (), Precedes (), Succeeds (). Operations on aggregates in ASA include logical operations, ordering operations, and arithmetical operations. The logical operations on aggregates are: Union ( [ ), Intersection ( \ ), Difference (n), Symmetric Difference (M), and Exclusive Intersection (:) [23]. The result of logical operations in ASA depends on aggregates compatibility. For example, an Intersection of two aggregates A1 and A2 is the aggregate A3 which includes only common components of both aggregates and is formed according to the following rule:

Mulsemedia Data Representation Based on Multi-image Concept

483

1. If A1 +A2 then A3 includes elements of both aggregates, which are common for them, in every tuple: 
 

 A1 ¼ M1 ; M2 ; . . .; MN  a11 ; a12 ; . . .; a1l ; b11 ; b12 ; . . .; b1m ; . . .; w11 ; w12 ; . . .; w1n 
hh E D Eii  D  A2 ¼ M1 ; M2 ; . . .; MN  a21 ; a22 ; . . .; a2r ; b21 ; b22 ; . . .; b2q ; . . .; w21 ; w22 ; . . .; w2p D h E  A3 ¼ A1 \ A2 ¼ M1 ; M2 ; . . .; MN  a1l1 ; . . .; a1la ; a2r1 ; . . .; a2rb ; D E D Ei b1m1 ; . . .; b1mc ; b2q1 ; . . .; b2qd ; . . .; w1n1 ; . . .; w1nk ; w2p1 ; . . .; w2pl ð2Þ where a1li 2 a1 ; a1li 2 a2 ; i 2 h1; . . .; ai; a2rj 2 a1 ; a2rj 2 a2 ; j 2 h1; . . .; bi; b1mk 2 b1 ; b1mk 2 b2 ; k 2 h1; . . .; ci; b2qs 2 b1 ; b2qs 2 b2 ; s 2 h1; . . .; di; w1nu 2 w1 ; w1nu 2 w2 ; u 2 h1; . . .; ki; w2py 2 w1 ; w2py 2 w2 ; y 2 h1; . . .; li. 2. If A1 $A2 then A3 is a null-aggregate:   A1 ¼ M11 ; M21 ; . . .; MN1 jha1 ; a2 ; . . .; al i; hb1 ; b2 ; . . .; bm i; . . .; hw1 ; w2 ; . . .; wn i 

 A2 ¼ M12 ; M22 ; . . .; MK2 jhc1 ; c2 ; . . .; cr i; d1 ; d2 ; . . .; dq ; . . .; z1 ; z2 ; . . .; zp

ð3Þ

A3 ¼ A1 \ A2 ¼ ½½;jh;i ¼ A; : : 3. If A1 ¼A2 then A3 includes elements of both aggregates, which are common for them, only in tuples of common sets, thus, the number of sets shortens:
 A1 ¼ ½½M1 ; M21 ; . . .; Mx ; . . .; MN1 j a11 ; a12 ; . . .; a1l ; hb1 ; b2 ; . . .; bm i; . . .;
1 1  f1 ; f2 ; . . .; ft1 ; . . .; hw1 ; w2 ; . . .; wn i

 A2 ¼ ½½M1 ; M22 ; . . .; Mx ; . . .; MK2 j a21 ; a22 ; . . .; a2r ; d1 ; d2 ; . . .; dq ; . . .;
2 2 
 f1 ; f2 ; . . .; fv2 ; . . .; z1 ; z2 ; . . .; zp  D E A3 ¼ A1 \ A2 ¼ ½½M1 ; . . .; Mx j a1l1 ; . . .; a1la ; a2r1 ; . . .; a2rb ; . . .; D E ft11 ; . . .; ft1q ; fv21 ; . . .; fv2x 

ð4Þ

where a1li 2 a1 ; a1li 2 a2 ; i 2 h1; . . .; ai; a2rj 2 a1 ; a2rj 2 a2 ; j 2 h1; . . .; bi; ft1e 2 f 1 ; ft1e 2 f 2 ; e 2 h1; . . .; qi; fv2h 2 f 1 ; fv2h 2 f 2 ; h 2 h1; . . .; xi. Ordering operations include: Sets Ordering ⊨; Ascending Sorting "; Descending Sorting #; Singling k; Extraction ⋉; Insertion ⋊ [24]. For example, Ascending Sorting enables reordering of all tuples according to ascending order of a certain tuple (called a primary tuple) among all tuples of the hh
n iiN j aggregate. Thus, if A1 ¼ Mj j aij i¼1 and 9k such as 1\k\N; k 6¼ 2 and j¼1

484

Y. Sulema et al.

n1 [ nk [ nN ; n2 ¼ nk , then the result of Ascending Sorting operation of A1 according elements of tuple ak is the aggregate A2 such as: D E A2 ¼ A1 " ak ¼ ½½M1 ; M2 ; . . .; Mk ; . . .; MN j a1a ; a1b ; . . .; a1m ; . . .; a1x ; a1nk þ 1 ; . . .; a1n1 ; D E D E D E a2a ; a2b ; . . .; a2m ; . . .; a2x ; . . .; aka ; akb ; . . .; akm ; . . .; akx ; . . .; aNa ; aNb ; . . .; aNm  ð5Þ
nj where aka \akb \. . .\akm \. . .\akx , amj 2 aij i¼1 , j ¼ 1. . .N, m 2 ½a; b; . . .; m; . . .; x, 1 m n, and n ¼ nk if nj nk or n ¼ nj if nj \nk . The ordering operations have a specific importance in the ASA because they allow us to compose and to operate with complex data structures called multi-images [23–25]. 3.2

Multi-image Concept

A multi-image is a complex representation of multiple data sets describing an object (subject, process, etc.) of observation which is obtained (measured, generated) in the course of time. In mathematical sense, the multi-image is an aggregate, the first data tuple of which is a non-empty tuple of time values. These values can be natural numbers or values of any other type which can be used for evidence and monosemantic representation of time. Thus, the multi-image can be defined in the following way: I¼

hh D E D Eii T; M1 ; ::; MN j ht1 ; ::; ts i; a11 ; ::; a1n1 ; ::; aN1 ; ::; aNnN

ð6Þ

where T is a set of time values;s ni ; i 2 ½1; . . .; N . If there is a tuple, elements of which are constant in time, in reference to the multi_ and the tuple of image, the corresponding set in the tuple of sets is marked as M identical elements is defined by one element which is supposed to be valid for every time value. The precondition of a multi-image construction is that obtained (generated, measured) data sequences are of different modalities and they are recorded (generated, measured) with respect to time. Since a multi-image is an aggregate, to process its data, we can use any operations defined in ASA. In particular, the logical operations of ASA can be used for preparing complex data representation of a multi-image and the ordering operations enable processing of multi-image data with respect to time [23–25].

4 Mulsemedia Data Representation Let us apply mathematical approach of the Multi-Image Concept to MulseMedia data sequences. For this purpose, at first, we need to consider every data modality related to MulseMedia and give its formal representation.

Mulsemedia Data Representation Based on Multi-image Concept

485

Thus, monophonic digital audio signal is a tuple defined as follows: Smono ¼ hsk iKk¼0 ¼ hs0 ; s1 ; . . .; sK i

ð7Þ

where ak is a physical value such as sound pressure level; ak 2 R. Then stereophonic digital audio signal can be represented as the following tuple of tuples: Sstereo ¼





s1k ; s2k

K k¼0

¼



1 2 
1 2 
 s0 ; s0 ; s1 ; s1 ; . . .; s1k ; s2k

ð8Þ

where s1k is a sound pressure level value for the left channel; s2k is a sound pressure level value for the right channel; s1;2 k 2 R. Finally, multichannel digital audio signal [4] can be formalized as the following tuple:

K Smulti ¼ s1k ; s2k ; . . .; sQ ¼ k

1 2
1 k¼0  Q 2 s0 ; s0 ; . . .; s0 ; . . .; sK ; sK ; . . .; sQ K

ð9Þ

where sqk is a sound pressure level value for q-channel; q 2 ½1; . . .; Q; sqk 2 R. Let us consider images. Then a 2D image of N  M pixels can be defined as a tuple of the following tuples: C 2D ¼



1 2 N


1  M cn ; cn ; . . .; cM ¼ c10 ; . . .; cM n 0 ; . . .; cN ; . . .; cN n¼0

ð10Þ

m 3 where cm n is a tuple of pixel colour components; cn 2 Z ; m 2 ½1; . . .; M . m The definition of cn depends on a colour model. For example, if RGB model is used, cm n is defined as follows:


m m m cm n ¼ r n ; gn ; bn

ð11Þ

m m m m where rnm ; gm n ; bn are colour components in RGB model; rn ; gn ; bn 2 Z. The formal description of a 3D image depends on the technology of its reproduction [1, 7]. For example, if voxel graphics is supposed to be used for visualisation of the 3D image with the size of N  M  K voxels, then it can be defined as follows:

C 3D ¼

D

c1n;k

EK k¼1

D EK D E K

N 2 ; cn;k ; . . .; cM n;k k¼1

k¼1

ð12Þ

n¼1

m 3 where cm n;k is a tuple of colour components, cn;k 2 Z ; m 2 ½1; . . .; M : If the 3D image is a stereo image, then its description is:

Cstereo ¼




1 2 
1 2 N M ln ; ln ; . . .; lM n ; rn ; rn ; . . .; rn n¼0

m where lm n is left frame data component; rn is right frame data component.

ð13Þ

486

Y. Sulema et al.

A 2D video can be represented as an aggregate of an audio signal data tuple of a certain type (monophonic, stereophonic, multichannel) and a tuple of 2D images:   V2D ¼ RQ ; Z 3 j S; C

ð14Þ

where S is Smono , Sstereo or Smulti ; C is C 2D or C3D . A 3D video can be defined as a combination of a stereo image sequence and spatial sound (Q-channel audio signal):   V3D ¼ RQ ; Z 3 j Smulti ; C stereo

ð15Þ

Haptic data received from a data glove [12] can be represented as a tuple of movement intensity for every finger. Then if every finger movement is registered by one sensor, we can define haptic data tuple as follows: H¼





h1k ; h2k ; h3k ; h4k ; h5k

K k¼0

ð16Þ

where hik is a movement intensity value; hik 2 R; i 2 ½1; . . .; 5. Thermoceptional (environmental) data can be formalized as: E¼



T H W K ek ; ek ; ek k¼0

ð17Þ

H W where eTk is a temperature value, eTk 2 R; eH k is a humidity value, ek 2 R; ek denotes the W wind effect intensity, ek 2 R. Olfactory data can be formally defined as:

O¼



T I d s K ok ; ok ; ok ; ok k¼0

ð18Þ

where oTk denotes the scent types following the popular olfactory model in [6, 13, 16] oTk 2 MO ¼ {‘‘burnt’’,‘‘flowery’’,‘‘foul’’,‘‘fruity’’,‘‘resinous’’,‘‘spicy’’}; oIK denotes the olfaction intensity as in [28], oIK 2 MI ¼ flow; medium;highg; odk 2 R denotes the duration of olfactory stimuli, and osk 2 R denotes the synchronization skew of olfaction which is proved to be crucial [16, 18]. Finally, gustatory data can be formalized as: G¼





gTk ; gIk ; gdk

K k¼0

ð19Þ

where gTk 2 MG = {‘‘sour’’,‘‘bitter’’,‘‘salty’’,‘‘sweet’’} denote the gustation stimulus types [26], gIk 2 MI ¼ flow; medium; highg denote the gustation intensity [29], and gdk 2 R denote the duration of gustation stimuli [29]. In this paper, we assume for simplicity that every data structure defined in formulas from (3) to (15) is recorded (measured) simultaneously. Then, finally, we can compose a multi-image of the object of observation, such as:

Mulsemedia Data Representation Based on Multi-image Concept

I ¼ ½½T; RQ ; Z 3 ; R5 ; R3 ;hMo ; M1 ; R2 i; hMG ; MI ; Rijht1 ; . . .; ts i; Smulti ; C  stereo ; H;  G   E;  O;

487

ð20Þ

In the next section, we analyse how this mathematical approach can be applied in real conditions.

5 Results and Discussion One of the possible use cases for Multi-Image Concept application is a MulseMediabased remote lab (Fig. 1). Such a lab can be used for remote study as well as immersive demonstration of a certain phenomenon to remote learners [30–33]. A real-world object (subject, process, event, and phenomenon) can be characterized by multiple sequences of human-perceived information. We assume that the data sequences are obtained from several sensors. Every data sequence characterizes a certain aspect of the object’s nature and behaviour. At the

Fig. 1. Basic idea of complex representation of MulseMedia data as a multi-image of a realworld object

488

Y. Sulema et al.

same time, in terms of mathematics every data sequence is a tuple of values belonging to a certain data set. As mentioned above, for simplicity we assume that in one session of work in the lab all data sequences are recorded (measured) simultaneously. The result is that they can be composed in one complex data structure, namely, an aggregate. Since the object of observation exists in time, the aggregate can be supplemented with a tuple of time values which correspond to the time moments when values of other modalities are measured (recorded). The multi-image of the object defined with respect to a certain period of time. In the next session of work with the lab, we obtain the next multi-image of the same object and so on. After a number of observations and measurements, we will possess a family of multi-images of the object of study. Thus, we come to the task of multi-images analysis. Such analysis is based on logical operations [23], ordering operations [24], and relations defined in the Algebraic System of Aggregates according to the MultiImage Concept. The main advantage of this concept is that we can operate with multiple values defined in a certain time moment as with a complex value of a function of several variables. Thus, we always have an overall view on the object’s nature and behaviour in every moment of time. It allows us to compare multi-images, predict further state and behaviour of the object, model it, etc. Moreover, having the object’s multi-image for a certain time duration, we can reproduce the object virtually by using hardware for 3D visualization, spatial sound and other sensory data reproduction. Let us consider an example of multi-image representation and processing. Let V be a short video fragment recorded in a time interval T ¼ 1; . . .; 10 (in seconds) and presented as a data aggregate accordingly to (10). Note that for simplicity we assume that the quantity of 2D frames and the quantity of audio signal samples in this recorded fragment are equal as wellDas D we do not specify E

1 2 10 c10;k ; . . .; cM values in tuples but use their formal notations sk ; sk k¼0 and 0;k ; . . .; D E 10 c1N;k ; . . .; cM N;k ik¼0 instead. Then we can obtain a multi-image I1 which represents video data with respect to time:

10 I1 ¼ ½½T; R2 ; Z 3 j h1; 2; 3; 4; 5; 6; 7; 8; 9; 10i; s1k ; s2k k¼0 ; DD E D EE10 1 M ; . . .; c  c10;k ; . . .; cM ; . . .; c 0;k N;k N;k

ð21Þ

k¼0

Besides, let us assume that we need to enrich this video fragment by adding olfactory effects, namely, flowery scent of low intensity starting with the 3rd second of the video and lasting 3 s (all numbers in this example are invented) as well as fruity scent of medium intensity starting with the 7th second and lasting 4 s, with zero skew in both cases. Then it can be expressed in the following multi-image I2 : ð22Þ

Mulsemedia Data Representation Based on Multi-image Concept

489

Thus, we have two multi-images obtained from different sources: the first one is received from external device (video camera) and the second one is composed artificially (for example, by using a certain MulseMedia editor). Now we need to synchronize them and combine in one multi-image describing some immersive scene. For this purpose, we apply the following operations of the ASA: Union ( [ ), Sorting (") and Singling (k) [23, 24]. Since these multi-images are quasi-compatible [23, 24] the result of this complex operation is as follows:

ð23Þ

The multi-image I defined by (19) gives a full description of the immersive scene in its dynamics and with internal synchronization of multimodal data. It can be rendered by using a domain-specific language such as ASAMPL [25].

6 Conclusions The Algebraic System of Aggregates is a tool for complex data representation. It enables multimodal data processing by using a range of operations and relations. In particular, the logical operations allow us to construct different compositions of multimodal data, which in turn enables complex data representation for compound description of objects and processes in different areas. The Algebraic System of Aggregates is the basis for the Multi-Image Concept which enables overall description of an object (subject, process, phenomenon) of observation carried out in the course of time. In our opinion, the Multi-Image Concept creates a background for the development of new MulseMedia data models, new multimodal data compression and coding methods, new algorithms of MulseMedia data processing, and new protocols for MulseMedia data transmission. The formalization approach proposed by the Multi-Image Concept also enables better standardization for hardware and software to be used in immersive technologies. Future work can be focused on the development of data synchronization methods and data compression algorithms. Acknowledgement. This work has emanated from research supported in part by a research grant from Science Foundation Ireland (SFI) and is co-funded under the European Regional Development Fund under Grant Number 16/RC/3918.

490

Y. Sulema et al.

References 1. Sulema, Y.: Mulsemedia vs. multimedia: state of the art and future trends (invited paper). In: Proceedings of the 23rd IEEE International Conference on Systems, Signals and Image Processing IWSSIP2016, Bratislava, Slovakia, pp. 19–23 (2016) 2. Han, J.J., Kim, S.-K.: Text of white paper on MPEG-V. Communication Group (2014). http://mpeg.chiariglione.org/standards/mpeg-v 3. Yoon, K., Kim, S.K., Han, J.J., Han, S., Preda, M.: MPEG-V: Bridging the Virtual and Real World, p. 210. Academic Press, Cambridge (2015). ISBN: 978-0-12-420140-8 4. Timmerer, C., Müller, K.: Immersive future media technologies: from 3D video to sensory experiences. In: del Bimbo, A., Chang, S., Smeulders, A. (eds.) Proceedings of the International Conference on Multimedia (MM 2010), pp. 1781–1782. ACM, New York (2010) 5. Ghinea, G., Andres, F., Gulliver, S.R.: Multiple Sensorial Media Advances and Applications. In: New Developments in MulSeMedia, p. 344 (2012). https://doi.org/10.4018/978-160960-821-7 6. Ghinea, G., Timmerer, C., Lin, W., Gulliver, S.R.: Mulsemedia: State-of-the-Art, Perspectives and Challenges. In: ACM Transactions Multimedia Computing Communications and Applications, pp. 1–23 (2014). ISSN 1551-6865 https://doi.org/10.1145/261799 7. Kovács, P.T., Murray, N., Rozinaj, G., Sulema, Y., Rybárová, R.:. Application of immersive technologies for education: state of the art. In: Proceedings of the 9th International Conference on Interactive Mobile Communication Technologies and Learning (IMCL 2015), Thessaloniki, Greece (2015) 8. Murray, N., Lee, B., Qiao, Y., Muntean, G.M.: Olfaction-enhanced multimedia: a survey application domains, displays and research challenges. In: ACM Computing Surveys, vol. 48, no. 4, article 56 (2016) 9. Murray, N., Ademoye, O.A., Ghinea, G., Muntean, G.M.: A tutorial for olfaction-based multisensorial media application design and evaluation. In: ACM Computing Surveys, vol. 50, no. 5, article 67 (2017) 10. Waltl, M.: Enriching Multimedia with Sensory Effects, Saarbrücken, Germany, pp. 100 (2010) 11. Murray, N., Lee, B., Qiao, Y., Miro-Muntean, G.: The influence of human factors on olfaction based mulsemedia quality of experience. In: Special Session on User Factors in Multimedia Experiences at the 8th International Conference on Quality of Multimedia Experience (QoMEX 2016), pp. 1–6 (2016). https://doi.org/10.1109/qomex.2016.7498975 12. Ghinea, G., Ademoye, O.: A user perspective of olfaction-enhanced mulsemedia. In: Proceedings of the International Conference on Management of Emergent Digital Ecosystems (MEDES 2010), pp. 277–280. ACM, New York (2010) 13. Mihelj, M., Podobnik, J.: Haptics for Virtual Reality and Teleoperation, Intelligent Systems, Control and Automation: Science and Engineering (2012). https://doi.org/10.1007/978-94007-5718-9_2 14. Steinbach, E., Hirche, S., Ernst, M., Brandi, F., Chaudhari, R., Kammerl, J., Vittorias, I.: Haptic communications. Proceedings of the IEEE 100(4), 937–956 (2012) 15. Sulema, Y.: Haptic interaction in educational applications. In: Proceedings of the 9th International Conference on Interactive Mobile Communication Technologies and Learning (IMCL 2015), Thessaloniki, Greece (2015) 16. Murray, N., Qiao, Y., Lee, B., Karunakar, A.K., Muntean, G.M.: Subjective evaluation of olfactory and visual media synchronization. In: Proceedings of the 4th ACM Multimedia Systems Conference, pp. 162–171 (2013)

Mulsemedia Data Representation Based on Multi-image Concept

491

17. Murray, N., Lee, B., Qiao, Y., Muntean, G.-M.: Multiple-scent enhanced multimedia Synchronization. In: ACM Transactions Multimedia Computer Communication, p. 20 (2014) 18. Yuan, Z., Bi, T., Muntean, G.-M., Ghinea, G.: Perceived synchronization of mulsemedia services. IEEE Trans. Multimedia 17(7), 957–966 (2015) 19. Timmerer, C., Waltl, M., Rainer, B., Murray, N.: Sensory experience: quality of experience beyond audio-visual. In: Möller, S., Raake, A. (eds.) Quality of Experience: Advanced Concepts, Applications and Methods, pp. 351–365. Springer, Heidelberg (2014) 20. Yuan, Z., Ghinea, G., Muntean, G.-M.: Beyond multimedia adaptation: quality of experience-aware multi-sensorial media delivery. IEEE Trans. Multimedia 17(1), 104–117 (2015) 21. Yuan, Z., Chen, S., Ghinea, G., Muntean, G.M.: User quality of experience of mulsemedia applications. In: ACM Transactions on Multimedia Computing, Communications, and Applications - Special Issue on MulSeMedia: Advances and Applications, vol. 11, issue 1s, article no. 15 (2014) 22. Timmerer, C., Waltl, M., Hellwagner, H.: Are sensory effects ready for the world wide web? In: Proceedings of the Workshop on Interoperable Social Multimedia Applications (WISMA 2010), CEUR Workshop Proceedings (CEUR-WS.org), Aachen, Germany, pp. 57–60 (2010) 23. Dychka, I., Sulema, Y.: Logical operations in algebraic system of aggregates for multimodal data representation and processing. Res. Bull. Natl. Tech. Univ. Ukraine “Igor Sikorsky Kyiv Polytech. Inst. 6, 44–52 (2018) 24. Dychka, I., Sulema Y.: Ordering operations in algebraic system of aggregates for multiimage data processing. Res. Bull. Natl. Tech. Univ. Ukraine “Igor Sikorsky Kyiv Polytech. Inst. 1, 15–23 (2019) 25. Yevgeniya, S.: ASAMPL: programming language for mulsemedia data processing based on algebraic system of aggregates. In: Advances in Intelligent Systems and Computing, vol. 725, pp. 431–442. Springer, Heidelberg (2018). https://doi.org/10.1007/978-3-31975175-7_43 26. Fraenkel, A.A., et al.: Foundations of Set Theory, p. 415. Elsevier, Amsterdam (1973) 27. Maltsev, A.I.: Algebraic Systems. Nauka (1970). (in Russian) 28. Kulik, B.A., Zuenko, A.A., Fridman, A.Y.: Algebraic approach to intellectual processing of data and knowledge. Izdatelstvo Politekhnicheskogo Universiteta, p. 235 (2010). (in Russian) 29. Kelling, S.T., Halpern, B.P.: Taste judgements and gustatory stimulus duration: taste quality, taste intensity, and reaction time. Chem. Senses 13(4), 559–586 (1988) 30. Wenshan, H., Liu, G.-P., Zhou, H.: Web-based 3-D control laboratory for remote real-time experimentation. IEEE Trans. Ind. Electron. 60(10), 4673–4682 (2013) 31. Tawfik, M., et al.: Virtual instrument systems in reality (VISIR) for remote wiring and measurement of electronic circuits on breadboard. IEEE Trans. Learn. Technol. 6(1), 60–72 (2013) 32. Potkonjak, V., et al.: Virtual laboratories for education in science, technology, and engineering: a review. Comput. Educ. 95, 309–327 (2016) 33. Esquembre, F.: Facilitating the creation of virtual and remote laboratories for sci-ence and engineering education. IFAC-PapersOnLine Sci. Direct 48(29), 49–58 (2015)

Cryptographic Systems and Threats in e-Commerce Javier Sánchez Guerrero1(&), Sandra Carrillo Ríos2, Darwin García Herrera3, and Julio Mocha-Bonilla1 1

3

Facultad de Ciencias Humanas y de la Educación, Universidad Técnica de Ambato, Ambato, Ecuador {jsanchez,ja.mocha}@uta.edu.ec 2 Facultad de Ingeniería en Sistemas, Electrónica e Industrial, Universidad Técnica de Ambato, Ambato, Ecuador [email protected] Universidad Católica de Cuenca, Unidad Académica, Azogues, Ecuador [email protected]

Abstract. XXI century companies must adapt their business models to run on the internet. At present the transactions made on the WWW are countless ideas of commercial applications on it seem endless. This paper describes the concepts related to the terminology used in e-commerce, cryptosystems that are used to protect information, new attacks on cryptosystems and the main strategies to give users confidence and trust on transactions for buying and selling, which are the problems to be solved by information technology. This study concludes with recommendations related to creating new philosophies to online businesses, involving government policies and security, in order to pass the physical trading of e-Business with complete confidence by the end customer. Keywords: e-commerce


Cryptosystems
Online
Internet
Cyber-attacks

1 Introduction 21st century companies are heading towards a world in which the ones that best adapt are those that survive the exponential connected world, where technology and digitalization are the main managers. Technology changes rapidly and companies compete digitally. In the next decade, countries will grow more than 10% of companies on the Internet, becoming one of the largest sectors of the economy as a whole [1]. In the 19th century, the idea of selling products through catalogues was generated, which created a new system of product offering. In 1962, the massive reforms of telemarketing began, seeking to attract customers through telephone calls. In the mid1980s, direct sales in telemarketing emerged, with the appearance of the Internet [2]. From the 1990s onwards, the internet and globalization became popular, with technology being the catalyst for sales, making business in the palm of the hand [3]. In 2005, Apple and Android applications created new methods for e-commerce, aimed at customer loyalty, expanding the market, and above all creating a massive sales network that involves business growth [4], Electronic commerce does not involve schedules. © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 492–506, 2021. https://doi.org/10.1007/978-3-030-49932-7_47

Cryptographic Systems and Threats in e-Commerce

493

Works 24 h a day, 365 days a year. It operates permanently an electronic agent that is able to provide the required data, take orders or offer a variety of services. Most people do not know what they are facing when surfing the Internet, in terms of computer security. The goal of computer security is to maintain the authenticity of computer- managed information. For this reason, the way computers communicate in order to generate the passage of sensitive data when making Internet purchases must be protected by reliable mechanisms or cryptosystems, which allow the users of these services to be given. The risks facing information are: Spam, Spyware, Virus and Phishing [5]. Hence, the lack of direct human presence in online business increases distrust in the system, because not knowing who is behind the transaction or simply trying to understand that it is a computer that executes a transaction, is a problem for most customers [2]. The transformation of the economy as a whole has begun to adapt to the new environment, as by 2020 more than 20 million people will be incorporated into the use of the Internet. The automation of tasks already has positive effects on society, however; there is an increase in productivity at very high levels, but employment rates remain flat [1]. The Internet is not present in things, they don’t emit information. But, the tendency is for things to do it. Clocks, glasses, gadgets will be able to emit information, such as sensors that help support data that converted into real-time information, would be of great help to customers [6]. In the next few years, new objects will be connected to the Internet to transmit information, which will allow companies to obtain informative advantages due to the information they have, since when it is generated by users, they can obtain a profile that allows them to better serve them and to know the particular needs of each one [7], especially in providing security and trust in online business transactions. This is still one of the drawbacks related to the platform that supports the whole business, since statistically it has been demonstrated in 1996 that half of the people do not buy by this means due to lack of security or confidence in the environment [2]. New threats in terms of cyber-attacks on banks are developing day by day. These attacks are part of the platform for managing e-business and e-commerce. This is manifested in three new forms of cyber attacking detected, Carnabak 2.0, Metel and GCMAC. These attacks target banking and financial institutions, which in turn provide credit or debit card service, fundamental support for e-commerce, using covert APT style and customized malware along with legitimate enterprise software and some innovative systems, which generates cyber- attack fraud [8]. Faced with this latent scenario, different security alternatives are presented based on cryptography, and as a component the crypto analysis, which focuses on establishing the way in which information systems, which must be secure, and which cannot be attacked by intruders. In other words, systems can be benefited by cryptography or bone encryption algorithms, where sensitive information can be hidden [48]. At present there is also talk of the Internet of things, this being an interesting innovation for the new business model on the Internet. For [49] it is necessary that the devices have unique protocols, which must be verified, this being the only form of safe and proper operation. Many security systems need to deploy patches that reduce bandwidth consumption or reduce the possibility of attacks when someone could steal the credentials to access the device’s operation.

494

J. Sánchez Guerrero et al.

The reality of Internet business processes has three complementary approaches: environment, business model and the technological model that supports the process. In addition to the service of online businesses, they must not only offer products but also propose solutions to the problems of potential customers. Software is the basic process tool. Digital purchases offer products and services, becoming digital empires, made up of hundreds of service companies that aim to digitize all information. Trying to reduce the level of mistrust through IT strategies helps to strengthen e-commerce and customer peace of mind. In this paper, we describe the different strategies that online businesses have adopted to survive on the platform to prevent hacking attacks. In addition, a detail of the new threats to crypto systems that allow for the communication of sensitive data such as those required for electronic commerce payments is presented. It also describes conclusions and recommendations, which as a result of theoretical research, can be taken into account to improve online business strategies.

2 State of the Art 2.1

Definitions

Electronic Commerce –‘‘E-commerce is the process of buying and selling goods and services electronically, through transactions over the Internet, networks and other digital technologies” [9]. Cryptography and Cryptosystem - As mentioned by [10] in their article: System of distribution of keys by means of quantum cryptography to avoid attacks of the type ‘‘Man in the middle”, cryptography is the art of converting a message to a confused form, so that not everyone is able to interpret it. This can be achieved by using some algorithm, which is called cryptosystem, and its function is to combine or mix the message with some additional information and thus generate a cryptogram, that is to say an encrypted message. The additional information used to change this message is known as a key. Cyberspace - In the last few decades, the term cyberspace has been frequently used to refer to all activities related to computers and communication networks, especially the security of the information generated by them. As an emerging area, conflicts in cyberspace such as cyber warfare between countries, cyber-exercises and cyberterrorism are being taken into account with great diligence. For the Department of Defense, 2016, cyberspace is” a global domain within the information environment consisting of an interdependent network of IT infrastructures, including the Internet, telecommunication networks, computer systems and embedded processors and controllers. 2.2

The New Era of E-commerce

The “Law of the Numbers of the Global and Regional Digital Economy” allows us to understand the development of electronic commerce in various countries of the world. In the US, for example, for this year according to Forrester’s February 2012 publication

Cryptographic Systems and Threats in e-Commerce

495

[11], e-commerce will increase by 62%, while in Europe based on the same source will be up to 78%. The proposal in Latin America is much smaller, where other parameters such as the difficulty of bringing assets to the formal banking system and the high cost of Internet in these countries influence other parameters [12]. Social business comes from the concept of social media - word of mouth -, it is a bidirectional system where the user - client is incorporated into a business system and in the definition of products and services [13]. A representation of the digital strategy that must be followed to maintain an online business is shown in Fig. 1:

Fig. 1. Strategy of business

In Fig. 1, the three steps must be combined and updated in order for the business to thrive online. Retention can be seen as the fundamental step, as users should be kept on a regular basis by offering social networking services or working on online promotions that invite them to continue as a business customer. But not only does online business work with customers, it can also help in the interaction between businesses to sell services to each other, consumer to consumer, used goods sales, and consumer to business or sales of product ideas. E-commerce integrates sales, marketing and operations with website activities. A strategy model for Internet business allows to improve operations and meet the objectives [14]. For e-commerce, strategies that address the company’s digital awareness and revised targets to determine whether the company is advancing into the digital age should be considered. We must understand what happens inside the company as well as outside of it. Then understanding what happens outside the company allows interconnection between several digital companies and formalizing their processes, with the aim of generating greater productivity and profits [15, 16]. The e-commerce strategy contains the processes to generate and improve, according to the image and identity of the organization, this allows to obtain creative achievements and baselines. The design of the strategy allows the use of technology and work with human resources. The technological architecture must be in line with e-commerce objectives by clearly identifying the business problem, and versions for each of the digital competition channels. However, Internet businesses must generate trust and security in transactions [17]. This implies that there must be an excellent prospecting system to

496

J. Sánchez Guerrero et al.

help build confidence before proceeding with transactions. If the mobile forces are organized, this generates a quantity of information and computing capacity, which obtains different business models, and it is there where it is possible to analyze the information from the concept of big data [18]. This is an advantage of the online business as the current trend is that online prices are much lower than those in the physical store. All of this changes the way businesses are shown and distributed. In 2010, technology sales were low relative to mobility, while in 2012, one example of the high impact was that sales of iPads were 20 million, while some companies are up in sales, others like E-Books will go from 10 million to 5 million units, with a decrease of 36% per year. To survive, some companies have adapted to technological advances, now cameras are sold as part of cell phones [19]. Digital markets are transactional environments that serve to bring together services and products. They are flexible and efficient, and have the ability to adjust prices dynamically, where the price of the product depends on the conditions of the seller’s supply and offer. Digital goods, newspapers, magazines and books can be stored and distributed as digital products, whose delivery system is less expensive. The purchase process in ecommerce consists of five steps: 1) The customer navigates through the purchase page, 2) The customer selects the product, 3) The customer must confirm his order, 4) The customer enters his payment information, 5) The customer confirms the entire transaction and the server confirms the process with a virtual invoice [20]. The sales forecast of traditional companies has nothing to do with the current sales forecast of digital companies [21], have two ways to go; complement their activities based on the real world in other derivatives in the virtual world, or simply risk being replaced. Companies face zip car companies. These are personal companies that rent cars when their owner is not using them, while working or on vacation [22]. In the area of health, it is linked to the mobile device. More than 4000 applications allow to monitor the physical state of the users, and even to keep records of them so that they can be later analyzed by doctors [23]. Mobile Help allows to finance all types of devices that allow the capture of information, as it can store all information for later analysis [24]. The most popular means of commerce has been the Inter- net, through computers and also through portable devices such as smart phones and tablets, which must take greater attention in security to ensure the reliability of the passage of information. For [25], mobile phones use WEP and WAP protocols use cryptosystems in their transport layer for sending and receiving information. In an insecure environment, but these protocols have presented vulnerabilities due to cryptographic attacks such as the manin-the-middle attack or brute force attacks, which evidence the flaws in the encryption algorithms, are usually symmetrical cryptosystem. The Retail sector is confronted with multi-channel purchasing, which incorporates mobile phone shopping, product photos and price comparisons in all stores. It is a multi-channel mix within a single digital channel [26]. Sensors that issue alerts to drivers regarding their driving habits, in order to predict tire expense, are currently being tested, which may cause decreased revenue to vehicle maintenance companies [27], but increase data and information management on the Internet.

Cryptographic Systems and Threats in e-Commerce

497

Hyundai’s agreements with Google Maps for in-vehicle advertising transmission is another philosophy of advertising communication that will appear in the coming years. Reference data from the study, highlights Denmark and the United Kingdom as leaders with the largest volume of people buying through the web in their respective populations with 77%, 30 percentage points above the European average. Spain, on the other hand, matched with the Czech Republic and Poland, is ranked 16th of the 28 countries studied with 15 percentage points below the European average with 32%. On the other hand, the current mismatches between limited supply and unsatisfied demand will be gradually corrected and only those companies that have strategies to anticipate the market will survive this phenomenon and win [28]. The overall goal of online business is to have a website to sell and have a return on investment. Digital advertising is handled in Google, where you can search for services. The keywords used by the user are the most important since companies are the ones that pay to appear in the highlighted results related to the service keywords that the user is looking for. Portals where customers can sell their stuff also function as an online business. The process must have security features that give the user confidence in the online business [2]. 2.3

Cyberattacks

The frank and physical process that occurs when making a purchase in a commercial store, tries to be emulated by the digital media, taking into account all parties involved and at the same time that sensitive information is being generated for those involved. In this way, with the wide distribution of the Internet and electronic commerce, the emergence of cryptography, which is the art of coding and encoding information, and together with cryptanalysis, which is the art of breaking or decrypting such codes, cryptology originates. In e-commerce, as well as in other cyberspace activities, the sender initially selects an encryption function and then selects the plain text to be transmitted; the text is encrypted and then transmitted to the recipient, who also knows that encryption key and thus decrypts the original message [10].

Fig. 2. Diagram of a quantum key communication system

Figure 2 shows the sending of messages described in the previous paragraph, such as the sending of messages between Alice and Bob, where the first encodes the

498

J. Sánchez Guerrero et al.

message, it travels through a secure medium and finally Bob decodes and reads it. However, there is another character who is Eve, the cyber attacker, the intruder who is waiting to hear and intercept the communication between Alice and Bob [29] (Fig. 3).

Fig. 3. Digital fraud

There are online shops that act as intermediaries between the seller and the customer, including the well-known auction sites E-Bay and Olx. The sale of books online led to the appearance of Amazon, a company that not only specializes in selling this product, but transformed to meet the needs of online customers. Research models have been proposed, which allow us to obtain the usefulness of the product, ease of use, the attitude of customers and the influence on customer satisfaction. This is in order to identify a purchasing profile so that online business sites can recommend users to purchase based on their previous preferences [30, 31, 32] (Fig. 4).

Fig. 4. The connections between different users

Cryptographic Systems and Threats in e-Commerce

499

The various online connections and relationships between users have made the Internet grow and therefore the number of online businesses has increased [5]. Social media projects have improved the way in which market research is used, which implies proposing several product lines in a single portal that can be accessed by online businesses, such as Gravy, Train, Lives and Meow Mix [33]. New technologies support the development of e-commerce, trying to provide security and confidence for users. The objectives of the latest advances in computer algorithms that contribute to increasing confidence in online business are shown in the following table:

Table 1. Technology strategies for online business. Objective Dynamic method for eliminating Vulnerabilities in online shopping Designing strategies to provide security for online transactions Algorithm for reducing attacks on online businesses Methods to preserve the privacy of big data information

Author (Liu and Mackie 2004) (Gong and Wang 2015) (Meng 2015) (Ghonakhloo et al. 2015) (Wang et al. 2015) (Li and Shi 2015) (Reda, Rahmani, Rahmani, Bourara, 2015)

The technologies are designed to improve transaction security and provide users with confidence, including in large amounts of information, since the greater the amount of data, the greater the risk of loss of information is [34–36] The algorithms listed in Table 1 work with encryption and encryption mechanisms, which convert the original messages into codes that are difficult to analyze by hackers. They use public and private keys, which are also decoded and require even quantum processes for conversion to original message. The reduction of attacks using cryptanalysis techniques is also necessary for online businesses, since it is a question of bank data in particular being fully protected and from there the appearance of patents that exchange credit cards for other devices that provide greater security to the user when purchasing over the Internet [36]. The protocols developed by Visa and MasterCard for security give way to secure electronic transactions [37]. The SET protocol is one of the strategies adopted to pro- vide confidence to ecommerce on the Internet. E-money can be used without being connected to the financial institution, and this is what provides anonymity to the person who buys and builds trust, as this can be done offline. This process is suitable for micro-payments, as transactions of less than $1 can be generated [34, 35]. But if there are problems with the computer or device, the money is at risk of being lost.

500

J. Sánchez Guerrero et al.

To solve this, a settlement agreement must be generated with the money issuer in order to retain transactional data. The use of e-Wallet is another means of online payment. In this wallet you store electronic money, credit cards, debit cards, shipping address data and contacts. This streamlines the buying process, as repeating credit card numbers can become a slow and boring process for the customer, while if all the information is in the wallet it speeds up the entire online business. Smart cards are connected to the computer, and the e-commerce site pulls the information from that card, where much of the customer’s personal information is stored. This information is encrypted, which secures the online system [38, 39]. There are some regulations for the development of e- commerce, once the business is under way you have to take into account the various regulatory and specific aspects such as safety, advertising and consumer protection. This implies that the money is stored offline, and can be used on both virtual and physical sites. As a joint task, the criteria of the legal contracting of services and everything that revolves around the legalization of contracts with an adaptation to new technologies must also be handled 2.4

New Threats to Cryptosystems in E-commerce

Historically, one of the needs of humanity has been to hide sensitive information that it wishes to transmit only to certain pairs of interest, which is why the most varied security mechanisms have been developed. In the 80s, the use of asymmetric key and DES systems began to be used, which in turn immediately gave way to a public market with the encryption of credit and debit card data that generated confidence to withdraw money from any ATM or make payments in any store [41]. Once the WWW (Word Wide Web) takes shape, consolidating the growth of companies on the Internet, [41], ensures that in the last 10 years threats have arisen in the form of attacks on cryptosystems contained in a chip through lateral channels, this means that attacks on credit cards, debit cards and other electronic identifiers, are made by physical access to the electronic circuit or its disruption through manipulation of the power supply, introduction of transients, as well as reading of localized electromagnetic radiation, reaction to excessive heat or cold in order to read the keys contained therein, totally or partially. Each of the techniques and methods used to give enough robustness to cryptosystems have been sufficiently reliable and safe for years, however, [42] indicates that the systems that base their algorithm on private and public key are threatened by advances in cryptanalytic techniques and quantum computing, but they are still the ones that provide the right framework of security and speed. Several authors agree with this criterion, such as [43] who indicate that: “Quantum computers would pose a lethal threat to the most widely used public-key cryptographic systems, given that the process of multiplying two numbers and factoring a number would become problems of similar complexity with a quantum computer”. However, new attacks and new ways of attacking are invented by hackers. An example of the online cyber-attacks taking place worldwide can be seen in Fig. 5:

Cryptographic Systems and Threats in e-Commerce

501

Fig. 5. Cyber-attacks are carried out in one country

Cyber-attacks don’t stop, and all you can do is protect yourself. The most attacked country is the United States. New cyber-attacks can give rise to cyber war. In 2007, the US had an attack that threatened to paralyze the bank and the government itself. A cyber war does not imply movement, and it is not very difficult to obtain evidence against the person responsible [44]. The work of software developers is intended to protect users from virus attacks and computer robots. Computer complexity has increased. A botnet is an army, where a bot or computer robot is just a soldier. This bot forces the computer to perform a task over and over again. This bot is controlled by the bot master, and he coordinates the attack, ordering his robots what to do. A bot downloads another program from the internet and runs it, and can cause the computer to perform tasks not programmed by its owner, and can even be used to commit computer crimes. The main problem here is that when boots work together, they increase computer traffic which causes any system to crash. Cyberspace becomes an ever-growing unexplored space. Settlers are responsible for their own safety. Aurora (persistent and lasting threat), was a project that generated a lot of outbound network traffic in Microsoft. It was believed that the redundancy system was developed in China, which caused the company to leave the site and start working in Hong Kong. The bug that was used was a security hole known to Microsoft, whose Appt (fault application) can even be auctioned on the black market. In 2007 in Estonia, an attack was made on its computer systems by means of denial of services tools, which was the beginning of a cyber-war between Russia and Estonia. The attack on the Iranian nuclear power plant was carried out by manipulating its SCADA systems. Such systems are only alert and not control systems of the plant. The point is that the systems under attack recorded data that were not in line with reality, so a national alert was generated, since Siemens’ S7 system sent modified data from the

502

J. Sánchez Guerrero et al.

system’s PLC, and what was delivered to SCADA was wrong, thus achieving an unstable system. In short, cyber-attacks today can lead to a cyber-war. Computer software is used to attack national security companies or entities from different countries worldwide. In the end, tools must always be developed to identify such attacks and alert human operators.

3 Conclusion and Future Works This paper describes the main strategies that online busi- nesses need to implement to stay active on the Internet. It goes from the change of being a physical business to the digital format and the acquisition of customers and the maintenance of business in the world of e-commerce. These are the phases that every online business should have and try to maintain during its digital life [40] (Fig. 6).

Fig. 6. Data on complaints received by the Attorney

The main problem encountered by this type of business is the lack of trust in the platform and the insecurity in data management. This research describes this problem, and takes a theoretical tour of the main solutions that can be adopted to keep business active on the Internet [22, 36]. The main objective of this paper is to describe in a theoretical way the technological methods and processes and algorithms that allow online businesses to have tools to correctly secure transactions. However, it is not possible to count on total security processes, since every day hackers update their algorithm development methodology to counter the platform’s security, and although it is manifested that by means of quantum techniques, the algorithms of computer attack do not progress, since it is necessary to have the necessary precautions against possible attacks. With the cyber war, the point is to identify the origin of the attacks and verify the security systems. At the same time, trained personnel must be able to intuit or recognize the possible cyber-attack. This would make it possible to make the right decisions for the managers of an organization. Finally, it can be said that it is necessary to be constantly vigilant about new forms of attacks and that the authorities at world level consolidate their efforts to maintain secure communication. The implementation of a National Cyber security Action Plan

Cryptographic Systems and Threats in e-Commerce

503

will allow the U. S. government to thoroughly review old networks, which will help control data leaks and cyber-attacks 3.1

Future Work

Strategies to increase the security of online transactions are the concern of recent years, and it is in this area that scientific advances are taking place. The appropriate treatment of information using encryption and encryption tools, which are impossible to decode, is the main objective of scientists as a basic aid to preserve online business strategies [22]. It is about finding a connection between online business retention strategies in the network and security that should build customer confidence [45, 46]. These new technology framework proposals will usher in the new era of online business where security mechanisms will use ubiquitous data to certify the authenticity of a transaction [21, 47]. The quality of online transactions will be improved with new offers to customers, which contain buying and selling patterns, which will improve product offerings and suggestions [1]. Ultimately it will be a new online business world where distrust is expected to be a term erased from the virtual online dictionary. Countries’ security systems must be studied. For [50] once it is clear the different alternatives of encryption or hiding the information, there are still failures in existing methods, so the study of Quantum Cryptography in an interesting area of research. The way in which major digital organizations can be infected seems to be easy, as it can be done from a flash memory. Although the work of trying to identify even the tiniest detail of a cyber-attack seems utopian, countries should invest more in researching tools to ensure that their systems are as secure as possible. Hence, new cyber-attacks need to be further explored, as this type of study can serve as a basis for governments to be informed of new ways of accessing their important data and information. A derivative work of this study would be research based on the best cryptosystems that have given better results with evidence worldwide and therefore generate greater confidence when making purchases online or generate electronic commerce.

References 1. Carr, N.: Automation for the people? Glass Cage Autom. US 276 (2014) 2. Ferreiro, T., Mario, T.: El Comercio Electrónico en Internet. Distribución Consumo (1997) 3. Kane, G., Palmer, D., Philips, N., Kiron, D.: Is your business ready for a digital future? ProQuest (2015) 4. Alpert, S., Karat, J., Karat, C.-M., Brodie, C., Vergo, J.: User attitudes regarding a useradaptiv ecommerce web site. User Model. User-Adapt. Interact. 13(4), 373–396 (2003) 5. Gong, S., Wang, Y.: Research on security strategy of electronic commerce industry websites. In: International Conference on Advances in Mechanical Enginnering and Industrial Informatics (2015) 6. Payam, B., Sheth, A., Singh, V., Hauswirth, M.: Physical-cyber- social computing: looking back, looking forward. Internet Comput. 19, 7–11 (2015) 7. Laru, J., Naykki, P., Jarvela, S.: Four stages of research on the educational use of ubiquitous computing. Learn. Technol. 81, 69–82 (2015)

504

J. Sánchez Guerrero et al.

8. Redacción. (09 de Febrero de 2016). Comunicaciones hoy. Obtenido de Comunicaciones hoy: http://profesionaleshoy.es/tics/2016/02/09/los-bancos-se-enfrentan-a-tres-nuevos-cibera taques-carbanak-2–0-metel-y-gcman/19216 9. Laudon, K., Laudon, J.: Sistemas de información gerencial. Organización y tecnología de la empresa conectada en red. Printice hall, México (2002) 10. Guillén, E.P., Navarro Gasca, J.J.: Sistema de distribución de claves mediante criptografía cuántica para evadir ataques del tipo “Man in the middle”. Ciencia e Ingeniería Neogranadina 16, 64–73 (2006) 11. Reitsma, R.: Which Market Leads In En l´ınea Retail Sales: UK or US? Forrester, Cambridge (2012) 12. Instituto Latinoamericano de Comercio Electrónico. (3 de Diciembre de 2015). El eCommerce en América Latina se enfrenta a dos mitos. Buenos Aires, Buenos Aires, Argentina 13. Rodríguez, M., Peterson, R., Ajjan, H.: Crm/Social Media technology: impact on customer orientation process and organizational sales permormance. In: Finding the New and Polishing the Old, pp. 636–638 (2014) 14. Martínez, S.: Plan de negocios para la creación de MediaSuite: comercio electrónico, social media y mobile marketing. Universidad de San Andres, Buenos Aires (2015) 15. Sila, I.: The state of empirical research on the adoption and difussion of business-to-business comercio electrónico. IndeScience (2015) 16. Guang, W., Pinsonnealult, A., Tomiuk, D., Wang, S., Liu, Y.: The impacts of social trust on open and closed B2B comercio electro´nico: a Europe-based study. Inf. Manag. 52, 151–159 (2015) 17. Sobihah, M., Mohamad, M., Azman, N., Wan Ismail, Z.: Comercio electrónico service quality on customer satisfaction, belief and loyalty: a proposal. Mediterr. J. Soc. Sci. 6, 260 (2015) 18. Aguiar, R., Benhabiles, N., Pfeiffer, T., Rodriguez, P., Viswanathan, H., Wang, J., Zang, H.: Big data, loT,…buzz words for academia or reality for industry. In: MobiCom, pp. 550–551. ACM, New York (2015) 19. Judge, S., Floyd, K., Jeffs, T.: Using mobile media devices and apps to promote young children’s learning. In: Young Children and Families in the Information Age (2014) 20. Lala, H.: Blog, news, and ecommerce: does genre matter for taggers? North Am. Symp. Knowl. Organ. 4, 88–99 (2013) 21. Rahm, L., Fejes, A.: Ubiquitous computing, digital failure and citizenship learning in Swedish popular education. Citezensh. Teach. Learn. 10, 127–141 (2015) 22. Castro, V.: Innovation Importance Across Dimensions for Business-to-Business and Business-to-Consumer Firms: A Case Study- Based Analysis. Wharton University, Pensylvania (2015) 23. Rahmani, A., Thanigaivelan, N., Tuan Nguyen, G., Granados, J.: Smart e-health gateway: bringing intelligence to internet-of-things based ubiquitous healthcare systems. In: Consumer Communications and Networking Conference, pp. 826–834. IEEE, Las vegas (2015) 24. Doryab, A., Frost, M., Faurholt-Jepsen, M., Kessing, L.V., Bardram, J.E.: Impact factor analysis: combining prediction with parameter ranking to reveal the impact of behavior on health outcome. Pers. Ubiquit. Comput. 19(2), 355–365 (2014). https://doi.org/10.1007/ s00779-014-0826-8 25. Barbosa, C.A., Tibaquira, Y.A., Montiel, C.G.: Implementación del criptosistema de curva elíptica en un prototipo de aplicación móvil para Comercio electrónico. Revista Tekhne 8, 5– 12 (2011)

Cryptographic Systems and Threats in e-Commerce

505

26. Vaananen, K., Olsson, T., Hakkila, J.: Towards Deeper understanding of user experience with ubiquitous computing systems: systematic literature review and design framework. In: Human-Computer Interaction, pp. 384–401 (2015) 27. Silva, F., Analide, C., Novais, P.: Traffic expression through ubiquitous and pervasive sensorizatin - smart cities and assesment of driving behaviour. In: International Conference on Pervasive and Embedded Computing and Comunication Systems. ESEO, Angers (2015) 28. Palau, C.M.: El comercio electro´nico como estrategia de negocio. Economía Ind. 326, 11– 20 (1999) 29. Elliot, C.: Quantum cryptography. IEEE Secur. Priv. 2, 57–61 (2004) 30. Leonard, L., Jones, K.: Consumer to consumer ecommerce: acceptance and intended behavior. CIIMA 14, 1 (2015) 31. Pradyut, V,H., Debleena, G., Atin, G.: A comparative study on factors shaping buying behaviour on B2B and B2C comercio electrónico platforms in India. J. Mark. Consum. Res. 5, 1–10 (2015) 32. Rianto, L., Nugroho, L., Santosa, I.: Analysis of Comercio electrónico user behaviour of indonesian students: a preliminary study of adaptive comercio electrónico. In: Computational intelligence and Efficienty in Enginnering Systems, pp. 365–375 (2015) 33. Turban, E., King, D., Lee, J., Liang, T.-P., Turban, D.: Marketing and advertising in comercio electrónico. Bus. Econ., 403–456 (2015) 34. Liu, C., Mackie, B.: Developing secutiry for comercio electrónico applications: a teaching case. Commun. IIMA (2004) 35. Meng, Y.-Q.: Study on the information security based on comercio electrónico. In: International Conference on Advances in Mechanical Enginnering and Industrial Informatics (2015) 36. Wang, G., Musau, F., Guo, S., Andullahi, M.: Neighbor similarity trust against sybil attack in P2P comercio electrónico. Para. Distrib. Syst., 1045–9219 (2015) 37. Lu, S., Smolka, S.: Model checking the secure electronic trans- action (SET) protocol. In: Modeling, Analysis and Simulation of Computer and Telecomunication Systems, pp. 358– 364. IEEE, College Park (1999) 38. Kanhekar, H., Mane, S.: Digital wallet. Wireless and Microwave Technology 4, 62–68 (2015) 39. Pasandideh, M., Motahhari-Nejad, H.: Approach to Comercio electrónico in communications era. J. Soc. Issues Humanit. (2014) 40. Torres, M.B.: Formas contractuales en el comercio electrónico. Información Comercial Española ICE Revista de economía, 173–186 (2004) 41. Montoya, F.: Testimonio de medio siglo: de la Per lustración al Cifrado cuántico. In: RECSI 2012 (2012) 42. Orue López, A.B.: Contribución al estudio del criptoanálisis y diseño de los cripto sistemas cuánticos, Madrid (2013) 43. González, S., Martínez, C.: Las matemáticas de la seguridad. Arbor Ciencia Pensamiento y Cultura 183, 419–425 (2007) 44. Salsona, M.: EEUU ante el reto de los ciberataques. IEEE 37 (2013) 45. Alpert, S., Karat, J., Karat, C.-M., Brodie, C., Vergo, J.: User attitudes regarding a useradaptive ecommerce web site. User Model. User-Adapt. Interact. 13, 373–396 (2003) 46. Bonneau, J., Herley, C., Oorschot, P., Stajano, F.: Passwrods and the evolution of imperfect authentication. Commun. ACM 58, 78–87 (2015) 47. Islam, S.: An algorithm for electronic money transaction secutiry (three layer secutiry): a new approach. Int. J. Secur. Appl. 9, 203–214 (2015)

506

J. Sánchez Guerrero et al.

48. Sánchez, J., Correa, R., Buenano, H., Arias, S.Y., Gómez, H.: Técnicas de cifrado: una descripción teórica y futuras propuestas. In: 2016 Tercera Conferencia Internacional sobre eDemocracia y Administración Electrónica (ICEDEG) (2016). https://doi.org/10.1109/ icedeg.2016.7461697 49. Guerrero, J.S., Albán, R.V., Guachimboza, M., Quinde, C.P., Ríos, M.N., Ruiz, L.A.J.: Cryptography applied to the internet of things. In: Teaching and Learning in a Digital World, Advances in Intelligent Systems and Computing, vol. 715 (2017). https://doi.org/10.1007/ 978-3-319-73210-7_47 50. Buenano, H., Guachimbosa V.V.H., Ruiz, J., Mera, J.S., Guerrero, J.S.: A reliable security alternative: quantum cryptography. In: 2019 Sixth International Conference on eDemocracy & eGovernment (ICEDEG) (2019). https://doi.org/10.1109/icedeg.2019.8734385

Work in Progress. SportSWARES, Towards an Intelligent Way of Physical Training Thrasyvoulos Tsiatsos(&), Ippokratis Apostolidis, Nikolaos Politopoulos, Agisilaos Chaldogeridis, and Ioannis Stamelos Aristotle University of Thessaloniki, Thessaloniki, Greece {tsiatsos,aposti,npolitop,achaldog, stamelos}@csd.auth.gr Abstract. There is a developing interdisciplinary research field trying to provide advanced technological support to various sports activities. This support is focused mainly to professional elite athletes but is considering also about training in sports academies, gyms even about amateur athletes practicing at home. Advanced training can now be carried out using low-cost technology that can produce useful data for researchers and coaches. Sports data collected from athlete’s skeletal tracking, movement analysis and physiological measurements may prove a valuable tool for motor and technical skills development. In addition, there is an increasing interest from coaches and sports academies to enrich sport activities by utilizing the use of serious exer-games and applications of augmented reality in order to provide a more intelligent, personalized and immersive training environment. In this paper a smart platform is proposed which collects athlete’s bio-signals and data from visualization of skeletal movement, it analyses them and it provides affective and sport analytics feedback through application interfaces. Keywords: Sport technology
Skeletal tracking
Motion analysis
Affective computing

1 Introduction Sports and physical activities deal with the human body, its characteristics and its movements. In general sports contribute to a better quality of life, developing healthier human bodies and improving cognitive functions. Recent research is trying to integrate modern technology into sports. Computer technology may prove a valuable supportive tool for: • • • •

Athletes of all ages to improve their technical skills. Improve amateur sports. Advanced coaching. Psychological support for athletes under pressure or amateurs as well.

Moreover, motion capture, computer vision and computer animation techniques have made a remarkable progress providing very realistic real - time human movement representations in augmented reality applications and exergames. After 2010, the © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 507–513, 2021. https://doi.org/10.1007/978-3-030-49932-7_48

508

T. Tsiatsos et al.

tendency that emerged was exergames as a profitable market and an advanced training facility. Exergames or Active Video Games are innovative tools that have the potential to turn a traditional sedentary activity, such as playing video games, in physical exercise [1]. The skills that kids and adolescents acquire during playing exergames can be transferred to real world activities and benefit physical, social, and cognitive development. However, in sports apart from the human motor skills development, emotion is another crucial factor of human health in general [2] and therefore has a significant impact on athlete’s bodies, mental health, and behaviour. It is commonly accepted that, if an athlete experiences positive emotions during a sport activity, is more like to achieve a better performance. Thus, athlete’s affective awareness may prove a very significant tool of effective training providing support to self-regulation as well as to coach’s efficient interventions. Furthermore, a very popular and advanced research area named Internet of Things (IoT) can apply many advanced technology techniques like human affective awareness, motion capture and skeletal tracking to various human activities making them smarter. Several applications of IoT are related to: • • • •

smart home, e-health, smart transportation, wearable technology etc. [2].

The aim of this research proposal is the development of a technological platform which can be considered as a building block of interfaces to various applications providing psychological support and promoting the development of motor skills. Thus, this dynamic platform will deepen in athlete’s (amateurs or professionals) affective response like anxiety, utilizing human body physiological reactions and provide a realtime affective feedback through a wearable wristband. Moreover, the proposed system will support physical activities by applying real-time vision techniques to biomechanical analysis of skeletal tracking and human posture recognition utilizing one or more depth cameras. Thus, this IoT platform aims to facilitate a smart gym – room. All data from affective feedback and biomechanical analysis collected and analysed by this platform, will be available through specific application interfaces to third party web or augmented reality applications and exergames supporting self – regulation and self improvement. The following section is dealing with an overview of the related research area. Then the research motivation is presented and another section is devoted to the design and system architecture. The final section is dealing with the future steps.

2 Related Work A modern attitude of sports science and physical education examines sport activities through bio-technological interventions [3]. This view is related to: • The development of training methods.

Work in Progress. SportSWARES, Towards an Intelligent Way of Physical Training

509

• The development of sports materials (e.g. sports shoes, training vests, basketball balls with integrated sensors). • The collection of various information data on sports activities and their statistical processing providing intelligent and supportive feedback like sport analytics etc. • The production and use of pharmaceutical products as food supplements. • Facilitated ergonomic centers • The development of fitness equipment • Improvements on athlete’s safety in certain sports [4]. • The reinforcement or the suspension of athlete’s skills in cases where certain human functions are substituted by technological tools [5]. The rapid development of electronics and computer science makes it possible to use a wide range of technological solutions to support sporting activities. The cost of these solutions varies from very cheap to very expensive [6]. However, this variety may create an additional difficulty in choosing the right product for each particular occasion. Often the prospective customer is a coach or a gym trainer and the process of choosing the right product is complicated requiring some technological knowledge. There have been developed technological applications utilizing GPS, speedometers, gyroscopes and other sensors, mainly supporting the training of team sports [6–8]. In addition, several methods of movement analysis have been introduced, which process videos of various sporting activities in the area of team sports, like football or basketball matches [6, 9, 10]. Moreover, the visual processing is applied to the movement analysis of an athlete (isolated) in order to collect data related to the biomechanical analysis of h/her motions and technique [6, 11, 12]. Similar processing also applies to the collection of data relating to the performance of an athlete or a whole team during sporting activities. Therefore, in the aforementioned cases, technology has substituted the human observation which is prone to mistakes in many cases. These data are presented as raw material or analysed after statistical processing in real - time or offline for better information and awareness [6, 13]. Information derived from an external source in relation to an athlete and known as “augmented feedback” is considered to be particularly effective during the state of athlete’s technique training [6, 14]. Modern sports science does not focus on physical exercise only; it examines ways to develop the psycho-mental skills of the athlete in order to achieve better results. According to scientific studies, psychological skills in sports can be developed under specific training, improving the performance of an athlete [15]. Besides, it is a common perception that the human psychological condition directly affects h/her physical function. Modern athletic psychology deals with the development of skills and techniques in order to achieve stress management, self-control, self-confidence and positive emotions. These techniques are not necessarily addressed only to high-level athletes [15], therefore, they could be applied to any athlete of any age. According to athletic psychology, the younger aged athletes can comprehend better the psychological techniques [15]. Furthermore, the persistent monitoring of athlete’s physiological state is a crucial factor to identify periods of optimal training, to follow recovery status and to get emerged of any potential overtraining [16]. In Greece as far as it is commonly known, technological tools are rarely used on sport activities, examining a wide range of them from young athletes’ academies to

510

T. Tsiatsos et al.

professional sports. Moreover, it could be supported that during training period greater emphasis is given to physical exercise than on the development of mental skills. In most cases the coach through visual observation verifies if a physical exercise is performed well and correctly without any significant technological support since most of the time the financial resources of a sports club are very limited, and they cannot afford for advanced technological solutions. In addition, usually the coaches are stressed with athlete’s psychological support but in many cases, they lack the necessary knowledge, the appropriate support for the task, or even the skills required. However, as technology is rapidly expanded and developing more cost-effective technological solutions are appearing as a challenge for an outdated condition to change.

3 Motivation The specification of the most suitable and appropriate training program for every case, as stated in [17], is a very crucial factor for better performance. Furthermore, the authors are suggesting that personalized training programs adapted to specific athlete’s needs are more probable to succeed. However, there are many difficulties in scheduling an appropriate personalized training program as it is a very hard process to perceive every trainee’s psycho - physical condition, especially in team sports or sport academies. SportSWARES and its features may approve a significant tool to contribute to the specification of the most suitable training program, to monitor the training progress and to support self – regulation and assessment of training. The proposed system could be utilized as advanced equipment of a smart training room where the athlete can be trained using skill development applications interfaced to SportSWARES platform in conjunction with his/her bio-signal measurements (Fig. 1).

Fig. 1. Smart training room

Work in Progress. SportSWARES, Towards an Intelligent Way of Physical Training

511

4 System Design and Architecture SportSWARES platform will rely on research works dealing with bio-signal processing and with skeletal tracking and motion capture such as the bio-feedback system [18] and the instructional mirroring application [19]. The bio-feedback system for anxiety awareness classifies human bio-signals applying a machine learning algorithm and displays real-time anxiety levels in interfaces executed in Windows or Android environments. This system utilizes three biosignals, a) galvanic skin response (GSR), b) skin temperature (SKT) and c) heart rate (HR). Various cognitive and physiological functions of the human body can affect the aforementioned bio-signals. In particular, stressful situations may cause an increment of GSR and HR and a reduction of SKT. Instructional mirroring uses the open source Vitruvius library [20], Microsoft Kinect for Windows SDK 2.0 and Microsoft Kinect sensor V2 (depth camera) in order to detect, analyse and support athlete’s technique while executing free shot in basketball training. This research work was a first approach to construct a supportive tool functioning as a scaffold to athletes’ practice. The core idea of this application is that many athletes use to stand in front of a mirror and practice on an exercise in order to calibrate themselves and improve their technique. Thus, the motivation of this work was to try to take advantage of this real life practice and extend it to a real time virtual mirror which will instruct and support the athlete with corrective signs. SportSWARES will utilize the experience gathered from these aforementioned projects and extend the techniques used by them. This system will collect data from athlete’s bio-signals and movements in real time in order to detect his/her psychological and physical condition. This system is consisting of the following subsystems: • Skeletal tracking and motion detection subsystem utilizing depth cameras: This subsystem will be connected to one or more 3D depth cameras. The information received from every camera can be processed using Vitruvius library in order to apply user skeletal tracking. • Bio- signal collection subsystem utilizing sensors of galvanic skin response (GSR) and heart rate (HR) in wearable form: This subsystem will implement a wireless connection to a device in the form of a wristband. This device will consist of an analog to digital converter (ADC) on a microcontroller board (e.g. Arduino board) connected to sensors collecting galvanic skin response (GSR) and heart rate (HR) bio-signals. Furthermore, this subsystem will collect the time intervals between successive heartbeats called “R-R intervals” in order to calculate heart rate variability (HRV) features. • Bio - signal analysis and classification subsystem: This subsystem will process the received bio-signals applying filtering techniques, normalization and feature extraction. Then a machine learning algorithm will be applied using a pre-trained model in order to classify the bio-signal features into anxiety levels. Moreover, HRV metrics especially standard deviation of normal to normal intervals (SDNN) or root mean square of successive differences at rest (rMSSD) is significantly associated with physical capability of following a training schedule. Autonomic nervous system (ANS) function is significantly related to HRV and stress. Thus,

512

T. Tsiatsos et al.

HRV is frequently used in the athletic world to identify periods of optimal training and to monitor recovery status and any potential overtraining [16]. • Visual and biomechanical analysis subsystem: This subsystem will utilize skeletal tracking and Vitruvius library in order to apply joint detection, joint angles and distance calculations and model user’s movements. • Open access application interface (API) providing affective feedback and biomechanical analytics to a variety of applications like augment reality applications, web applications with sport analytics and exergames: The application interface will provide real-time athlete’s bio-feedback information like anxiety levels and physical capability as well as biomechanical metrics through the implementation of sockets and Web APIs. This subsystem will provide also a REST Web API applying many functions for storing and updating athlete’s measurements in a database (Fig. 2).

Fig. 2. SportsWARES architecture

5 Discussion and Future Steps SportSWARES, up to the moment this article is being written, is at the stage of the development. This platform can form and facilitate a smart training room. In this room valuable information about athlete’s psycho-physical condition and activity will be gathered and it will be provided through the open access API to augmented reality applications or exergames opening-up new prospects on common sport practice and training. Finally, SportSWARES long term evaluation in sports academies, fitness centers or even during house training, will present more safe results of its effectiveness.

Work in Progress. SportSWARES, Towards an Intelligent Way of Physical Training

513

References 1. Katmada, A., Politopoulos, N.: The effect of computer games in physical education and health of children and youth. A literature review. In: 2nd Panhellenic Conference “Education in ICTs” Athens, Greece (2014) 2. Myroniv, B., Wu, C-W., Ren, Y., Christian, A.B., Bajo, E., Tseng, Y.C.: Analyzing user emotions via physiology signals. Data Sci. Pattern Recogn. (2017) 3. Patsantaras, N.: Amhqώpimo rώla, Ahkηsirlό1 jai Utrijή Acxcή rsi1 pqoopsijέ1 sη1 Wηuiajή1 epovή1, Hleqokόcio Epiloquxsijώm Diakέnexm (2015). https://slideplayer. gr/slide/11142751/ 4. Miah, A.: From anti-doping to a ‘performance policy’ sport technology, being human, and doing ethics. Eur. J. Sport Sci. 5, 51–57 (2005) 5. Sheridan, H.: Tennis technologies: de-skilling and re-skilling players and the implications for the game. Sport Soc. 9, 32–50 (2006) 6. Giblin, G., Tor, E., Parrington, L.: The impact of technology on elite sports performance. Sensoria: J. Mind Brain Cult (2016). https://doi.org/10.7790/sa.v12i2.436 7. Dellaserra, C.L., Gao, Y., Ransdell, L.: Use of integrated technology in team sports: a review of opportunities, challenges, and future directions for athletes. J. Strength Conditioning Res. 28(2), 556–573 (2014). https://doi.org/10.1519/jsc.0b013e3182a952fb 8. Cummins, C., Orr, R., O’Connor, H., West, C.: Global positioning systems (GPS) and microtechnology sensors in team sports: a systematic review. Sports Med. 43(10), 1025– 1042 (2013). https://doi.org/10.1007/s40279-013-0069-2 9. Barris, S., Button, C.: A review of vision based motion analysis in sport. Sports Med. 38(12), 1025–1043 (2008). https://doi.org/10.2165/00007256-200838120-00006 10. Wilson, B.D.: Development in video technology for coaching. Sports Technol. 1(1), 34–40 (2008). https://doi.org/10.1002/jst.9 11. Choppin, S., Wheat, J.: The potential of the Microsoft Kinect in sports analysis and biomechanics. Sports Technol. 6(2), 78–85 (2013). https://doi.org/10.1080/19346182.2013.819008 12. Rosenhahn, B., Brox, T., Kersting, U., Smith, A., Gurney, J., Klette, R.: A system for marker-less motion capture. Künstliche Intelligenz 1, 45–51 (2006) 13. Harrison, N., Neumayr, T.: iPad available in nine more countries on May 28. Apple (2010). http:// www.apple.com/pr/library/2010/05/07iPad-Available-in-Nine-More-Countries-on-May-28.html 14. Sigrist, R., Rauter, G., Riener, R., Wolf, P.: Augmented visual, auditory, haptic, and multimodal feedback in motor learning: a review. Psychon. Bull. Rev. 20(1), 21–53 (2012). https://doi.org/10.3758/s13423-012-0333-8 15. Psouni, L.: Ahkηsijή Wtvokocίa. Pύkη Wtvokocίa1 (2009). https://www.psychology.gr/ sports-psychology/368-athlitiki-psychologia.html 16. Aubert, A.E., Seps, B., Beckers, F.: Heart rate variability in athletes. Sports Med. 33(12), 889–919 (2003) 17. Tzetzis, G., Lola, A.: Kimηsijή lάhηrη jai amάpstnη. [ebook] Athens: Greek Academy Libraries Association (2015). http://hdl.handle.net/11419/329 18. Apostolidis, I., Karakostas, A., Dimitriou, T., Tsiatsos, T., Tsolaki, M.: Advanced biofeedback and collaborative techniques to support caregivers of Alzheimer patients. In: Intelligent Networking and Collaborative Systems (INCoS), pp. 683–688. IEEE (2014) 19. Apostolidis, H., Politopoulos, N., Stylianidis, P., Chaldogeridis, A., Stavropoulos, N., Tsiatsos, T.: Instructional mirroring applied in basketball shooting technique. In: Interactive Mobile Communication, Technologies and Learning, pp. 603–611. Springer, Cham, November 2017 20. Lightbuzz: Vitruvius, creating stunning motion apps in minutes (2017). https:// vitruviuskinect.com/. Accessed 19 July 2019

Greek Traditional Dances Capturing and a Kinematic Analysis Approach of the Greek Traditional Dance “Syrtos” (Terpsichore Project) Efthymios Ziagkas1(&), Vasiliki Zilidou1, Andreas Loukovitis1, Styliani Douka1, and Thrasyvoulos Tsiatsos2 1

Department of Physical Education and Sport Science, Aristotle University of Thessaloniki, 57001 Thermi, Thessaloniki, Greece {eziagkas,sdouka}@phed.auth.gr, [email protected], [email protected] 2 School of Informatics, Aristotle University of Thessaloniki, 54124 Thessaloniki, Greece [email protected] Abstract. Intangible Cultural Heritage (ICH) content means “the practices, representations, expressions, knowledge, skills – as well as the instruments, objects, artifacts and cultural spaces associated therewith”. Although ICH content, especially traditional folklore performing arts, is commonly considered worthy of preservation by UNESCO and the EU Treaty, most of the current research efforts are on the focus is on tangible cultural assets. The aim of Terpsichore project is to, analyze, design, research, train, implement and validate an innovative framework for affordable digitization, modelling, archiving, e-preservation and presentation of ICH content related to folk dances, in a wide range of users. The Aristotle University of Thessaloniki was involved in the performance of Greek traditional dances and the three-dimensional capturing and editing. The three-dimensional recordings were held at the Laboratory of “Motor Behaviour and Adapted Physical Activity”. For data capturing we used the Vicon 3D recording system. Two Greek traditional dances from different regions of Greece were performed and captured. The results of the threedimensional capturing of movement showed that the assessment of left toe trajectories through markers gives us the ability to identify the differences between two types of the Greek traditional dance “Syrtos”, “Syrtos Makedonikos” and “Syrtos Kalamatianos”. The findings of this work add a new approach to improve Greek traditional dances identification through 3D motion kinematic data analysis. They represent optical and mathematical representation of the rhythm of Greek traditional dances which could be used for the purposes of Terpsichore project. Keywords: Traditional dances
Human movement analysis cultural heritage
Capturing
Vicon


Intangible

© Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 514–523, 2021. https://doi.org/10.1007/978-3-030-49932-7_49

Greek Traditional Dances Capturing and a Kinematic Analysis Approach

515

1 Introduction Intangible Cultural Heritage (ICH) content means “the practices, representations, expressions, knowledge, skills – as well as the instruments, objects, artifacts and cultural spaces associated therewith”. Intangible Cultural Heritage (ICH) is a very important mainspring of cultural diversity and a guarantee of sustainable development, as underscored in the UNESCO Recommendation on the safeguarding of Traditional Culture and folklore of 1989, in the UNESCO Universal Declaration on Cultural Diversity of 2001 and in the Convention for the Safeguarding of the Intangible Cultural Heritage [1]. Improving the digitization technology regarding capturing, modelling and mathematical representation of performance arts and especially folklore dances is critical in: (i) promoting cultural diversity to the children and the youth through the safeguard of traditional performance arts; (ii) making local communities and especially indigenous people aware of the richness of their intangible heritage; (iii) strengthening cooperation and intercultural dialogue between people, different cultures and countries. Although ICH content, especially traditional folklore performing arts, is commonly considered worthy of preservation by UNESCO and the EU Treaty, most of the current research efforts focus on tangible cultural assets, while the ICH content has been given less emphasis. The primary difficulty stems by the complex structure of ICH, its dynamic nature, the interaction among the objects and the environment, as well as emotional elements (e.g., the way of expression and dancers’ style). Of course, there have been some notable efforts such as the i- Treasures project which provides a platform to access ICH resources and contribute to the transmission of rare know-how from Living Human Treasures to apprentices [2] and the RePlay project, whose goal is to understand, preserve, protect and promote traditional sports [3]. Towards this direction, the Terpsichore project, a European funded research and innovation program from the European Union’s Horizon 2020, aims to study, analyze, design, research, train, implement and validate an innovative framework for affordable digitization, modelling, archiving, preservation and presentation of ICH content related to folk dances, in a wide range of users (dance professionals, dance teachers, creative industries and general public). The main goal of the Terpsichore project is to leverage such past efforts towards the transformation of intangible cultural heritage content to 3D virtual content, through the development and exploitation of affordable digitization technology. Towards this direction, fusion of different scientific and technological fields, such as capturing technology, computer vision and learning, 3D modeling and reconstruction, virtual reality, computer graphics and data aggregation for metadata extraction, is necessary. Exploring the digitization technology regarding folklore performances constitutes a significant impact at European level. On one hand, the multi-cultural intangible dimension of Europe is documented, preserved, made available to everybody on the Internet. On the other hand, the multifaceted value to the ICH content for usage in education, tourism, art, media, science and leisure settings are added. Currently, digital technology has been widely adopted, which greatly accelerates efforts and efficiency of Cultural Heritage (CH) preservation and protection. At the same time, it enhances CH in the digital era, creating enriched virtual surrogates. Many research works have been

516

E. Ziagkas et al.

proposed in the literature on archiving tangible cultural assets in the form of digital content [4]. Although the aforementioned significant achievements for improving the digitization technology towards a more cost-effective automated and semantically enriched representation, protection, presentation and re-use of the CH via the European Digital Library EUROPEANA, very few efforts exist in creating breakthrough digitization technology, improving the e-documentation (3D modelling enriched with multimedia metadata and ontologies), the preservation and re-use of ICH traditional music and fashion, folklore, handicraft, etc. 3D modelling relies on computational geometry techniques such as skeleton extraction, division of space into subspaces and mesh reconstruction. Authors has presented an approach to recover body motions from multiple views using a 3D skeletal model, which is an a priori articulated model consists in kinematic chain of segments representing a body pose [5]. 3D rendering is a necessary process for visualizing modeled content. However, real-time rendering of detailed animated characters, especially in crowded simulations like dance, is a challenging problem in computer graphics. Textured polygonal meshes provide high-quality representation at the expense of a high rendering cost. To overcome this problem, several techniques focusing on providing level-of-detail representations have been proposed. Image-based pre-computed impostors [6] render distant characters as a textured polygon to accelerate rendering of animated characters. A much more memory-efficient but viewdependent approach is to subdivide each animated character into a collection of pieces, in order to use separate impostors for different body parts [7]. In a three-level-of-detail approach is described, combining the animation quality of dynamic meshes with the high performance offered by static meshes and impostors [8]. One other technique in adopts a relief mapping approach to encode details in arbitrary 3D models with minimal supporting geometry [9]. Terpsichore targets at integrating the latest innovative results of photogrammetry, computer vision, semantic technologies, and time evolved modeling, combined with the storytelling and folklore choreography. An important output of the project will be a Web-based cultural server/viewer with the purpose to allow user’s interaction, visualization, interface with existing cultural libraries and enrichment functionalities to result in virtual surrogates and media application scenarios that release the potential economic impact of ICH. The final product will support a set of services such as virtual/augmented reality, social media, interactive maps, presentation and learning of European Folk dances with significant impact on the European society, culture and tourism. Greek traditional dances (national dances) have been for centuries one of the most valuable and most expressive cultural assets of Greek society. During the long path of the Greek nation, the side of traditional culture contributed decisively to maintaining of national identity of Greek nation, especially the difficult moments, as was the main means of expression of social and also, to a certain extent, of religious events Greek population. In traditional society, in the context of customary life, there were many opportunities for dancing [10]. In Greece there is a large number of dances which is characterized by diversity. During the years, every place created its own dance repertoire influenced by the particular way of life. The local citizens used to express and communicate to each other, so

Greek Traditional Dances Capturing and a Kinematic Analysis Approach

517

that they have managed not to lose their roots and the connection with their progenitors [11, 12]. Syrtos (Makedonikos) is one of the most famous folklore dances in the whole Greece and one of the most basic dances of local repertoires. It is danced by men and women in an open circle. The basic pattern of dance performed in twelve movements. The hand grip is with hands tied by palms, in W position, and sometimes is customary above the hand positions exist only between the first and second, while the rest have their hands tied by palms down. All over Greece there are a lot of different songs accompanied the dance. The musical tempo is at 7/8 beat. It is discerned into seven main choreographic steps: Initial Posture (IP), Left Leg Back (LLB), Cross Legs (CL), Cross Legs (CL), Cross Legs (CL), Initial Posture (IP), Right Leg Back (RLB) [11]. Syrtos Kalamatianos is one of the most popular Greek folklore dances and it is danced in the south area of Greece, Peloponnese. It is danced in an open circle by men and women. Its basic dance motif is performed in twelve movements, with intense movements by men and softer by the women. The first dancer could perform different dancing figures. The dancers are holding by hands by the palms in the W position (sometimes this hand position is performed only by the first and the second dancer, while the rest of the dancers have their hands lowered from the palms). In all of Greece there are many different songs that accompanied Syrtos Kalamatianos. The music tempo is at 7/8 beat. Syrtos Kalamatianos is also discerned into seven main choreographic steps which are the followings: Initial Posture (IP), Cross Legs (CL), Cross Legs (CL), Cross Legs (CL), Cross Legs (CL), Initial Posture (IP), Cross Legs Backwards (CLB) [11]. In a previous work we have proposed a method for Greek Traditional dances through rhythm pattern analysis of dancing steps [13]. Although, we manage to identify other Greek traditional dances, due to the same rhythm pattern of “Syrtos Makedonikos” and “Syrtos kalamatianos”, however the proposed method was unable to identify the differences between those two dances. The aim of this paper is to describe Terpsichore objects and scopes as well as to present a new method to improve Greek traditional dances identification through 3d motion kinematic data analysis.

2 Methods 2.1

Participants

At the recordings participated four professional dancers of traditional dances, two men and two women who were asked to perform, one by one, the steps of Syrtos Makedonikos and Syrtos Kalamatianos dances, guided at first by counting the rhythm orally and then, by the proper music for each dance. All performances and captures took place in the Laboratory of Motor Behaviour and Adapted Physical Activity at the Department of Physical education and Sport Sciences at Aristotle University of Thessaloniki. All four participants were chosen because of their expertise in the performance and didactics in Greek traditional dances and their previous experience in human motion capturing process.

518

2.2

E. Ziagkas et al.

Instruments

In the Laboratory of Motor Behaviour and Adapted Physical Activity we performed the Greek traditional dance capturing using the VICON system (Nexus Vicon, Oxford, UK). This system consists of 10 high precision and sampling camcorders to record human motion (Bonita 3, Nexus Vicon, Oxford, UK). Motion capturing systems are widely used in biomechanics sports, computer graphics and computer animation. The effectiveness of motion capturing systems depending on their system setup and are sensitive against variations. Marker properties, optical projections, video-digital conversion, camera configuration, lens distortion, calibration procedures. A set of spherical reflective markers are attached to the research object, in our case is the dancer. The reflective markers are tracked by a number of grey scale cameras which are placed around the research area and via the Vicon software is calculating and calibrating the 3D position for each reflective marker. The Vicon system consists both hardware and software components. The hardware includes 10 high precision and sampling camcorders to record human motion, and an analogue data acquisition module. The software includes Vicon Workstation that collects and processes the motion and analogue data. We also captured the performance of Greek traditional dances via a video camera. 2.3

Procedure

We captured dancing trials from four different dancers performing several parts of traditional Greek dances. Dancers performed and we captured “Syrtos Makedonikos”, “Syrtos Kalamatianos. After collecting somatometric data of the dancers, we calibrated the Vicon system capturing area and started capturing. The capturing procedure lasted about an hour for each dancer. The capturing frequency of the Vicon system was set to 100 Hz. For the 3d video capture we used the PIG (plug in gait full body) market placement using 36 reflective markers placed on specific anatomical points of the body of dancers. Before capturing the dancers, we performed a static subject calibration (Fig. 1). After the 3d video capturing, we labelled reflective markers based on the full body plug in gait model and exported data concerning, segments, joints and trajectories of each market. After the post capturing data processing we exported a c3d format type file and a txt format type file containing those entire variables for further analysis. In order to synchronise the 3d video capture and the digital video capturing we synchronised the two devices clocks.

Greek Traditional Dances Capturing and a Kinematic Analysis Approach

519

Fig. 1. The Vicon motion capturing system topology Table 1. The seven main choreographic steps of the Greek traditional dance “Syrtos Makedonikos”. Syrtos Makedonikos

Initial Posture (IP)

Left Leg Back (LLB)

Initial Posture (IP)

Right Leg Back (RLB)

Cross Legs (CL)

Cross Legs (CL)

Cross Legs (CL)

3 Results After editing the captured content, we selected the seven main choreographic steps of each Greek traditional dance. Using the synchronized c3d data from Vicon and the video from the camcorder we created two tables, one for each dance. Each table present snapshots of dances time series from both Vicon and digital camcorder (Table 1 and Table 2).

520

E. Ziagkas et al.

Table 2. The seven main choreographic steps of the Greek traditional dance “Syrtos Kalamatianos”. Syrtos Kalamatianos

Initial Posture (IP)

Cross Legs (CL)

Initial Posture (IP)

Cross Legs Backwards (CLB)

Cross Legs (CL)

Cross Legs (CL)

Cross Legs (CL)

As may be seemed from the tables, the only differences between “Syrtos Makedonikos” and “Syrtos kalamatianos” exist on the second and on the last main choreographic steps. From kinematic respect, the trajectory of the left toe’s reflective marker represents those differences between “Syrtos Makedonikos” and “Syrtos kalamatianos” on the second and on the last main choreographic steps. More specifically, on the second main choreographic step (frames 38–128) the left toe’s reflective marker mean displacement on the y axis was 1526,07 ± 61,28 for “Syrtos Makedonikos” dance and 1306,19 ± 22,70 for “Syrtos Kalamatianos” dance. On the last main choreographic step (frames570-629) the left toe’s reflective marker mean displacement on the y axis was 1519 ± 19,89 for “Syrtos Makedonikos” dance and 1574,98 ± 14,93 for “Syrtos Kalamatianos” dance. Data concerning the displacement of left toe on the y axis for “Syrtos Makedonikos” and “Syrtos kalamatianos” are presented in Fig. 2. As shown in Fig. 2, the two dances have the same rhythmic pattern, but the displacement for each of them is represented by the axis y.

Greek Traditional Dances Capturing and a Kinematic Analysis Approach

1700

521

The differences of toe markers trajectories on y axis between "Sirtos Kalama anos" and "Sirtos Makedonikos"

Y axis trajectoty

1600 1500 Syrtos Makedonikos

1400 1300

Syrtos KalamaƟanos

1200 1100 1 34 67 100 133 166 199 232 265 298 331 364 397 430 463 496 529 562 595

1000

Fig. 2. The differences of toe markers trajectories on y axis between “Syrtos Kalamatianos” and “Syrtos Makedonikos”

4 Discussion The aim of this paper is to describe Terpsichore objects and scopes and to further present a new method to improve Greek traditional dances identification through 3d motion kinematic data analysis. In addition to our previous work [13], the findings of this paper add a new approach to improve Greek traditional dances identification through 3d motion kinematic data analysis. This new 3d motion kinematic data analysis show that analysing the displacement of the left toe reflective marker trajectory is an ideal tool in order to identify the differences between dances of the same rhythm pattern such as “Syrtos Kalamatianos” and “Syrtos Makedonikos”. 3D modeling is critical for encoding the complex 3D reconstructions of performing arts into a set of compact semantic signatures in a similar way that a music song is encoded using a music score. For this reason, computational geometry algorithms are used to decode the spatial-temporal trajectory of the performances. This includes methodologies for positioning both in 3D and temporal space. Then, semantic signatures and respective spatial-temporal associations are extracted to represent the performances with high level concepts. It is clear that semantic analysis aids the digitization and computer vision process and vice versa. The visualization of scores fosters the understanding of the dance and it helps visitors, dancers and choreographs to comprehend the structure and the intension of the dance. A more formalized documentation of the dance will be supported with an automated mapping of capturing data to formal and abstract chorographic notations (“Symbolic Representations”) [14]. The main goal of the Terpsichore project is to transform the intangible cultural heritage content to 3D virtual content, through the development and exploitation of the state of art digitization technology. Towards this direction, fusion of different scientific

522

E. Ziagkas et al.

and technological fields, such as 3D capturing technology modeling and reconstruction, virtual reality, computer graphics and data aggregation for metadata extraction, have to be used [14]. The findings of this paper represent and optical and mathematical representation of the rhythm of Greek traditional dances which can be used for the purposes of Terpsichore project.

5 Conclusion The results of the three-dimensional capturing of movement showed that the assessment of left toe reflective marker trajectory displacement gives us another useful tool in order to improve the ability to identify Greek traditional dances. Finally, data from the recordings were distributed to the program partners in order to further analyze them for the fulfillment of the project objectives. With the completion of the program, a set of services will be created including: virtual and augmented reality applications, interactive maps, presentation and learning of European traditional dances with a huge impact on European society culture and tourism. In this paper we have presented the concept of the Terpsichore project. Through the described approach Terpsichore aims to study, analyze, design, research, train, implement and validate an innovative framework for affordable digitization, modeling, archiving, e-preservation and presentation of Intangible Cultural Heritage content related to folk dances, in a wide range of users, including dance professionals, dance teachers, creative industries and the general public. Acknowledgements. This work has been supported by the H2020-MSCARISE project “Transforming Intangible Folkloric Performing Arts into Tangible Choreographic Digital Objects (Terpsichore)” funded by the European Commission under grant agreement no 691218. The authors would like to help all partners for their contribution and collaboration.

References 1. Kyriakaki, G., Doulamis, A., Doulamis, N., Ioannides, M., Makantasis, K., Protopapadakis, E., Hadjiprocopis, A., Wenzel, K., Fritsch, D., Klein, M., Weinlinger, G.: 4D reconstruction of tangible cultural heritage objects from web-retrieved images. Int. J. Heritage Digit. Era 3, 431–452 (2014). https://doi.org/10.1260/2047-4970.3.2.431 2. Dimitropoulos, K., Barmpoutis, P., Kitsikidis, A., Grammalidis, N.: Extracting dynamics from multidimensional time-evolving data using a bag of higherorder linear dynamical systems. In: 11th International Conference on Computer Vision Theory and Applications (VISAPP 2016), Rome, Italy (2016) 3. Linaza, M., Moran, K., O’Connor, N.E.: Traditional sports and games: a new opportunity for personalized access to cultural heritage. In: 6th International Workshop on Personalized Access to Cultural Heritage (PATCH 2013), Rome, Italy (2013) 4. Li, R., Luo, T., Zha, H.: 3D digitization and its applications in cultural heritage. In: Ioannides, M., Fellner, D., Georgopoulos, A., Hadjimitsis, D.G. (eds.) EuroMed 2010. LNCS, vol. 6436, pp. 381–388. Springer, Heidelberg (2010)

Greek Traditional Dances Capturing and a Kinematic Analysis Approach

523

5. Menier, C., Boyer, E., Raffin, B.: 3D skeleton- based body pose recovery. In: Proceedings of the Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT 2006), Washington, DC, USA, pp 389–396 (2006) 6. Tecchia, F., Loscos, C., Chrysanthou, Y.: Image-based crowd rendering. IEEE Comput. Graph. Appl. 22, 36–43 (2002) 7. Kavan, L., Dobbyn, S., Collins, S., Zára, J., O’Sullivan, C.: Polypostors: 2D polygonal impostors for 3D crowds. In: Proceedings of the 2008 Symposium on Interactive 3D Graphics and Games, New York, NY, USA, pp 149–155 (2008) 8. Pettré, J., Ciechomski, P., Maïm, J., Yersin, B., Laumond, J., Thalmann, D.: Real-time navigating crowds: scalable simulation and rendering. Comput. Animation Virtual Worlds 17, 445–455 (2006) 9. Andújar, C., Boo, J., Brunet, P., Fairén, M., Navazo, I., Vázquez, P., Vinacua, À.: Omnidirectional Relief Impostors. Comput. Graph. Forum 26, 553–560 (2007). https://doi.org/10. 1111/j.1467-8659.2007.01078.x 10. Douka, S., Vavritsas, N., Anastasios, A., Papadopoulos, P.: Common Elements and Differences of Ancient and Greek Traditional Dance, Athletic History and Philosophy V, Thessaloniki, pp. 191-202 (2012) 11. Douka, S.: Educational material for Lifelong Learning, in Greek Traditional Dance. Aristotle University of Thessaloniki, Department of PE and Sport Science (2014) 12. Douka, S.: The Character of Dance in Classical Antiquity, Thessaloniki: Department of PE and Sport Science, Aristotle University of Thessaloniki (1998) 13. Ziagkas, E., Stylianidis, P., Loukovitis, A., Zilidou, V., Lilou, O., Mavropoulou, A., Tsiatsos, T., Douka, S.: Greek traditional dances 3D motion capturing and a proposed method for identification through rhythm pattern analyses (Terpsichore Project). In: International Conference on Strategic Innovative Marketing and Tourism, ICSIMAT 2019, 17–21 July 2019 14. Doulamis, A., Voulodimos, A., Doulamis, N., Soile, S., Lampropoulos, A.: Transforming intangible folkloric performing arts into tangible choreographic digital objects: the terpsichore approach, pp 451–460 (2017). https://doi.org/10.5220/0006347304510460

Game Based Learning

Educational Mobile Applications on Computational Thinking and Programming for Children Under 8 Years Old George Terzopoulos(&), Maya Satratzemi, and Despina Tsompanoudi University of Macedonia, Thessaloniki, Greece {gterzopoulos,maya,despinats}@uom.edu.gr

Abstract. Mobile technology has the ability to change the way education incorporates computers in the teaching process. Mobile devices have distinct features compared to computers, such as portability, touchscreen and a variety of sensors that makes them more suited for young children. Computational thinking is an essential skill in teaching children how to code but it is also a universal problem-solving technique. In this paper we present mobile applications as educational resources that teach young children under 8 years old how to code and advance their computational thinking abilities. We also study whether these applications have proven their educational effectiveness in the classroom based on related scientific research. Finally, we propose characteristics that should be incorporated in mobile applications in order to improve their educational value. Keywords: Mobile applications Programming


Computational thinking
Tablets


1 Introduction In the years to come, computer programming will be considered an equally important and useful skill as reading and writing and a new literacy for the 21st century [1]. The number of devices that can be programmed is constantly rising and our everyday life is overwhelmed by smart devices that execute some sort of code. Although demand is high for skilled staff in computer programming and in the Information Technology (IT) sector in general, there is not enough qualified personnel to cover the jobs that are being created. According to the US Department of Labor, by 2020 there will be 1.4 million new jobs in the IT industry [2]. Additionally, the employment-oriented service, LinkedIn, reports that for 2019 the most sought-after skills will be related to software development and technology [3]. While this gap is being created in the workplace, it is striking that only 8% of Science Technology Engineering Mathematics (STEM) students choose to study in the computer industry [4]. The lack of interest among the candidate students in the IT sector is due to many factors and starts from the early school years. In most countries of the world, only a small proportion of students in primary and secondary education learn how to code [5]. Computer programming interventions has shown that children as © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 527–538, 2021. https://doi.org/10.1007/978-3-030-49932-7_50

528

G. Terzopoulos et al.

young as 5 years old can understand the programming concepts of sequencing, logical ordering, cause-and-effect relationships [6] and basic concepts of programming [7]. At the same time, in recent years, there has been a plethora of tablet-based educational applications to introduce programming to children. This is due to the fact that children are more familiar with them. According to a study of 350 children aged 6 months to 4 years at a pediatric clinic in an urban, low-income, minority community in the US, almost all children (96.6%) used mobile devices, and most started using before they reach the age of 1 year [8]. This study aims to explore educational mobile applications that teach coding skills to kindergarten and early elementary children (4–8 years old), since according to McCoy et al. [9], early interventions can have compounding effects later in life and influence personal and academic outcomes. Furthermore, in order to depict the educational value of each application, related studies that have evaluated the above applications in the classroom are presented. Our specific research questions (RQ) for the study are as follows: RQ 1: Which mobile applications for teaching programming are available for children under 8 years old and what are their main characteristics? RQ 2: Which of the above mobile applications have been evaluated in a classroom environment? The remainder of this article is organized as follows: Background information about programming and mobile devices is given in the next section. Then, the methodological approach used for finding relative educational mobile applications and research papers is depicted. Next, the characteristics of mobile applications for programming are presented, which are followed by the educational value of the above mobile applications based on related studies. Finally, an interpretation of the findings of this study follows, and recommendation of new areas for future research.

2 Background Mobile devices can enhance educational effects when used in the classroom since the overall effect of using mobile devices is better than when using desktop computers or not using mobile devices at all [10, 11]. Geist [12] studied the use of iPads by 20 2–3 year-olds and found that the touchscreen operation was more intuitive than using a mouse and physical keyboard. Couse and Chen [13] integrated tablets to 3–6 year-old children’s classroom and found that their engagement with the drawing activity assigned was surprisingly high. Fallon [14] evaluated 40 educational mobile applications with 5 year-old children and found that productive learning occurs when applications provide communicating learning objectives in ways children can access and understand. Thus, mobile devices can easily be used by young children. The theory of constructivism [15] argues that people produce knowledge and form meaning based upon their experiences and that children think differently than adults, hence they need the opportunity to construct their own knowledge. The theory suggests that children build (construct) their own knowledge through experience rather than being taught by teachers. Papert [16], applied the theory of constructivism to the emerging field of children’s computer programming. He used the term “computational

Educational Mobile Applications on Computational Thinking

529

thinking” (CT) and described the relevance of CT to various aspects of everyday life. Papert created the computer programming language “LOGO” specifically for use by children in order to teach programming concepts. According to Wing [17], computational thinking is a problem-solving approach that involves decomposing problems and using algorithms to solve problems. Programming allows children to think about concepts of sequencing, order, and logic in approachable ways, and to apply these foundational concepts to any technology they encounter in their world [18]. Tablets can be useful in education. Huber et al. [19] concluded that 4 to 6-year-old children are capable of improving at problem-solving tasks through practice on touchscreen devices. Oladunjoye [20] studied the usage of iPads by children 3 to 6.5 years old. The result of the study was that the teachers’ positive attitude towards the use of iPad helped to enhance and facilitate the development of literacy skills in the children. Additionally, the use of iPad promoted social interaction amongst the children. There are many apps that are designed to teach coding skills to young children, although research on the effectiveness of these applications on preschool and early elementary children is limited. Children at this age, can create and study third party ready-made programs in a way that demonstrates a deep understanding of the basic programming concepts according to the study of [21].

3 Methodology The first step of the research was to locate educational mobile applications that teach programming. The following resources were used: • Official online application stores for mobile applications: Microsoft, Google and Apple stores. • Websites with catalogues and reviews of educational mobile applications: Educational App Store [22] and Common Sense Media [23] Although numerous mobile applications for coding were found, only 19 of them satisfied the following criteria: • Targeted at children younger than 8 years old • Have been updated in the last 5 years • Are not associated with any hardware (i.e. robots) The following mobile applications were excluded from the study since they did not meet the above criteria: Coda Game and Hopscotch: Coding for Kids is targeted at children older than 9 years old, while LEGO® MINDSTORMS® Fix Factory and Dynamic ART Lite are suitable for children older than 8. Coddy luck and Surf Score are found in the literature as educational mobile applications, but they are now deleted from the available stores. TouchLogo has not been updated in the last 5 years, while Blue-Bot, Dash & Dot, Cozmo + Code Lab, Thymio Robot, meeperBOTS, COJI and Cubelets robots, are associated with hardware so they were excluded from the study.

530

G. Terzopoulos et al.

The second step of the research was to retrieve sufficient and high-quality papers regarding educational mobile applications and their usage in the classroom. The snowball technique was used as described in [24]. The technique has the following steps: • First, perform a search in Google Scholar, IEEE Xplore, Scopus and gather the initial start set of relevant papers • For the initial start set of papers, iterate through backward and forward snowballing. Backward snowballing uses the reference list to identify new papers to include. Forward snowballing refers to identifying new papers that cite the paper being examined. Using backward and forward snowballing, new papers that are identified in the iteration are put into a pile to go into the next iteration. In this study, papers that were not in English language and low-quality papers were excluded. Compared to other older surveys [25, 26], we have included up to date information about the mobile applications and presented studies that have evaluated them in the classroom. We also identified the supported computational concepts of the mobile applications using the 7 computational thinking concepts as stated in [27]: • • • • • • •

Concept 1. Sequence: identify a series of steps in order to complete an activity Concept 2. Loops: Running the same sequence multiple times Concept 3. Events: One thing causing another thing to occur Concept 4. Parallelism: Instructions happening at the same time Concept 5. Conditionals: The ability to make decisions based on certain conditions Concept 6. Operators: Support for mathematical, logical, and string expressions Concept 7. Data: Storing, retrieving, and updating values

4 Mobile Applications for Programming Table 1 summarizes basic characteristics for each mobile application presented in this study such as platform availability, last update date, price, targeted age and computational thinking concepts available. Table 1. Reviewed mobile applications Application

Platform (Updated)

Price

Age Computational thinking Concepts

A.L.E.X. [28]

IOS (2/10/2017) Android (13/9/2014) IOS (21/1/2019) Windows (12/4/2016) IOS (25/9/2017)

Free with in-app purchases

4+

1

Free

6+

1, 2, 5

Free Lite version, Paid full version

5+

1, 2,5

Cargo-Bot [29]

Cato’s hike [30]

(continued)

Educational Mobile Applications on Computational Thinking Table 1. (continued) Application

Platform (Updated)

Code Karts - Pre-coding IOS (22/1/2019) logic [31] Android (19/2/2019) Codeable Crafts [32] IOS (6/12/2017) Android (9/7/2015) CodeSpark Academy IOS (4/6/2019) with The Foos [33] Android (4/6/2019) Daisy the Dinosaur [34] IOS (2/11/2016) Hopster Coding Safari IOS (9/2/2019) for kids [35] Kidlo Coding for kids IOS (3/5/2019) [36] Android (11/4/2019) Kodable [37] IOS (28/6/2019) Lightbot Jr: Coding Puzzles [38] Move the turtle [39] Robot School. Programming for kids [40] Robozzle [41]

Run Marco! [42]

ScratchJr [43]

Think & Learn Code-apillar [44]

Tommy the Turtle, Learn to Code: Kids Coding [45] Tynker [46]

Price

Age Computational thinking Concepts

Free with in-app purchases

4+

1, 5

Free

: wk ¼ thi thi1 . . .th 2 C

ð2Þ

In T, these subsequences can follow each other in any order. Thus, we can represent the input sequence in the following way: T ¼ w1 w2 . . .wk

ð3Þ

758

I. Dychka et al.

where wi ¼ t1 t2 . . .tn is considered as a n-digits vector in the notation PA and k is a number of subsequences (2) in T. In order to compress the input data represented as the sequence, (3) is transformed into a new sequence U, which consists of subsequences ui ¼ c1 c2 . . .cm as follows: U ¼ u1 u2 . . .uj

ð4Þ

where j is a number of subsequences ui in U. In other words, compression results from transforming n-digits vector wi in the notation PA into m-digits vector ui in the notation PXinf :   nðPA Þ ! m PXinf

ð5Þ

where n [ m. Compression rate is calculated as a ratio of input data sequence length to compressed data sequence length and is called a compression coefficient: nlog3 PA ½ ðsÞ  UPX ðPA Þ ¼  inf m log3 PXinf

ð6Þ

The proposed method of data compression can be used for random symbol sequences input using a keyboard. Information will be represented in a form of 2DBsymbol of smaller area, which ensures its space-saving placement on a carrier.

6 Discussion and Results The developed application implements a user interface shown on Fig. 2. We suppose the following use case for this application: on the stage of exhibition preparation a virtual objects location map is created and information about the virtual objects is collected. This information is presented as high-density tricolor barcode markers obtained by using the proposed method explained in the previous section. Then these markers are linked to the objects according to the exhibition map. Let us consider an example of creating a barcode marker for an AR exhibition where the Large Hadron Collider is presented as AR exhibit. The information about this exhibit is an input textual information consisted of 82 alphanumeric characters, which are targeted to be represented in the form of 2DB-symbol: THE LARGE HADRON COLLIDER WAS BEING BUILT DURING 1998-2008. IT LIES IN A TUNNEL 27 km LONG.

In this textual string, we consider 8 subsequences, among which both 6 subsequences of 12 characters and one incomplete subsequence of 8 characters belong to the set L, and one incomplete subsequence of 10 characters belongs to the set D. 12-digits and 8-digits subsequences consists of elements of the textual alphabet AL , and 10-digits subsequences includes elements of the numeral alphabet AD .

Augmented Reality Application Based on Information Barcoding

759

Fig. 2. Application user interface

Thus, there are two types of transformation (5): • transformation “12” ! “7” for textual subsequences • transformation “16” ! “7” for numeral subsequence. These transformations are conditioned by the cardinality of chosen alphabets and the compression coefficient [20]. Due to incompleteness of some subsequence, in practice, the transformation becomes “8” ! “5” for textual subsequence and “10” ! “4” for numeral subsequence. As we need to switch between 2 alphabets, AL and AD , 2 switcher symbols has to be inserted in order to switch from the textual alphabet to the numeral alphabet and vice versa. Accordingly, a sequence of ternary subsequences which represent 53 barcode patterns, both informational (textual data) and auxiliary (switcher symbols between alphabets), is obtained as follows: 0020110 2110112 1101211 0201200 0101021 0100102

2020100 1022222 2122111 0120010 1102220 2201220 0001022 2000022 0220120 0010110 2222011 0112220 0201110 0110010 1110022 2000202 2001022 2010011 2220222 0220220 2212110 2201010 2120122 0122012 2020200 0010210 2022222 0111020 2020100 0000020 0020000 0020000 0210021 1102012 0120120 0010102 0011000 2000000 2222002 1020122 0012121 0022222

0211001 0020022 2012012 2100100 0222220

760

I. Dychka et al.

In order to result in a 2DB-symbol, which can be used by the application, each of 53 ternary sequences is corresponded to a barcode pattern consisted of 7 tricolor cells. Since a barcode symbol is required to have a rectangle shape, three more barcode patterns, which represent a Pad symbol, i.e. a placeholder, are added to 53 barcode patterns of the barcode symbol. The final 2DB-symbol of the required information is presented at Fig. 3. It consists of 392 tricolor cells considering that there are 7  56 barcode patterns. The dimension of the barcode symbol is 28  14 cells.

Fig. 3. The 2DB-symbol presenting information about an exhibit

Thus, the resulting 2DB-symbol contains general background information about the object being a part of virtual museum exposition. Accordingly, it can be used as a marker for the developed augmented reality application.

7 Conclusion The proposed AR application based on information barcoding allows us to achieve two goals, namely, to generate markers for AR objects location and orientation and to store information about these objects in their AR markers. To store more information in each marker, we propose a tricolor barcoding method, which allows us to increase information representation density up to 25% [20] depending on chosen parameters and alphabets. For handling AR objects and their visualization, we use the algorithm of object orientation using a gyroscope the and algorithm for scaling the map with gestures, which are also presented in this paper. Further development can be focused on both improvement of objects shading algorithms and advancing a marker by adding resistance to interference and including additional auxiliary information in order to ensure quick and error-free scanning.

Augmented Reality Application Based on Information Barcoding

761

References 1. Billinghurst, M., Clark, A., Lee, G.: A survey of augmented reality. Found. Trends® Hum.Comput. Interact. 8(2–3), 73-272 (2015) 2. Three different types of AR explained: marker-based, markerless & location (2018). https:// www.blippar.com/blog/2018/08/14/marker-based-markerless-or-location-based-ar-differenttypes-of-ar 3. Mekni, M., Lemieux, A.: Augmented reality: applications, challenges and future trends. Appl. Comput. Sci. 20, 205–214 (2014) 4. Bertolami, J., Mann, S.A., Bronder, M.L., Dougherty, M.A., Craig, R.M., Lee, M.W.: U.S. patent no. 8,839,121. U.S. Patent and Trademark Office, Washington, DC (2014) 5. Bacca, J., Baldiris, S., Fabregat, R., Graf, S.: Augmented reality trends in education: a systematic review of research and applications (2014) 6. Qiao, X., Ren, P., Dustdar, S., Liu, L., Ma, H., Chen, J.: Web AR: a promising future for mobile augmented reality—state of the art, challenges, and insights. Proc. IEEE 107(4), 651–666 (2019) 7. Shea, R., Fu, D., Sun, A., Cai, C., Ma, X., Fan, X., Liu, J.: Location-based augmented reality with pervasive smartphone sensors: inside and beyond Pokemon Go! IEEE Access 5, 9619– 9631 (2017) 8. Grillo, A., et al.: High capacity colored two dimensional codes. In: Proceedings of the International Multiconference Computer Science and Information Technology, 18–20 October 2010, pp. 709–716 (2010) 9. High Capacity Color Barcodes (HCCB). http://research.microsoft.com/en-us/projects/hccb/ 10. Querini, M., et al.: 2D color barcodes for mobile phones. Int. J. Comput. Sci. Appl. 8(1), 136–155 (2011) 11. Bulan, O., et al.: High capacity color barcodes using dot orientation and color separability. In: Media Forensics and Security, vol. 7254 (2009). https://doi.org/10.1117/12.807742 12. Cattrone, P.: Two-dimensional color barcode and method of generating and decoding the same. U.S. Patent 7 478 746, 20 January 2009 13. Bagherinia, H., Manduchi, R.: A theory of color barcodes. In: IEEE International Conference on Computer Vision Workshops, 6–13 November 2011, pp. 806–813 (2011). https://doi.org/ 10.1109/iccvw.2011.6130335 14. Bulan, O., et al.: Method for encoding and decoding data in a color barcode pattern. U.S. Patent 8 100 330, 24 January 2012 15. Parikh, D., Jancke, G.: Localization and segmentation of a 2D high capacity color barcode. In: IEEE Workshop on Applications of Computer Vision, 7–9 January 2008, pp. 1–6 (2008). https://doi.org/10.1109/wacv.2008.4544033 16. Barrus, J., Wolff, G.J.: Embedding barcode data in an auxiliary field of an image file. U.S. Patent 7 150 399, 19 December 2006 17. Industrial Augmented Reality: Vuforia. PTC. https://www.ptc.com/en/products/augmentedreality 18. Apple Developer Documentation. https://developer.apple.com/documentation/ 19. ARCore overview. https://developers.google.com/ar/discover/ 20. Onai, M.V., Sulema, O.K., Dychka, A.I.: Data encoding based on tricolor matrix barcodes. KPI Sci. News 2(2), 37–45 (2019) 21. Dychka, I., Sulema, O.: Data compression and representation as multicolor barcodes. In: Proceedings of 15th International Conference on ICT in Education, Research and Industrial Applications. Integration, Harmonization and Knowledge Transfer (ICTERI 2019) (2019)

Mobile Health Care and Training

Work in Progress: The Impact of the Project OnBoardMed on Development of Study Courses in Maritime Emergency Management The Results of the Pilotproject Inese Barbare(&) Rigas Technical University, Riga, Latvia [email protected]

Abstract. Non-technical skills are a set of human cognitive and social skills which are integrated and used along side with technical skills. T In each of these fields of work it is important to define the most valuable non-technical skills for a specific profession and how to further develop this set of skills during early studies and the learning process as a whole. The project OnBoard-Med aims at developing more aligned vocational education programmes in the Central Baltic area. The shipping industry has become multinational, therefore recognizing problems in education cannot easily be solved on a national level. A cross-border approach is needed to tackle the issue. The project develops study courses in maritime emergency management, medical treatment and occupational safety. education institutions. Blendedlearning tools include face-to-face actions, e-learning courses, simulations and, of course, mobile communication, especially by emergency simulations (maritime emergency management, medical communication by accidents on sea etc.). The project results in harmonized and improved education and non-technical skills. Participants get excellent professional skills by using blended learning tools - combining lessons on e-learning platform OPTIMA and simulations faceto-face. Further, the results will help local employees to enter the European and international labor market. Keywords: Maritime education INTERREG CBS project


Non-technical skills
E-learning


1 Introduction 1.1

Non-technical Skills for Emergency Industry Needs

Non-technical skills are a set of human cognitive and social skills which are integrated and used along side with technical skills. Non-technical skills manifest themselves during routine procedures and non-standard situations [4]. The analysis of accidents and routine work helps to assess how nontechnical skills affect working environment and people [3]. The term “non-technical skills” was first discussed in the aviation © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 765–772, 2021. https://doi.org/10.1007/978-3-030-49932-7_71

766

I. Barbare

industry in the 60s and 70s as part of investigating plane crashes and other accidents [13]. As a result it was decided that non-technical skills along with technical skills is an important part of decision-making, communication and teamwork [9]. The aim of the article is to compare the non-technical skill level of current ship captains and upcoming officers in charge of navigational watch (students) in order to determine the themes in need of improvement for the overall study process. Improving different themes in both early and late years of studies will further help current students develop the necessary non-technical skills for their future profession in the maritime industry. 1.2

Non-technical Skills in Modern Pedagogy in Maritime Education

The explanation of non-technical skills in modern pedagogy requires a new approach. It is important to understand not only the nature of these skills, but also to see the way of obtaining them. These problems are associated with the integration of new knowledge and skills, innovations in the field of human cooperation pedagogy and the actualization of competence training during studies. It is ever so important to establish a balance between acquiring knowledge to perform certain practical activities and learning to actually think and assess the situation. The modern professional quality of education is more focused on the acquisition of specific information and the ability to put it into practice, which is how it is being done so far, as well as personality self-development and the use of self-experience, which has not been given much attention. Consequently, it is likely that upcoming officers in charge of navigational watch do not understand the following two concepts: learning as a purposeful process of cognition and learning as a process of developing specific knowledge and skills [12]. The correlation of non-technical skills and technical know-how is highly positive - as for an individual person’s social skills as well as a whole team’s set of skills in comparison with their level of technical knowledge and individual skill [1].

2 Maritime Emergency Management Skills Improvement Theory analysis suggests that non-technical skills are abilities in the process of different engineering activities, by using personal resources independently or cooperatively in a team, to get information and be able to evaluate it critically, make decisions in order to provide targeted and reliable operation and assessment of the many technical processes. Non-technical skills complement the technical knowledge and skills in everyday and emergency situations. As one of the most appropriate means to achieve a set goal, there are certain signs and expressions of the actions, which can be used to accurately assess both individual work, as well as teamwork. Signs are required to identify positive or negative task execution and afterwards suitable for detailed analysis of the causal link. This practice in the analysis process of plane crashes is described by B. Klampfer’s team of researchers from the Swissair airline training center [7], but it can be successfully adapted to the maritime industry, identifying the incidents caused by human error.

Work in Progress: The Impact of the Project OnBoardMed

767

Non-technical skills are abilities in the process of different engineering activities, by using personal resources independently or cooperatively in a team, to get information and be able to evaluate it critically, make decisions in order to provide targeted and reliable operation and assessment of the many technical processes. 2.1

IMO Guidance and Standards for Seafarers Training

Shipping is by nature a genuinely international activity, with strict requirements for harmonized and standard equal training regardless of where this is conducted. In this respect the United Nation’s International Maritime Organization (IMO) provides guidance to those performing the training through codes and conventions such as the STCW95 (Standards for Training of Seafarers Watchkeeping). Lives are endangered every time a disaster strikes on the European continent. Occurring incidents will often be of an international scale, and therefore will enhance the demand for a common understanding and readiness for crisis management in a multinational and interdisciplinary perspective [6]. The course in Crowd and Crisis Management (CCM) is expected to strengthen the overall knowledge of human behavior in crisis situations. The course includes among other things, leadership, human behavior and reactions in a crisis situation, as well as other aspects of handling a crisis and an emergency on board a ship, in compliance with the STCW requirements. Therefore, the focus is on cooperation between different groups of professionals and the improvement of their respective understanding of each other’s performances and organizations during crisis and rescue operations. To realize differences in professional roles, to understand one’s own professional role in comparison with others, attitudes that influence acting, possible defending of one’s own professional preservers and so on are important. A multi-professional team consists of different experts bringing specific know-how to the team. The aim is to combine knowhow from different experts is to achieve a better rescue result. 2.2

Shortly About the Project OnBoardMed

The project OnBoardMed – Harmonization of on Board Medical Treatment, Occupational Safety and Emergency Skills in Baltic Sea Shipping aims at developing more aligned vocational education programs in the Central Baltic area. A cross-border approach is needed to tackle the issue. The project develops study courses in maritime emergency management, medical treatment and occupational safety. The courses are targeted at maritime (seaman, deck officer, maritime engineer) and nursing (registered nurse, emergency care nurse and public health nurse) education institutions. The project is a joint cooperation venture of Turku University of Applied Sciences (Finland), Åland University of Applied Sciences (Finland), Estonian Nautical School (Estonia), Rīga Stradinš University (Latvia) and Latvian Maritime Academy (Latvia). The project is funded by the EU Interreg Central Baltic programme 2014–2020.

768

2.3

I. Barbare

Background and Preliminary Ideas for the CCM-Course

Educational research shows that people have different learning styles or preferences, and that adults learn most effectively when the learning has an experiential component. In other words they learn by doing. Active learning enables students to construct meaning and deep understanding rather than simply recording knowledge. Experiential learning is all about encouraging students to observe, reflect, analyses, synthesis, evaluate, and apply what they have learnt. It’s not just about having practical experiences but using those experiences to gain through to higher levels of learning. In maritime emergencies, especially onboard passenger vessels, appropriate and timely behavior of people is important for safety. The crews onboard, especially the officers, need to understand the behavioral patterns of mates and passengers around them to be able to handle and communicate effectively in order to manage the situation. The company should be prepared to handle external players such as authorities, relatives and media. The company should also have routines for taking care of crew members suffering from stress [8, 10]. All that is mentioned above could be contained in a Crisis Management Plan. Important issues are comprehension, planning and training.

3 Maritime Emergency Management Skills Improvement 3.1

Design and the Main Topics of the Course

In maritime emergencies, especially onboard passenger vessels, behavior of people onboard is important for safety [5]. The crew needs to understand the behavioral patterns and have practices that aid in reducing panic and in minimizing the risks. Learning objectives of the course: the skills of management and organization of large accumulations of persons in an emergency, including accident potential, safety plan, safety organization, management of crowds, catering for persons who need special assistance, stress management, human behavior in emergencies, and efficient management of emergency situation. The course should cover the three phases before, under and after a crisis. Good crisis management starts with analyzing risks and making plans to handle different situations [11]. It could be beneficial to point out the differences between a Safety Management System and a Crisis plan even though these could be quite similar. First phase can contain the making of crisis management exercises. Second phase is about human reactions. Why and how we react and how to manage different reactions. Third phase covers the follow up after a crisis situation. It is important to have a debriefing session after every crisis situation. This is a tricky part of a Crisis Management education. 3.2

Prequestionnaire About Lack of CCM-Course

Survey was organized by project first stage in January 2017 (N = 74; 93,1% male and 6,9% female, largest groups of ranks are deck officers 36,8%, sea captains 23,5%, others 14,7% and chief officers 13,2%. (see Fig. 1). Other refer to Human Resources

Work in Progress: The Impact of the Project OnBoardMed

769

Managers and inspectors. 6,4% of all target group members have participated in a CCM-course (Crowd and Crisis Management). No-answers comes mostly from tankers. The course is not mandatory for people working on tankers. There are some differences in maritime educations in the Nordic and the Baltic. For example, every deck- and engine student in Åland Maritime Academy are participating in a CCMcourse on the management level. Most common answers for course duration 2,3 and 5 days. Only in Germany has had 5 days course. Åland had a couple of 2 day courses for shore personnel earlier. Places varies quite a lot but 6 respondents have participated in Åland courses. On board courses and Computer Based Training-courses are also among the answers. 75% confirm that there is a Crisis Management Plan (Fig. 1). Is there a understanding of what a Crisis Plan? Is there a confusion between Crisis Management and contingency plans in the Safety Management System? 80,6% tells that they are well aware of the plan and they know what to do according to the plan.

Fig. 1. Having crisis management plan

70,6% tells there is Crisis Management exercises onboard. If all statements above are correctly understood and answered, then the situation is quite satisfactory. 69,4% has experienced a critical situation onboard and the severity lies between moderate to serious. Severe and critical are 12,5% each (Fig. 2).

Fig. 2. Experience in critical situations

770

I. Barbare

When it comes to different kinds of critical situations there are two main groups. Fire stands for about 33% and technical-, cargo- and navigational related situations are about 28%. 5 situations are related to Man-Over-Board. There are 4 situations related to crew and passengers from drunken captain to fight amongst passengers. Bomb-threat and piracy are also experienced. All these situations give a basis for a part of a CCMcourse. You have to understand the need for identifying risk factor and make good plans. When it comes to reactions among the crew most of the answers point to activeness (80,4%) (Fig. 3). For the passengers 10,4% were active, 31,3% were passive and 10,4% were in panic (Fig. 4). Other comments are 31,3% and most of them are about passengers not knowing about the crisis situation (sleeping or not informed). These percentages follow the research results from Dr David Canter [2] quite well. Canter gives a distribution of 25% active, 60% passive, 10% paralyzed and 5% panic.

Fig. 3. Crew reaction in critical situations

Fig. 4. Passengers reaction in critical situations

Work in Progress: The Impact of the Project OnBoardMed

771

The situations were commonly handled well. The main reasons for good handling are knowledge (58,2%), experience (67,3%) and training (54,5%). Situations not handled well comes from lack of experience (44,4%), insufficient training (38,9%) and stress (50%). According to the survey, there has been a high degree of follow up after a crisis (83,6%) on board. Most common answers are Debriefing conducted on board (30,4%) and Discussion meeting on board (26,1%). 3.3

Results After the First Pilot Course of Improved CCM-Course

The second stage of the project OnBoardMed was pilot course and survey of participants about course content, methodology, online course quality, self-assessment about personal experience. The survey showed following results. Actually, the materials online were used with strong interest (4–20 h online learning) and valuated as wellconsidered supply and availability of materials for training of practical skills (Fig. 5). Analyzing the results of the post-project survey, it has to be concluded that the teaching methods should be improved at a later stage of the project. Emergency simulations only half of the pilot project participants found it useful to improve their practical skills. In turn, group work was more supported. An impressive proportion of learners have already gained experience in online learning, but they do not support it by giving more preference to face-to-face classes. Overall, the learning process is good, even brilliantly planned and supported.

Fig. 5. Assessment of learning methods and process

One of the most important aspects of course acquisition is the student’s communication with the instructor - how the group work is organized, whether there is a sufficient preparation phase before the simulation, feedback on the result achieved and, above all, whether the intended course goal has been achieved. The results are positive. Future course improvement work will focus on time planning and feedback.

772

I. Barbare

4 Conclusions The project will result in harmonized and improved education and skills. The expected improvements in education and safety are significant, and will hopefully became the standard in the field. Participants will get excellent professional skills, and when applicable, will also be able to receive international certification from the courses. Further, the results will help local employees to enter the European and international labor market. The first stage results are showing that the project multinational team have to improve simulation and online part of the Maritime Emergency Management course. Especially interesting and useful is cooperation between maritime education institutions – online discussions, surveys, tests on blended learning tools are the most competitive methods for teachers and for students as well. The next steps after improving the courses would be accreditation of the final version of courses in national level education authorities.

References 1. Brunckhorst, O., Shahid, S., Aydin, A., Khan, S., Mcllhenny, C., Brewin, J., Sahai, A., Bello, F., Kneebone, R., Khan, M.S., Dasgupta, P., Ahmed, K.: The relationship between technical and nontechnical skills within a simulation-based ureteroscopy training environment. J. Surg. Educ. 72, 1039–1044 (2015). https://doi.org/10.1016/j.jsurg.2015.04.002 2. Canter, D.F.: Human Behaviour (ed.): Fulton ISBN 1-85346-105-9 (1990) 3. Flin, R., Maran, N.: Basic concepts for crew resource management and non-technical skills. Industrial Psychology Research Centre, University of Aberdeen, King’s College, Old Aberdeen; Department of Anaesthetics, Royal Infirmary of Edinburgh (2015) 4. Flin, R., Martin, L., Goeters, K.M., Hörmann, H.J., Amalberti, R., Valot, C., Nijhuis, H.: Development of the NOTECHS (non-technical skills) system for assessing pilots’ CRM skills. Hum. Factors Aerosp. Saf. 3(2), 95–120 (2003) 5. Gregory, D., Shanahan, P.: The human element: guide to human behavior in the shipping industry. The Stationery Office (TSO), Norfolk, UK, 120.lpp (2010) 6. Hetherington, C., Flin, R., Mearns, K.: Safety in shipping: the human element. J. Saf. Res, 37, 401–411 (2006) 7. Klampfer, B., Flin, R., Helmreich, R.L., Häusler, R., Sexton, B., Fletcher, G., Field, P., Staender, S., Lauche, K., Dieckmann, P., Amacher, A.: Enhancing performance in high risk environments: recommendations for the use of behavioral markers, Swissair Training Centre, Cīriche, p. 30 (2001) 8. Lyk-Jensen, H.T., Spanager, L., Malene, R., Ǿstergaard, D., Dieckmann, P.: N-ANTS Handbook. The Danish Institute for Medical Simulation. 16.lpp (2014) 9. Moffat, S., Crichton, M.: Investigating non-technical skills through team behavioral markers in oil and gas simulation-based exercises. Procedia Manuf. 3(2015), 1241–1247 (2015) 10. Mundt, A., Spanager, L., Malene, R., Ǿstergaard, D., Dieckmann, P. (2014). SPLINTSdk User Guide. The Danish Institute for Medical Simulation, 11.lpp 11. O’Connor, P., Flin, R.: Crew resource management training for offshore oil production team. Department of Psychology, Safety Science, pp. 591–609 (2003) 12. Žogla, I.: Didaktikas teorētiskie pamati. RaKa, Rīga (2001)

An Approach for Supporting Space Orientation of the Blind Using Ontologically-Based Object Map Dariusz Mikułowski(&) and Marek Pilski(&) Institute of Computer Science, Siedlce University of Natural Sciences and Humanities, Konarskiego 2, 08-110 Siedlce, Poland {dariusz.mikulowski,marek.pilski}@uph.edu.pl

Abstract. Mastering skills of spatial orientation is a very important issue for the blind people. However, there are widely available or dedicated systems that support independent movement of them, i.e. the solutions based on GPSnavigation or the use of ultrasonic sonars or cameras, there is the lack of approaches that supports learning of spatial orientation in a smart way by helping blind people with recognition of real objects or assisting them with moving along unknown routes. To make a step forward to fulfill this gap, we propose an approach based on Object Map Ontology and binaural sounds. It allows us to describe unknown objects on the street and shows a new path for the blind user. The core of this proposal is a special ontology that stores knowledge about different objects that can be found in the city and set of binaural sounds that are integrated with these objects. The route that is a sequence of objects taken from the ontology is translated into a series of binaural sounds and auxiliary text descriptions. The preliminary studies have confirmed that blind users such as students but also adults are interested in developing such a solution. Investigations showed that the proposed system would be even useful for sighted trainers who teach spatial orientation. The initial Object Map Ontology that covers an exemplary root was created and expressed as a set of OWL classes, relations, and individuals. The preliminary sound classification was created and a couple of binaural sounds was recorded. Keywords: Spatial orientation of the blind
Binaural sound representation Training of blind students
Interactive environment




1 Introduction Spatial orientation is one of the key skills that should be familiar to every blind person. To move independently in open space, a blind user has to learn different techniques of spatial orientation such as: listening to the environment, making a respective travel direction, recognizing and avoiding obstacles. He should also learn respective behavior in potentially dangerous situations such as road crossing etc. In order to learn these techniques, the student should participate in spatial orientation training. Nowadays, such training is carried out by traditional techniques, that

© Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 773–784, 2021. https://doi.org/10.1007/978-3-030-49932-7_72

774

D. Mikułowski and M. Pilski

use a mock-up that consists of dummy objects such as models of the buses, poles, stop vehicles, etc. However growing development of modern technology has resulted in solutions that support independent movement of the blind, to our knowledge, no electronic smart assistance systems have yielded with supporting learning these skills by young blind students or recently blind adults. Only one of the few exceptions may be the system implemented in Poland in 2012 [1]. The most of existing systems such as TotuPoints [2] usually support people who already have advanced spatial orientation skills and they work directly at the time when the user moves in a real environment. As far as we know, there are not solutions that support learning of spatial orientation in simulated environments. To overcome this problem, we propose technology which, on the one hand, will allow blind students to learn spatial orientation skills, and with another - will allow the trainers to help their students with performing this difficult task. Our solution is based on a few main phenomena and components. In the beginning we have noticed that about 50 percent of blind people have the ability of so-called an echolocation. This phenomenon states with hearing sounds that are reflected from obstacles that are nearby of the user. Another idea that we use is the binaural sounds. There are sounds recorded in standard way using two stereo channels, but so that, listening to them, the user has the impression of 3-dimensional space. Applying above features in combination with semantic technologies, we propose a special Object Map Ontology that stores information about different objects in the real world. That assists blind students with proper space recognition and navigation. There are objects such as sidewalks, pedestrian crossings, poles, roadways, building entrances, etc. An important property of this map is its hierarchic structure which allows to automatically assign a route between any two points on the map. Such a route can then be converted into sequence of binaural sounds and text descriptions that describe the pass of the root for the user. To check the accuracy of our idea we have developed preliminary studies with the participation of a couple of blind and partially sighted respondents. They have shown that the spatial orientation trainers, parents, and students would like to use such kind of tool for learning and teaching spatial orientation skills. Moreover, using this system would be safer and less stressful way of learning spatial orientation for beginner students than the currently used traditional solutions. In the next sections, we will describe the method that we have developed to create such ontology map and applying it to support better technique of learning and teaching spatial orientation skills.

2 Related Works and Approaches It has become common, that, before going to the trip, a user looks for a route using services such as Google Maps and explores the route using his electronic device. Services such as Google Street View takes users in a virtual and visual way to places, he wants to explore. Thus, you can explore unknown places in a safe way before you reach them. Unfortunately, such virtual environments are usually inaccessible to blind people. Therefore, attempts are made to solve this problem in different ways. The first

An Approach for Supporting Space Orientation of the Blind

775

of them is learning by playing. This approach was used in the project [3]. Where a special sound-based computer game - Space Simulator has been implemented. It was used as the means to learn of spatial orientation. In experimental studies, it was found that the addictive nature of the software encouraged and engaged blind users to actively explore the virtual environment. In accordance with authors’ explanation, the blind participants of the game transferred their skills to the real world. Another expert [4] shed light on our current understanding of the positive and negative ways in which playing video games can affect cognition and behavior, and explain how this knowledge can be harnessed for educational and rehabilitation purposes. In the next project, the authors provide several games supporting the training process related to the perception of distance and angles [5]. A Smart Cane slowly replace the outdated white cane. The stick is equipped with electronics that can alert user if he faces obstacles. The system provides real-time information by vibrating or voice on the nature of the obstacles, which blind may encounter. Smart Cane can have value-added features to maximize user independence and safety by installed GPS modules or communicate with the user’s smartphone. The next solution presents a virtual reality system [6] developed for orientation and mobility training of people who are newly blind. The system allows users to interact with different virtual structures and objects via auditory and haptic feedback. A number of novel assistive technologies and electronic travel devices have been developed. From the simplest based on a single sensor such as BuzzClip - small and discreet wearable device offers essential head level obstacle detection and can be attached to many forms of clothing by clip. The device uses ultrasound to detect obstacles and notifies the user of these obstacles through vibrations. Another project [7] presents a prototype of an electronic device that helps with navigation of blind person by means of sound signals. Sounds are meant to provide the blind with a simplified map of the object depth in their path. Yet another proposition is based on sensory substitution devices [8]. Nowadays the smartphones [9, 10] are very popular devices so no wonder that they are also used to support independent movement. There are some popular applications such as GPS based travel aids e.g. Nearby Explorer, GetThere, Lazarillo on Android and Ariadne GPS, BlindSquare on iOS. Sometimes very sophisticated devices worn by the user on shoulders, equipped with GPS, compass, loudspeakers, haptic pointer interface (HPI) [11] or cameras [12, 13] are proposed. Support for the mobility of the blind with walking difficulties has been proposed in [14]. It uses a special walking frame equipped with a gyroscope, accelerometer and Microsoft Kinect sensor as a cheap and effective method for acquiring information from the environment. A completely different approach was proposed in [15]. Instead of equipping a blind user with a series of sensors to acquire data on the structure of the environment, it is proposed to build an intelligent city so that its elements send data about themselves to safely help with orientation in the real environment.

776

D. Mikułowski and M. Pilski

3 Proposal of the VRS System To facilitate learning and teaching of spatial orientation of the blind, we propose a system based on the concept that we called the Virtual Reality Sound (shortly VRS). It used the echolocation ability that there are many blind people, binaural sounds and hierarchical map of the environment that is stored as a proper ontology. The concept of this system we describe in the following subsections. 3.1

How It Works

In this section, we will explain the predicated working of our solution using simple example. Let suppose that a blind student of the first year John wants to learn how to overcome the route, from the building of the Institute of Computer Science of the university he is attending to the Railway Station. To achieve this goal, John wants to take help of the spatial orientation trainer and VRS system. To learn this route, they do the following. The trainer runs his application that is built into VRS system and creates a route for the student. For this purpose he can use any program that uses GPS coordinates – then imports the root to VRS system. To finalize, the trainer complements this root by adding another object and other data that will be relevant for the student, such as transitions through the streets, alarms, street lights, lawns, bus stops, fence posts, etc. An exemplary route from Institute of Computer Science to the Railway Station (included individuals from presented ontology) is shown in Fig. 1. As we can see the route passes through two crossroads and three pedestrian crossings. Next, the produced root is processed into sequence of binaural sounds and textual descriptions. So now the trainer can play for his student a sound-text representation of the route. The fragment of this representation is presented in the listing: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.

Text: Open the exit door from the Institute of Computer Science. The sound of cars driving right to left and left to right heard in front of the student. Text: turn left 90 degrees and go about 80 m. to the nearest crossroad. The sound of cars driving back to front and front to back heard on the right side of the student. Front of the student. The sound of steps heard in front of the student. The sound of the building wall heard on left side of the student. Text: you’ve reached the crossroad, turn 90 degrees to the left And walk about 100 m to the bus stop. The sound of cars driving back to front and front to back heard on the right side of the student. Front of the student. The sound of cars driving right to left and left to right heard in front of the student. And so on.

Then, after exploration almost the entire route, the student can safe to go to the real environment without the help of our system. However, the route is known to him and thanks to the use of the training he will be able without problems to overcome the actual route on their own.

An Approach for Supporting Space Orientation of the Blind

777

Fig. 1. A route from the Institute Of Computer Science to Railway Station with marked individuals from the OWL ontology.

3.2

VRS System Concept

To support the training of spatial orientation, as described above, we propose a system that is based on the VRS. The main part of it is the Object Map Ontology that we will present in more detail in the next section. The system has separate user interfaces for the trainer and for the student. The trainer interface is an application that will allow him to create routes for the student using graphics widgets or outside applications such as SketchUp or Google Maps. This interface will generate a graphical representation of the route, which is then converted to the form of individuals and relations of the ontology. Then, using an inference layer of the system, particular objects of the map are equipped with a number of prerecorded binaural sound samples and textual descriptions. This sequence of sounds and text descriptions is transferred to the student’s interface that consists of application installed on the smartphone and headphones. That can play binaural sounds and text descriptions of the generated route. Due to this solution a student will be able to listen to sound representation of generated route and, in this way, get to know it before going to real trip.

778

D. Mikułowski and M. Pilski

4 Ontology Realization, Object Map Structure, and Schema Because ontologies are known as a very good way of describing different kinds of knowledge, we consider them as the approach significant tool to our needs. As a language for describing our ontology we undertake OWL (Web Ontology Language), which has been described, standardized and published by W3C consortium. In order to achieve our goal, we propose the specially developed Object Map Ontology and set of binaural sound classification. The ontology consists of classes that represent elements of city space that are important from the point of view of the blind user. They represent elements of infrastructure that the pedestrian can move (e.g. pavements, pedestrian crossings, stairs) and the locations that can become of his destination e.g. addresses, GPS coordinates, railway stations, etc. There are also classes representing architectural elements of the city, which pedestrians are moving along (e.g. bus stops, monuments, real estates, streets), etc. Another category of objects in the ontology map provides useful information for the blind user. There are objects such as road signs, traffic lights and another that may act as information points e.g. ATMs or shops. Another important category may be an obstacle (e.g. bollard, advertising column, light pole or car park cutting into the pavement). OWL classes are interpreted as the sets that contain individuals. Currently, the proposed ontology contains 47 classes that can be used to define any part of the city. The selected class hierarchy looks as follows. At the top of the hierarchy is the owl: thing class, from which they inherit the following classes: 1. City Components – the class represents the physical and stationary elements of the city: 1:1. Elements Of City Architecture – general class representing typical objects along the pavement, selected subclasses of this ontology: 1:1:1. Advertising Column – frequently it is located in the middle of a square or pavement; 1:1:2. ATM; 1:1:3. Bollard – a dangerous obstacle often set on pavements at pedestrian crossings, blind people can fall into it; 1:1:4. Bus Stop – it may happen that the place where the bus stops is cut into the pavement, it is dangerous for the blind; 1:1:5. Pole – street lamps, electric pole, information pole they are obstacles on the walking route; 1:1:6. Road Sign – usually traffic signs represent information in the form of a graphic symbol or text that is inaccessible to the blind: Information Sign, Prohibition Sign, Warrant Sign; 1:1:7. Traffic Light; 1:1:8. Trash Can – also a potential obstacle; 1:2. Infrastructure For Pedestrians – these elements are used to set the route for the blind, he will move on them: 1:2:1. Lift; 1:2:2 Pavement;

An Approach for Supporting Space Orientation of the Blind

2. 3.

4. 5.

779

1:2:3. Pedestrian Crossing – the zebra crossing may be equipped with traffic lights, sound signals or not; 1:2:4. Stairs; 1:2:5. Road Infrastructure, along with sub-classes: Bridge, Cross Roads, Street, Traffic Circle, Tunnel; Location – usually every element of city architecture has its location (address) and geo-location coordinates; Real Estate – it can be a destination, a Point Of Interest, it is also other buildings along which the blind moves on its way. Note that the walls reflecting sounds coming from the environment and may be used by blind to maintain the right direction of movement, e.g. Bank, Railway Station, School, Shop, etc.; Vehicle – the class represents moving objects, but which can be assigned a specific sound that they emit. e.g.: Bus, Car, etc.; Voice – every place on the route can be assigned a binaural sound, e.g. an MP3 or WAV format file.

Of course, the above class hierarchy proposal is not closed, it can be expanded with new classes as needed. OWL individuals represent objects in the domain that we are interested in. For the needs of the presented approach, 35 individuals have been identified. These individuals represent the selected objects of a real simple short route from the Institute of Computer Science to the Railway Station in Siedlce town (Poland). In principle, a pair of individuals have been defined for each presented earlier classes. For example, individuals which will be used to set up a route, e.g. pavements and pedestrian crossings: • pavementSienkiewicza1 (the type of Pavement) – a first section of the pavement on the left side of Sienkiewicza Street; • pavementArmiiKrajowej2 (the type of Pavement) – a second section of the pavement on the left side of Armii Krajowej Street; • pedestrianCrossing3Maja1 (the type of PedestrainCrossing) – first pedestrian crossing through the 3Maja street at the intersection of 3Maja and Armii Krajowej streets. Individuals which may be an obstacle on a fixed route for the blind walker, e.g.: • trashCanSienkiewicza1 (the type of TrashCan) – trash can located on the edge of the pavement on the street side; • busStop01SiedlceArmiiKrajowejSienkiewicza (the type of BusStop) – bus shed located in the middle of the pavement, it can be walked around from both sides; • poleArmiiKrajowej (the type of Pole) – street lamp placed at the edge of the pavement from the street side; • advertisingColumnArmiiKrajowej3Maja (the type of AdvertisingColumn) – a largediameter advertising column located on the edge of the pavement from the side of a lawn. Individuals that inform the user about the possibility of movement, e.g. trafficLightArmiiKrajowej3Maja1 (type of TrafficLight) – information about the possibility of crossing the street safely.

780

D. Mikułowski and M. Pilski

An individual which represents an object that a user can use, e.g. atmSiedlce ArmiiKrajowej (type of ATM) – ATM to withdraw money for a ticket. Individuals that a blind user can recognize by listening to his specific sound that its object emits, e.g. bus1scania (the type of Bus) – object with associated sound of a stopping bus and the sound of the doors opening. Summarizing, we can create an ontological representation of the city by mapping physical objects from the real world to individuals. OWL data properties define the data types that can be assigned to the domain (in this case to the OWL classes). For example, the Address class has defines data properties such as: address_country_name (xsd:string), address_post_code (xsd:string), address_city_name (xsd: string), address_building_number (xsd:integer), address_flat_number (xsd:integer) and so on. Parking class has a data property parking_capacity, the next class named Voice has filename, filePath, fileURI and fileType data properties which represent recorded sounds. OWL object properties are binary relations (between two OWL things) on OWL individuals. To provide a more smart and better description of environment in the ontology map the relevant relations have been also defined: • appliesToPedestrian – the property is used to determine the application of a thing to another thing. For example, this property has been used to define a restriction that says that only one object (individual) of the Traffic Light class can apply to exactly only one object (individual) of the Pedestrian Crossing class. An example of expression from the ontology: TrafficLight appliesToPedestrian exactly 1 PedestrianCrossing. • hasDirectNeightbor (symmetric properties) – when the pavementSienkiewicza individual is related to pavementArmiiKrajowej individual (on crossing streets) via the hasDirectNeightbor property, we can infer that pavementArmiiKrajowej must also be related to pavementSienkiewicza via the hasDirectNeightbor property. In other words, when pavementSienkiewicza has a direct neighbor pavementArmii Krajowej, then pavementArmiiKrajowej must have a direct neighbor pavementSienkiewicza. This property allows determining the next elements of the route. An example of expression from the ontology: CityComponents hasDirectNeighbor some CityComponents. • hasLocation – each object (individual) can have exactly one location on the map expressed by address and/or GPS coordinates. An example of expression from the ontology: CityComponents hasLocation exactly 1 Location. • isLocatedOn – this property is mainly used to express that something is located on the route that the blind is moving. For example, a trash bin is located on the pavement and may be an obstacle for him. An example of expression from the ontology: TrashCan isLocatedOn exactly 1 Pavement. • recordedInTheLocation – sounds can be recorded in various places around the city. This property is used to assign a sound to a specific location. For example, the sound can represent cars driving on the street. Considering the direction of the user’s movement, this sound can be played on the left or right channel. An example of expression from the ontology: Voice recordedInTheLocation exactly 1 Location.

An Approach for Supporting Space Orientation of the Blind

781

Fig. 2. Selected properties representation from the OWL ontology.

As was mentioned above, in order to explain these different objects for the blind user, most of such classes can be assigned with binaural sounds, sound samples recorded from real world. In addition, a standard ontology mechanism has been used OWL annotations. These annotations are the text comment that you can add to any ontology element. The text can be read by speech synthesizer (Fig. 2). Summarizing, the routes consisting of ontology objects generated automatically from this map can then be translated into a series of binaural sounds and auxiliary text descriptions. The audio information can be played for the blind user. In this way, the user is able to build the representation of the route in his imagination what allows him to learn how to safely get to his destination in the real world.

5 Preliminary Investigations of Suitability To determine directions of research for the proposed solution the preliminary investigations were conducted among the blind potential users. The goal was to show whether the blind will find this solution as useful for supporting the independent exploration of outdoor environment and learning a spatial orientation. The research was conducted in the form of an online questionnaire and email sharing with several respondents. The questionnaire was fulfilled by 35 respondents of middle age (most of them were between 26 and 50 years old). For research purposes, in the questionnaire, the proposed solution was called “Roots audio-descriptor for the blind”. The questionnaire introduction has provided a brief description of the concept of the proposed solution. The main part of the questionnaire has contained the following three questions. Question 1. What is your opinion, does the audio-descriptor of roots would be useful for blind users to facilitate them with easier orientation in unfamiliar places? Question 2. What is your opinion, does the audio-descriptor of roots would be useful for spatial orientation trainers to support them with teaching of the blind students? Question 3. What is your opinion, does the audio-descriptor of roots would be useful for blind students during learning of spatial orientation to facilitate them with the recognition of new roots?

782

D. Mikułowski and M. Pilski

The summary of the obtained answers to above questions is presented in the following Table 1. Table 1. Summary of obtained answers Question 1 Question 2 Question 3 Absolutely useful 29 16 28 Rather useful 6 13 6 I have no opinion 0 4 0 Rather not useful 0 1 1 Absolutely not useful 0 1 0

As we can see in the table, all answers to question 1 were positive which mean that 29 people (82%) responded that the program would be useful and 6, that it would be rather useful. Talking about the potential usefulness for instructors of spatial orientation (question 2) we can notice that 45.7% of respondents answered that it would be useful and 37.1%, rather it would be useful. There were also 4 (1.4%) answers of undecided respondents to this question. The answers to the third question, concerning the suitability of proposed solution for students of spatial orientation, was also the most positive. 28 respondents (80%) replied that, in their opinion, the program would be useful and 6 (17%) that, rather useful. The respondents were asked also to enter his/her suggestions and remarks to the presented concept. 18 respondents (more than 50%) took advantage of this opportunity. Most of the comments (9 respondents) suggested embedding into application detailed descriptions suitable for the blind user, such as information about intersections, crossings, traffic islands, roadways, lawns, and nearest service points and shops. Two respondents indicate to integrate the program with Google Maps, Totupoints or another existing navigation system. Another two respondents suggested that the program should be navigation software with special functions for the blind. However these opinions are not consistent with any other respondent suggestion that binaural sounds will interfere with the sounds that reach the user from the environment. This leads to the fact that they will interrupt in navigation. Another interesting suggestion was to add the possibility of informing the user about slope of the root, and the suggestion to add the possibility of listening to the routes section by section. To summarize, we can say that the response to the questionnaire among the blind was very positive and confirmed the potential usefulness of the proposed solution. The most critical comments came in the email exchange with some respondents. People, who understood that the program will be designed to assist in navigation in real-time exposed the opinion, that the binaural sounds will be stumbling block in such navigation. In addition, supporters of the traditional methods for learning spatial orientation argued that such a program will not make it easier to learn, and a better solution is to do it with real models.

An Approach for Supporting Space Orientation of the Blind

783

Despite these critical comments, it should be noted that the above results of the research showed that people are open to new opportunities would be ready to use such a solution.

6 Summary Supporting the training of spatial orientation of blind people electronically isn’t an easy problem. As a step towards solving it we propose an ontology-based system using audio representation of city map which we have presented in this paper. The preliminary studies that were conducted among a few dozen of the blind users showed that there is a great interest in this idea. These investigations were shown also that such solution would be useful for students, spatial orientation trainers, and parents to facilitate the development of these skills for the blind children. Although work is still in the initial state, we plan to extend and optimize the complete ontology object map. Adding the support for creation routes from external sources, such as, for example, Scathup, Open Street Map or Google Maps are also planned. After developing the pilot version of our system, the investigations within blind students and trainers are planned to prove significance of our technique for supporting spatial orientation learning.

References 1. An orientation park for the blind was opened (2012). https://www.portalsamorzadowy.pl/ fundusze-europejskie/otwarto-park-orientacji-dla-niewidomych,37714.html. Accessed 06 July 2019 2. Totupoint - an app for the blind and visually impaired (2018). https://teleexpress.tvp.pl/ 39973064/totupoint-aplikacja-dla-niewidomych-i-slabowidzacych. Accessed 06 July 2019 3. Merabet, L.B., Connors, E.C., Halko, M.A., Sánchez, J.: Teaching the blind to find their way by playing video games. PLoS ONE 7(9), 1–5 (2012) 4. Bavelier, D., Green, C.S., Han, D.H., Renshaw, P.F., Merzenich, M.M., et al.: Brains on video games. Nat. Rev. Neurosci. 12, 763–768 (2011) 5. Maidenbaum, S., Amedi, A.: Blind in a virtual world: mobility-training virtual reality games for users who are blind. In: Virtual Reality (VR), pp. 341–342. IEEE (2015) 6. Lahav, O., Schloerb, D.W., Srinivasan, M.A.: Newly blind persons using virtual environment system in a traditional orientation and mobility rehabilitation program: a case study. Disabil. Rehabil. Assist. Technol. 7, 420–435 (2012) 7. Kubanek, M., Depta, F., Smorawa, D.: System of acoustic assistance in spatial orientation for the blind. In: International Multi-Conference on Advanced Computer Systems, pp. 266– 277. Springer (2016) 8. Johnson, L.A., Higgins, C.M.: A navigation aid for the blind using tactile-visual sensory substitution. In: 28th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 6289–6292 (2006) 9. Akbar, I., Misman, A.F.: Research on semantics used in GPS based mobile phone applications for blind pedestrian navigation in an outdoor environment. In: International Conference on Information and Communication Technology for the Muslim World (ICT4M), pp. 196–201. IEEE (2018)

784

D. Mikułowski and M. Pilski

10. Bulat, J., Glowacz, A.: Vision-based navigation assistance for visually impaired individuals using general purpose mobile devices. In: International Conference on Signals and Electronic Systems (ICSES), pp. 189–194. IEEE (2016) 11. Loomis, J.M., Marston, J.R., Golledge, R.G., Klatzky, R.L.: Personal guidance system for people with visual impairment: a comparison of spatial displays for route guidance. J. Vis. Impair. Blind 99(4), 219–232 (2005) 12. Tang, H., Zhu, Z.: A segmentation-based stereovision approach for assisting visually impaired people. In: International Conference on Computers for Handicapped Persons, pp. 581–587. Springer (2012) 13. Manduchi, R., Coughlan, J., Ivanchenko, V.: Search strategies of visually impaired persons using a camera phone wayfinding system. In: Miesenberger, K., Klaus, J., Zagler, W.L., Karshmer, A.I. (eds.) ICCHP. LNCS, vol. 5105, pp. 1135–1140. Springer, Heidelberg (2008) 14. Kalischewski, K., Wagner, D., Velten, J., Kummert, A.: Orientation and positioning with inertial sensors for walking frame guidance. In: 9th International Workshop Multidimensional (nD) Systems (nDS), pp. 1–5. IEEE (2015) 15. Montanha, A., Escalon, M., Dominguéz-Mayo, F., Polidorio, A.: A technological innovation to safely aid in the spatial orientation of blind people in a complex urban environment. In: International Conference on Image Vision and Computing (ICIVC), pp. 102–107. IEEE (2016)

Soupa and Integration of Ontologies Verl for Conceptualizing Contexts in Video Surveillance and Ubiquitous Computing Susana Arias T1(&), Xavier Arias2, Claudia Cartuche3, and Lozada J. Francisco4 1

3

Facultad de Ciencias de la Salud, Universidad Técnica de Ambato, Ambato, Ecuador [email protected] 2 Universidad Regional Autonoma de los Andes, Km 5 1/2 vía Baños, 180215 Ambato, Ecuador [email protected] Departamento de Arquitectura y Artes, Universidad Técnica Particular de Loja, Loja, Ecuador [email protected] 4 Facultad de Ingenieria en Sistemas, Electronica e Industrial, Universidad Técnica de Ambato, Ambato, Ecuador [email protected]

Abstract. Research describes the design and Ontology for Ubiquitous Surveillance Systems (OSU). OSU arises from the combination of Video Event Ontology (VEO), used generally to conceptualize events surveillance [2] with Standard Ontology for Ubiquitous and Pervasive Applications (SOUPA), used generally to conceptualize events ubiquitous computing [3]. The goal of combining VEO and SOUPA is complementarity, because there classes SOUPA that can be used to describe events surveillance (Person, for example), while there are classes VEO that can be used to describe events ubiquitous computing (Object, for example). OSU allowed understand and conceptualize, better, the events recorded in CASAS [4] projects and start [5]. Keywords: Ontology


Event
Activity
Ubiquitous computing

1 Introduction Ubiquitous computing is the correct distribution of elements: computers, software and electronic devices, so that these elements are not physically visible, help people in their daily lives. To achieve this objective elements are interconnected and properly synchronized. For example, a home device ubiquitous computing can interface and synchronize lighting and heating with an environment control such that depending on the evolution of the time of day and its characteristics, this system reacts and vary the temperature and conditions light in a house or building, continuous and progressive [1] way. © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 785–799, 2021. https://doi.org/10.1007/978-3-030-49932-7_73

786

S. Arias T et al.

For optimal function of the elements necessary to conceptualize both its features and operation, as the context in which they are located. Incorrect conceptualization makes the software not efficiently process the data collected by the devices, synchronization is not accurate, there are failures and slow the assistance to be given to people, etc. To avoid errors in the conceptualization, this research conceptualizations ontology VEO [2] and SOUPA [3] are combined in a single ontology called OSU [#]. The combination of VEO and SOUPA for its complementarity, allows classes of SOUPA can be used to describe events of video surveillance (person, for example) and VEO, are used to describe events Ubiquitous computing (object, for example). Besides described, OSU has its own classes and properties, and allows the use of fuzzy logic in conceptualizing events. In order to describe in detail, the design and operation of OSU, this research is explained by the following sections: 1. State of the art. This describes why the VEO were selected and SOUPA ontologies as components OSU. 2. OSU design. a description of the components and characteristics of OSU in conceptualizing classes, properties and contexts surveillance and ubiquitous computing is performed. 3. Experimentation. OSU benefits specified when it conceptualizes the events recorded in CASAS [4] projects and start [5]. 4. Conclusions and future work.

2 State of the Art In [6] an ontology is defined as a “specification of a conceptualization, whose main purpose is to allow communication between computer systems in a way that it is independent of technologies, information architectures and application domain”. In other words, an ontology is the combination of data structures, content and meaning which aims to represent application domains consistently and unambiguously [7]. Domains related to video surveillance; ontologies play a decisive role in the conceptualization of its components. Consider the CARETAKER [11] and CAVIAR [12] ontologies that allow conceptualizing events from data recorded by the sensors (security cameras, motion sensors, etc.). In domains ubiquitous computing the SOUPA [2] is regarded as ontology model [2, 8, 9]. SOUPA has the following ontologies: Friend-Of-A-Fried (FOAF) [10], DAMLTime and sub-ontology temporal [11] Ontologies space management [12] Ontologies for calculating regions RCC [12] COBRA-ONT [2], MoGATU BDI [2] and Rei ontology management policies [2]. OSU like ontologies named in the preceding paragraphs, specialized in the domains for which it was designed (Ubiquitous computing and video surveillance), but unlike them, improved detailed representation of events, combining classes VEO and SOUPA, and adding classes and properties, depending on the characteristics of the context to represent.

Soupa and Integration of Ontologies Verl for Conceptualizing Contexts

787

One way to get an ontology to conceptualize a context, is to describe natural language each of its elements, and then use this description to design the ontology. Note that for the VEO ontology, in [2] structured Video Event Representation Language (VERL) to describe in detail the elements of a video surveillance context language is used; there was the only procedure for obtaining the classes and properties of its ontology. Also, to describe the events that are recorded on video surveillance in [2] structured Video Event Modeling Language (VEML), in order to interact with VERL ontology and obtain proper context semantic description language was used. However, ontology VEO and the VERL and VEML, languages conceptualise only some elements of its domain, and did not consider others who are considered by SOUPA, rinsed with the case of the architectural layout of the building in which find a person’s daily activities thereof, etc., which characterizes its importance in video surveillance [13, 14]. SOUPA also can use VEO classes to describe, in detail, events out of context, e.g. To determine whether a person is alone or in company, determine whether a person took an object or left forgotten, etc., this proves the importance for ubiquitous computing [4, 5]. Concretemos: this study classes VEO and SOUPA combined to achieve greater detail in conceptualizing related to ubiquitous computing and video surveillance, contexts with the OSU ontology is obtained, whose design is explained in the following pulled apart.

3 OSU Design OSU is made up of various kinds, among the most important are: 3.1

Class Person

Class Person (CPVEO) is conceptualized by VEO, among other things, as a moving object. A person is not only that, but it has attributes (properties): names, special interests, mail, profession, etc. ANDn the world there are many people who share the same attributes (name, height, interests, etc.) causing ambiguity when people search is performed, if there is any attribute that identifies them as exclusive [3]. FOAF solves this problem by adding at the email people with a unique identification code. Conceptualizations of VEO and FOAF are complementary, so that both are taken into account at OSU as if they were one. To achieve this, he converted to CPVEO in a class defined linked to class person FOAF (CPFOAF), with the necessary and sufficient condition that CPVEO  CPFOAF. In Fig. 1 the conceptualization of the Person class is observed.

788

S. Arias T et al.

Fig. 1. Conceptualization of the Person class

3.2

Object Class

In this class OSU is conceptualized in the same manner as in VEO. 3.3

Class Event

VEO conceptualizes two types of events: simple and compound. A simple event is not broken down into other, e.g. the shift positions in a space. A compound event consists of simple events, e.g. in Fig. 2, the Person_walk event requires change_of_spatial_position event execution, to indicate that the person changing spatial position (PE), which may be walking.

EVENT Person _walk (change_of_spatial_position ((x, y), (x1, y1)))

Fig. 2. Event compound conceptualized VEP

3.4

Class Sensor

Table 1 is an example of information registered by sensors located in the homes [4] and the placeLab Project [16] Project. His attributes are the basis for conceptualizing the sensor type, as follows:

Table 1. Data generated by motion sensors Date Hour Sensor State 27/02/2008 12: 43: 27.416392 M08 ON 27/02/2008 12: 43: 27.8481 M07 ON

Soupa and Integration of Ontologies Verl for Conceptualizing Contexts

789

5. Date and Time took the defined class time: temporalUnit of DAML-Time. VEO contributed to the conceptualization of milliseconds (ms) and nanoseconds (ns). OSU was added in the necessary and sufficient condition: {time: unitSecond time: UnitmSecond time: UnitnSecond time: unitMinute time: unitHour time: unitDay time: unitWeek time: unitMonth time: unitYear}. Figure 3 shows the conceptualization:

Fig. 3. New class defined time: TemporalUnit.

6. The class time: Instant was modified so that it can make use of class date, in order to conceptualize the moments in which motion sensors are activated. In Fig. 4, the tab time: Instant_10 is an instance of this class.

Fig. 4. Sensor class conceptualization

7. ContextualObject, e.g. in Fig. 4, the tab VEO: Sensor_14 as sensor element is a context to conceptualize the sensor as part of a context, the VEO class used. 8. The last step is to conceptualise the sensor name, status and type. name string sensor type, sensor status type boolean and string type of sensor format: This property was created at OSU. 3.5

Conceptualizing Contexts Video Surveillance OSU

The description of classes person, object, event and sensor affirms the proposal for OSU [#]. This section classes and properties that OSU used to conceptualize surveillance contexts described. a. Conceptualisation of tracking people The OpenCyc and RCC SOUPA [17] ontologies, [18–20] conceptualize architectural plans; its important class, most used in this research is Construction. OSU took

790

S. Arias T et al.

this class, modified some of its properties and added new order to identify the exact site where the sensors are located. The modified and new properties described below: 1) Construction: hasSensor(new). Indicates the plant (plant 1, plant 2, etc.) where the sensor is located. 2) Construction: PlacementX, Construction: PlacementY, Construction: PlacementZ. They are the coordinates X, Y, Z sensor in the plane. 3) AmlRequeriments: Compose. It is the place where the sensor (wall, window, door, etc.) will be located. The construction tab: Sensor Fig. 5 indicates that the sensors are located somewhere in a building.

Fig. 5. New Sensor class

The time tab: Instant Fig. 4 has been renamed to j.0: Instant in Fig. 5, to indicate that temporary inferences made in order to know the instant the sensor is or was activated, the place activation and places by which a person walks (tracking). b. Conceptualizing events related to walking speed of people: one person follows another Since it is possible to recognize people in a video [21], OSU may conceptualize the video event e1 = A person follows the person B. The variables involved include speed and direction of the walk [22, 23], distance between two [21], and time intervals, these variables were added to the person class in OSU. Figure 6 is an example of e1.

(A)

(B)

Fig. 6. A person follows the person B. Source YouTube: http://www.youtube.com/watch?v= EqhGxxeyB2I

Soupa and Integration of Ontologies Verl for Conceptualizing Contexts

791

In the example (context) Two persons A and B. 6 (a) (timeslot 1) A and B have a value of walking speed and distance between them, different from that observed in 6 (b) (timeslot 2) is observed, its direction remains constant. OSU and the following rules conceptualise the stated context: 1) A follows B If (Speed (A) = Speed (B)) and (Direction (A) = Direction (B)) R (1) 2) A B continues and reaches for (event E11) If (Speed (A)  Speed (B)) and (Direction (A) = Direction (B)) R (2) Using the above rules, it is easily distinguished that A follows B. But in other cases, this facility does not exist, as they try to go unnoticed [24], and not to leave evidence, can not be conceptualized their actions, being necessary inferring the instants a note that follows leaves B. Algorithm 1, OSU uses classes and DAML-Time Є SOUPA, conceptualizing those moments: Count = 0 Repeat hasTime (? t) If (? T> = threshold) Then OSU: has_a_walk_speed (A, VC1?) OSU: has_a_walking_direction (? B? DC1) LEAST (VC2, s?)

sqwrl: LEAST (DC2, s1?)

swrlb: equal (VC1, VC2?)

sqwrl:

swrlb: equal

(DC1, DC2?) → sqwrl: select (VC1, VC2, DC1???) If (? VC1

? VC2

? DC1

? DC2) then

Count ++ endif endIf threshold = threshold + Incthreshold Until #observations If Count = #Observations then a sqwrl: select ( "Person A Person B following at") endif

Treatment algorithm 1. Temporary Event "person is someone else"

Algorithm 1 in the following for temporarily conceptualize e1 are presented: 1) The current time in seconds hasTime (? X,? T) 2) Create time windows to check the occurrence of e1, with If (? T  threshold) Then. The threshold value should be set in the algorithm and its unit in seconds [24] 3) Compare walking speed of A and B. VEO: has_a_walk_speed get walking speed of A and B. sqwrl: Least gets the value of the rate hike and swrlb A: equal (VC1, VC2?) Compares the walking speed of A and B. 4) Compare the direction of A and B walk. VEO: has_a_walking_direction obtains the address of A and B. walk sqwrl: LEAST (DC1, s?) Gets the value of the direction of walk and swrlb A: equal (? DC1, DC2) compares the address hike and B.

792

S. Arias T et al.

V 5) If walking speeds and directions of A and B are equal, (If (? VC1 = VC2 ? DC1 = DC2) then), then the count counter is incremented by 1, its value is the number of moments that A follows B. 6) Repeat steps 1 through 5, the number of times indicated in #observations. This value is set in the algorithm and represents the number of seconds to ensure sufficient A follows B [24]. 7) If count equals #observations (If Count = #observations then), A follows B. Algorithm 1 in a context of video surveillance is conceptualized. In the next section, on the benefits of OSU applied to ubiquitous computing contexts are detailed.

4 Experimentation CASAS conceptualizing contexts [4] project The events recorded in the CASAS [4] project (project) are as follows: 1) Call by telephone. Participants walk around the room with the phone in hand, looking for a number in the phone book, dial the number and hear a message on the phone. The recorded messages correspond to recipes that participants write in a notebook. 2) Wash hands: Participants walk in the kitchen and near the sink to wash your hands. They use soap and dry your hands with a paper towel. 3) Cooking: Participants cook oatmeal soup according to the recipe. To do this participants pour water into a pot according to the measure, pour the oatmeal and then add sugar and raisins. 4) Eating: In the dining room, participants oatmeal and soup are served a medicine. 5) Clean: Participants take the dishes and take them to the sink to wash them with soap and water. The events are recorded in the project from 12:43 pm to 12:54 pm. Context is performed tasks at lunch, OSU conceptualized as follows: 1) Call by phone: a) OSU walking participant uses the property: has_a_walk_speed or track a person (see section E). b) Participant walking through the dining room: V V SOUPA: isComposedOf (DR1, S?) OSU: hasState (S, state?) sameAs (S, V VEO: MovementSensor_08) swrlb: equal (state, true?) ! sqwrl: select (“person walking in Dining Room”) R (3) R (3): • The SOUPA property: isComposedOF obtains sensor data. • OSU property: hastSate, get the sensor status.

Soupa and Integration of Ontologies Verl for Conceptualizing Contexts

793

• SameAs compares the property if the sensors (swrlb: equal (state, true)) correspond to an existing sensor in the dining room1. • The swrlb property: equal, determines whether the sensor was activated and therefore the participant walks through the dining room. c) Participant takes the phone book to find a number: V V SOUPA: isComposedOf (DR1, sameAs (S, V S?) OSU: hasState (S, state?) OSU: Phone_Book_Sensor_I08) swrlb: equal (state, true?) ! sqwrl: select (“person seek a telephone number”) R (4) d) Participant telephoning:

V V j.4: isComposedOf (DR1, S?) VOSU: hasState (S, state?) sameAs (S, OSU: Usage_Phone_Sensor_Asterisk_26) swrlb: equal (state, true?) ! sqwrl: select (“person call by phone and writing the recipe”) R (5) 2) Handwashing: a) Participant walk from the dining room to the kitchen, use tracking persons described in section E. b) In the event participant washing hands in sink, AD1-A (or B-AD1) sensor should be active during the proposed in [25] time, as described in Algorithm 2:

If j.4: isComposedOf (DR1, S?)

OSU: hasState (S, state?)

sameAs (S, OSU: AD1-A)

swrlb: equal (state?,

OSU: hasState (S, state?)

sameAs (S, VERL: AD1-A)

swrlb: equal (state?,

True) then OSU: has_instant (S, t1?) endif If j.4: isComposedOf (DR1, S?) False) then OSU: has_instant (S, t2?) Endif if (t2-t1)> = Average_time [25] then sqwrl: select: ( "person washing the hands") endif

Algorithm 2. Conceptualization of a person washing hands in the kitchen sink

3) Cook: To conceptualize this event, the sensors food preparation I01, I02, I03, I04, I05; or stove burner sensor AD1-C must be active:

1

This rule has only been existing M08 sensor located in the dining room and not at all for purposes of space.

794

S. Arias T et al.

V V j.4: isComposedOf (DR1, OSU: hasState (S, state?) sameAs (S, OSU: V S?) cooking_sensor_AD1-C) swrlb: equal (state, true?) ! sqwrl: select (“person cooking”) R (6) 4) Eat and take medicine: This event uses tracking a person (me see section E) from the kitchen to the dining room and to see if the participant take medicine, R (7) verified that the I06 sensor is active: V V j.4: isComposedOfV(DR1, S?) OSU: hasStateV(S, state?) sameAs (S, OSU: MovementSensor_08) swrlb: equal (state, true?) j.4: isComposedOf (DR1, S1?) V V V OSU: hasState (S1, State1?) sameAs (S1, OSU: I06) swrlb: equal (State1, true?) ! sqwrl: select (“person eating”) R (7) 5) Clean: Use tracking a person (see section E) to conceptualise the participant walked from the dining room to the kitchen and Algorithm 2 is applied, varying the time that the water faucets are open, according to what was proposed in [25]. OSU conceptualized the context tasks carried out at lunch, describing logically each of their events. Algorithm 3 appoints the steps followed:

ContextGet the context describes the events L (E) For each event and in L (E) do Conceptualize classes and properties Check the Relationship Between Time and space classes Get rules and algorithms to conceptualize space and time to each event Eend ForDetermine the execution order of events eCheck the actual results with Data

Algorithm 3. Algorithm for conceptualizing a context

The following section describes the Algorithm 3 is used in another context of ubiquitous computing. A. Conceptualisation of contexts: “Help in managing external information and development task for someone with Alzheimer” External information and task management for people with Alzheimer described in the project Restart [5], are conceptualized by OSU as follows: a) Conceptualisation of aid in managing external information An example of this context is the system that helps a person with Alzheimer (person) to learn about the distance to the bus station and departure time thereof. The system records the hours that the person takes the bus and departure time thereof. When a person walks into the station, the system must:

Soupa and Integration of Ontologies Verl for Conceptualizing Contexts

795

• Check if the time is right. • If the walking speed of the person is the right time to reach the station. Algorithm 3 shows system operation as follows: 1) Event Description • The person walks toward a bus stop. • The system was registered previously, that at this time of day the person often take the bus home. • Real-time information shows that the bus It leaves in 5 min, and next in an hour. • If he system predicts that the person will lose the bus current pace of walking, you should intervene and advise the person to walk faster. 2) Conceptualizing events, OSU uses the following classes and properties: • • • •

Person DAML: TIME Interval, Instant. OSU property: has_direction_walking to get the address of the person. The SOUPA property: has_instant to define the time interval of execution of the event. • OSU property: has_event for taking the bus home event must occur in the previous time interval. The same property can be used for the event, the bus leaves within 5 min. • OSU property: has_speed_walking 3) The rules to be used to define the context are: • The person walks toward a bus stop V OSU: has_walking_direction (D, P?) sqwrl: same_as (D, Bus_direction?) ! sqwrl: select (“person walking at bus stop”) R (1RI) • The system registered earlier that at this time of day the person often take the bus home. V V V SOUPA: interval (I1, S?)V sqwrl: makeSet (s, I?) OSU: has_instant (I, s?) sqwrl: makeSet (II, I?) sqwrl:: EQUAL ! j2 (II, 10,00,00?): has_event E, “person take a bus for home”) R (2Ri) • Real-time information shows that the bus It leaves in 5 min, and next in an hour. V V SOUPA: has_Event (A, B?) sqwrl: makeSet (s, A?) j2: has_instant (I, s?) V sqwrl :: greatherthan (? 5, II, I) ! sqwrl: select (“The bus start in five minutes”) R (3RI) • Thesystem predicts that the person will lose the bus current pace of walking. The system alerts the person to walkVfaster. V SOUPA: has_speed_walking (V, P?) Verl: distance_from (D, BS?) sqwrl: V divide (VR, D, 5,0,0?) sqwrl: less ! sqwrl (V, VR?) Select (“person to walk faster”) R (4RI)

796

S. Arias T et al.

b) Help in the development of tasks for people with Alzheimer’s An example of this context is described below: The current time is in the morning, the person entered the bathroom and turn on the faucet sink. Take a few minutes, and remains motionless. In the morning the events of “tooth brushing and bathing” should have begun, but it has not happened that way. The system intervenes verbally ask the person if he has taken the toothbrush. It should be noted that a message must not be issued immediately, unless it is deemed necessary, and it should not be scheduled according to time but according to events that have not been made [26]. Algorithm 3 shows system operation as follows: 1) The description of the context is obtained from [26] 2) Events that are part of the context are: • • • • • •

The system interprets times are in the morning. The person entered the bathroom. The person turned on the sink. The person remains motionless. The system holds the event brushing and bathing morning. Issue a help message if events brushing and/or bath did not happen.

3) Conceptualizing events, OSU uses the following classes and properties: Person DAML: TIME Interval, Instant. Sensor ADC-3 (sink sensor) and M38, M39, M40, M41 (bath motion sensors [4]) The has_state property gets the status of the sensors. The has_instant property gets the start time of an event. The has_event property gets the events that must occur, depending on the moment. 4) The rules to be used to control the context are: • The system interpret times are in the morning V V Time (? T) sqwrl: lessthan (t, 09,00,00?) sqwrl: greatherthan (t, 08,00,00?) ! sqwrl: select (“Morning”) R (1CA) • The person entered the bathroom V V SOUPA: isComposedOf (DR1,VS?) OSU: hasState (S, state?) sameAs (S, OSU: MovementSensor_38) swrlb: equal (state, true?) ! sqwrl: select (“person in bathroom”) R (2CA) • The person turned on the sink V V SOUPA: isComposedOf (DR1, sameAs (S, V S?) OSU: hasState (S, state?) OSU: WaterSensor_ADC3) swrlb: equal (state, true?) ! sqwrl: select (“person turned on the sink”) R (3AC) • The person stands still V V SOUPA: isComposedOf (DR1,VS?) OSU: hasState (S, state?) sameAs (S, OSU: MovementSensor_39) swrlb: equal (state, false?) ! sqwrl: select (“person rMotionless emains”) R (4CA)

Soupa and Integration of Ontologies Verl for Conceptualizing Contexts

797

• Event brushing and bathing V morning V SOUPA: interval (I1, S?) sqwrl: makeSet (s, I?) SOUPA: has_instant V V V (I, s?) sqwrl: makeSet (II, I?) sqwrl :: greathethan (? II, 08,00,00) sqwrl: lessthan ! OSU (II, 09,00,00?): has_event (E, I1?) R (5CA) sqwrl: equal (E “? toothbrushing and bathing”) ! sqwrl: select (“Question: Has the person pick up the toothbrush?”) R (6A) The execution order of the rules is as follows:

If R (2CA) and R (3AC) then If R (4CA) then Execute R (5CA) Execute R (6CA)

endif endif Algorithm 4. algorithm 1CA (4). Algorithm for conceptualizing the example of representation of

tasks

Algorithms 3 and 4 are the result of detailed conceptualizing contexts in this section. However, there are other conceptualizations that will be conceptualized by OSU in future work described below.

5 Conclusions and Future Work This research has used the VEO and SOUPA ontologies, to give rise to an OSU ontology. The aim was conceptualizing videogilancia contexts. To detail the benefits of OSU, tracking people conceptualized context. Its efficiency was proven in the contexts “Help in managing external information and development task for someone with Alzheimer’s.” During the following work, a sub-ontology of scheduling, in order to achieve greater detail in conceptualizing related to people with Alzheimer design contexts. OSU assumes that speed and direction of walking people can be obtained using the methods described in the literature [21–23] These methods will be part of classes in a subsequent investigation, as will also the mathematical magnitudes related to SOUPA: RCC.

References 1. Tapia, D.: Multiagent architecture for ambient intelligence environments. Ph.D. thesis, University of Salamanca (2004) 2. Nevatia, R., Bolles, B.: A hierarchical video event ontology in OWL. Final report (2004). http://www.qword.com/search.php?s=404&q=rrc.mitrenwrrcowl-events-final-report.pdf. Accessed 03 Mar 2011

798

S. Arias T et al.

3. Chen, H., Perich, F., Finin, T., Joshi, A.: SOUPA: Standard Ontology for Ubiquitous and Pervasive Applications. In: International Conference on Mobile and Ubiquitous Systems: Networking and Services, pp. 258–267 (2004) 4. HOUSES Project: http://ailab.wsu.edu/casas/ 5. “Start Over”. Practical exercises of cognitive stimulation for people with Alzheimer’s. http://es. scribd.com/doc/9409653/Volver-a-Empezar-ejercicios-para-alzheimer. Accessed 24 Mar 2011 6. Held, A., Buchholz, S., Schill, A.: Modeling of context information for pervasive computing applications. In: Proceedings of the 6th World Multiconference on Systemics, Cybernetics and Informatics (SCI2002) (2002) 7. Chen, H., Finin, T., Anupam, J.: An ontology for context-aware pervasive computing environments. Special Issue Ontol. Distrib. Syst. Knowl. Eng. Rev. 18(3), 197–207 (2004) 8. Toro, C., Vaquero, J., Posada, J.: User experience, ambient intelligence and virtual reality in industrial maintenance and domain using protégé. In: 11th International Protégé Conference, June 2009 9. Devaraju, A., Hoh, S.: Ontology-based context modeling for user-centered context-aware services plaform. In: International Symposium on Information Technology, pp. 1–7 (2008) 10. Brickley, D., Miller, L.: FOAF Vocabulary Specification. Revision 1.47 edition, September 2003 11. Hobbs, J.: A daml ontology of time (2002). http://www.cs.rochester.edu/ferguson/daml/ daml-time-20020830.txt 12. Randell, D., Cui, Z., Cohn, A.: A spatial logic based on regions and connection. In: Proceedings of the 3rd International Conference on Knowledge Representation and Reasoning (1992) 13. “Vigilance & Security” Take of: http://www.vigilanceandsecurity.com/. Accessed 15 Mar 2011 14. Preibusch, S., Hoser, B., Gurses, S., Berendt, B.: Ubiquitous social networks - opportunities and challenges for privacy-aware user modeling. In: Proceedings of the Data Mining for User Modelling Workshop, June 2007 15. Silva, G., Ishikawa, T., Yamasaki, T., Aizawa, K.: Person tracking and multicamera video. retrieval using floor sensors in a ubiquitous environment. In: International Conference on Image and Video Retrieval a Ubiquitous Environment (2005) 16. PlaceLab Project: Take of http://architecture.mit.edu/house_n/placelab.html. Accessed 05 June 2010 17. Ontology for ambient intelligence. http://www.ontospace.uni-bremen.de/ontology/ modSpace/AmI.html. Accessed 14 Mar 2011 18. Horridge, M., Patel-Schneider, P.F.: Manchester OWL syntax for OWL 1.1. OWL: Experiences and Directions (OWLED 08 DC), Gaithersberg, Maryland (2008) 19. Masolo, C., Borgo S., Gangemi, A., Guarino, N., Oltramari, A.: Ontologies library. WonderWeb Deliverable D18, ISTC-CNR (2003) 20. Hois, J., Bhatt, M., Kutz, O.: Modular ontologies for architectural design. In: Proceedings of the 2009 Conference on Formal Ontologies Meet Industry. IOS Press, Amsterdam (2009) 21. García, C., Romero, N., Quiroga, J.: Detection and Tracking people in a crosswalk. In: XIV Symposium in Signal Processing, Images and Artificial Vision (2009) 22. Daamen, W., Hoogendoonr, S.: Free speed distributionsfor pedestrian traffic. In: Transportation Research Board Annual Meeting (2006) 23. Silva, G., Ishikawa, T., Yamasaki, T., Aizawa, K.: Person tracking and multicamera video. retrieval using floor sensors in a ubiquitous enviroment. In: International Conference on Image and Video Retrieval a Ubiquitous Environment (2005)

Soupa and Integration of Ontologies Verl for Conceptualizing Contexts

799

24. Techniques for monitoring and escort. http://es.scribd.com/doc/32479808/Tecnicas-deseguimiento-y-escolta. Accessed 18 Mar 2011 25. Rashidi, P., Cook, D.: Mining sensor streams for discovering human activity patterns over time. In: Proceedings of the IEEE International Conference on Data Mining (2010) 26. Patterson, D., Etzioni, O., Fox, D., Kautz, H.: Intelligent ubiquitous computing to support alzheimer’s patients: enabling the cognitively disabled. In: Proceedings UbiCog’02: First International Workshop on Ubiquitous Computing for Cognitive Aids (2002)

A Small Robotic Step for the Therapeutic Treatment of Mental Illnesses Hector F. Gomez A1(&), Diego Freire2, Elena Malo3, and Francisco Naranjo Cobo4 1

2

3

Universidad Técnica de Ambato, Ambato, Ecuador [email protected] Universidad Regional Autonoma de los Andes, Km 5 1/2 vía Baños, 180215 Ambato, Ecuador [email protected] Departamento de Arquitectura y Artes, Universidad Técnica Particular de Loja, Loja, Ecuador {semalo,cpcartuche}@utpl.edu.ec 4 Facultad de Sistemas, Universidad Técnica del Norte, Ibarra, Ecuador [email protected]

Abstract. Advances in Psychological therapies to treat mental illness have found their peak in the search for new treatments, especially when they are combined with technology. Technology gives some hope of improvement in treatments and patients. The main purpose is generate positive emotions and in this way, not only learn but also manage to retain learning in order to be applied in concrete and real facts. In this work we show the viability in the development of routines that can help improve the quality of life of people suffering from Alzheimer and Autism, diseases that lend themselves to the programming of routines related to the daily life of patients. The assignment can be considered as impressive because working with a robot awakens positive emotions in those who are developing robotic therapies. Keywords: Alzheimer


Autism
Robot
Routine

1 Context The advances in psychological therapies to treat mental illnesses have been of vital importance in recent times, especially by the combination with technology. The development of applications that allow interacting with patients suffering from mental illnesses is very important to improve their quality of life and the development of routine activities has benefits like reduces Anxiety, increase self esteem and confidence, allows some independence of them [1]. Appealing to new mechanisms gives some hope of improvement in treatments and patients. The main objective of this work is to show the scientific community the ease that computer programs have at present to develop support routines. Investigators are interested in finding out which robot was used, what were the goals of the application, how the robot was controlled, what kind of behaviours the robot exhibited, what kind of actuators the robot used and what kind of sensors the robot used © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 800–805, 2021. https://doi.org/10.1007/978-3-030-49932-7_74

A Small Robotic Step for the Therapeutic Treatment

801

[2]. This need is driving engineers to study human behaviour toward other humans and toward robots, leading to greater understanding of how humans think, feel, and behave in these contexts, including our tendencies for mindless social behaviours, anthropomorphism, uncanny feelings toward robots, and the formation of emotional attachments. Human robot interaction is a fascinating field and one in which psychologists have much to contribute, both to the development of robots and to the study of human behaviour [3]. We focus specifically on two diseases: Alzheimer’s and Autism. Therapeutic pet robots designed to help humans with various medical conditions could play a vital role in physiological, psychological and social-interaction interventions for children with autism spectrum disorder [4]. The therapies that can deal with this type of disease, find their support in computer components and robotics [5]. Therefore, Alzheimer’s disease is the most frequent type of dementia and its numbers increase, its symptoms have a great impact on patient’s life and society, so we would prepare routines for patients suffering Alzheimer disease. There is currently no cure, but there are several medications available to treat symptomatology and non-drug therapies that are proving to bring numerous benefits to these patients, such as the routine of the Eva robot that demonstrate that there are other methods by which we can treat people with Alzheimer’s. providing benefits such as temporal and spatial orientation, in which with an Eva routine you can know when you have to wake up because you will know that it is the right time in the morning, the ability to recognize the clothes to wear and the memory through retention for know where to go, and how to go home again [6]. The ultimate detection methods for cognitive impairment have a prediction of to 92.65 1.18 over the Alzheimers Disease (AD) Neuroimaging Initiative dataset, with a specificity of 92.19 1.56 and sensitivity of 93.11-1.29, and 96.68-1.44 over the Open Access Series of Imaging Studies dataset, with a sensitivity of 97.72-2.34 and specificity of 95.61-1.67 [7]. This makes possible to apply technological therapies proposed in this work as an early therapy approach. The following sections describe the state of the art of robot use in the diseases mentioned, as well as the methodology applied to create the first routines of the NAO, which allowed experimenting with the EVA robot, the applications were developed by students of second level of the career of Psychopedagogy1, in which a high level of attention and interest was observed in the development of new ways of applying the research methodology for real cases of therapeutic help. The objective of [8] is to present the development of our pilot experiment protocol where children with ASD will be exposed to the humanoid robot NAO. Anticipated results are the real initial response and reaction of ASD children during the HRI with the humanoid robot. The routines developed by our team are based on the fact that performing physical actions repeatedly allows the person to learn these movements better so that they can do them without having to think about what they are doing; it is so in the case of Alzheimer; so the repetition of activities in the daily routine will make the patient retain them for much longer; because they are autonomous for the subject. In addition, exercise can modulate brain tissue, increasing nerve cells in critical areas such as the hippocampus (one of the most affected by Alzheimer’s). The programming environment on which the tools will be developed in Choreography (Fig. 1):

1

Routines: https://www.facebook.com/multimodalclass/.

802

H. F. Gomez A et al.

Fig. 1. General choreography interface

Fig. 2. EVA in the laboratory

The simulator shown in Fig. 1 allows you to check the routines before they are executed in EVA. Our laboratory (Fig. 2) located in the Faculty of Human Sciences and Education at the Technical University of Ambato, allows to test the routines developed in the simulator, and to directly recognize the behaviour of the robot. The developed routines can be requested in our work group communicating with the authors of this research work. The surveillance camera will allow observing patients in direct relation with EVA.

A Small Robotic Step for the Therapeutic Treatment

803

2 Developed Routines Routine 1: 1. Relaxation music throughout the routine so that the child sits in a pleasant and quiet environment to perform certain movements and the most effective way to get the child’s attention. 2. Lift the arms one by one. 3. Lower the arms as a whole. 4. Head movement Right-Left. 5. Flex the legs with the arms raised (this movement will be repeated 3 times). 6. Take a spin. 7. Sit down. 8. He will say thank you. Routine 2: The creation of the routine for a person with Autism in the Nao Robot was developed with the intention of integrating a mechanism that allows the individual to remember and establish a daily grooming routine and in this way can carry this cycle passively and appropriately in the environment, without disturbing the sensitivity of the subject. Autism is a neurobiological disorder of development that affects the human being in terms of its relations with the outside world, thereby infringing its social growth negatively, so that new techniques have been developed for its treatment, among them we have the implementation of a new invention known as Robot Nao, it was created with skills to help people in their learning and ensure a quality life. Among the characteristics of this disorder are: Difficulty to speak with people and relating to society. Usually they do not look at the person who is speaking to them. May have limited interests and repetitive behaviour. They have an established and permanent routine. Create their own world. This routine consists of three phase the first is for brushing the teeth, the second is to take a shower and the third is to wash your hands. Each of these movements was created in a timeline giving autonomy to the routine. The first phase: The robot enters the bathroom with the child and they pose in front of the mirror they approach the toothpaste and toothbrush they continue to take the toothbrush to their teeth and begin with the brushing movement they continue spitting and washing the toothbrush. The second phase: The robot shows the child with a clear language that it is time to take a shower so the child together with the robot go to the shower, the robot teaches the child how to take off his clothes and get in the shower, open the tap and the movements you must do, such as taking the shampoo, rubbing it over your head and proceeding to wipe himself, get out of the shower and dry himself.

804

H. F. Gomez A et al.

The third phase: The robot will indicate to the child how to wash their hands, it will stretch its hands in front of the wash will take the soap will open the water tap will rub their hands, rinse them and finally close the tap and dry. Keep in mind that the whole routine was done with every detail so that the child does not forget to do anything, and in this way begins to memorize this grooming cycle. Routine 3: For the realization of the present routine one of the characteristics of autism was taken into account, which is the lack of orientation in time and space, in addition to considering as documentation the project carried out by the School of Informatics and the Faculty of Psychology of the UNED who used robotics in peak-therapeutic interventions in children with autism spectrum disorders (ASD). The routine is based on the morning schedule since getting up at 6 am and asking the child to get up next to the robot, likewise asking the child to change the sleepwear for the school uniform while the robot gives the order illustrates movements very fine so that the child can replicate them, at the end of the routine the well-made phrase is repeated, which stimulates the child, as the next step is the breakfast in the same way Eva leads the child holding his hand and showing the movement that must be done to take the food to the mouth, after this action we have the brushing of teeth where Eva shows the child how to take the paste with the brush and reminds him that the movement should be from left to right and simultaneously from top to bottom, The last thing that contains our routine is to drive the child to the door so he can go to school, which marks the end of this routine prepared for morning hours. This procedure has been carried out in order to be able to work together with the therapy and to strengthen the memory capacity of the child, to try to capture their attention and to fix their attention focus on the correct things to do.

3 Conclusions and Future Works The Nao robot called Eva is very useful for the creation of various routines that allow a person with autism to develop in an appropriate environment and without disturbing their sensitivity. The detailed explanation of each movement belonging to the different parts of the routine causes the child to adequately understand all the movements that must be done to achieve a correct cleanliness and in this way begin to retain the cycle in his memory to determine it as his permanent routine. This work shows the ease of choreography software for the programming of routines that can be implemented in the therapies of patients with Alzheimer’s and autism. It is observed that the behaviour of the robot can be modelled in order to achieve the process of interaction with patients, thus achieving a partial result that is the interaction of the robot with the students who programmed their routines. This was the first impact that was found during the experimentation, the attention that the students have in a class guided by a robot and to which they must teach routines that later will be implemented in the treatment of a disease became one of the fundamental findings of this process. In future work we

A Small Robotic Step for the Therapeutic Treatment

805

intend to configure the behaviour of the robot using ontologies to establish patterns of routines that can be combined in the treatment of patients. Well, we believe that certain robotic Alzheimer´s routines can be applied in autism and in the same way in the opposite way.

References 1. Esther Heerema, M.: The benefits of routines for people with Dementia, 29 May 2019. https:// www.verywellhealth.com/using-routines-in-dementia-97625 2. Erich, F., Hirokawa, M., Suzuki, K.: A systematic literature review of experiments in socially assistive robotics using humanoid robots, Robotics (2017) 3. Broadbent, E.: Interactions with robots: the truths we reveal about ourselves. Ann. Rev. Psychol. 68, 627–652 (2017) 4. Bharatharaj, J., Huang, L., AL-Jumaily, A., Elara Rajesh, M., Krageloh, C.: Investigating the effects of robot-assisted therapy among children with autism spectrum disorder using biomarkers. In: Materials Science and Engineering (2017) 5. Taylor, H., Pettitt, J.: How robot therapists can fill a gap in health care, 21 July 2016. https:// www.cnbc.com/2016/07/21/how-robot-therapists-can-fill-a-gap-in-health-care.html. Accessed 20 Nov 2017 6. Roca, J.: Alzheimer y otras patologías neurodegenerativas. Un reto del siglo XXI, Universidad Carlos III de Madrid UCM3, Madrid (2019) 7. Alam, S., Goo-Rak, K., Kim, J.-I., Chun-Su, P.: Twin SVM-based classification of alzheimer’s disease using complex dual-tree wavelet principal coefficients and LDA. J. Healthcare Eng. 2017, 12 (2017) 8. Shamsuddin, S., Yussof, H., Ismail, L., Akhtar Hanapiaj, F., Mohamed, S., Ali Piah, H.: Initial response of autistic children in human-robot interaction therapy with humanoid robot NAO. In: International Colloquium on Signal Processing and its Applications (2012) 9. Robles-Bykbaev, V., et al.: An intelligent ecosystem to support the development of communication skills in children with autims: an experience based on ontologies. In: MultiSensory Stimulation Rooms, and Robotic Assistants. IGI Global Diseminator of Knowledge (2017)

The Use of Gamification in Evaluating Children’s Emotional Intelligence Abbas Narimani1(&), Ali Khaleghi1, Hadi Haedar1, and Farzad Semnani2 1

2

Imam Khomeini International University, Qazvin, Iran {abbas.narimani,h.haedar}@edu.ikiu.ac.ir, [email protected] Science and Research Branch, Islamic Azad University, Tehran, Iran [email protected]

Abstract. Intelligence is the carrying out of abstract thought, like the ability of learning and adopting to the environment. In general this ability would be defined by a common factor. Deferent types of intelligence, generally are distinguished according to the information on which they operate. Emotional Intelligence (EI) is a type of intelligence which operates on emotional information. It is mentioned that personality disorder is linked to various aspects of EI. The common method for studying the status of individual’s EI is presented in questionnaires which are usually boring and exhausting for kids to fill them out. MSCEIT is one of this assessment tools which has performance-based measurement approach. To make the assessment process more engaging and entertaining for kids, we are going to gamify the whole process. In the first step, we have been designed and implemented a mini-game, which its main mission is to measure kids’ EI by focusing on gamifying Faces and Facilitation tasks which are presented in MSCEIT. As the information gathering process finished, we are going to analyze this information by implementing classification methods using machine learning algorithms to improve the speed and accuracy of the assessment process. Keywords: Emotional intelligence learning
Assessment


Gamification
MSCEIT
Machine

1 Introduction Salovey and Mayer defined emotional intelligence as the: “Ability to monitor one’s own and other’s feelings and emotions, to discriminate among them and to use this information to guide one’s thinking and actions.” [1]. Emotional feeling is kind of person’s ability to interact with his feelings and others. Thus, emotional intelligence is a very important factor in predicting people’s performance in educational activities, occupation, ability to negotiate, control anxiety, trust and leadership ability. It also has a direct impact on the cognitive abilities of individuals that can directly affect the performance of individuals in the workplace [2].

© Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 806–813, 2021. https://doi.org/10.1007/978-3-030-49932-7_75

The Use of Gamification in Evaluating Children’s Emotional Intelligence

1.1

807

Emotional Intelligence Assessment Methods

Emotional intelligence is divided into two general concepts. A model based on the ability that emotional intelligence is presented as a set of cognitive abilities and a mixed trait-ability model that is a combination of personality traits [3]. Over the past decades, different methods have been developed to measure emotional intelligence. These tests are separated from two perspectives. First, they are divided into two conceptual bases mentioned above. Second, these ways use two different measurement methods of performance and self-report. In the research [3] mentioned, if emotional intelligence is introduced as an ability, the performance test is more valid, and in cases where emotional intelligence is considered as a set of unrelated ability attributes The selfreport may be appropriate. In performance test the ability and capacity of the emotional intelligence of the person measured by placing him in different positions. In this type of test, the cognitive ability of individuals is directly measured. While in the self-report test, more attention is paid to personality and non-cognitive characteristics [3]. A critique on self-report is it actually showing the expected performance level instead of the actual performance level of emotional intelligence [3]. So far, various tests have been developed to measure emotional intelligence [3]. Among these tests, the MSCEIT which introduced by Mayer, Caruso & Salovey, is based on the measurement of emotional intelligence performance [4]. The validity of this test has been reviewed in several studies and has been validated by all [3, 5]. According to Mayer and Salovey, Emotional Intelligence has four main components: (1) the ability to understand your feelings and others (perceiving emotions); (2) the ability to create a feeling and then to use it (Using emotion to facilitate thoughts); (3) the ability to understand complex emotions and how to transfer a feeling from one level to other level (Understanding emotions); (4) the ability to manage self-emotions and others [6]. Each of the above abilities is measured by two tasks [7]. The ability to understand feelings is measured by the faces and pictures task. The ability to facilitate thoughts is tested through sensation and facilitation tasks. Understanding emotions is measured by blends and changes, and managing emotions is measured by emotion management and emotional relationships [7]. For example, in the face task, the person is presented with an image of a face, and he is asked to indicate which feeling or feelings are perceived from the image, and for the picture task, either for example, an image of one scene is displayed or the perceived feelings of that image are asked. In the sensation task, respondents display a feeling, then they express the sense of it. For instance, they may generate a feeling of envy and decide how cold or hot it is. In the facilitating task, respondents will discuss the feelings that accompany a cognitive task or behavior. For example, how euphoria feeling accompanies going to the park activity. In the blend task, people will also figure out how to combine several feelings and shape a new sense. In the section of changes task, they choose the emotion that emerges as a result of the exacerbation of a feeling. In the emotion management section, people are also asked to express what action they do that will trigger a particular feeling. Finally, in the emotional relationships task, the respondents point out the practical responses that have the most impact on their feelings [7].

808

A. Narimani et al.

Using this test and other tests to measure emotional intelligence for children can be difficult and boring. In addition, the need for attending a clinic to test also makes the test process more difficult and the availability of this test can help and strengthen parents to measure their children emotional intelligence. Gamification approaches can be used to reduce costs, as well as motivate these tests, as well as increase the availability of this test [8, 9]. Also, with the help of the methods of machine learning, it is possible to reduce the need for personal contact and communication with cognitive clinics. 1.2

Using Game to Help Assessment

Generally, for assessment of emotional intelligence questionnaires are used and provided to the people. In self-report method, a person answers the questions related to his mental and emotional states. Also in the other-report method, the opinion of others toward the individual that is being tested are asked in the form of questionnaires. Another method for assessment of emotional intelligence is the ability test, in this method instead of asking you questions like how kind are you or how kind is a person in the view of others, individual is put in a situation that needs a reaction [3]. This method seems more trustworthy because it observes the reaction of the person in a specific situation [3]. Due to the above mentioned facts, it can be expressed that the assessment process of emotional is time consuming and costly. Furthermore, putting someone in a situation like exam can affect his performance negatively due to the stressful conditions. Information and communication technology, have high potentials to facilitate the mentioned assessments such as better standardization of management, increase of timing presentation and response latency, facilitating the data gathering and management, ability to record and monitor dynamic indicators (such as cognitive markers, reaction time latency or other cognitive indicators like walking patterns). A promising approach in this state is the usage of games for diagnosis or teaching that basically are not used for entertainment purposes [8]. These games are known as serious games and are used in various applications like education, rehabilitation or military training so on and so forth. These games include an interesting and entertaining component as motivational element for a purpose other than entertainment such as learning, assessment or teaching. Furthermore, these games should be entertaining and challenging and have positive effect on adults.

2 Background 2.1

Gamification in Health

Gamification has been introduced in the second half of 2010 as a new trend in business and marketing [9]. The growth of this concept is obvious by examining the number of research papers and articles presented in this area, especially its use in the health area [10]. Gamification is used as a tool for using game design elements in non-game context and activities that are often used to change the attitudes and behavior of

The Use of Gamification in Evaluating Children’s Emotional Intelligence

809

individuals [11, 12]. Gamification means that the software product is designed with components such as game components, but the product itself has goals and functions that are not similar to the goals or functions of the game. [13] gamification tries to participate, sustainability and achievements by promoting the motivational power of games, [14–16]. Gamification has grown dramatically with advances in some of the digital technology factors, such as ease of access and affordability, and new forms of data tracking and expanding the use of games that have a deep root in the culture of humanity [17]. The main components of the gamification are [18]: 1. 2. 3. 4. 5. 6. 7. 8.

Score: A numerical representation of how subjects function Badges: Visual representation of the achievements of the subjects during the game Game process: shows the situation of testable parameters Avatar: Visual representation of subjects or the symbol of persons Level: shows the degree of difficulty of each stage based on the player’s experience Social graphic: Indicates the way and the relationship between different players Mission: sign of the status and activities that must be done through the game. Competition: A system to increase players’ motivation to perform a specific mission.

In fact, in gamification, we are looking for a way that with the help of game features such as scoring system, competition, avatars, graphic, mission, and game level to reach a goal beyond entertainment [19]. 2.2

Developed Games in This Area and Cognitive Domains

Various games have been developed to assess cognitive abilities, but no particular attention has been paid to the evaluation and improvement of emotional intelligence. There are dispersed activities in this field. For example, researchers have designed and implemented a game of six mini-games to improve and enhance emotional intelligence in deaf or with hearing loss children. The study notes that there are improvements in parts, but there is a need for further investigation [20]. Also, Revelion has gamified the emotional intelligence process called Emotify. In this game, people are placed in different situations and they are asked to make their own reaction to this situation. Or in other situations, they are asked to recognize the feelings of other people in the game and report it. In the research [21], the game is designed to measure the impact of emotional intelligence on playing, as well as the learning process of children and their relationship with each other. The study found that people with higher emotional intelligence will have better performance during play and in the learning process than others. 2.3

Machine Learning

Machine learning (ML) is one of main branches of artificial intelligence and the main goal of it is innovate and Discover and extract patterns and algorithms that allow machines to learn without direct planning. Machine learning is two model: Supervised ML Which teaches when a curriculum is taught in a predefined set, such as educational samples or training collections, and

810

A. Narimani et al.

Unsupervised ML That’s when the program is presented with data, it trains itself. When the program is presented with data, it uncovers patterns and relationships in those data and educates itself [22]. In traditional way, after conducting emotional intelligence test, experts and psychologists manually check the obtained information and determine the level of people. Using machine learning in this section, in addition to facilitating the process of leveling and categorizing individuals, it extracts possible new patterns of existing data.

3 Process of the Design and Implementation of Mini Games The process of design and implementation of this game is formed in several phases through multiple sessions with psychologists and experts and the technical team, discussing the implementation, game frame and the required components by psychologists. Here we will explain this process through multiple phases. In the first phase, we generally discus the characteristics of emotional intelligence and how to test and strengthen it. At the first session of this phase, experts and psychologists discussed their desired components for measuring children’s emotional intelligence, and discussed how to transform these components in to a game. For example, during this meeting, in the first stage it was decided to implement two tasks, face tasks and a facilitating task in the form of a mini-game. The initial idea raised in this section was that the questions asked in the form of “What are you feeling at this position?” and the child chooses among the faces that each one passes an emotion will answer these questions. In this way, by choosing a mask, his ability to detect face emotion is measured, and he is also given a facilitation by connecting a feeling to a position. The implementation team concluded after the meeting that the challenge or question by playing characters and the use of simple mechanics such as choosing from existing options or interconnecting options could be the best way to get child’s response and answer. To do this, after describing the position or question by character, the child is asked to choose the correct option according to her feelings or opinions. The idea of the game design team was reviewed at another meeting with psychologists and the format of the game was confirmed by the experts to begin the implementation stages and visual design of the game. Most of the sessions in the second phase consisted of discussions regarding the game architecture, implementation method and character design. The Unity game engine was used to implement the game since it is well suited for 2 dimensional game designs and has quite good performance and efficiency. Since we needed to collect data related to child’s behavior during game play, a logging system was designed to monitor the child’s interactions with the game. In addition to receiving information about the child-selected option, this system also provides additional information such as the amount of time spent by the child in each question to answer it, the time taken to complete the whole minigame, whether each question was answered correctly or not, and it will also be logged if the child has not answered a question and passed it. To make it more interesting and engaging for kids, we also used a specific leaderboard as a competition element which is one of the main elements of Gamification. After passing each question, the status of the player will be presented in the aforementioned leader-board. Desire for status could keep kids motivated and encourage them to answer next questions (Fig. 1).

The Use of Gamification in Evaluating Children’s Emotional Intelligence

811

Fig. 1. Game character explains a situation and asks the player to choose the face which is closest one to his/her feeling in that situation.

Fig. 2. After passing each question, the status of the player will be presented in a leaderboard. Here it shows that 44% of players chose the laughing face.

In Phase three, after the implementation of the demo version, the result of the implementation was presented to the psychologists, and after interacting with the implementation version, they amended their corrections for each question with the technical team. The modifications were applied until the experts and the technical team reached an agreement to release the final version of the game ready to test the children (Fig. 2). The fourth phase will be test and the analysis of the results. To perform test, certain number of children will be selected and first, their emotional intelligence will measure

812

A. Narimani et al.

by the MSCEIT test. After this step, children will play with the mini-game and the results will be collected. After this stage, by using machine learning and Supervised Learning method, collected data will be analyzed without the involvement of experts, and the results obtained in this way will be compared to the results obtained through the MSCEIT questionnaire.

4 Conclusion Emotional intelligence is a kind of person’s ability to interact with his and others feelings. In this way, emotional intelligence is a very important factor in predicting the performance of individuals in the educational, occupational and their future affairs. Measuring the status of children’s emotional intelligence can help experts and psychologists to adopt an appropriate procedure to strengthen it. Among the methods for measuring emotional intelligence, the MSCEIT test is a valid performance-based test. With the use of Gamification, we’ve tried to make the process of measuring more engaging and engaging for the children and also increased the availability of this test by making it available through the smartphone platform.

References 1. Mayer, J.D., Salovey, P.: What is Emotional Intelligence? In: Salovey, P., Sluyter, D. (eds.) Emotional Development and Emotional Intelligence: Educational Implications, p. 331. Basic Books, New York (1997) 2. O’Boyle, E.H., Humphrey, R.H., Pollack, J.M., Hawver, T.H., Story, P.A.: The relation between emotional intelligence and job performance: a meta-analysis. J. Organ. Behav. 32 (5), 788–818 (2011) 3. Goldenberg, I., Matheson, K., Mantler, J.: The assessment of emotional intelligence: a comparison of performance-based and self-report methodologies. J. Pers. Assess. 86(1), 33– 45 (2006) 4. Mayer, J.D., Salovey, P., Caruso, D.: Emotional intelligence as Zeitgeist, as personality, and as a mental ability. In: Bar-On, R., Parker, J.D.A. (eds.) Handbook of Emotional Intelligence: Theory, Development, Assessment, and Application at Home, School, and in the Workplace, pp. 92–117. Jossey-Bass, San Francisco (2000) 5. Mayer, J.D., Salovey, P., Caruso, D.: Emotional intelligence: theory, findings and implications. Psychol. Inq. 15, 197–215 (2004) 6. Ingram, A., Peake, W.O., Stewart, W., Watson, W.: Emotional intelligence and venture performance, J. Small Bus. Manag. 57, 780–800 (2017) 7. Mayer, J.D., Salovey, P., Caruso, D.: Measuring emotional intelligence With the MSCEIT V2.0 (2003) 8. Khenissi, M.A., Bouzid, Y., Essalmi, F., Jemni, M.: A learning game for deaf learners. In: 2015 IEEE 15th International Conference on Advanced Learning Technologies, pp. 418–422 (2015).. https://doi.org/10.1109/icalt.2015.98 9. Seaborn, K., Fels, D.I.: Gamification in theory and action: a survey. Int. J. Hum. Comput Stud. 74, 14–31 (2015)

The Use of Gamification in Evaluating Children’s Emotional Intelligence

813

10. Hamari, J., Koivisto, J., Sarsa, H.: Does gamification work? - a literature review of empirical studies on gamification. In: Proceedings of Annual Hawaii International Conference on System Sciences, pp. 3025–3034 (2014) 11. Deterding, S., Sicart, M., Nacke, L., O’Hara, K., Dixon, D.: Gamification. using gamedesign elements in non-gaming contexts. In: Proceedings of the 2011 Annual Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA 2011, p. 2425 (2011) 12. Wu, M.: Gamification and Serious Games (2017) 13. Kasurinen, J., Knutas, A.: Publication trends in gamification: a systematic mapping study. Comput. Sci. Rev. 27, 33–44 (2017) 14. Garris, R., Ahlers, R., Driskell, J.E.: Games, motivation, and learning: a research and practice model. Simul. Gaming 33(4), 441–467 (2002) 15. Sweetser, P., Wyeth, P.: GameFlow: a model for evaluating player enjoyment in games. Comput. Entertain. 3(3), 3 (2005) 16. Hsu, C.L., Lu, H.P.: Why do people play on-line games? an extended TAM with social influences and flow experience. Inf. Manag. 41(7), 853–868 (2004) 17. Deterding, S.: Gamification for motivation. Interactions 19, 14–17 (2012) 18. Wood, L.C., Reiners, T.: Gamification. In: Encyclopedia of Information Science and Technology, pp. 3039–3047 (2015) 19. Lumsden, J., Edwards, E.A., Lawrence, N.S., Coyle, D., Munafò, M.R.: Gamification of cognitive assessment and cognitive training: a systematic review of applications and efficacy. JMIR Serious Games 4(2), e11 (2016) 20. Nakpong, N., Chanchalor, S.: Interactive multimedia games to enhance the emotional intelligence of deaf and hard of hearing adolescents. Int. J. Instr. 12(2), 305–320 (2019) 21. Yang, J.C., Quadir, B., Chen, N.-S.: Effects of children’s trait emotional intelligence on digital game-based learning. Int. J. Hum. Comput. Inter. 35, 374–383 (2018) 22. Greenes, R.A.: Clinical Decision Support: Clinical Decision Support The Road to Broad Adoption (2014)

Using Gamification Based on Mobile Platform in Therapeutic Interventions for Children with Dyslexia Mahsa Behnamghader(&), Ali Khaleghi, Pegah Izadpanah, and Farzaneh Rahmani Imam Khomeini International University, Qazvin, Iran {mahsa.behnamghader,pegahizadpanah}@edu.ikiu.ac.ir, [email protected], [email protected]

Abstract. Dyslexia or Reading disorder is one of the most common learning disabilities which has a significant impact on the whole life of people who have dyslexia, for instance on the student’s educational outcomes and grades. Therapeutic intervention Specific Learning Disorders (SLD) usually takes more time alongside day-to-day educational activities at school. Many of these people, because of the difficulty and coercion they spend on extra time to carry out the types of activities that are usually involved, as well as the high costs these interventions impose on their families, makes them less likely to be motivated and have a low level of confidence. If a reading disorder can be practiced correctly, it can be resolved. So using gamification could potentially increase the motivation of the dyslexic students. The purpose of the project is using gamification on gamified interventions for the reading disorder for children ages 6 to 8 in the mobile platform. In this way, applying gamification techniques would increase motivation, participation, and desire of the individual to continue the process of cognitive rehabilitation. Keywords: Specific learning disorder Gamification
Technology


Therapeutic intervention
Dyslexia


1 Introduction Learning disorders are in the category of neurodevelopmental disorders or language processing disorders that occurs due to the specific functions of the brain [1]. This disorder is often associated with abnormal functions of the brain in the field of speech [2]. Specific learning disorder is one of the most common neurodevelopmental disorders in childhood [3]. In the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders with final changes, the term “learning disorder” was named specific learning disorder that includes reading (Dyslexia), writing (Dysgraphia), and calculation (Dyscalculia) disorders [1]. Early symptoms of this disorder appear in pre-school years, but the official diagnosis of the disorder is done after the official start of the academic process and, if not properly assessed, it has a potential impact on an individual’s inconsistent performance and creates weakness in multiple areas of his/her life [4]. © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 814–824, 2021. https://doi.org/10.1007/978-3-030-49932-7_76

Using Gamification Based on Mobile Platform in Therapeutic Interventions

815

Children with learning disorders are not lazy or retarded children, they have high intelligence like other children, however, they only have difficulty in analyzing their brain information [1]. This disorder, due to the individual’s poor performance and the development of difficulties in the individual’s social aspects, in the long run, causes depression and anxiety. About 15% to 20% of the world’s population suffers from the specific learning disorder [4]. Nearly 70% to 80% of these people have dyslexia, which have difficulties in reading and writing [5]. Considering the prevalence rate and increased incidence of learning disability in reading, as well as its significant effect on the individual’s academic process, the need for improvement and intervention in this field becomes more important. Also, the individual’s academic process will affect his/her career future, social status, and interpersonal communication in the long run. Failure in the mentioned cases all will have a negative effect on the individual’s confidence, self-belief and motivation [1]. We can improve reading disorder and reduced its severity by applying exercises and correct interventions. There are various intervention methods for improving learning disability in reading. These interventions are used to improve the various aspects of reading disorder such as phonological awareness, phonetics, reading fluency, vocabulary and comprehension strategies [5]. People with learning disabilities in reading skill have difficulty in the following areas: 1) 2) 3) 4) 5) 6) 7)

Correct grasp and pronunciation of sounds Strengthening auditory memory and visual sequence Visual perception and distinction, memory and visual sequence Reading signs Exceptional words Tense, verb, and pronoun Understanding words: sentence, phrase, and text

In the field of intervention, interventional games to improve learning disorder include video games, serious games, as well as games with a gamification approach and other tools. The use of various technologies in cognitive therapeutic and rehabilitation issues is increasing day by day, thus our focus in this paper is on gamified interventions. The difference of gamification with targeted and serious games is that in the first case, the design and preparation of the game are only aimed at achieving a specific goal, while in gamification, the essential and required elements are combined with a gamified design [6]. There are various achievement systems in gamification, such as rewards and medals. The main purpose of these achievements is to motivate the individual to perform certain tasks that the person performs to achieve a particular achievement. In other words, gamification, in its core, is trying to increase motivation in the individual [7]. In research in the field of learning of patients with learning disorders, motivation is considered as an essential and pivotal element in teaching children with learning disorders [6]. The general objectives of this research are to examine and analyze the effectiveness of gamification of intervention method, to extract essential elements and characteristics

816

M. Behnamghader et al.

from the intervention method and those with learning disabilities for gamification of interventions. This is to examine the percent of the effectiveness of gamified intervention methods compared to non-gamified samples and the amount of long-term and short-term effectiveness of gamified intervention methods on an individual. Our basic assumption is that gamification of interventions has a significant impact on the individuals’ lack of motivation in learning and increases their level of participation and motivation in the learning process. Assuming that the existing cognitive interventional rehabilitation methods are effective, the following research questions come up: • Is it possible to increase the person’s motivation to start or continue the intervention method using gamification elements such as bonuses, scores and more …? • Does the proposed method based on the mobile platforms increase the effective participation of the individual in the implementation of interventions? • Does the proposed method in this age range improve the educational/educational process of children with disabilities? • Does personalization and localization of intervention increase its effectiveness? Research method • Initial internal validation: by health experts • Initial external validation: on a limited statistical population and under the supervision of health experts Limited numbers of rehabilitation centers provide intervention services for specific reading disability, and a wide range of patients with this disorder are deprived of such services or unable to afford the relevant costs. On the other hand, mobile and tablet devices are the most commonly used devices among the majority of different people of the community, so we decide to choose mobile as a platform for this research. In this study, we want to provide an appropriate implementation platform for the public by using standard gamified interventions under the supervision of experts in this field, so we would be able to take a step in the rehabilitation and improvement of these people. Regarding the studied research in the field of applying the gamification approach to the intervention process of dyslexic patients, fewer studies were conducted and further investigation was needed. Therefore, in this research, we intend to design an intervention game based on the characteristics of these individuals by using gamification to analyze its effectiveness with other approaches.

2 Literature Review The learning disability in reading skills negatively affects people with dyslexia since early childhood from various aspects. These negative effects have increased the importance of preparing structured interventional programs to meet the needs of these people. There are currently limited intervention programs or modules for children with dyslexia in Iran. In schools, teachers mainly focus on improving literacy skills such as reading and writing skills among dyslexic students, but they have little emphasis on the

Using Gamification Based on Mobile Platform in Therapeutic Interventions

817

training of cognitive skills. The teachers’ tendency for using traditional methods in teaching children with the disorder to overcome class problems cannot provide them with encouraging results and fail to overcome reading difficulties [8]. Brownell [9] says the traditional way of learning and teaching for children with dyslexia is not effective because it is stereotyped, teacher-oriented, and passive. Irwin suggested that teachers should emphasize the multisensory teaching and training method [10]. In addition, the number of interventional studies to improve cognitive function in dyslexic children in the country is relatively limited [8, 11, 12]. According to the conducted studies, a limited number of researchers have designed interactive multimedia computer programs for students to help them identify words and read [13–15]. However, no significant research has yet been done on modules to enhance cognitive skills such as attention, memory, and performance, and the importance of further studies is felt with regard to the limited number of researches and importance of this point. Categorizations already conducted in the field of dyslexia are cited in the studies below: Wagner and Torgesen categorized three cognitive-diagnostic functions of language: phonological awareness, phonological working memory, and rapid automatized naming, which are strongly related to reading and writing abilities [16]. In addition, visual skills, motor coordination, and visual-motor coordination are also crucial in the writing process [17]. Gomez found in his study that dyslexic children did not have good motor skills, which affected calligraphy and letter writing in copying tasks [18]. Most of the styles are intuitive. However, we invite you to read carefully the brief description below. Daud and Abas have developed an application for mobile phones called “Dyslexia Baka” in the Malay language. The purpose of this application is to help children with reading disorders to identify and recognize alphabets. The heuristic evaluation shows that the application in terms of content, approach, and multimedia elements has an average score of 4.4 to 4.8 [19]. Skiada et al. also developed a mobile app called “EasyLexia” that improves some of the basic skills of dyslexia in the English language. The preliminary assessment of this application with five children with dyslexia showed promising results in these areas, as the students showed progress in performance over a short period of time [20]. Purkayastha et al. used the Orton-Gillingham (OG) teaching approach in the “Dyscover” application, which is based on HTML5. The application utilizes new ICT technologies such as touch screens, accelerometers, gyroscope, voice recognition, audio production, and etc. in a multisensory approach. The application also personalizes the implementation of OG using Gardner’s theory of multiple intelligences [21]. Some studies incorporate phonics reading techniques in their application. Ahmad, et al. developed “Bijak Membaca”, an application for dyslexic children which applies phonics reading technique and multisensory approach with interactive multimedia, such as text, graphics, video, and animation. The heuristic evaluation of this application is considered success in integrating all of the learning strategies mentioned before [13]. “JollyMate” was developed to implement the phonics techniques as well. The application was designed as a digital notepad/school notebook that recognizes the handwriting characters for writing. Jolly phonics system is also used to teach letter-sound and letter formation [22]. The web-based game of “fastforword” provides tens of

818

M. Behnamghader et al.

cognitive rehabilitation intervention exercises for primary, secondary and high school students, in which each exercise is specifically designed to strengthen the skill of recognition and fast learning. Rello has designed a game in Android and iOS called “Dysexia”, which includes word training in the Spanish language. The content of this application includes linguistic and pedagogical criteria and corpus linguistics [23]. Di Tore developed “Madrigale” as a game that focuses on phonological training and visuo-spatial attention in reading disorder [24]. Gooch, in a platform called “ClassDojo”, used gamification to increase student motivation. In their research, they use medals as an element of gamification. The application focuses on the best teaching method by using a gamified application [25]. According to the studies conducted, it can be concluded that therapeutic interventions with the method of the serious game or game therapy and gamification, each have a significant effect on the improvement of a person with learning disabilities in reading. These effects include increased motivation, enhanced cognitive skills, and increased self-efficiency. Regarding the studied research, it can be concluded that gamification has a positive impact, but since the number of research done in this field is limited in proportion to the other cases. so we need the more and specialized research in different aspects.

3 Methodology and Process of the Design and Implementation of Mini Games Commonly, after identifying people with a learning disorder in reading skills by the experts in the field of cognitive science, several interventions appropriate to the mentioned disordered areas are proposed for the purpose of empowering and improving the disorder. In this research, we try to design interventional games for special learning disabilities in reading skills with a gamification approach. For a closer investigation, the age range for this study is considered to be 6 to 8 years old children. The focus of the proposed project can be a ground for the cooperation of cognitive science specialists, child psychologists and game developers, and it is expected that more effective and appropriate interventions will be identified and effectively gamified in this field via the created conditions. One of the main steps in designing cognitive rehabilitation games in the present study is gamification of intervention methods in accordance with the problems caused by this disorder for the people suffering from learning disorders is reading skill. A precise study of existing interventions to further understand cognitive aspects and to empower interventions on a person is one of the main requirements of this research. Since the main focus and purpose of this research and project is to evaluate and improve the learning disorder in reading, we first need to look at various types of cognitive rehabilitation interventions and scenario writing. The steps for doing this phase are as follows:

Using Gamification Based on Mobile Platform in Therapeutic Interventions

819

• Reviewing different problems caused by learning disorder in reading skill • Investigating factors affected by the disorder on the person suffering from learning disability • Studying the existing appropriate interventions for this disorder, which are selected according to the individual’s problems • Studying the existing cognitive games and categorizing them based on different problems these individuals with the disorder have • Choosing intended areas and interventions for gamification In order to move in line with the research goals, in addition to studying articles and academic books in this field, meetings with cognitive science specialists should be held in order to select the best intervention model for gamification. The purpose of these meetings is to raise awareness and increase knowledge of what each intervention entails, and the improvement and rehabilitation that come in the following. Understanding these relationships helps boost efficiency in designing and writing a scenario. In this way, the coordination of the written scenario with each intervention and its effectiveness will increase. 3.1

Reviewing the Designed Scenarios

To design the intended gamified intervention game scenario, we used the ideas drawn up by cognitive and child psychologists to create a game that matches the conditions of the children, who are the main audience of the game. In order to make the games attractive for the children of both sexes, it was important to use exciting features favorable for most children, to encourage their participation as much as possible. The purpose of this research was to use the platform of the popular Super Mario game. In the following, some of the designed scenarios are elaborated. In the scenario shown in Fig. 1, a word is shown to the player in the form of audio or text. The player must search for the letters of the word.

Fig. 1. In this mini-game, the player searches for the word of the declared word to continue the game

820

M. Behnamghader et al.

• Mechanics The player can take control of Mario’s movements and try to solve the challenge offered in the scenario. Mario chooses the desired block by hitting it with his head. Mario can jump over obstacles in his path and collect golden stars that are the same points. • Notifications The things that encourage and create competition among the players are displaying the scores, the remaining time, the number of lives, playing background music in the game and playing a special sound for each movement, displaying (text or audio) messages, and playing the pronunciation of each letter after hitting its associated block. • Notes – After finding the block with the correct letter, the player selects it with a jump and earns a score. – If the player selects the block containing the wrong letter, he/she loses a life. – In addition to the mission required to be accomplished in this scenario, the player is required to collect golden stars to increase his/her score. – At the end of the game, the remaining time and the earned points are recorded, thus the player can compete with other players. • Levels – In the next level, it is possible to ask the player to choose the letters of a word only by voice commands or by displaying the word (book). – In higher levels, letters with visual similarities will be used as traps among the blocks (such as c and o, or b and d). – At the last level of this game, the player is required to put the word’s letters in reverse order. • The game’s purpose Visual perception and distinction, memory sequence. In the next scenario appearing in Fig. 2, a word is shown to the player in the form of audio or text. The player must select blocks containing elements, of which the number of syllables is the same as the declared word. • Mechanics The player can take control of Mario’s movements and try to solve the challenge offered in the scenario. Mario chooses the desired block by hitting it with his head. Mario can jump over obstacles in his path and collect golden stars that are the same points. In addition, he can kill the moving owls (presumed enemies) by jumping over them. • Notifications The things that encourage and create competition among the players are displaying the scores, the remaining time, the number of lives, playing background music in the game and playing a special sound for each movement, displaying (text or audio) messages, and playing the pronunciation of each letter after hitting its associated block.

Using Gamification Based on Mobile Platform in Therapeutic Interventions

821

Fig. 2. The player must select blocks containing shapes that have the number of syllables or floods (parts) equal to the word declared.

• Notes – After finding the block with the correct letter, the player selects it with a jump and earns a score. – If the player selects the block containing the wrong letter or being hit by the moving owls (presumed enemies), he/she loses a life. – In addition to the mission required to be accomplished in this scenario, the player is required to jump over the moving owls and kill them to increase his/her score. – At the end of the game, the remaining time and the earned points are recorded, thus the player can compete with other players. • Levels – In the next level, a certain word is played for the player vaguely and rapidly. – Blocks containing pictures can be replaced with the pictures’ names. • The game’s purpose Focuses on individual phonemic elements, More advanced sound-letter identification. In the scenario shown in Fig. 3, a bunch of objects is shown to the player in the form of voice or text. The player must select blocks that are among the shown bunch of objects. • Mechanic The player can take control of Mario’s movements and try to solve the challenge offered in the scenario. Mario chooses the desired block by hitting it with his head. Mario can jump over obstacles in his path and collect golden stars that are the same points. Also, he can kill the moving mushrooms (presumed enemies) by jumping over them.

822

M. Behnamghader et al.

Fig. 3. The player must select blocks that have been declared in the group.

• Notifications Of the things that encourage and create competition among the players is displaying the scores, the remaining time, the number of lives, playing background music in the game and playing a special sound for each movement, displaying (text or audio) messages, and playing the pronunciation of each letter after hitting its associated block. • Notes – After finding the block with the correct letter, the player selects it with a jump and earns a score. – If the player selects the block containing the wrong picture or being hit by the moving mushrooms (presumed enemies), he/she loses a life. – In addition to the mission required to be accomplished in this scenario, the player is required to jump over the moving mushrooms and kill them to increase his/her score. – At the end of the game, the remaining time and the earned points are recorded, thus the player can compete with other players. • Levels Blocks containing pictures can be replaced with the pictures’ names. • The game’s purpose Contribute to understanding the concept of objects, understanding the concept of words.

4 Conclusion Given the research done on the various interventions and their impact, as well as the limitations on access to service centers, our goal is to provide an appropriate intervention method for assessing the validity of research hypotheses. According to the research, the effect of gamification on cognitive interventions in this field is limited and it is merely referred to as a platform for intervention, and no specific study has been

Using Gamification Based on Mobile Platform in Therapeutic Interventions

823

done in this regard. Hence, there is a need for more focused studies on the impact of gamification and the role of mobile in rehabilitation. Doing this research, it is expected that a more accurate level of impact, more relevant considerations, and more details will be obtained that will provide a ground for more focused studies. Finally, based on the obtained results a cognitive product that is available, effective and affordable compared to other products in the market can be delivered that is an appropriate option for the specified age range.

References 1. American Psychiatric Association, Diagnostic and Statistical Manual of Mental Disorders (DSM-V®), Washington, DC (2013) 2. Hargreaves, H., Rowbotham, M., Philips, M.: A Handbook on Learning Disabilities. Integra Is Funded by Ontario’s Ministry of Children and Youth Services (2009) 3. Moll, K., Kunze, S., Neuhoff, N., Bruder, J., Schulte-Körne, G.: Specific learning disorder: prevalence and gender differences. PLoS One 9(7), e103537. (2014) 4. International Dyslexia Association. Frequently Asked Questions About Dyslexia (2011) 5. IDA, “Fact Sheets On Dyslexia and Related Language-Based Learning Disabilities,” The International Dyslexia Association (2012). http://www.interdys.org/FactSheets.htm 6. Chorney, A.I.: Taking the game out of gamification. Dalhousie J. Interdiscip. Manag. 8, 1– 14(2012) 7. Rose, J.S.: Identifying and teaching children and young people with dyslexia and literacy difficulties. DCSF Publ., pp. 1–213, June 2009 8. Subramaniam, V., Mallan, V.K., Che Mat, N.H.: Multi-senses explication activities module for dyslexic children in Malaysia. Asian Soc. Sci. 9(7), 241–267 (2013) 9. Brownell, M.T., Yeager, E., Rennells, M.S., Riley, T.: Teachers working together: what teacher educators and researchers should know. Teach. Educ. Spec. Educ. J. Teach. Educ. Div. Counc. Except. Child. 20(4), 340–359 (1997) 10. Dyslexia, I.G.: British Manila School SEN Conference (2000) 11. Ambrose, P.P., Cheong, L.S.: Effects of the clay modeling program on the reading behavior of children with dyslexia. Asia-Pacific Educ. Res. 20(3), 456–468 (2011) 12. Zakaria, Z.: The application of visual art activities in the teaching and learning of the malay language to dyslexic primary school children. In: Lisbon. World Conference on Arts Education Building Creative Capacities for the 21st Century 2006, 6–9 March (2006) 13. Ahmad, S.Z., Ludin, N.N.A.N., Ekhsan, H.M., Rosmani, A.F., Ismail, M.F.: Bijak Membaca Applying phonic reading technique and multisensory approach with interactive multimedia for dyslexia children. In: IEEE Colloquium on 2012 Kota Kinabalu. Humanities, Science and Engineering (CHUSER) (2012) 14. Sidhu, M.S., Manzura, E.: An effective conceptual multisensory multimedia model to support dyslexic children in learning. Int. Inf. Commun. Technol. Educ. 7(3), 34–50 (2011) 15. Majzub, R.M., Abdullah, M.A., Aziz, Z.: Effects of a multisensory programme on dyslexic students: Identification and mastery of the alphabet. Res. J. Appl. Sci. 7(7), 340–343 (2012) 16. Wagner, R.K., Torgesen, J.K.: The nature of phonological processing and its causal role in the acquisition of reading skills. Psychol. Bull. 101, 192–212 (1987) 17. Volman, M.J.M., Schendel, B.M.V., Jongmans, M.J.: Handwriting difficulties in primary school children: a search for underlying mechanisms. Am. J. Occup. Ther. 60(4), 451–460 (2006)

824

M. Behnamghader et al.

18. Gomes, Z.G.: Neurofisiologia da linguagem oral e escrita. In: Zorzo, J., Capellini, A.S. (ed.) Dislexia do desenvolvinmento c outros disturbios de leituraescrita. Sao Jose dos Campos (SP), Pulso, pp. 35–42 (2009) 19. Daud, S.M., Abas, H.: Dyslexia baca’ mobile app - the learning ecosystem for dyslexic children. In: Proceedings - 2013 International Conference on Advanced Computer Science Applications and Technologies, ACSAT 2013 (2014) 20. Skiada, R., Soroniati, E., Gardeli, A., Zissis, D.: EasyLexia: a mobile application for children with learning difficulties. In: Procedia Computer Science (2013) 21. Purkayastha, S., Nehete, N., Purkayastha, J.: Dyscover - an orton-gillingham approach inspired multi-sensory learning application for Dyslexic children. In: World Congress on Information and Communication Technologies, pp. 685–690 (2012) 22. Khakhar, J., Madhvanath, S.: JollyMate: assistive technology for young children with Dyslexia. In: 12th International Conference on Frontiers in Handwriting Recognition, pp. 576–580 (2010) 23. Rello, L., Bayarri, C., Gorriz, A.: What is wrong with this word? dyseggxia: a game for children with dyslexia categories and subject descriptors. In: 14th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 6–7 (2012) 24. Di Tore, P.A. Di Tore, S., Ludovico, L.A., Mangione, G.R.: MADRIGALE: a multimedia application for dyslexia and reading improvement GAmifying learning experience. In: 2014 International Conference on Intelligent Networking and Collaborative Systems, pp. 486–491 (2014) 25. Gooch, D., Vasalou, A., Benton, L.: Exploring the use of a gamification platform to support students with dyslexia. In: 6th International Conference on Information, Intelligence, Systems and Applications (2015)

Combined Approach to Diagnose ADHD: Gamifying Conners Rating Scale Ali Khaleghi(&), Fatemeh Heydari, Maedeh Takhttavani, Hadi Haedar, and Alireza Soltaninezhad Imam Khomeini International University, Qazvin, Iran [email protected], [email protected], {S956369001,H.haedar}@edu.ikiu.ac.ir, [email protected]

Abstract. Attention Deficit Hyperactivity Disorder (ADHD) is one of the most common childhood disorders is affected 5.9% to 7.1% of children. It seems that early diagnose of ADHD can prevent the number of negative impacts on the children’s lives. By gamifying the traditional approaches, we can motivate children to participate in the diagnostic test. In this paper, we present an assessment tool that is being developed to diagnose ADHD children by gamifying Conners Parent Short Form Assessment Reports. In first phase, by the time the ADHD statements were categorized, the various possible scenarios have been written. When the scenarios are gotten confirmation, the mini-game will be designed. In the design phase, participatory design model (PD) and a usercentered design approach (UCD) will be used to understand and realize what children are really like. In the implementation phase, we have two main steps: development and debugging. After the game design and developed are completed, the calibration phase will be start. In this phase. By using machine learning technique on the data, which have been saved, in the log file we will design the diagnosis model. Final phase is the test phase which is done to evaluate the game diagnosis accuracy. Keywords: ADHD


Diagnostic tool
Serious game
Gamifying

1 Introduction Diagnosis of attention deficit/hyperactivity disorder is difficult and complicated. One of the common ways of assessing and evaluating ADHD is using the standard questionnaires completed by child’s parents and teachers which is called traditional method. The psychologist analyzes and evaluates the given responses to each question, Whether he/she is a child with ADHD disorder or not [1]. There are some problems in traditional evaluations, including the tedious, rigorous, and repeatability of the evaluation process, the lack of positive feedback from children to participate in the evaluation tests [2], Hawthorne effects [3] And mismatch between the teachers and parents response to the questionnaire questions [1]. Another approaches for assessing ADHD is observing the child’s behavior and decision-making based on assessment forms filled up by child’s parents and teachers in response to the child’s behaviors and reactions [4]. It is possible © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 825–835, 2021. https://doi.org/10.1007/978-3-030-49932-7_77

826

A. Khaleghi et al.

to use gamification concept to implement the ADHD evaluation questionnaires, and diagnose disorder through recording the child’s behavior and interactions during the game, based on the child’s real behavior. In Sect. 2 short definition of ADHD will be given, its diagnosis approach and the consequences of delay in diagnosis. In Sect. 3 gamification concept will be explained. Relying on our studies in Sect. 4 related work in using gamification as a diagnosis tool will be conducted. Finally, in the 5th section our approach and the outlines of this project are explained.

2 ADHD ADHD is the most common childhood neurobehavioral disorders. Studies have indicated that eight to 12% of worldwide children are affected by this disorder [5]. Studies have shown that the prevalence of ADHD can range from 2.2 to 17.8 percent [6], and around 20% of parents of children with ADHD have symptoms of ADHD themselves [7]. Inattention and lack of concentration, doing the excessive activity, having impulsive behaviors are the problems that ADHD children suffer from. The prevalence of ADHD is greater in boys than girls. These children have difficulty in communicating with their peers and often become alone because of this [8]. ADHD is divided into three different types: inattentive types, hyperactive-impulsive type and combined type. Children diagnosed with inattentive ADHD have difficulty with attention skills. children diagnosed with hyperactive-impulsive ADHD are usually overactive and have impulsive behavior and children who are diagnosed with combined type exhibit mixed behavior from inattentive and hyperactive-impulsive [8]. 2.1

The Consequences of Delay in Diagnosis

ADHD has the significant impacts on the lives of ADHD children and their families. One of these effects is the cost of the treatment of these children, which affects the economy of their families and the community. According to studies conducted in the United States, the annual cost for these children is between $143 billion and $266 billion million, which most of these costs are borne by families, and these costs were mainly associated with health care and educational services for children. ADHD and learning disabilities are rarely recognized until children between the ages of four to seven years. When they start to read and do other school skills, ADHD symptoms such as inattention, hyperactivity and impulsiveness appear [9]. These Delays in a clinical diagnosis of ADHD consequence in more long-term costs: people who are not diagnosed until adulthood incur higher costs than their same-sex sibling does [10]. Also, the diagnosis of ADHD over the age of 18 is associated with an increased risk of death syndrome [11]. If ADHD is not treated promptly, it causes emotional problems and anti-social behaviors, as well as the problem of overall performance and childhood ADHD symptoms may persist until adulthood [12]. Children with ADHD are prone to social problems [13]. Studies on the social problems of children with ADHD show that most of these children had parental dissatisfaction and social inequalities [14]. There are multiple communication problems children with ADHD and their peers. More than half of the children with ADHD are rejected by their peers, while healthy children are

Combined Approach to Diagnose ADHD: Gamifying Conners Rating Scale

827

rejected by 10 to 15% [12]. Children with ADHD with 50 to 70% suffer from Dependent Personality Disorder [5]. Studies have shown that 56 to 76% of children with ADHD do not succeed in establishing the mutual friendships, while 10 to 32 percent of healthy children have this problem [13]. Increasing introversion leads to poor social interactions in adulthood [12]. 2.2

How to Diagnosis ADHD

The approach for diagnosing ADHD is clinical; there is not an objective approach [4]. 18 behavioral symptoms of ADHD disorder is in Book DSM-V [10]. A number of experts for diagnosing ADHD use a combination of the methods such as; medical examination, DBRS, WISC-IV, CBCL, TRF and direct observation of the behavior of children [17]. In addition to these methods, computer programs can also be used as a tool for evaluation including CPT, TOVA and some CNB programs like the MicroCOG or the CogTest. However, the ability of CPT to the diagnosis of ADHD is limited. One of the other measurement tools is Conners Comprehensive Behavior Rating Scale (Conners 2008). In this way, the ADHD behavioral symptoms, which found in DSMV, measure using a form. The form is filled in by the parents or teacher of the child. However, this method may have limited predictive validity [18].

3 Gamification and Cognitive Science Games by using their attractive stories, rewards and challenges can provide a situation in which players can get a good experience that is hard to get it in their daily life [15]. One of the main features of the game is that it can attract the attention of the players, stimulate their curiosity and help them with the challenges and fantasies to increase their motivation to achieve a particular goal. If Games are designed and developed with goals beyond entertainment, it can be used to motivate the pleyers to achieve the goal. This is where we can use the concept of gamification. Gamification refers to the use of game elements such as scoring, competition, avatars, graphics, mission, and game level in non-game concept where the gaol is somting other than gaming and entertaments. By Using the gamification and its components, it’s possible to reinforce or change a particular behavior [2,16]. According to Jim et al., For some reasons such as increasing motivation to participate in assessment and training tests, stimulating the brain to carry out its activities, increasing the ability to use cognitive tests and teaching and using methods and Treatment for different ages, health researchers, cognitive science and game developers are trying to design and develop the games in this field [2].

4 Related Work In order to use gamification in human computer interaction technology to diagnosis ADHD in children, beside knowing the role of gamification in diagnosis ADHD, we needed to have a deep study on different parts of gaming industry such as design

828

A. Khaleghi et al.

models and machine learning. These 2 fields are the key points in our approach. In follow you could find some general information, which helped us make the decision to choose the best methods to use. 4.1

Gamification and the Children with ADHD

We live in information and technology age and the healthcare environment has changed. It is important to pay attention to the points such as the patient’s tendencies and their impact on the treatment process [17]. Extensive research has been conducted on the use of computer tools and programs in assessing and diagnosing ADHD. the literature shows that psychologists and specialist for some reason, including encouraging participate in an assessment test, tend to use combined methods which include both computer programs and traditional approaches [18–21]. In 2012, McCallum, as a result of his studies, points out the role of computer games in the future of health systems, games will be able to log player performance during the game. The collected data can be used in health diagnostic systems [17]. Many games have been developed in the field of psychology and brain health, which can be categorized into two types: assessment and training games. In both type, the focus is on evaluating and enhancing working memory, decision-making, skills, executive function, attention and precision, and deterrent functions [2]. Since children with ADHD also have a disorder in these cases [22], it is possible to use games for Assessing and training ADHD children. According to Pascal and et al., in 2012, one usage of games is their employment as a diagnostic tool by implementing traditional tools and psychological tests in the form of a game. by increasing the Children motivation we can diagnosis ADHD at the early age [23]. The game employmenting as a diagnostic tool can have a precision equivalent to some of the traditional evaluation tests [24]. One of the benefits of using games as a psychological test is that it reduces Hawthorn effect [22]. This effect occurs when an individual is aware of the fact that he\she is being assessed and he/she may behave and try to act differently, in this case the validity of the diagnosis may be reduced. but using computer games, the child is drowned in the game without feeling that she/he is being assessed [3, 22]

5 Design Principle According to studies on user interface design for children, to identify the features and rules of the game’s design, it is essential to consider the children’s age, the level of their literacy and their understanding of abstract concepts [25]. Beside that if we want to design a game for children with cognitive disorders, paying attention to others aspect such as Type of disorder is crucial [26]. Several studies have been conducted to identify the proper features of designing the game and digital media environment for children with special needs. Most of the research has focused on the various aspects of game design, including sound engineering, visualization, graphics, story and scenarios in a variety of ways [25]. In designing childish user interface, you should use symbols and icons that are familiar to children. for example, to return to the home page, you can use the home

Combined Approach to Diagnose ADHD: Gamifying Conners Rating Scale

829

icon. Symbols and objects that represent a particular concept or person must always be fixed. for example, help character must be the same in different levels of the game. Any changes in the symbols and objects which used in the user interfaces design causes distraction of children, especially those with cognitive impairment, including those with autism spectrum disorder [25]. Alongside using graphical objects and icons in designing childish interface, text is also important. As the children specially the younger ones can’t properly read and write, it seems better to use visual and vocal representation of the text [25]. Beside using this principle and guideline some of the researchers believe that for designing an appropriate digital tool for special group of person, understanding and realizing what they really like, it is better to use their interest, feeling, mental pattern and opinion in game designing process. On the other hand, it is better to involve the users in the design process. There are different theories and models to participate. User-contend design, participatory design and interaction design are models which can be used. In next section we will overview on this models and theories [22].

6 Machine Learning Usages Machine learning algorithms are widely used in everyday life. machine-learning approaches can be greatly used in clinical psychology and psychiatry so that in computer-based psychotherapy, machine learning can be greatly beneficial [27]. The gold standard in the translational science is applying statistical models built in a sample on another instance [28]. In machine learning, in addition to the transfer ability, there is the ability to simulate, optimize, and even generalize [27]. The automatic determination and analysis of parameters is one of the factors, which influence the rate of machine learning. Several models can be used to increase accuracy.

7 Machine Learning in a Healthcare Systems Machine learning has been significantly succeed in diagnosing diseases using largedimensional data, in particular the structure and function of the neurological data [29]. Although Former studies were focused on Alzheimer and Schizophrenia, recent studies have been expanded into diagnose spectra such as Anxiety disorders [30], Anorexia [31], substance abuse [32], and special types of peculiarities [33]. Research also expands with the use of nominating modalities such as Genetics [34], Metabolomics [35], and proteomic data [36]. In general, existing literatures suggest that machine learning can be used to identify people with mental disorders based on brain data with a precision of over 75% [29].

830

A. Khaleghi et al.

8 Current Project Approach In this section, we illustrate the steps will be taken to develop a game as a diagnostic tool Based on the Conners rating scale questionnaire. In order to assess and diagnose ADHD, different aspects such as working memory, response inhibition, executive functions, time estimation, reaction time and waiting behavior should be investigated. As mentioned above, the ultimate goal of this project is to design a serious game to diagnose ADHD. This game must be able to classify players into three groups: healthy children, children with the disorder, and children suspected to have the disorder. The purposes of this project are keeping and increasing the children’s motivation, diagnosing ADHD at the early age and having a game that both ADHD children and control group can play it. To reach these goals, we will use research through design and iterated extensively in the design and development phase, which have five phase. It is worth to mention that in this project the order of phases is important. If there is a problem in each phase backtracking method will be used. This means that, during the project life cycle, in each phase may be necessary to go back to the previous phase to fix the problems. 8.1

First Phase: Writing Scenarios

Designing a game according to ADHD symptoms makes it necessary to do a comprehensive study on ADHD questionnaires. After selecting Conners questionnaire, each question was reviewed and categorized. By the time the ADHD statements were categorized, the various possible scenarios were written. For each question, a number of scenarios were designed. Scenarios were reviewed by a number of health professionals and game developers. In some cases, the transfer of knowledge between gamers and game designers required a common language, so we used shapes, diagram and flowcharts in many cases. After studying the scenarios, a brainstorming was conducted to select the best scenario. Eventually, the approved scenario was transferred to the second phase. 8.2

Second Phase: Game Design

When the scenarios are gotten confirmation, the mini-game is being designed. In the design phase, participatory design model (PD) and a user-centered design approach (UCD) are being used to understand and realize what children are really like. Participatory design model is being used to make it more user-friendly and meet users need. First step in this phase is to design a prototype. In order to design the prototype, we use a guideline and principle which have been collated through literature review. In addition to the literature review we use user-centered design. In order to do this, we will go to the clinic and interviews with children and ask them about their feeling and interest. We can ask them some question about the color, the character, the shape and other object which will be used in game design. By evaluating the result and the features which have been collected through literature, the main core of the game will be design. The second and the third phase have a direct and close connection. We use game design document (GDD) to allow the developers easily understand what is really

Combined Approach to Diagnose ADHD: Gamifying Conners Rating Scale

831

wanted by the designer. In this document all the design features will be written by the details. In fact, the GDD is an agreement between the designers and developers. By the time the GDD papered, the next phase will be started and the first prototype will be developed. After the prototype developed, by using the participatory design model we can improve our product. In the participatory design model, we can use expert’s opinions and experiences. By the collaboration between designer, developer and Cognitive Science Specialists we can design appropriate game to reach our goal. 8.3

Third Phase: Implementation

In the implementation phase, which is the longest phase of the development lifecycle, we have two main steps: development and debugging. In the first step, we will develop the mini-game that have several parts such as mini-game modeling, log system implementation, graphics and sound design. After the game has been developed, debugging step will start. In debugging step, if there is a problem we should use Backtracking methods and find a way to solve the problem. In the implementation phase, game engines are used to design and develop the mini-games. Game engine allows us to swiftly implement games with more efficient tools. In this project, developers are using the Unity program as a game engine. Unity is a game engine that can support popular scripting languages like JavaScript and C#. Furthermore, Unity is a multi-platform gaming engine, which can support several kinds of outputs like android, windows and etc. to log game data, database experts suggested the Unity Database (UDB) for easy and optimal storage of data. Our developers have chosen Unity due to the fact that our developers and designers have had the working experience in unity engine. Unity have the collaboration option which could help us to work faster and more accurately. 8.4

Fourth Phase: Calibration

We will choose a group of children whom their ADHD has been determined by cognitive tests or clinical diagnoses. This group will be asked to play the game we designed (70 children between 7 and 12 years old, some with and some without any disorder). Their performance during the game will be logged without knowing about being assessed. By using the machine learning technique for the data, which have been logged in the log file, the diagnosis model will be designed. We will take 100 labeled data which will be using as a training data in machine learning algorithm. 8.5

Fifth Phase: Test

In order to test the game and estimate algorithm precision in ADHD diagnosis, we will select a control group. The control group is a group of children aged between seven to twelve who have been diagnosed as having ADHD or being healthy through clinical evaluation tests. Through comparing the results of our game with the traditional methods the game precision will be investigated. To test the pattern obtained from the phase 4 we will give 30 unlabeled data to evaluate precision.

832

A. Khaleghi et al.

9 Pilot Study: AGE1 As a pilot project, we have been successful in designing one of the Minigame based on first statement of Conners sating scale. Conners sating scale is a questionnaire for diagnosing of attention deficit and Hyperactivity Disorder based on the symptoms which are mentioned in the Diagnostic and Statistical Manual of Mental Disorders, 5th Edition (DSM_V). The first question in this questionnaire is that “the children is careless and his/her attention is distracted easily “According to the question, our goal is to understand whether the child’s senses are easily thrown or not. With extensive research on the effects of playing game, and under psychologist supervision, the mini game was designed. At the start of the game the character become visible and give the game guideline for child. Figure 1 the first game scene which will show at the start of the game. The game scene is:

Fig. 1. REV conference logo

Combined Approach to Diagnose ADHD: Gamifying Conners Rating Scale

833

The character asks the child to fill a glass with water by pushing the button. While child pushing the button, the glass will be field with water. as a distracter an attractive butterfly was designed. The distracter was designed to measure whether butterfly easily distract child from button or not. 5. During the game, system logs various variable like number of click on objects, and time period the button was being pushed.

1. 2. 3. 4.

The general Schema of this Mini-game showed in a Fig. 1. In this figure you can see three different scenes that illustrate the Mini-game procedure. First scene: game scene which will show at the start of the game. Second scene: one of the game scene: how the player should fill a glass of water third scene: one of the game scene which show to the butterfly distract the player’s attention.

10 Outlines At the end of this project, we have a game that can engage children to participate in the assessment test. This game will be configurable. ADHD children are categorized into three types. So by configuring the time and difficulty of each level, we can get more accurate diagnostic information.

11 Conclusion Experts and Psychologists use different approaches including questionnaire and computer programs such as games to diagnose ADHD. As there are some problems with this approaches, by combining gamification concept and questionnaire based assessment. We try to gamify questionnaire to use for assessment propose. Previous studies in the field of diagnosis through the game imply that it is not far from reaching the desired result by using gamified questioners. In this article we present our approach to gamifying a clinical Conners rating scale in our opinion game can be an effective tool to use as a diagnostic tool. Previous studies in the field of diagnosis through the game imply that it is not far from reaching the desired result by using gamified questioners.

References 1. Santos, F.E.G., Bastos, A.P.Z., Andrade, L.C.V., Revoredo, K., Mattos, P.: Assessment of ADHD through a computer game: An experiment with a sample of students. In: Proceedings - 2011 3rd International Conference Games Virtual Worlds Serious Applications VS-Games 2011, pp. 104–111 (2011) 2. Lumsden, J., Edwards, E.A., Lawrence, N.S., Coyle, D., Munafò, M.R.: Efficacy, gamification of cognitive assessment and cognitive training: a systematic review of applications and jim. JMIR Serious Games (2016) 3. McCarney, R., Warner, J., Iliffe, S., Van Haselen, R., Griffin, M., Fisher, P.: The hawthorne effect: a randomised, controlled trial. BMC Med. Res. Methodol. 7, 1–8 (2007)

834

A. Khaleghi et al.

4. Gualtieri, C.T., Johnson, L.G.: ADHD: is objective diagnosis possible? Psychiatry 2(11), 44–53 (2005) 5. Faraone, S.V., Sergeant, J., Gillberg, C., Biederman, J.: The worldwide prevalence of ADHD: is it an American condition? World Psychiatry 2(2), 104–113 (2003) 6. Skounti, M., Philalithis, A., Galanakis, E.: Variations in prevalence of attention deficit hyperactivity disorder worldwide. Eur. J. Pediatr. 166(2), 117–123 (2007) 7. Kooij, S.J.J., et al.: European consensus statement on diagnosis and treatment of adult ADHD: the European network adult ADHD. BMC Psychiatry 10(1), 67 (2010) 8. American Psychiatric Association, Diagnostic and Statistical Manual of Mental Disorders. Arlington. p. 991 (2013) 9. Glascoe, F.P.: Early Detection of Developmental and Behavioral Problems (2000) 10. Sayal, K., Prasad, V., Daley, D., Ford, T., Coghill, D.: Review ADHD in children and young people : prevalence, care pathways, and service provision. Lancet Psychiatry 0366(17) (2017) 11. Østergaard, S.D., et al.: Predicting ADHD by assessment of Rutter’s indicators of adversity in infancy. PLoS ONE 11(6), 1–10 (2016) 12. Oord, S.V.D., Bul, K.C.M., Ingmar, H.A., Franken, L.J.V., Kato, P.M., Danckaerts, M., van den Heuvel, R., Willems, A., van Oers Helga, J.J., Rens van Slagmaat, A.M.: Development and user satisfaction of ‘‘plan-it commander’’ a serious game for children with ADHD. GAMES Heal. J. Res. Dev. Clin. Appl. 4, 502–512 (2015) 13. Hoza, B.: Peer functioning in children with ADHD why are peer relationships important. J. Pediatr. Psychol. 32(6), 655–663 (2007) 14. Mikami, A.Y.: The importance of friendship for youth with attention-deficit/hyperactivity disorder. Clin. Child. Fam. Psychol. Rev. 13(2), 181–198 (2010) 15. Bioulac, S., Arfi, L., Bouvard, M.P.: Attention deficit/hyperactivity disorder and video games: a comparative study of hyperactive and control children. Eur. Psychiatry 23(2), 134– 141 (2008) 16. Reiners, T., Wood, L.C.: Gamification in education and business. Gamification Educ. Bus. (October), 1–710 (2015) 17. McCallum, S.: Gamification and serious games for personalized health. Stud. Health Technol. Inf. 177, 85–96 (2012) 18. Shipstead, Z., Redick, T.S., Engle, R.W.: Is working memory training effective? Psychol. Bull. 138(4), 628–654 (2012) 19. Hulme, M.M.-L.: Is working memory training effective? a meta-analytic review. Dev. Psychol. 49, 270–291 (2013) 20. Chacko, M.R.A., Bedard, A.C., Marks, D.J., Feirsen, N., Uderman, J.Z., Chimiklis, A., Rajwan, E., Cornwell, M., Anderson, L., Zwilling, A.: A randomized clinical trial of cogmed working memory training in school-age children with ADHD: a replication in a diverse sample using a control condition. J. Child Psychol. Psychiatry 55, 247–255 (2014) 21. Rapport, M.D., Orban, S.A., Kofler, M.J., Friedman, L.M.: Do programs designed to train working memory, other executive functions, and attention benefit children with ADHD? A meta-analytic review of cognitive, academic, and behavioral outcomes. Clin. Psychol. Rev. 33(8), 1237–1252 (2013) 22. Gongsook, P.: Interactive diagnostic game for time perception : Timo’s adventure game (2016) 23. Pascual, M.F.: Diagnosis of the attention deficit disorder using ‘D2’ and ‘Symbols Search’ tests through a game-based tool, pp. 116–119 (2012) 24. Faraone, S.V., Newcorn, J.H., Antshel, K.M., Adler, L., Roots, K., Heller, M.: The groundskeeper gaming platform as a diagnostic tool for attention-deficit/hyperactivity disorder: sensitivity, specificity, and relation to other measures. J. Child Adolesc. Psychopharmacol. 26(8), 672–685 (2016)

Combined Approach to Diagnose ADHD: Gamifying Conners Rating Scale

835

25. Chiasson, S., Gutwin, C.: Design Principles for Children’s Technology (2005) 26. Zaki, N.A.A., Wook, T.S.M.T., Ahmad, K.: Therapeutic serious game design guidelines for stimulating cognitive abilities of children with speech and language delay. J. ICT 2(2), 284– 312 (2017) 27. Dwyer, D.B., Falkai, P., Koutsouleris, N.: Machine learning approaches for clinical psychology and psychiatry. Annu. Rev. Clin. Psychol. 14(1), 91–118 (2018). annurevclinpsy-032816-045037 28. Johnson, D., Deterding, S., Kuhn, K.-A., Staneva, A., Stoyanov, S., Hides, L.: Gamification for health and wellbeing: a systematic review of the literature. Internet Interv. 6, 89–106 (2016) 29. Arbabshirani, M.R., Plis, S., Sui, J., Calhoun, V.D., Mind, T.: Single subject prediction of brain disorders in neuroimaging: promises and pitfalls. Neuroimage 145, 137–165 (2017). https://doi.org/10.1016/j.neuroimage.2016.02.079 30. Hilbert, K., Lueken, U., Muehlhan, M., Beesdo-Baum, K.: Separating generalized anxiety disorder from major depression using clinical, hormonal, and structural MRI data: a multimodal machine learning study. Brain Behav. 7(3), 1–11 (2017) 31. Lavagnino, L., et al.: Identifying neuroanatomical signatures of anorexia nervosa: a multivariate machine learning approach. Psychol. Med. 45(13), 2805–2812 (2015) 32. Whelan, R., Garavan, H.: When optimism hurts: Inflated predictions in psychiatric neuroimaging. Biol. Psychiatry 75(9), 746–748 (2014) 33. Visser, R.M., Haver, P., Zwitser, R.J., Scholte, H.S., Kindt, M.: First steps in using multivoxel pattern analysis to disentangle neural processes underlying generalization of spider fear. Front. Hum. Neurosci. 10(May), 1–13 (2016) 34. Tognin, S., et al.: Using structural neuroimaging to make quantitative predictions of symptom progression in individuals at ultra-high risk for psychosis. Front. Psychiatry 4(JAN), 1–9 (2014) 35. Setoyama, D., et al.: Plasma metabolites predict severity of depression and suicidal ideation in psychiatric patients-a multicenter pilot analysis. PLoS ONE 11(12), 1–16 (2016) 36. Diniz, B.S., et al.: Circulating biosignatures of late-life depression (LLD): towards a comprehensive, data-driven approach to understanding LLD pathophysiology. J. Psychiatr. Res. 82(Lld),1–7 (2016)

Using Gamification Based on Virtual Reality Mobile Platform for Treatment of Adults with Amblyopia Fateme Hosseinnia(&), Ali Khaleghi, and Kamran Mahmoudi Imam Khomeini International University, Qazvin, Iran {S966169006,kmahmoudi}@edu.ikiu.ac.ir, [email protected]

Abstract. Amblyopia or lazy eyes is a unilateral or bilateral Disorder and reduction of best-corrected visual acuity (BCVA) which affected approximately 3% of the world’s population. To improve the visual deficits caused by Amblyopia, the traditional treatments have to be implemented before the termination of the critical visual development period which is within the age of 6– 12 years. Despite traditional opinions that amblyopia treatment is only effective for children through treatment such as eye patching, video games could be highly engaging and improve vision and attention in the adult with amblyopia. In this study, after examining the factors influencing amblyopia treatment, we used different experimental approaches that have been used to enhance visual function in adults with amblyopia including video game play and dichotic training. A Contrast based game has been developed for clinical trials used for that purpose to enhance visual function in adults suffering from amblyopia. The therapeutic procedure at first includes a clinical visit for assessing the patient’s visual related parameters and the second is using a proposed 3D video game and a game-set comprise a VR mobile-set and android device. This study represents that the proposed 3D video game might be useful as an effective treatment for amblyopia in adults. Keywords: Amblyopia


Videogames
Gamification
Adult
Virtual reality

1 Introduction Amblyopia or the Lazy eye is referred to as a condition where the vision of one or in some rare cases both eyes are not normal or the vision has a high drop despite refractive error correction (best-corrected visual acuity). Clinically, the difference in vision over two lines from the Snellen chart between the two eyes can cause Amblyopia [1]. In this disease, there is no organic problem in the eye, And because of a lack of coordination between the two retinal images in the brain, visual impairment is achieved. Amblyopia is a developmental disorder of the central nervous system that results in an inability to process visual images and thus reduce visual acuity. Typically, amblyopia occurs in association with an eye or visual pathway disorder. Childhood is the most sensitive stage in the development of the body and the eye [2]. © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 836–843, 2021. https://doi.org/10.1007/978-3-030-49932-7_78

Using Gamification Based on Virtual Reality Mobile Platform

837

Amblyopia is a major public health problem for the prevalence among children and its lifelong impact, and its effects can be profound, such as learning disruption and failure in education and communication. Both amblyopia and its treatment can have a significant impact on the quality of life. The prevalence of lazy eye population varies from 0.7 to 1.9 percent in children ages 6 to 71 months, while school-based studies typically report a range of 1.0% to 5.5% in older children. According to studies, bilateral amblyopia is less common than unilateral amblyopia, but the reported ratio differs significantly from 5% to 41% of all amblyopia cases [2]. The current clinical treatment for Amblyopia involves refractive correction, followed by a non-amblyopia eye obstruction to encourage the use of the amblyopic eye. This treatment, if performed in the first years of life, can help to treat children. However, the progression of binocular vision is limited in conventional treatments, and this kind of treatment eliminates binocular vision because of fellow suppressed eyes [4]. At present, clinic treatment for adults is not efficient and treatment is limited to the duration of the critical period [5]. The usual treatment methods (eye patch method) are not effective in children older than 12 years of age and adults. In addition, patching the eyes may reduce Binocular and 3D vision and can cause social and psychological problems such as decreased self-confidence or skin sensitivity [6, 7]. Also, about 25% of children are experiencing relapse due to the disorder after a year of treatment, or reverse amblyopia may occur and the healthy eye get amblyopia, it is advisable to minimize the time and amount of eye patch [8].

2 Alternative Treatments Several studies over the past twenty-one years have shown that perceptual learning can be an effective new method for amblyopia treatment. Elinor Gibson defined perceptual learning as “a relatively permanent and consistent change in an array’s understanding of a stimulus after experimenting with this array” [9]. Adults can improve the performance of doing sensory tasks through rehearsal exercises or perceptual learning, and this learning is presented as a flexible neurotransmitter whose results are visible in the cerebral cortex [9]. Recent research has shown that practicing repetitive visual work can improve visual acuity in adults with amblyopia. The common therapies with eye reduce binocular vision, which is why new therapies try to force the brain to use both eyes simultaneously. Bipolar images are such that the images visible to the amblyopic eye differ from those of the fellow eye. This is done either through glasses with red and blue color filters or by providing high-contrast images in an amblyopic eye and low-contrast images to the fellow eye. Other common therapeutic interventions, including therapeutic vision (also called “orthopedics” or eye exercises), are defined as a medical program involving visual activities for improving vision and Binocular vision. This

838

F. Hosseinnia et al.

method includes computer programs, charts, filters, metronomes, anti-repression activities, and eye-to-hand coordination exercises. A relatively new solution has been studied in recent years, the use of therapeutic games as serious or gamified games or therapeutic software.

3 Gamification Therapeutic Approach Gamification has been defined using game design elements such as bonuses and penalties and scoring on non-game issues and fields [7]. The main idea of gamification is to use the special design features or motivational features of entertainment games in other systems to further interact with the desired system. Which has been used in recent years in various fields such as business areas, social education, medicine, treatment of mental and intellectual disorders, etc. [10]. Today, the approach to gamifying therapeutic and therapeutic methods has been very much taken into consideration and has provided the field of collaboration between physicians and game design specialists [11]. In a study that looked at video game effects on adults with lazy eye in 2013, 58 adults were randomly selected to practice with a game called “Medal of Honor,” and the Tetris game. All patients were significantly improved compared to patients with routine treatment. The study concludes that games that are less challenging have more effective interventions in adult treatment [12]. In a study conducted in 2015 on 25 adults to compare the impact of binocular game and watching videos in 3D, the results of doing this test on patients for 40 h show improved the visual acuity in patients who performed the video game was much more than those who watched the 3D movie and their visual acuity improved about 28%. Also, the reading speed and contrast accuracy increased in these patients [13]. Another study was conducted in 2018 on twenty-one people with a unilateral lazy eye disorder, 12 of whom had the disorder due to severe refractive errors, and 9 of them with strabismic lazy eye. 11 of the participants should play this game by patching the eye and 10 others should play the game in Binocular fashion. After 6–10 weeks, visual acuity improved by 38% for the Binocular group and 15% for children with monocular and eye patches. The Depth perception improved in the Binocular method by 17% and in the monopoly 15%. As a result, this test showed a significant improvement in Binocular treatment [14]. It is argued that by providing visual stimuli separately, but simultaneously in each eye, so that the amblyopic eye (for example, by reducing the contrast of the image to the non-amblyopia eye), both eyes cooperate, and they come to a visual understanding in a particular point of contrast balance. Preliminary data have shown that this binocular treatment not only increases the fusion of the two eyes but also improves the amblyopic eye vision in some adult patients beyond the critical period of visual growth [8]. In the present study, we tried to identify the criteria and treatment methods presented in previous studies and the effective measures for adult interventions, then these criteria were presented as a gamified binocular virtual reality game. Also, in order to increase engagement and motivation to enhance the effectiveness of treatment, these criteria are Provided in a ninja fruit game using the principles of gamification.

Using Gamification Based on Virtual Reality Mobile Platform

839

4 Game Design Methodology People with lazy eyes have problems such as reduced corrected vision, depth detection, 3D vision, and binocular vision, and the present study seeks to improve these problems. Because there is no definitive and clinical treatment for adults, and for a closer look, the focus of this study will be on adults. In this research, the Ninja Fruit game platform has been used in a virtual reality environment. The following steps should be taken to design a suitable game and to study its effectiveness. I. EVALUATION

OF EFFECTIVE THERAPEUTIC CRITERIA

Before designing the game, we must examine and evaluate the criteria and effectiveness of various interventions and treatments, as well as consulting with ophthalmologists to select the appropriate factors and procedures for gamification. 1. The color of game elements: The retina contains light receptor cells, which include two types of bars and cones. The cones are located in the center of the retina and the bars are located on the outer edge of the retina. In the retina, light is detected and converted into electrical signals. These signals are transmitted to the brain through the nerves of vision. The brain processes these signals and turns them into images. Stimulating the cones enables the brain to understand the colors. The color depends on the amount of stimulation of each type of cone. According to previous studies, red, yellow, green, blue and orange colors stimulate the retina and most of the games designed in this area use these colors. These colors are also widely used in therapeutic filters of lazy eyes [15, 16]. 2. Since patients with lazy eye have eye movement problems, a variety of eye movements that include fast voluntary movements and follow-up movements should be used in the game. 3. The size of the game elements should be adjusted according to the patient’s visual acuity and according to the Snellen chart. 4. The game is designed based on binoculars and virtual reality. High-contrast images are shown to the amblyopic eye and low contrast images are shown to the fellow eye with the virtual reality glasses. After consultation with visionary laxatives, the contrast ratio of the images in the fellow eye is initially are set equal to the amblyopic eye and is reduced by 10% at each stage and according to the patient’s progression. 5. The CAM method is an active lazy eye treatment that has been used for around 1978 to date. In the visual cortex, different cells are sensitive to spatial frequencies in different directions [17]. This allows the optic neurons to be activated by a network of square wave lines rotating slowly in front of the eye. According to this principle, for the first time, the “low visibility stimulator” or “CAM” was constructed by Campbell. In this device, a circular plate is rotated slowly and rapidly by an engine, for example, one round per minute, plates containing a grid of lines with different spatial frequencies or transparent screens with different color schemes can be installed on this rotating plate (Fig 1).

840

F. Hosseinnia et al.

Fig. 1. Cam vision simulator

II. APPLYING

THE CRITERIA OBTAINED FROM STUDY IN THE GAME

After identifying the effective therapeutic properties, these factors should be designed in the form of a Ninja Fruit game. The color of the environmental elements and the fruits that are in the game for fragmentation are mainly used in previously identified colors. Consider the size of the elements, at the beginning of the game, the patient’s eye number is recorded and the elements and size of fruits are adjusted accordingly. To apply various eye movements and enhancement of movability, the movement of bombs and fruits is adjusted to the eye movements so that the patient’s eyes are forced to chase the fruits and to grind them. The contrast ratio of the images is also adjustable according to the patient’s file, progress rate and expert opinion at the beginning of each treatment session. In order to use the CAM treatment in some step parts of the background are set up as different spatial frequencies and tailored to the visual acuity of the patient. During the game, the environment is changed over the course of a few moments, and the background changes as the cam-spatial frequencies. Patient scores are recorded and to enhance the engagement game they can compete with other patients. During the game, the speed of the movement of fruits and the direction of entering the scene changes (Fig. 2). After recording the patient’s information, the contrast level and size of the game elements are adjusted accordingly in the game (Fig. 3). In moments of the play, for example, when a player loses a fruit or explodes a bomb, they appear in the environment for moments of spatial frequencies that are spinning.

Using Gamification Based on Virtual Reality Mobile Platform

841

Fig. 2. a) Register patient information b) Adjust the size and contrast of the fellow eye according to the recorded information

Fig. 3. The use of frequencies of the long CAM method

5 Required Equipment and Software The game was designed as a virtual reality game using Unity’s toolkit that enables the patient’s binocular vision and gives the ability to adjust the contrast between the two eyes. This game was designed to work on Android and iOS operating systems so that later on it can be used outside of the clinic and by the patient. The equipment needed to play the game is an Android or iOS phone that will use a suitable virtual reality glasses so that the player can play the game with refractive vision correction, and motion controls are used in order to move the sword designed in the game to cut the fruit of the game in the Ninja Fruit platform.

842

F. Hosseinnia et al.

6 Game Evaluation and Test Steps The game will be reviewed on a number of adults with unilateral amblyopia without strabismus or mild strabismus in two months and two 30-min sessions per week in Noor Alborz Eye Hospital. The visual acuity, the contrast of vision, and the depth of vision of the patient are measured at the first session, and then at the start of each treatment, the patient’s visual acuity and visual acuity are recorded. The medications are regulated by the therapist. Then, after the last session, the full ophthalmic exam is performed for the extent of the effect of the treatment and the patient’s improvement. This measurement will be repeated after one and six months for the duration of the treatment mendacity.

7 Conclusion In this article, we presented a prototype of a mobile-based virtual reality game designed specifically for the treatment of amblyopia for adults, which was designed using effective gamification rules and effective therapeutic interventions. It is hoped that the interactive content of the game and the competition between the patients participating in the test would involve patients during the course of the treatment. The design of the game will be further enhanced so that patient information is first recorded by a therapist at the medical center and the settings are made by the specialist in the clinic on the control platform of the game and the patient will be treated without a clinic. The ability of researchers and experts to track data about patient compliance and progress is essential for this. Adjusting the amount of playback contrast is automatically performed by the system. We hope that by applying these changes to the game and re-testing it will get better results than the currently available games.

References 1. Sengpiel, F.: Plasticity of the visual cortex and treatment of amblyopia. Curr. Biol. 24(18), R936–R940 (2014) 2. Wallace, D.K., et al.: Amblyopia preferred practice pattern®. Ophthalmology 125(1), P105– P142 (2018) 3. Camos, E.C.: Pediatric ophthalmology and strabismus: editorial overview. Curr. Opin. Ophthalmol. 2(6), 661–662 (1991) 4. Birch, E.E.: Amblyopia and binocular vision. Prog. Retin. Eye Res. 33(1), 67–84 (2013) 5. Thompson, B., Chung, S.T.L., Kiorpes, L., Ledgeway, T., McGraw, P.V.: A window into visual cortex development and recovery of vision: introduction to the vision research special issue on amblyopia. Vis. Res. 114, 1–3 (2015) 6. Levi, D.M., Li, R.W.: Perceptual learning as a potential treatment for amblyopia: a minireview. Vis. Res. 49(21), 2535–2549 (2009) 7. Fortenbacher, D.L., Bartolini, A., Dornbos, B., Tran, T.: Vision therapy and virtual reality applications. Adv. Ophthalmol. Optom. 3(1), 39–59 (2018) 8. Narasimhan, S., Harrison, E.R., Giaschi, D.E.: Quantitative measurement of interocular suppression in children with amblyopia. Vis. Res. 66, 1–10 (2012)

Using Gamification Based on Virtual Reality Mobile Platform

843

9. Gibson, E.J.: Perceptual Learning. Learning (1962) 10. Deterding, S., Dixon, D., Khaled, R., Nacke, L.: From game design elements to gamefulness. In: Proceedings of the 15th International Academic MindTrek Conference Envisioning Future Media Environments - MindTrek 2011, p. 9 (2011) 11. King, D., Greaves, F., Exeter, C., Darzi, A.: ‘Gamification’: influencing health behaviours with games. J. R. Soc. Med. 106(3), 76–78 (2013) 12. Belchior, P., et al.: Video game training to improve selective visual attention in older adults. Comput. Hum. Behav. 29(4), 1318–1324 (2013) 13. Vedamurthy, I., et al.: A dichoptic custom-made action video game as a treatment for adult amblyopia. Vis. Res. 114, 173–187 (2015) 14. Gambacorta, C., et al.: An action video game for the treatment of amblyopia in children: a feasibility study. Vis. Res. 148, 1–14 (2018) 15. Mitarai, G.: Neural mechanism of color vision. J. Inst. Telev. Eng. Jpn. 29(12), 968–978 (2011) 16. Kim, D.Y.: The interactive effects of colors on visual attention and working memory: in case of images of tourist attractions. In: International CHRIE Conference Track, p. 1 (2010) 17. Johnson, B.D.: Nanotechnology: opportunities for photonics. Photonics Spectra 36(12), 44– 45 (2002) 18. Shah, B.K., et al.: A randomized trial of a binocular iPad game versus part-time patching in children aged 13 to 16 years with amblyopia. Am. J. Ophthalmol. 186, 104–115 (2017)

Linear Programming Model Applied to the Optimization of Nutritional Diets for Athletes Julio A. Mocha-Bonilla1, Victor Hugo Guachimbosa2, Carolina Guachimbosa Santiago3, and Javier Sánchez Guerrero1(&) 1

3

Facultad de Ciencias Humanas y de la Educación, Universidad Técnica de Ambato, Ambato, Ecuador {ja.mocha,jsanchez}@uta.edu.ec 2 Facultad de Ingeniería en Sistemas, Electrónica e Industrial, Universidad Técnica de Ambato, Ambato, Ecuador [email protected] Carrera de Contabilidad y Auditoría, Universidad Israel, Quito, Ecuador [email protected]

Abstract. The present work fulfills the objective of providing a tool of computer technology to the service of the personal health, an optimized diet in function of costs and quantities by rations that must be consumed to fulfill the diet in a high performance sportsman, caloric requirements in function of the type of physical activity. The method used consisted of taking a sample of athletes who maintain a regular training regime of four hours a day; the results include the application of the Linear Programming Model of Operations Research, which is a mathematical matrix model of deterministic type for the optimization of resources at minimum cost, with maximum utility. This model was applied to the formulation of mix or optimization of the nutritional diet for athletes, generating an optimal nutritional formulation that meets the technical specifications and minimum, maximum or fixed nutritional requirements (restrictions = rows). These nutritional requirements must be compared or met from the mixture or combination of a certain number of available food ingredients or inputs (decision variables = columns) that in turn possess or contribute to the mixture formula with their respective technical characteristics, contents and nutritional inputs. This matrix structuring of the model and its corresponding processing using the so-called Simplex Method, allows to obtain an optimal nutritional diet for athletes, taking also as a sample, the formulation of the optimal diet for breakfast, by virtue of the multiple possibilities of daily formulation for the athlete. Subsequently, the results are obtained and displayed through the link with a software and Linear Programming application used for the optimization of the diet, using online tools that give reliable results, which in turn are analyzed and validated prior to implementation. Keywords: Models of operative research optimization
Calorie intake


Linear programming
Diet

© Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 844–866, 2021. https://doi.org/10.1007/978-3-030-49932-7_79

Linear Programming Model Applied to the Optimization of Nutritional Diets

845

1 Introduction From the archaic era to the present day, the so-called eating disorders have been reflected, in terms of the diet commonly called eating habit, it is essential today to assume food stability, since an imbalance can create health problems [1]. Intense physical activity leads the elite athlete to maintain an imbalance between energy demands and income in macro and micronutrients. A top-level athlete trains four hours a day, which means a high nutritional requirement [2]. Prior to the specific development of the arguments contemplated in the initial conception of the present study, it has been considered convenient to present a sketch that synthesizes by means of an agile interpretation, the fundamental aspects inherent to each one of the particularities to optimize the nutritional diets in people [3] mentioned in [4], in our case for competitive sportsmen, whose formulation in all its amplitude is presented in the same content of the investigation. The descriptive synthesis, part of the basic study of schematizing the main characteristics established in the realization of the work and analysis of the linear programming model, which constitutes the essential structure of the study. The real purpose of the execution of the analytical process derives from the need to provide those who are interested in the objective review of the application developed, a summarized vision of its content, so that its observation allows to have relevant elements of judgment that in turn facilitate in the first instance the global understanding of the investigative task carried out. This will enable a quick conception of the activities carried out with absolute adherence to the techniques stipulated as foundations of the development of the elaborated model, whose optimization is generated as a consequence of the use of the capacities and facilities of the automatic processing based on the data. It should be noted that the study involved the direct or indirect participation of professionals in the different areas of specialization, specifically in the sciences of physical activity and sports nutrition, as well as professionals in the area of linear programming, therefore the problem treated belongs to a multidisciplinary and interdisciplinary research. The practical application developed is structured on the basis of the use of the basic principles of a technique of great importance within the context of Operations Research known as Linear Programming, whose aspect must be resolved in the initial stage of the implementation of internal point algorithms used to optimize them [5]. The main characteristics and elements are specified through the definition of real data of a specific situation that serves as the object of study. Therefore, the achievement of the practical model of linear programming became feasible thanks to the availability of existing technical information in the Career of Pedagogy of Physical Activity and Sport, with its research project called “Technology applied to physical activity and sport”, through which, based on the data provided, the case of Minimization of Costs could be executed in a problem of mixture formulas, which in specific form is concretized in the formulation of a nutritional diet for high-performance athletes at low cost, specifically taking as a sample, the formulation of an optimal diet for the athletes’ breakfast, during the training period.

846

J. A. Mocha-Bonilla et al.

The data requirement included in the model consists of a description of the types of food ingredients or food inputs available to obtain 700 g of the aforementioned food, and the specification of their respective costs per portion corresponding to 100 g in each product. In addition, the stipulated information contains the technical specifications or nutritional requirements that the formula to be obtained must satisfy; as well as the technical characteristics, contents and nutritional contributions of each ingredient or food input available with respect to these specifications and the maximum limits in which certain ingredients or food inputs available may intervene for the formulation of the optimal diet for the breakfast of an athlete in training, since the diet can improve or hinder the sports performance [6]. The data obtained based on the information, energy and nutritional needs of the athlete [7] allow the resolution of the problem through a computerized application of the simplex method of linear programming, which in turn makes feasible the achievement of an optimal solution of energy intake. In order to achieve this, we worked definitively with 7 decision variables (ingredients or available food inputs) and 12 restrictions (technical specifications, nutritional requirements and intervention limits of ingredients or nutritional inputs), which after being processed by the indicated method, through 22 iterations we obtain optimal responses to the proposed model; the same ones that have been reviewed, verified and analyzed by the team of researchers of the mentioned study, so that the implementation of the model and its obtained solutions offer a real diet in the short term. Therefore, the application of the Linear Programming Model to optimize a diet, breakfast optimized according to the lowest cost but with great caloric contribution in high performance athletes, is presented as an objective of study. In short, the practical computerized model of linear programming allowed to count on a accumulation of valuable information that can constitute an efficient means of support for the generation of an adequate decision making tending to the rationalization and optimization of the use of the available resources for the professionals of nutrition and sportsmen of high performance, in function of planning the feeding to reach a good level of muscular mass and lean mass to obtain a good state of health and sport performance [8]. The need to achieve the most correct solution possible, within the practical model of linear programming, determined that it can be used using the properties of the simplex method, the most efficient method for solving algebraic operations [9], which allows us to study the effects generated in the optimal solution found, based on the data initially presented for solving the problem of linear programming, in the formulation of a nutritional diet for high-performance athletes, in the specific case, an optimal breakfast, which is linked to improving physical fitness, well-being and performance through dietary evaluation [10]. The analysis that properly stands out is feasible thanks to the use of the exposed method, and to the availability of information, which constitutes a process to improve the nutritional diets according to the caloric contribution at the lowest cost. The required data, as well as the formulation of the problem; the construction of the model itself, its resolution and tests are detailed in the presentation of the respective results.

Linear Programming Model Applied to the Optimization of Nutritional Diets

847

2 State of the Art The optimization is directly derived from the application of the principle of the Simplex method, the problem is to maximize or minimize a certain function called (objective function) that depends on a set of variables [11]. Considering that Operations Research is a branch of applied sciences that provides certain services to business productivity, the realization of this study is based on the optimization of the results expected to be obtained as a product of various processes based on sound decision-making that allow an appropriate combination in the use of human, material, technological and financial resources, in short to design more efficient and effective alternatives within the food and welfare [12]. Operations Research [13] has importance at the present time mainly due to the use of technology in the development of its techniques and applications [14], since it has been determined that it can be reached with greater ease and efficiency when using the technological contributions of science, nutrition and treatment of the various fields of action to optimize the performance of the athlete, through a correct intake and an efficient nutritional recommendation that facilitates the normal metabolic functioning of the human being [15]. After this brief analysis, it can be stated that Operations Research is a set of technical-scientific instruments whose application is reflected with many advantages in decision-making, which in turn determine the possibility of selecting the best alternative that facilitates the solution of certain specific problems in order to achieve optimization in the use of resources and also within this context are implicit mathematical models as the basis of the study presented. This determined, that the essential part of the present investigation concentrates in the study of one of the most important models of the Investigation of Operations, the application of Linear Programming; whose resolution, when applying computerized resources is carried out in a specialized and reliable way, by virtue of which it establishes the feasibility of choosing those final results that represent a contribution to the fulfillment of the objectives of the study. Thus, of the many techniques that comprise the Operations Research area, the Linear Programming model is undoubtedly one of the best known and applied in the field, as it has the greatest potential for real practical application, the purpose of which is the optimization of resources under certain restrictions and/or limitations of a technical or economic nature under nutritional standards [16]. Linear programming has made it possible to create interactive applications with users, i.e. linear programming includes planning that presents well-defined sequences in order to reach a specific objective, in the specific case, the improvement of users’ bodies [17]. Within the risk factors of cancer and the consumption of a healthy diet, the use of the method of linear programming is indispensable, because creating a healthy and economic menu, accessible for people suffering from these diseases, helps greatly to reduce the costs in the feeding since the linear program for diet constitutes an innovative solution to the problems of feeding [18].

848

J. A. Mocha-Bonilla et al.

The theoretical and technical aspects are mathematically represented in the general structure, form or mathematical formula of the Linear Programming Model detailed in Table 1 below. Table 1. Structure, form or general mathematical formula of the linear programming model Element

Function objective

Restrictions

Shape General

Z = C1X1 + C2X2 + … + CjXj + … + CnXn

a11 X1 þ a12 X2 þ . . . þ a1j Xj þ . . . þ a1n Xn  ;  ; ¼ b1 a21 X1 þ a22 X2 þ . . . þ a2j Xj þ . . . þ a2n Xn  ;  ; ¼ b2 .. .. .. .. .. . . . . . ai1 X1 þ ai2 X2 þ . . . þ aij Xj þ . . . þ ain Xn  ;  ; ¼ bi .. .. .. .. . . . . am1 X2 þ am2 X2 þ . . . þ amj Xj þ . . . þ amn Xn  ;  ; ¼ bm

Summarized

Z¼

n P

Cj  X j

j¼1

m n P j¼1 i¼1

Vectorial Symbology

Z¼~ C~ X

aij  Xj  ;  ; ¼ bi

*

A X , , = ~ b aij = Coefficients of decision variables in constraints Z = Objective function that is optimized (MAX) o (MIN) A = Matrix of the coefficients of the decision variables Cj = Coefficients of the decision variables in m # of rows = # of restrictions the target function n = # of columns = # of decision variables Xj = jésima Decision Variable b = Restrictions limits vector ~ C = Vector coefficients of decision variables bi = restrictions limits in F.O. *

X = Vector for decision variables Non-negativity condition of the decision variables: Xj  0

3 Methodology Ratifying the above, the following is a specific presentation of the particularities of the Linear Programming model which, when implemented, definitively certify that the predetermined approaches have been effectively complied with as the starting point of the research carried out here. It is a question of clearly typifying the definition of a procedure, which at the same time constitutes a similar guide for the implementation of Operations Investigation models to be developed in the future, using certain essential techniques and phases that allow the realization of practical applications that signify a contribution to the improvement of the delimited scope of study. The practical applicability of the procedure made it possible to comply correctly with the proposed objective; since with this purpose certain stages of study of the model are developed in a particular way, which, being interdependent in their followup, will finally determine the achievement of practical applications of proven validity. In synthesis, the process that characterizes the structuring of the focused model consists with certain particular adaptations, of the following phases to execute with an indispensable interrelation between them; these are: Analysis and obtaining of the required data, Formulation or approach of the problem, Construction of the model,

Linear Programming Model Applied to the Optimization of Nutritional Diets

849

Obtaining of the solutions of the model with use of a specific software of free origin, Test of the model and of the solutions and Implantation of the model and of the solutions. 1. Analysis and obtaining of the required data The modeling procedure begins with the analysis of the data required for the conformation of the model and also explains how to obtain such data. This allows a clear visualization of how the model will be structured according to their respective conditions and characteristics. 2. Formulation or approach of the problem At this stage, the aim is to define the problem by describing several aspects that are essential to achieve consistency in the detail of the problem. Basically, these aspects will include the determination of the objective that is expected to be achieved with the resolution of the problem, as well as the establishment of decision possibilities and the delimitation of the restrictive conditions that the system must comply with as requirements. It considers the global problem and all the alternatives and limitations that allow to obtain an optimal solution; defining it with this purpose in its true dimensions when determining those factors that affect it such as variables, limitations, suppositions and purposes of the same one. 3. Construction of the model The development of this phase is in direct relation with what was proposed in the previous one, since the establishment of the model that represents the system under study depends indispensably on it. To this end, all possible elements are detailed, assigning them representative mathematical symbols, for them and depending on the relationships that are presented, to construct equations or inequalities, which in turn express in a specific way the objective function and the restrictions, including variables and numerical constants in their structure. The construction of the mathematical model used represents a certain complexity; resulting in mathematical expressions with a certain degree of difficulty, aspects that finally generate models in the form of a system of linear equations and inequalities. 4. Obtaining model solutions with the use of the computer Once the model is constructed, certain methods or resolution techniques are used to find exactly in mathematical form the specific values that optimize the results that are intended to be achieved with the application of the model. In the developed model, being mathematical in nature, its resolution is achieved by applying specific techniques, which in this study have been implemented in function of specialized computer software that allows to obtain the most exact possible solutions to each problem and that would not have been feasible to achieve without the use of these computerized methods. It is important to point out that, in the model and resolution techniques used, the functions used are linear and depend on the variables used and on the existing restrictions that condition the possible values of the variables and directly affect the optimization of the objective equation. In certain cases it is not possible to put all the existing restrictions, because the model would lose efficiency, but those considered convenient for obtaining a real solution to the problem are included.

850

J. A. Mocha-Bonilla et al.

Iterative solution methods were used, which are feasible to perform, only thanks to the application of computer techniques, and especially because of the considerable number of variables and constraints that must be handled and that manually or with the use of common calculators is really impossible to get solutions through these methods. Likewise, when determining the optimal solution in the model used, the values obtained for the decision variables are those that provide the best value of the objective function, complying with all restrictions. In other words, the objective function in this model facilitates the determination of the optimum solution to the problem posed. 5. Testing the model and the solutions. The mere construction of the model and its resolution does not merit its application in the respective realities studied, but the validity and certainty of the model must be tested in relation to the real system in operation that it represents. Then, the results obtained from the model should be compared with the real results achieved in the system, through the relationship of the data that exist on its performance and those indicated by the respective model. By testing or verifying the model, certain particularities may be determined that reflect as far as possible its deficiencies; this will allow corrections to be established with a high degree of reliability. 6. Implementation of the model and the solutions. In the implementation phase of the model and the solutions, not only must the report be delivered with the results of the investigation and the explanation of the solutions obtained; but also, the results of the work carried out must be put into execution, this will give the last evaluation of the same, and will allow to analyze if the models and their solutions are advantageous or not so that they can be implanted. In this sense, it would be advisable to use and apply the results of the model developed, together with the criteria of experience and valuable knowledge possessed by the working group, technicians and specialists of the system under study.

4 Results In carrying out this approach to linear programming, certain main and common characteristics of the problems analyzed in this study were established, such as the possibility of constructing mathematical models that are expressed through a system of linear equality and inequalities (use of linear equations or inequalities) that are the result of the representation of restrictions on the availability of resources, more than one objective function, also of a linear type, that seeks to achieve optimization; in this case, the minimization of costs. Hence, the purpose of the model presented is to find the “optimal solution” that minimizes the corresponding objective function and satisfies the stipulated restrictions. In short, the model aims to combine the ingredients or food inputs available, in appropriate quantities in a mixing formula that must meet the technical specifications or nutritional requirements of athletes, at minimum cost. That is to say, it is a question of obtaining a nutritious and economic food; as well as the balance between the nutritional needs or requirements of the sportsmen and the technical characteristics, contents or nutritional contributions of each one of the ingredients or available nutritional inputs.

Linear Programming Model Applied to the Optimization of Nutritional Diets

851

In order to implement the modeling procedure, the search for an optimal formula for the calculation of a nutritious food is taken as a basis, taking as a demonstrative sample, the formulation of the optimal diet for an athlete’s breakfast. 1. Analysis and Obtaining of the Required Data. This model of optimization of the formula of mixture for the obtaining of the mentioned food, consists of certain indispensable elements for its implementation, these are: – A function to be optimized, which in this case specifically aims to minimize the total costs of the mixture; – The amount of grams of mixture to be prepared, expressed as an equation; – The costs corresponding to the ingredients or available food inputs that intervene in the mixture formula, in dollars per portion of 100 g; – The content with respect to the nutritional conditions that present as technical characteristics, or nutritional contributions all and each one of the ingredients or available food inputs that compose the formula, and that are the ones that can be disposed in the study; – The technical specifications or nutritional requirements that must be met in the formulation of the mixture. All the information proposed in this phase of the modeling procedure has been compiled and classified through the following sources: food composition tables, ration forms, reports on the availability of food ingredients or inputs, and records on the costs of the food ingredients or inputs involved in the formulation of the required food. These data are those available as of March 31, 2019, and the elaboration of data tables under the responsibility of the work team. With the indication of all these data, which are required for the construction of the model itself, we can continue with the implementation procedure of the mixing formula model; thus we would have the second phase. Problem Formulation This phase of the modelling procedure basically includes all the conditions and elements of the case under study. Therefore, the approach of the problem is summarized in the following terms: It is necessary to obtain the optimal formula of mixture that allows the calculation of 700 g of a nutritional diet for the breakfast of sportsmen and women. In such a way that the final product is obtained at a minimum cost and meets or fulfills the technical specifications or nutritional requirements with their values or minimum limits that are represented by (bi), where i = 1 to 4; exposed in Table 2: Table 2. Technical specifications or nutritional requirements i 1. 2. 3. 4.

Nutritional specification Limit values (bi) Minimum Energy (Calories) 601.1 Proteins (Grams) 38.7 Carbohydrates (Grams) 71.0 Fat (Grams) 18.1

852

J. A. Mocha-Bonilla et al.

All these food ingredients or inputs require the determination of their respective costs per serving; thus we have to represent the number of food ingredients or food inputs available with the symbol j, so the costs are symbolized by Cj, where j = 1 to 7, as shown in Table 3. Table 3. Costs of available food ingredients or inputs j 1 2 3 4 5 6 7

Food ingredients or inputs Cj = $/portion of 100 g Brown rice 0,15 Chicken 0,50 Eggs 0,35 Skimmed milk 0,45 Toast with jam 0,55 Beans 0,40 Fruit portion 1,00

These costs will constitute non-nutritional specifications, together with the mass of nutritional food that needs to be prepared, which for the case at hand is 700 g. In addition, the formulation of this mixture requires compliance with certain specifications on the minimum, maximum and fixed limits in which certain food ingredients or inputs may be involved, which are needed in the calculation of this food and which are set out in Table 4 below: Table 4. Limits on the content of available food ingredients or inputs j

Raw material

1 2 3 4 5 6 7

Brown rice Chicken Eggs Skimmed milk Toast with jam Beans Portion of fruit

Maximum content limits (portions/day) 1 3 2 2 2 2 2

Maximum content limits (grams) 100 300 200 200 200 200 200

Finally, the technical characteristics, contents or nutritional contributions of each ingredient or food input must be indicated, with respect to the previous technical specifications or nutritional requirements, as shown in Table 5. These combinations of availability of the technical characteristics, contents or nutritional contributions of the food ingredients or inputs and the technical specifications or nutritional requirements, we represent them with the symbolism: Aij

Linear Programming Model Applied to the Optimization of Nutritional Diets

853

Table 5. Technical characteristics, contents or nutritional contributions of each raw material, with respect to technical specifications or nutritional requirements A i

1 2 3 4

j Ingredients or Inputs Food. Contribution Nutritional

1 Brown rice

Energy (Calories) Proteins (Grams) Carbohydrates (Grams) Fat (Grams)

2 Chicken

3 Eggs

4 Skimmed milk

5 Toast with jam

6 Beans

7 Fruit Portion

124

230

160

351

402

142

54

2

27

24

35

12

13

0

25

0

4

48

75

19

11

1

13

10

15

3

1

0

In this way, the problem of formulating the nutritional diet of the breakfast for sportsmen would be raised, which allows to undertake with the construction of the respective model that constitutes the third phase. 2. Construction of the Model In this stage it is necessary to illustrate each and every one of the elements exposed previously; but now in mathematical form, and thus we have as structure of the model three fundamental aspects that are: a) Definition of Decision Variables: They constitute the unknowns as an unknown optimal quantity of grams of each of the ingredients or food inputs available, j = 1 to 7, whose values must be found, as a determination of the optimal solution of the model. These variables, which can be controlled by the respective technical personnel, are mathematically represented as: Xj, where: each Xj represents the amount in grams of each ingredient or food input that will intervene in the formulation of the nutritional diet of the breakfast for athletes, as indicated in Table 6:

Table 6. Definition of decision variables Variable Xj X1 X2 X3 X4 X5 X6 X7 X1 a X7  variables)

Unknown optimum quantity of raw material (j) Brown rice Chicken Eggs Skimmed milk Toast with jam Beans Fruit portion 0 (non-negative condition of the decision

854

J. A. Mocha-Bonilla et al.

b) Formulation of the Objective Function: As this aspect consists of presenting the purpose of optimization of the model through a linear mathematical expression (unknowns with exponent = 1), we would have that the objective of the model is to minimize the function (Z) of costs (Cj) of the ingredients or food inputs that integrate the nutritional diet of the breakfast for sportsmen (j), according to the following formulation: Minimize Cost: Z ¼

7 X

Cj  Xj

j¼1

Where Cj is the cost per ration of each raw material j, and Xj is the unknown optimal quantity (in grams) of each ingredient or food input that intervenes in the following mathematical formula: ðMINÞZ ¼ 0; 15Xl þ 0; 50X2 þ 0; 35X3 þ 0; 45X4 þ 0; 55X5 þ 0; 40X6 þ 1; 00X7 In other words, the objective function tries to minimize costs, which is equivalent to the sum from 1 to 7 of the products resulting from the multiplication of costs by the respective unknown quantity of each raw material (Table 7). c) Formulation of Restrictions: The restrictions as well as the objective function are specified in the form of restrictive mathematical expressions that can be linear equations or inequalities; for which we use arithmetic operators of the form: =,  ,  . This is determined by the particular characteristic of the type that is each specification, nutritional or nonnutritional; for example, we have that mass is an equality therefore the arithmetic operator will use =; and in case the specification is a minimum the operator to use will be  ; and if it is a maximum it will be represented by the arithmetic operator  (inequalities). This way we would have the formulation of the diverse restrictions that represent each and every one of the specifications that the final product will have to fulfill, in the following way: 1. The only non-nutritional technical condition is the required mass of the formulation of the breakfast diet for sportsmen and women, in such a way that being mass = 700 g, it is necessary to indicate that this condition must comply with the expression: Mass ¼

7 X j¼1

Xj ¼ 700 g

Linear Programming Model Applied to the Optimization of Nutritional Diets

855

In this way the mass specification should be expressed as the sum of the masses of each and every one of the ingredients added in the formula, and which are represented by the following Restriction N.- 1. Restriction No: 1: Xl þ X2 þ X3 þ X4 þ X5 þ X6 þ X7 ¼ 700 This restriction, and all those determined below, are integrated in Table 8. The other technical specifications are nutritional (Table 2), they are established taking into account their minimum or maximum character and are expressed through a set of inequalities added in Table 7 and thus determined: The technical nutritional specification Energy, which must be a minimum of 601.1 cal, can be expressed as follows: ðMinimum ContentÞ Energy  601:1 cal This minimum technical nutritional specification of Energy, is the sum of the technical characteristics or content of Calories of each one of the ingredients or food inputs used; being A1j the nutritional contribution of Energy that contains the ingredients j = 1 to 7 (Table 5). It is therefore necessary to indicate that this condition must comply with the expression: Energy ¼

7 X

A1j  Xj  601; 1

j¼1

Therefore, the specification on the minimum content of Energy is expressed by the following inequality: Restriction No: 2: 124X1 þ 230X2 þ 160X3 þ 351X4 þ 402X5 þ 142X6 þ 54X7  601; 1 Similarly, the minimum protein requirement greater than 38.7 g is described by inequality (  ): ðMinimum ContentÞ Protein  38:7 g Being: 7 P Proteins ¼ A2j  Xj  38; 7 j¼1

856

J. A. Mocha-Bonilla et al.

Then, the specification on the minimum protein content is expressed by the following inequality: Restriction No: 3: 2X1 þ 27X2 þ 24X3 þ 35X4 þ 12X5 þ 13X6 þ 0X7  38; 7 Likewise, the minimum requirement of Carbohydrates greater than 71.0 g is described by inequality (  ): ðMinimum ContentÞ Carbohydrates  71; 0 g Being: 7 P Carbohydrates ¼ A3j  Xj  71; 0 j¼1

Hence, the specification on the minimum Carbohydrate content is expressed by the following inequality: Restriction No: 4: 25X1 þ 0X2 þ 4X3 þ 48X4 þ 75X5 þ 19X6 þ 11X7  71; 0 Similarly, the minimum fat requirement greater than 18.1 g is described by inequality (  ): ðMinimum ContentÞ Fat  18; 1 g Being: 7 P A4j  Xj  18; 1 Grasa ¼ j¼1

Hence, the specification on the minimum fat content is expressed by the following inequality: Restriction No: 5: 1X1 þ 13X2 þ 10X3 þ 15X4 þ 3X5 þ 1X6 þ 0X7  18; 1 All of these specifications are expressed by a set of inequalities summarized in Table 7. Similarly, as in the formulation of this nutritional diet for an athlete’s breakfast, it is required that the variety of the diet be specified as the maximum levels of the food ingredients or inputs involved, we have to indicate them below:

Linear Programming Model Applied to the Optimization of Nutritional Diets

857

1. Brown rice ðMaximum ContentÞ Brown rice  100 g Then, the specification on the maximum oats content is expressed by the following inequality: Restriction No: 6: X1  100 2. Chicken ðMaximum ContentÞ Chicken  300 g Therefore, the specification on the maximum content of Chicken, is expressed by the following inequality: Restriction No: 7: X2  300 3. Eggs ðMaximum ContentÞ Eggs  200 g That is to say, the specification on the maximum content of Eggs, would be expressed by means of the following inequality: Restriction N 8: X3  200 4. Skimmed Milk ðMaximum ContentÞ Skimmed Milk  200 g Therefore, the specification on the maximum content of Skimmed Milk, would be expressed by the following inequality: Restriction No: 9: X4  200

858

J. A. Mocha-Bonilla et al.

5. Toast with jam ðMaximum ContentÞ Toast with jam  200 g It is thus that the specification on the maximum content of Tostadas with jam, would have the following inequality: Restriction N 10: X5  200 6. Beans ðMaximum ContentÞ Bean  200 g Hence the specification on the maximum content of Bean, would have the following inequality: Restriction N 11: X6  200 7. Fruit Portion ðMaximum ContentÞ Fruit Portion  200 g Hence, the specification on the maximum content of Fruit Portion would have the following inequality: Restriction N 12: X7  200 In Table 7 we present the Model constructed with the Summary of the Objective Function and of all the Restrictions developed mathematically; and that in matrix form would be expressed by the following model: Mathematical model of the formulation of the optimal nutritional diet of the breakfast for sportsmen and women. ðMinimizeÞ Z¼

7 P

Cj Xj

j¼1

12

Subjectto:

7 P j¼1 i¼1

Aij  Xj ¼;  ;  bi Xj  0

Linear Programming Model Applied to the Optimization of Nutritional Diets

859

Summarizing: Table 7. Construction of the linear programming model for the optimal nutritional breakfast diet for athletes

X4

X5

X6

X7

Limits

1X3 160X3

1X4 351X4

1X5 402X5

1X6 142X6

1X7 54X7

= 700,0 ≥ 601,1

2Xl

27X2

24X3

35X4

12X5

13X6

0X7

≥

38,7

25Xl

0X2

4X3

48X4

75X5

19X6

11X7

≥

71,0

13X2

10X3

15X4

3X5

1X6

0X7

≥

18,1

≤ ≤ ≤ ≤

100,0 300,0 200,0 200,0

Carbohydrates (Grams)

5

Fat (Grams)

1Xl

6 7 8 9

Brown Rice Chicken Eggs Skimmed Milk Toast with Marmalade Beans Fruit Portion

1Xl

Eggs

Chicken

4

Beans

1X2 230X2

Skimmed Milk

1Xl 124X1

Proteins (Grams)

11 12

X3

≤ ≥ =

3

10

X2

Toast with Marmalade

Restricciones Mass Energy (Calories)

X1

Fruit Portion

1 2

0,15X l + 0,50X2 + 0,35X3 + 0,45X4 + 0,55X5 + 0,40X6 + 1,00X7

Brown Rice

(Minimize) Z = Subject to: A j Variables de i Decisión

1X2 1X3 1X4 1X5

bi

≤ 200,0 1X6 1X7

≤ 200,0 ≤ 200,0

5 Obtaining Solutions to the Practical Problem with the Use of the Computer In the practical demonstration of the problem of linear programming focused on obtaining the optimal nutritional diet formula for breakfast for sportsmen and women, once the indispensable data for the conformation of the model is known and properly processed and prepared, and the problem has been formulated and the construction of the model has been established, its resolution consists of achieving an optimal and economic formula (minimum cost) and the balance between the nutritional needs of sportsmen and women and the nutritional contribution of the available food ingredients or inputs. As the practical case involves the use of a considerable number of food ingredients or inputs, technical specifications or nutritional requirements and intervention limits of food ingredients or inputs, the optimal desired formula can be obtained through the use of the computer. The use of computing facilitates the practical application of the linear

860

J. A. Mocha-Bonilla et al.

programming model, mainly because the time needed to perform the required calculations is really minimal in relation to the excessive time that would be used to perform them manually and even with the use of calculators, and what is more in practical cases such as the treaty, its resolution is really impossible without the use of the computer, which also allows overcoming arithmetic errors of calculation to obtain a correct and reliable solution. The fundamental aspects that determined the computerization of the model for its resolution constitute the feasibility of working with a good number of variables and restrictions that are constructed in the form of linear equations and inequalities to which the Simplex Method is applied (Dantzig, 1951), for cost minimization, this method allows to solve problems of linear programming (PL) determining an optimal solution in function of maximum profits to the minimum cost, in the particular case using an online application such as that of PHPSimplex. The application of the computerized model of linear programming to the formulation of diets for athletes is a method that notably facilitates the problem of combining in a very short period of time the ingredients or food inputs in appropriate quantities resulting in a formula that must meet the requirements or technical nutritional specifications of athletes, at minimum cost. As in the solution of the model the computerized resources mentioned before have been used, the answers that are obtained are the optimal ones and that would have been impossible to obtain without the efficient use of these technical instruments. Next, the results obtained in PHPSimplex in the computer are presented, as a solution to the problem initially posed, through the summary of the data introduced to the computer, the mathematical construction of the model, the last iteration of mathematical calculations and the formula or optimal solution of the nutritional diet of the breakfast for sportsmen and women. That’s how it is: Summary of data entered into the computer:

Linear Programming Model Applied to the Optimization of Nutritional Diets

861

Mathematical construction of the model

Last iteration of mathematical calculations

Optimal formula or solution of the nutritional diet of the breakfast for sportsmen and sportswomen

The optimal formula obtained consists of the food ingredients or food inputs that have been selected and that cover the nutritional requirements and minimum cost of the balanced food that must be prepared with their respective amount in grams, and the total cost of the formula in national currency units, as shown in Table 8, which presents the values that optimize the objective function:

862

J. A. Mocha-Bonilla et al. Table 8. Optimal solution: nutritional diet breakfast for athletes

Decision Selected food ingredients variable or inputs Brown rice X1 X2 Chicken X3 Eggs X4 Skimmed milk X5 Toast with marmalade X6 Beans X7 Fruit portion Value of the function being optimized: (MIN) Z = Minimum Nutritional Breakfast Diet Cost for Athletes

Optimum quantity 100,00 0,00 200,00 200,00 0,00 200,00 0,00 255,00 U.S. cents per 100-gram serving = $2.55 per serving or breakfast

6 Testing the Model and the Solutions Obtained Prior to the practical implementation of the linear programming model, an evaluation must be carried out both of the solutions it has reported and of the model itself. In this way, we can state that in the case focused here the results obtained have the characteristic that they not only approach or approximate the results that have been obtained previously in the formulation of balanced foods, but also optimize them; and therefore the model used establishes with exactitude the required answers, hence its reliability and validity are absolutely demonstrated. This aspect also underpins the correct mathematical construction of the model. Having determined the optimal solution to the problem, in this case the optimal formula of the nutritional food for sportsmen and women; the values obtained for the decision variables, i.e. for each of the selected food ingredients or inputs, are those that provide the best value of the objective function or optimal total cost by multiplying the calculated quantity of each selected food ingredient or input by its respective cost. Also complying with all restrictions or technical specifications or stipulated nutritional and non-nutritional requirements, and technically establishing the minimum cost of the nutritional breakfast diet for athletes. The practical computerized application object of study, constitutes an advantage in front of the system of work that has been developing the project of investigation in technology of the physical activity and sport as far as formulations of nutritional diets are concerned, since although it is certain that they are in capacity to provide technical applications that fulfill the nutritional requirements using own methods of their specialty, this they establish it without repairing in the cost of the same ones; instead in the results that we obtain through the application of the model of linear programming it is fulfilled to total with the technical specifications or nutritional and non-nutritional requirements determined for the required diet. In addition, the cost of the formula is minimized, in relation to the selected food ingredients or inputs.

Linear Programming Model Applied to the Optimization of Nutritional Diets

863

These aspects can be demonstrated taking as an example the nutritional specification Energy, whose content is one of the most intimately related to the efficiency of the nutritional diet and that for the case is stipulated in a minimum of 601.1 cal in the diet and that was defined by the Restriction N° 2; Thus: Restricción N° 2: 124X1 þ 230X2 þ 160X3 þ 351X4 þ 402X5 þ 142X6 þ 54X7  601; 1 Replacing the values obtained from the respective decision variables per serving and multiplied by the nutritional inputs of the food ingredients or inputs that have been selected for the nutritional breakfast diet for athletes (Table 8), we have 124ð1; 00Þ þ 230ð0Þ þ 160ð2; 00Þ þ 351ð2; 00Þ þ 402ð0Þ þ 142ð2; 00Þ þ 54ð0Þ  601; 1 Solving 124 þ 0 þ 320 þ 702 þ 0 þ 284 þ 0  601; 1 1:430; 0 [ 601; 1 Formulation that proves what we wanted to demonstrate, that is to say that the sum of the products of the value of the food ingredients or selected inputs or the amount allocated to the nutritional diet for the value of Calories that each one of them contributes, in total is greater than the specified nutritional requirement. Similarly, if you wish, you can check compliance with technical specifications or nutritional requirements from the previous procedure for all other restrictions. Another type of test that demonstrates the validity of both the solutions and the model is the non-nutritional technical specification Mass, which for the treated case constitutes 700 g, which must be covered with the sum of the contents of all and yields one of the raw materials as demonstrated from equation N.- 1: Mass of Table 8; thus: Xl þ X2 þ X3 þ X4 þ X5 þ X6 þ X7 ¼ 700 Replacing it with the quantities of the selected raw materials, we have: 100 þ 0 þ 200 þ 200 þ 0 þ 200 þ 0  700 Solving: 700 ¼ 700 In the same way, in the optimal solution found it can be verified that the maximum limits in which certain food ingredients or inputs must enter, are also complied with exactly allowing the selection for the formula of around 77% of the number of food ingredients or inputs presented initially and that the same contribute with normal and feasible quantities of not causing damage to the athletes, who will be the consumers of the product that is elaborated from the nutritional diet obtained.

864

J. A. Mocha-Bonilla et al.

Likewise, as far as the total cost typified in the optimal solution is concerned, we can say that it is within the normal margins estimated for this type of nutritional food, with the cost obtained being 30% to 40% lower than the total costs at which these products have been formulated (only taking into account the cost of food ingredients or inputs).

7 Implementation of the Model and the Solutions Obtained From the antecedents indicated, we can state that the validity and correctness of the model and its solutions is demonstrated, which is of great importance since it constitutes the step that must be technically fulfilled before moving forward. After these activities have been completed and the reliability of the model has been assured, the complete report of the practical application of computerized linear programming on a mixture formula has been delivered to the Management of the research project in technology of physical activity and sport. This report was analyzed in detail and technically by the personnel of the mentioned unit and as a result the present investigation was fully tested; and therefore the validity and usefulness of the applied linear programming model was accepted, which will be tried to implant as soon as possible, for the formulation of nutritious foods for sportsmen and women.

8 Conclusions This work is the preliminary result of a broader research which is in progress and which is much more ambitious and comprehensive, for which a website has been used to apply the Simplex method. Being a work in progress, consider the creation of a formal website and a mobile application in the medium term, which considers more study variables. In conclusion, the present study implies the practical application of the linear programming model focused on the efficient use of limited resources, existing in the processes of nutrition and food diet in sportsmen and women, which were taken as the basis of the study, in order to obtain an optimal result, which allows the selection of a determined course of action among several alternatives and which concludes in the achievement of the pre-established objectives. The modeling procedure that is applied in the construction of the model, agrees with aspects and fundamentals of the scientific method, of which an interpretation has been made that is applicable to the specific study of linear programming and to the characteristics of the object of study where it is carried out. In such a way that the realization of the model clearly establishes a less complicated understanding and comprehension than the reality that they represent in the formulation optimization of diets for sportsmen and sportswomen. In the structuring of the present study, the aspects related to the use of computational resources were concentrated on the objective of applying them to serve as support in decision making in the specific field of dietary optimization, facilitating the

Linear Programming Model Applied to the Optimization of Nutritional Diets

865

use of a considerable number of decision variables and restrictions, as well as the resolution of the linear programming model; mainly due to the fact that the computerization of these applications allows to eliminate arithmetic calculation errors to obtain a correct solution and that the time needed to perform the required calculations manually and even with the use of calculators would be excessive, which has been overcome with the use of the capabilities offered by computers and applied software.

References 1. López-Espinoza, A., Moreno, A.G.M.: La transición del comportamiento alimentario: una explicación desde la teoría de la conducta. Universitas Psychologica 15(4), 1–10 (2018) 2. Villegas García, J.A., Zamora Navarro, S.: Necesidades nutricionales en deportistas. Archivos de medicina del deporte 169–179 (2018) 3. Melo, R.V., Ascón, J.E.G.: Aplicación de la Programación Lineal para optimizar el costo de una dieta balanceada. NGnosis Revista de Investigación Científica 4(1), 48–63 (2018) 4. Buitrago, O.Y., Ramírez, A.L.: Determinación de un Punto de Inicio en Algoritmos de Punto Interior en la Solución de Problemas de Programación Lineal. Información tecnológica 28(5), 23–30 (2017) 5. Segal, E., Elinav, E.: La dieta personalizada (Colección Vital): El nuevo paradigma nutricional para diseñar un plan de salud a tu medida (2019) 6. López-Sobaler, A.M., Vizuete, A.A., Ortega, R.M.: Papel del huevo en la dieta de deportistas y personas físicamente activas. Nutrición Hospitalaria 34, 31–35 (2017) 7. Olivos, O.C., Cuevas, M.A., Álvarez, V.V., Jorquera, A.C.: Nutrición para el entrenamiento y la competición. Médica Clínica Las Condes 23(3), 253–261 (2012) 8. De Armas Jacomino, L., Valdes-Ramirez, D., Pérez, C.M., Bello-Pérez, R.: ANÁLISIS DE HERRAMIENTAS INFORMÁTICAS LIBRES Y DE CÓDIGO ABIERTO PARA PROBLEMAS DE OPTIMIZACIÓN DE APILAMIENTO DE CONTENEDORES. Investigación Operacional 40(2), 153–164 (2019) 9. Mielgo-Ayuso, J., Maroto-Sánchez, B., Luzardo-Socorro, R., Palacios, G., Gil-Antuñano, N. P., González-Gross, M., EXERNET Study Group: Evaluation of nutritional status and energy expenditure in athletes. Nutricion hospitalaria 31(3), 227–236 (2015) 10. Anaut, D.O., di Mauro, G.F., Suárez, J.A., di Mauro, R.R.: Configuración Óptima de Redes de Distribución Primaria: Método Simplex. Información tecnológica 17(1), 85–92 (2006) 11. Pedraza, D.F.: Seguridad alimentaria familiar. Revista Salud Pública y Nutrición (2003) 12. Sasieni, M.: Investigación de operaciones; métodos y problemas (1976) 13. Fava, L., Vilches, D., Díaz, J., Pagano, M., Dapozo, R.R.: Tecnología aplicada al deporte de alto rendimiento. In: XX Workshop de Investigadores en Ciencias de la Computación (WICC 2018, Universidad Nacional del Nordeste) (2018) 14. Hernández Triana, M.: Recomendaciones nutricionales para el ser humano: actualización. Revista Cubana de Investigaciones Biomédicas 23, 266–292 (2004) 15. Galindo, M.: Producción de alimentos apoyada con programación lineal. Boletín electrónico, no. 2 (2007) 16. Bermúdez Colina, Y.: Aplicaciones de programación lineal, entera y mixta. Ingeniería Industrial Actualidad y Nuevas Tendencias 7 (2011)

866

J. A. Mocha-Bonilla et al.

17. Alaini, R.: Optimización de la dieta mediante programación lineal para desarrollar un plan de alimentos para la prevención del cáncer a bajo costo para adultos seleccionados en Kuala Lumpur, Malasia. BMC Salud Pública 19, 546 (2019) 18. Moncayo-Martínez, L.A., Muñoz, D.F.: Un Sistema de Apoyo para la Enseñanza del Método Simplex y su Implementación en Computadora. Formación universitaria 11(6), 29– 40 (2018)

White Blood Cells Detection and Classification Using Convolutional Neural Network Muaad Hammuda Siala(&), M. Samir Abou El-Seoud(&), and Gerard McKee(&) The British University in Egypt, Cairo, Egypt {Muaad144301,GerardMcKee}@bue.edu.eg, [email protected]

Abstract. Leukemia is one of the deadliest diseases in human life, it is a type of cancer that hits blood cells. The task of diagnosing Leukemia is time consuming and tedious for doctors; it is also challenging to determine the level and type of Leukemia. The diagnoses of Leukemia are achieved through identifying the changes on the White blood Cells (WBC). WBCs are divided into five types: Neutrophils, Eosinophils, Basophils, Monocytes, and Lymphocytes. In this paper, the authors propose a Convolutional Neural Network to detect and classify normal white blood cells. The program will learn about the shape and type of normal WBC by performing the following two tasks. The first task is identifying high level features of a normal white blood cell. The second task is classifying the normal white blood cell according to its type. Using a Convolutional Neural Network CNN, the system will be able to detect normal WBCs by comparing them with the high-level features of normal WBC. This process of identifying and classifying WBC can be vital for doctors and medical staff to make a decision. The proposed network achieves an accuracy up to 95.47% with a dataset including 10,000 blood cell images. Keywords: Image processing learning
Histology


Convolutional Neural Network
Deep

1 Introduction Leukemia is one of the deadliest diseases threatening humanity [1]. It is a form of cancer that hits blood and bone marrow. It starts in the bone marrow then expands to white blood cells and some of the leukemia’s start in other types of blood cells [2]. The blood cells are the main source for detecting leukemia. As is known, the human blood is divided into three main parts which are red cells, white cells, platelets [3]. The abnormal changes of number, shape, texture of the white blood cells is considered as a major symptom of having leukemia in the blood [4, 5]. Figure 1 shows the difference between healthy blood and blood Infected with leukemia. The main focus of this study is on the classification of normal white blood cells, which is the common problem that faces most researcher in biomedical engineering. The white blood cells are divided into five types, which are Neutrophils, Eosinophils, Basophils, Monocytes, and Lymphocytes [4]. © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 867–878, 2021. https://doi.org/10.1007/978-3-030-49932-7_80

868

M. H. Siala et al.

Fig. 1. The difference between healthy blood and leukemia [6].

The diagnose of Leukemia nowadays is done by taking one of the three main clinical tests. Physical test, Complete Blood Count test, Bone Marrow test. This paper focuses on the first stage of the Complete Blood Count (CBC) test. The first stage is the ability to determine the difference between normal WBC types. In CBC test doctors will look at the blood smear and focus on the normal and abnormal changes in white blood cells. The Technology has significantly contributed to the society’s healthcare in a positive way. In the context of this paper there are two main computer technologies/tools that can help in the process of diagnosing Leukemia. The first is big data - where huge amounts of medical data are available to researchers, scientists and engineers to construct effective technologies to diagnose and treat different diseases. Secondly, there is the environments and frameworks available for researchers, scientists and engineers to manipulate the data to obtain the required results. In order to diagnose Leukemia white blood cells need to be examined. The examiner will need to look for abnormal changes/shapes in white blood cells. By using image processing a white blood samples can be classified according to its type, later it will be checked if it contains any abnormality. The microscope image of a white blood cells sample is shown in blue in Fig. 2 [9].

Fig. 2. Blood cells sample - Monocyte cell appears in blue [9].

White Blood Cells Detection and Classification Using CNN

869

In previous researches, the most common algorithm used to detect and classify microscopic images goes through a number of steps: pre-processing; clustering; morphological filtering; segmentation; feature extraction, classification, and evaluation [7]. These common methods have many disadvantages including long development time, the selection of the features in order to obtain best accuracy; also, it is difficult sometimes to make an accurate decision on whether the cell is abnormal or not. On the contrary, deep learning, such as Convolutional Neural Network (CNN) reduces the processing time by eliminating some steps of the process. It discovers high-level features and it attempts to classify images at the same time. It means the convolutional neural network (CNN) could be considered a powerful technology to build a robust image classifier [8]. This paper uses a dataset that containing 10.000 microscope images (JPEG) [9]. The blood dataset is divided into four main types of white blood cells, namely Monocytes, Lymphocytes, Neutrophils, and Eosinophils. The Basophils cell images are not included in this dataset. There are approximately 2500 images for each type [9]. This blood dataset is appropriate to construct an image classifier using several techniques and methods to classify blood images into relevant classes. In addition, the convolutional neural network has the advantage of extracting high-level features from microscope images then to classify them with less time and less code implementation. However, the main challenge of building and designing a convolutional neural network is the high complexity, in terms of number of layers required to extract a high-level feature in the microscope images and how to classify these features. Therefore, the development of the network is challenging [8]. This paper will propose an application using a convolutional neural network, that detects white blood cells from microscope images and then classifies these blood cells into one of the four classes: Class A: Monocytes, Class B: Lymphocytes, Class C: Neutrophils, Class D: Eosinophils. l.

2 Related work Thanh, Vununu, Atoev & Lee in their study “Leukemia Blood Cell Image Classification Using Convolutional Neural Network” [8] in 2018, proposed a support system of early diagnosis of acute myeloid leukemia. The system is an image classifier that uses convolutional neural network-based method. They proposed convolutional neural network aims to detect normal and abnormal white blood cells in microscopic images [8]. In this study, they used a dataset that included 1188 blood cell image. Then proposed a simple data augmentation method just to expand the dataset. They applied eight methods. The methods they used to expand data are: Histogram equalization, Translation, Reflection, Rotation, Shearing image, Grayscale image, Blurring image, and Rotation image. Table 1 shows the purpose of using of these methods [8].

870

M. H. Siala et al. Table 1. The description of methods used in the previous study to expand data [8].

Methods 1. Histogram equalization 2. 3. 4. 5.

Translation Reflection Rotation Shearing image

6. Grayscale image 7. Blurring image 8. Rotation image

The discerption of methods To reinforce the contrast of a weak contrast image by tracing the distribution of pixel densities and extend the range of values to a required range of intensity values To shift an image along x-axis and y axis To reflected image through x-axis and y-axis To rotate image randomly Sheared image using mapping [x, y] coordinates of input to [x’, y’] coordinate of output Converting all images from RGB to grayscale By applying Gaussian filter on both RGB image and Grayscale image Rotate image according to rotation angle, then resize the output to be same with the input

The proposed architecture of convolutional neural network contains 7 layers. The first 5 layers are for feature extraction. The last 2 layers are a fully connected layer and a soft-max layer. These two layers are used for classifying the extracted features. The input size is 100  100  3 while the filter size or receptive field is 5  5. In this network there is no zero-padding. And only one stride in each iteration [8]. Figure 3 shows the architecture of the network.

Fig. 3. The architecture of the network [8].

According to experiment in [8], the authors used the same dataset what was used in the previous work mentioned in [17]. The dataset contains 108 image of blood cells divided into 59 normal blood cells and 49 abnormal blood cells. However, they obtained 1188 images by applying the eight operations (data augmentation) that are mentioned in Table 1 above. This convolutional neural network model achieved in an accuracy up to 96.6%. The experiment was conducted on Matlab with 70% of the obtained dataset (831 pictures) for training the model. and 30% of the dataset (357 pictures) for testing the model [8].

White Blood Cells Detection and Classification Using CNN

871

3 Proposed Solution The proposed solution section describes the algorithms and methods that are used to build the proposed convolutional neural network model. The first stage is Image Preprocessing which contains five methods: Images Read, Convert RGB images to Grayscale, Image Resize, Image Labeling, Image Pickling. The second stage is a Convolutional Neural Network Model. Figure 4 shows diagram of study stages.

Fig. 4. Proposed solution stages.

3.1

Data Pre-processing: A) Images Read By using CV2 and OS libraries we can read images from folder. Then categorize the folder into 4 files. Also, CV2 allows us to read all images as gray scale. Figure 5 show code of how-to reading images and categorized [10, 11].

872

M. H. Siala et al.

Fig. 5. Reading images from folder and categorized [10, 11].

B) Convert RGB images to Grayscale Gray-scaling is simply reducing the complexity of the image dimensions from 3D pixel values (Red, Green, Blue) to 2D pixel values (Black, White). Thus, by applying grayscale conversion on the image it will narrow pixel intensity, which will provide an easier way to deal with the image in the form of 1 for white color and 0 for black color [12]. The grayscale conversion is applied on microscopic image as shown in the Fig. 6.

Fig. 6. Blood image sample after been converted to grayscale.

C) Image Resize In this step we decreased the dimension of the image from [340  240  3] to [50  50  1]. Image resizing is important when we need to decrease the total number of pixels in image. D) Image labeling In this step, we are labeling data by using function append. Once the image is labeled its used for training advanced algorithm to recognize pattern. Figure 7 shows labeling the training set using append function.

White Blood Cells Detection and Classification Using CNN

873

Fig. 7. Labeling the training set using append function.

E) Image Pickling After images labeled and appended it in X and Y array. We have to pickle the data to be saved in external database. The main reason of pickling data is to convert a python object into byte stream to be stored in external file or external database. Furthermore, pickling images is helpful for transporting data over the network, since we use a TensorFlow as backend engine. (Pickle library). Figure 8 shows the pickling of the training dataset [13].

Fig. 8. Shows the pickling of the training dataset [13].

3.2

Convolutional Neural Network Architecture

After preprocessing has been applied on input images. The images would be entering in a convolutional neural network. The network contains five layers. Four layers for extracting features from an input image. Last layer (Output Layer) to classify the features gained from the previous four layers. The dimensions of the input image are [50  50  1]. The dimensions of the convolutional filter are 3  3. The MaxPooling filter is 3  3. The stride is 1. Then moving filter one pixel at a time with zero paddings [8]. The processes of this network divided into two parts: The first part is the layers for extracting features of the input image resulted from applying three operations on each layer (1-Convolutional Operation, 2- Activation, 3- Max-Pooling). The second part is last one layer on the network for image classification as shown in Fig. 9 (Table 2).

874

M. H. Siala et al.

Fig. 9. The architecture of convolutional neural network model. Table 2. The analysis of each layer in convolutional neural network Part Layer Layer Layer Layer Layer

1 2 3 4 5

Operations in the layers Convolutional Operation/ReLu Convolutional Operation/ReLu/Max Pooling Convolutional Operation/ReLu/Max Pooling/Dropout Convolutional operation/ReLu Flatten/Dense/Softmax

Part Part Part Part Part Part

1: 1: 1: 1: 2:

Features extraction Features extraction Features extraction Features extraction Classification

Part 1 of the Network: which are the features learning through 4 convolutional layers. The convolutional layer contains Convolutional Operation, Activation and Max Pooling respectively. Convolutional Operation is a dot operation on the receptive fields of image which is a multiplication of the 3  3 kernel (filter) on image and slide it across the image iteratively (Eq. 1) [14, 15]. hð x Þ ¼ f  g ¼

x X

f ðx  aÞgðaÞDa ¼ F ^

pffiffiffi  2pF j f jF jgj

ð1Þ

1

The figures below are an explanation of the convolutional operation with kernel 3  3. Subsequently multiply this kernel by the 5  5 matrix the green matrix in the Figs. 9, 10, and 11. In this example there are nine iteration to reach the final result (matrix) of the convolutional operation with stride = 1 move up and down. The result of each iteration is a summation of the multiplication of the respective field (yellow matrix). The Fig. 10 shows the respective field movements in first iteration. The Fig. 11 shows the respective field movements in second iteration. The Fig. 12 shows the respective field movements in third iteration.

White Blood Cells Detection and Classification Using CNN

875

Fig. 10. Convolutional multiplication in iteration = 1 [15].

Fig. 11. Convolutional multiplication in iteration = 2 [15].

Fig. 12. Convolutional multiplication in iteration = 3 [15].

Activation or Normalization (ReLu): ReLu is a specific kind of activation function. The convolutional operation may result some negative value in the matrix image. Then the ReLu will turns all negative value of the image to zero. the purpose of this kind of normalization is to make our model as a nonlinear model and able to learn [15]. Max Pooling: The max pooling function is used for reducing the dimension of the image by taking the maximum value of the receptive fields of image which is 2  2. Figure 13 shows that in the first iteration of pooling it takes a 35 as maximum value from the set of (35, 19, 13, 22). And the second iteration It takes a 25 from the set of (19, 25, 22, 16), and so on till reach the end of the input file [15].

876

M. H. Siala et al.

Fig. 13. Taking max value of the 2  2 respective fields [15].

After the three operations has been applied in the first three layers, the output of layer 3 will be the input in the layer 4 which is Probability Conversion layer. Part 2 of the Network: The second part of the convolutional neural network which is a classification image into their related class (Neutrophils, Eosinophils, Monocytes, Lymphocytes). Probability Conversion In most of the neural network’s architecture perform the probability using Soft-max function. Therefore, in this project I used dense fully connected layer to classify the features. Then the output of the Dense function is a set of discrete probability values for each of the classes that were trying to predict [16].

4 Results The dataset used in this paper is provided by Paul Mooney, Developer Advocate at Kaggle [9]. The dataset labels, quantity, and size are considered as one of the main factors that influence on the accuracy of white blood cells classification. The dataset is divided into four classes which are the four types of white blood cells (Neutrophils, Eosinophils, Monocytes, and Lymphocytes) [9]. The quantity and dimension of the training set and testing set is in Table 3. Table 3. The dataset quantity and dimension [9] Image class Neutrophils Eosinophils Monocytes Lymphocytes

Training set Testing set Image dimension 2,499 48 320  240 2,497 38 320  240 2,478 33 320  240 2,483 6 320  240

The experiments were conducted on Google Colaboratory as notebook, Anaconda as python distributor, Keras library, and TensorFlow as backend engine. The accuracy rate of the recognition of types of white blood cells by our proposed Convolutional Neural Network model achieved 95.47%.

White Blood Cells Detection and Classification Using CNN

877

5 Conclusion Due to its efficiency in diagnosing Leukemia at early stages, the convolutional neural network would be of a great value to the medical diagnostic system used to detect Leukemia in the blood cells. The availability of medical data is the main factor used to improve the complete diagnostic system. The Convolutional Neural Network is an efficient method of image classification because it eliminates most of the steps used in the traditional image classification techniques. The detection and classification of white blood cells is dependent on two factors. The first factor is the provided dataset by the medical unit, the second is the development of the convolutional neural network to handle and classify the dataset. The accuracy of proposed program is 95.4% which is highly reliable in white blood cells classification. In the future work, abnormal white blood cells will be used to train and optimize the proposed CNN model.

References 1. Amanda Chan, LiveScience Health: The 10 Deadliest Cancers and Why There’s No Cure, 07:44am 10 September 2010. https://www.livescience.com/11041-10-deadliest-cancerscure.html 2. Leukemia & Lymphoma Society: Leukemia Definition and types. https://www.lls.org/ leukemia 3. Blood Cells Types. https://en.wikipedia.org/wiki/Blood_cell 4. Health Encyclopedia, University of ROCHESTER Medical Center. What are White Blood Cells: Types of white blood cells (2019). https://www.urmc.rochester.edu/encyclopedia/ content.aspx?ContentID=35&ContentTypeID=160 5. By Mayo Clinic Staff, Mayo Clinic: Leukemia Diagnosis (2018). https://www.mayoclinic. org/diseases-conditions/leukemia/diagnosis-treatment/drc-20374378 6. Sabtu: Leukemia of The Blood Cells, March 2015. http://informaticsofhealth.blogspot.com/ 2015/03/leukemia-cancer-of-blood-cells.html 7. Vaghela, H.P., Modi, H., Pandya, M., Patdar, M.B.: Leukemia detection using digital image processing techniques. Int. J. Appl. Inf. Syst. (IJAIS) 10(1), 43–51 (2015). ISSN 2249-0868. Foundation of Computer Science FCS, New York, USA. www.ijais.org 8. Thanh, T.T.P., Vununu, C., Atoev, S., Lee, S.-H., Kwon, K.: Leukemia blood cell image classification using convolutional neural network. Int. J. Comput. Theory Eng. 10(2), 54–58 (2018) 9. Mooney, P.: Developer Advocate at Kaggle. Boulder, CO, USA (2018). https://www.kaggle. com/paultimothymooney/blood-cells 10. The Python Software Foundation. Miscellaneous operating system interfaces: The Python Standard Library. Generic Operating System Services (2019) 11. Mordvintsev, A., Abid, K.: OpenCV-Python Tutorials: Gui Features in OpenCV. Getting Started with Images (2013) 12. Coste, A.: University of Utah. Gray scale digital image (2018). http://www.sci.utah.edu/ *acoste/uou/Image/project1/Arthur_COSTE_Project_1_report.html 13. Agrawal, A.: Understanding Python pickling and how to use it securely, Tuesday, November 18th 2014. https://www.synopsys.com/blogs/software-security/python-pickling/ 14. Dettmers, T.: Understanding Convolution in Deep Learning (2015). https://timdettmers.com/ 2015/03/26/convolution-deep-learning/

878

M. H. Siala et al.

15. Raval, S.: Convolutional Neural Network.ipynb, San Francisco, CA (2019). https://github. com/llSourcell/Convolutional_neural_network/blob/master/convolutional_network_tutorial. ipynb 16. Keras Documentation Activation https://keras.io/activations/ 17. Thanh, T.T.P., Pham, G.N., Park, J.H., Moon, K.-S., Lee, S.-H., Kwon, K.-R.: Acute leukemia classification using convolutional neural network in clinical decision support system (2017)

Work-in-Progress: The Use of Big Data and Data Analytics in the Prevention, the Diagnosis and the Monitoring of Long-Term Diseases Oualid Mecili1(&), Barkat Hadj1, Farid Nouioua1, Samir Akhrouf2, and Rachid Malek3 1

3

University of Bordj Bou Arreridj, El Anseur, Algeria [email protected] 2 Mohamed Boudiaf University M’Sila, M’Sila, Algeria CHU de Sétif, University Ferhat Abbas Sétif 1, Sétif, Algeria

Abstract. Diabetes is a group of metabolic diseases in which a person has high blood sugar, either because the body does not produce enough insulin, or because cells do not respond to the insulin that is produced. The constant hyperglycemia of diabetes is related to long-haul harm, brokenness, and failure of various organs, particularly the eyes, kidneys, nerves, heart, and veins. This last can be monitoring by doctors over the internet and especially social networks. In this context, having a healthcare social media strategy is no longer optional, it’s a requirement. With the right strategy, social media will become a powerful tool to build trust, reach more patients, and spread valuable medical information. Furthermore, advances in mobile technology and the widespread use of smartphones and tablets will make an improvement in healthcare services at a rapid pace. In this paper, we propose a system that supports a community of diabetes people, their families and friends, doctors, nutritionist and anyone who might need or offer help and support to the community. Our objective is to develop a system that can predict, diagnose and monitor the diabetes or support diabetes self- management using machine learning algorithms and Big Data. Keywords: Big Data Mobile application


Diabetes
Social network
Health technologies


1 Introduction Diabetes is a chronic disease requiring ongoing medical care, a continuous patient education, and management. The diabetes is a very serious disease that can lead to a large number of long-term complications such as blindness, liver problems, heart disease and kidney. Early prediction of diabetes is very important for timely treatment which can stop the disease progressing to such complications.

© Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 879–887, 2021. https://doi.org/10.1007/978-3-030-49932-7_81

880

O. Mecili et al.

The International Diabetes Foundation (IDF) showed that in 2017 the MENA region had the second highest prevalence of diabetes (9.2%), with almost 40 million people with diabetes, after North America and Caribbean region. Algeria represents a prevalence of 6.9% with and annual cost of approximately 567 USD per diabetic person [1]. The increased prevalence of chronic (and non-communicable) diseases is today largely attributable to the convergence of an aging population with the persistence of several risk factors. Many of these risk factors can be mitigated by health interventions and education, and communication tools could support healthy lifestyle and behavior change. To obtain feedback and general information about the disease they suffer from, and to receive alerts to remind them to take their medications or to measure their blood glucose levels, more than 100,000 health related apps are also available in the market allowing people to record, track, and analyze vital signs and physical health data over time.

Fig. 1. Annual digital growth in Algeria January 2019.

Due to the large number of internet users and social networks in Algeria [2] as it is shown in Fig. 1, sharing health data among members of an online community seems also to be correlated to a better management of the disease they suffer from. Social networks and user-friendly mobile devices are one of the most significant recent developments in information and communication technology. These, can be used as a center-stone medium in diabetes prevention and treatment through appropriate awareness. As well as being successful and popular, social media hold a potential for interesting new use cases in health care. The mobile health applications (apps) are growing with over 31,000 health and medical apps available for download [3], and that lead to increasing market size [4] as it is shown in Fig. 2.

Work-in-Progress: The Use of Big Data and Data Analytics

881

Fig. 2. mHealth industry market size

Apps designed for health professionals can be used to diagnose diseases, consult data on medications, perform clinical calculations, search scientific evidence, exchange clinical experiences, improve the management of chronic diseases, and conduct health care research [5]. From a health care perspective, the use of mobile phones by clinicians could improve clinical communication, increase the practice of evidence-based medicine, and enable access to information tools at the point of care. This work presents the design and development of a computer system that helps patients with diabetes to predict and self-manage their condition with a close range to healthcare professionals. We aim to design and develop a specific-purpose social network to support a community of diabetes, in addition we propose a specific purpose social network to support the diabetes community in order to help them discuss their problems, and this discussion will be moderated by doctors and specialists. In the next sections of this paper, the related work is outlined and the project methodology is described. Finally, we present the system architecture, then we finish by a conclusion and future works.

2 Related Work Doctors and patients can greatly benefit from taking advantage of the ever-growing mobile platform through the various medical mobile app developments. By developing a medical mobile app to help monitor their health conditions and nutrition levels, provide more people with access to even the most basic levels of healthcare, millions of people rely on their mobile devices to help them simplify their daily life. Through a simple research on Google Play or Apple store, it is possible to find out many applications related to diabetes self-management. However, only a few of them are open to everyone, while others have only some premium functionality available at an additional cost.

882

O. Mecili et al.

The functionalities in all applications analyzed are timely insulin dosage or medication and registration of blood glucose levels some of these applications have indications suitable for diet, physical exercise, weight control, and blood pressure. Other support alert/notifications and integration with social media such as Facebook and Twitter. A study of an online community enabling type two diabetic patients reporting, charting and optional sharing of recent hemoglobin HbA1c values through a geographic display, showed that patient provided data were current with 83.1% of most recent HbA1c values reported obtained within the past 90 days [6]. El-Gayar [7] argues that it would be helpful to the patient to provide an individual analysis and interpretation of results. However, in the majority of the applications, it only provides insulin dosage suggestion, based on the data record, located in the patient mobile device [8]. The limitation of these applications is remarkable for the nonconsideration of clinical history of the patient. Furthermore, diabetes researchers consider that self-management education has a positive impact on clinical outcomes [9]. One issue that has not been taken into account by developers is usability. Usability is a quality or attribute that represents the ease which a human-computer interface issued and provides an effective, efficient, and satisfying experience [10]. The development of meaningful solutions that are reliable, as well as compliant with the law, is a critical step in the adoption of medical apps among patients and care providers. There are many different types of medical mobile apps, but most fall into 5 common categories [4] as it is shown in Fig. 3.

Fig. 3. Types of health care apps

Work-in-Progress: The Use of Big Data and Data Analytics

883

However, not all of these digital interventions are successful. Many medical mobile apps are not living up to customer expectations due to poor user experience, confusing user interfaces, and unsatisfactory functionality. In this context, the authors will develop a computer system with the aim to help diabetes prediction and self-management of the patients, with physician monitorization. This computer system is multiplatform, a mobile application for patients and a web application for doctors, nutritionist and gym experts.

3 Methodology Good quality design is important for any mobile application, but it’s especially important when developing an app for sensitive target audiences such as those using medical mobile apps. A number of factors need to be considered when designing an app that is easily adoptable by users, as well as reliable and secure. In order to develop a mobile health application successfully, it is important to find out what the target audience needs most and focus development around the implementation of those essential features. Patient-oriented medical app development should always target the end user’s needs and should focus on delivering a great user experience, as well without ensuring that a developed app is safe and secure for both patients and providers, the whole system becomes useless. To develop the system a cycle methodology was used which consist of four phases: conception, prototype development, tests and result analyses [11] as it is shown in Fig. 4.

Fig. 4. Cycle methodology

884

O. Mecili et al.

3.1

Conception Phase

Since many users in this sector will be elderly, might have sensory impairments or other disabilities, or may lack the technical expertise of younger mobile app users, it’s necessary to think ahead when we create our design like: • • • • • •

Color schemes Fonts Buttons size Text alignment Icons Images

And in order to solve the problem of trust, we can make use of encryption. It is widely considered the most effective way to protect sensitive data, whether is it at rest, in transit, or traversing multiple network connections. It can be used to protect: • • • •

Databases. Files on servers. Entire communication channels. And other potentially sensitive transmissions or storage of data.

We took the best practices associated with creating effective UI/UX for medical apps. Taking color schemes and font selection into account, as well as making smart decisions on various other elements and design details and functionalities will foster trust among users and facilitate immediate adoption of the system designed. Mobile Application Functionalities. • FINDRISC Questionnaire: number of questions that the answers can give a prediction whether the patient will develop diabetes or not as it is shown in Fig. 5. • Patient initial configuration: personal profile and clinical history. This functionality is configured by the healthcare professional through the web application that is automatically synchronized with the patient mobile application. • Daily registration: blood glucose levels, blood pressure and important annotation such as food intake changes, medication, physical exercise and others. • Notifications/Alerts: alert notifications about taking medication (insulin or others), date of medical consultation, assessment and others alerts that are important for the patient health. • Data analysis: The information that is recorded daily (blood glucose levels, blood pressure, and weight) are represented in a chart and table format. In conjunction with the patient representation values, the chart represents the normal values of the patient’s age. • Chat: allows the patient to communicate with the healthcare professional whenever it has a doubt. Also, it is possible to the patients to communicate with each other’s.

Work-in-Progress: The Use of Big Data and Data Analytics

885

Fig. 5. FINDRISC

• Education: presents a set of information about diabetes disease and healthcare that patient must have. The healthcare professional through the web application share with the patient mobile application, a set of specific healthcare information according to their needs. • Patient configuration: allow the healthcare professional insert, edit and query data of the patient. • Data transfer: the healthcare professional can transfer data to the patient mobile application such as: education and diet plan, physical exercises. Also, the web application receives the data about patient daily registration. • Alerts: the healthcare professional receives alerts when a patient has abnormal values related with blood glucose, blood pressure and others. Here is the proposed architecture system illustrated in Fig. 6.

886

O. Mecili et al.

Fig. 6. Architecture of the proposed system.

4 Conclusions and Future Work Diabetes is a chronic illness that requires constant monitorization and self-management through medication, diet, and exercise. Patients with diabetes face a challenge of preventing or monitoring these various components into their daily lives. The mobile technologies have made diabetes applications available and these have a great potential to support diabetes prediction and self-management. Furthermore, some applications are not suitable to patients needs once it has lack of personalized feedback and usability issues, particularly for elderly people. Along with these lines, the adoption of an application for diabetes prediction and self-management only makes sense if the patients feels secure and confident in its use and have a close support of their healthcare. But current researches enhance our understanding of how the lack of some designated core features may influence the impact of specific application features on clinical outcomes. This work presents the design of a system that can predict, diagnose and monitor the diabetes. It also supports diabetes self-management with the use of UI/UX which is suitable for everyone. As a future work, we intend to study the clinical effectiveness of the system and deploy the social network in order to gather enough data to conduct our future research.

References 1. International Diabetes Federation: IDF Diabetes Atlas, 8th edn. International Diabetes Federation, Brussels, Belgium (2017). http://www.diabetesatlas.org 2. https://datareportal.com/reports/digital-2019-algeria

Work-in-Progress: The Use of Big Data and Data Analytics

887

3. Ganasegeran, K., Renganathan, P., Rashid, A., Al-Dubai, S.A.R.: The m-Health revolution: exploring perceived benefits of WhatsApp use in clinical practice. Int. J. Med. Inform. 97, 145–151 (2017). [CrossRef] [Medline] 4. https://steelkiwi.com/ 5. Payne, H.E., Lister, C., West, J.H., Bernhardt, J.M.: Behavioral functionality of mobile apps in health interventions: a systematic review of the literature. JMIR mHealth and uHealth 3(1), e20 (2015) 6. Weitzman, E.R., Adida, B., Kelemen, S., Mandl, K.D.: Sharing data for public health research by members of an international online diabetes social network. PLoS ONE 6, e19256 (2011). https://doi.org/10.1371/journal.pone.0019256 7. El-Gayar, O., Timsina, P., Nawar, N., Eid, W.: Mobile applications for diabetes selfmanagement: status and potential. J. Diabetes Sci. Technol. 7(1), 247–262 (2013) 8. Fu, H., McMahon, S.K., Gross, C.R., Adam, T.J., Wyman, J.F.: Usability and clinical efficacy of diabetes mobile applications for adults with type 2 diabetes: a systematic review. Diabetes Res. Clin. Pract. 131, 70–81 (2017) 9. Norris, S.L., Lau, J., Smith, S.J., Schmid, C.H., Engelgau, M.M.: Selfmanagement education for adults with type 2 diabetes. Diabetes Care 25(7), 1159–1171 (2002) 10. Esteves, M., Pereira, A.: YSYD-You Stay You Demand: user-centered design approach for mobile hospitality application. In: 2015 International Conference on Interactive Mobile Communication Technologies and Learning (IMCL), pp. 318–322. IEEE (2015) 11. Hartson, R., Pyla, P.S.: The UX Book: Process and Guidelines for Ensuring a Quality User Experience. Elsevier, Amsterdam (2012)

An Interactive Augmented Reality Volume Rendering Mobile Application Amr S. Mady(&) and Samir Abou El-Seoud Faculty of Informatics and Computer Science (ICS), The British University in Egypt (BUE), Cairo, Egypt [email protected], {amr122437,samir.elseoud}@bue.edu.eg

Abstract. Throughout the past several years, the new technologies and enhancements related to medical visualization has been more than fascinating. In this study, we discuss a proposed idea of a handheld or a mobile Augmented Reality (AR) application. This application will ease the process of medical visualization. The criticality of a faulty medical imaging diagnosis could lead to severe consequences and outcomes to patients’ health. This criticality is what ignites new ideas to improve medical imaging. The proposed AR application will help doctors and radiologists in their daily basis. Still most doctors, surgeons, and radiologists use two-dimensional (2D) slices to visualize patients’, which is quite hard to image and map the 3D structure of a human anatomical structure with just 2D plain images. This process requires a lot of experience which sometimes might not be enough. It will help physicians by visualizing patients’ scans such as Computed Tomography (CT) or Magnetic Resonance Imaging (MRI) to be rendered as a three-dimensional (3D) object and not as a 2D object. Using the AR technology makes it much more straightforward in observing the rendered object. The proposed application reads a medical image file(s) such as CT or MRI files and renders its data as a 3D model in real world space, by using a volume rendering technique called ray-tracing. This technique works by reading data from a selected file or dataset. After reading a file, the technique traces a ray into object space from each pixel. Afterwards, it accumulates and calculates the color and opacity values of pixels along each ray. The Final step is to assign the values to its corresponding pixels. By completing this process, a selected dataset is rendered as a 3D object into real world space. With AR technology, seeing and interacting with the 3D object is much easier for the users. The expected outcomes of this study are to improve the process of pre-surgery medical images visualization. With interactive buttons and controllers included in the application and the use of several options like decreasing opacity, clipping, zooming, rotation, threshold … etc. will give more control to the users. Moreover, the proposed application enables users to customize his views to get the best of the 3D object for better visualization. This study discusses the in development mobile AR application that could be used as a new, more facilitating method of visualizing medical images. The application will deliver a more feasible way of interacting with visualized medical images, with buttons and sliders and virtual joysticks creating a better user experience (UX).

© Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 888–896, 2021. https://doi.org/10.1007/978-3-030-49932-7_82

An Interactive Augmented Reality Volume Rendering Mobile Application

889

Keywords: Volume rendering
Ray-tracing
Medical imaging
Computed Tomography (CT)
Magnetic Resonance Imaging (MRI)
Augmented Reality (AR)

1 Introduction Applications of medical imaging visualizations has received great attention in the recent years, and it began to spread into every small topic and aspect in medicine speedily, as stated in Ref. [1]. But the problem with the recent developments is the need of setting a big room full of heavy and expensive hardware just to accomplish the requirements of only running the developed application. But when it comes to anatomical visualization, applications should be lite sized and be compatible with many devices even portable ones like smartphones, to help ease the everyday life of surgeons and radiologists. Not all surgical rooms have the feasibility of installing such heavy equipment nor the money of buying such expensive hardware just in order to visualize patients, like many of the in-developed countries, poor cities, or states wouldn’t, and would rather hire more radiologists to visualize patients’ scans. And as procedures of interventional radiology using imaging guidance such as CT/MRI does not always meet the full satisfactions of doctors or surgeons. In some procedures, radiologists have to scan patients from different angles, which results in exposing them both the patient and the radiologist of excessive radiation. Afterwards, surgeons have to visualize the resulted images from the scans by investigating multiple images, which is both exhausting and time consuming. Several applications that uses AR technology were developed to provide a solution to this medical problem over the past recent years, see Ref. [2]. The fundamental encounter in these AR applications offering a real-time representation that is quite precise when compared to human organs [3]. In this study, we consider the following scenario. A patient is going to be undergoing a minimally invasive medical procedure where the specialist needs to process some procedure [1]. We additionally expect that a CT or MRI picture of the patient is accessible. However, it is difficult for most surgeons to map the 3D anatomical structure of the real patient with just 2D images. This happens frequently when a surgeon lacks some experience in a similar procedure. It requires skills in order to visualize in what way some 2D slices that viewed on a screen are located in a patient in a surgery room. The objective of the in-development application is mapping the anatomical structure of a patient’s body with a virtual augmented 3D model for an easier and more comfortable visualization. Our indevelopment system could be a one step forward in solving the problem of visualizing human bodies. The proposed mobile application that is currently in development, is lite sized and cross platform, enabling it to run on many Android and IOS smartphones, and doesn’t requires buying not setting any external hardware for it to work. It renders a 3D model of a selected medical imaging file like a CT or an MRI Digital Imaging and Communications in Medicine (DICOM) file [4], or a plain raw file, by volume rendering technique, to be displayed and placed on a selected surface position by the user using Google’s AR Core technology, check Ref. [5].

890

A. S. Mady and S. Abou El-Seoud

Modern innovations made with volume rendering in medical imaging visualization of human body has been making a revolution recently [6]. Most of the modern 3D computer games and other computer graphical software has been built by voxels. A voxel which is the 3D equal to a pixel and is the smallest element in a 3D model [7]. A great part in minimally invasive procedures over the recent years have been taken by several volume rendering applications. Volume rendering technique renders the data inside a volume object, volumetric data are transformed by the volume rendering technique to screen pixels to be displayed directly. And also, the technique controls the transparency of a volume to see through it. This work points at the development of an AR software that will be used in minimally invasive surgeries and interval procedures. Several groups and research labs formed of scientists and developers have also worked in the same field of medical imaging applications, but none of which went on developing such software in AR, neglecting its vast importance in medicine, some of these works are mentioned in Sect. 4. Our goal is to reduce the heavy load of scans visualization, as well as saving time and effort. This procedure will be much cheaper than the previous methods. Our application requires only a smartphone. Augmented Reality is the combination real world space and virtual objects. It allows users to see and interact with both physical and virtual items. Therefore, using AR in medical visualization applications is more practical than previous methods.

2 Materials and Methods 2.1

System Overview

In this study, the proposed in-development application outputs a 3D representation of medical images (e.g. CT, MRI). First, the software process medical images by using volume rendering ray-casting technique. Volume rendering is a term that is used to describe techniques that are used to render and visualize 3D data. The technique works as follows: Step 1: • Casting rays into a volume, while sampling the volume at certain intervals. • Samples are interpolated or reconstructed from voxel grids at each sampling location. • Trace each pixel along the casted rays. • Compute and accumulate color/opacity value along each ray in the process of pixel compositing. • Assign the obtained value to the pixels. Figures 1 and 2 illustrates the process of Ray-marching and compositing of pixels.

An Interactive Augmented Reality Volume Rendering Mobile Application

891

Fig. 1. Ray-marching process [8, 9].

Fig. 2. Compositing of pixels’ color/opacity along the ray [9, 10], where c is the color of pixel, and a (alpha) refers to the opacity.

Step 2: In this step, is where a compositing technique is used named alpha blending, i.e. the iterative computation of discretized volume integral. Figure 2 illustrates how alpha blending works while each ray goes through the object on its direction. The in-development software has cross-platform support, enabling it to run on most Android (Android Nougat and newer) and IOS (IOS 11 and newer) mobile phones. It will enable the software to augment the 3D object of the medical scans in real world space. Interaction with the augmented object will be performed using but-tons, sliders, and virtual joysticks. Users could manipulate the viewed 3D object generated from the medical images sliced by hiding parts of the object or view it in different ways with some GUI features to help the user to interact with it more easily, such as:

892

• • • • •

A. S. Mady and S. Abou El-Seoud

Increasing visibility Increasing and Decreasing Opacity Clipping (removing parts of the object) on the X, Y and Z axes. Rotation and Translation And with Google’s AR Core technology, change the position of the augmented on any spot on a surface. Briefly, the considered scenario may be summarized as follows:

1. Obtaining a volumetric medical data (e.g. DICOM, raw file, sequence of JPEG/PNG images). 2. Preprocess the selected medical data to become a raw file, if the selected data by the user wasn’t a raw file. 3. The data are stored afterwards on a smartphone. 4. The software will render the data from step 3 as a 3D object into reality using AR technology. 5. User will interact with the 3D object via several interactive GUI elements. Using the previously discussed scenario, physicians and radiologists will be able to review scanned images of a patient as a 3D object in the presence of them and they will be able to interact with it through interactive buttons, sliders, joysticks. They have the capability to zoom in or out or even to make parts of the object transparent as well as clipping parts of it. 2.2

Image Acquisition

First, CT or MRI scanners scan a patient. Then, the data outputted from the scanner(s) is sent to an online database, or a hospital’s archives to be stored and registered. Afterwards, the data has to be downloaded by a smartphone via wireless communication for processing and visualization. In this research, we have used CT datasets from online medical imaging database websites.

3 Results and Discussion 3.1

Samples and Results

In this section, we demonstrate results obtained from experimental study using the proposed application. In Fig. 3, a sample of abdominal CT slices obtained from single image from the CT data used in this experiment. These images demonstrate the anatomical structural of human internal organs (Fig. 6). The used method in this application was tested using two different 3D datasets of CT scans, samples from the used datasets were shown in Fig. 3 and Fig. 5. Figures 4 and 5 previews some sample screenshots from the in-development applications, rendering the sample datasets in AR, and showing some features of the applications like clipping and changing the 3D model’s shader’ opacity. And the recorded frames per second (FPS) of the application viewing the 3D model ranged between 10–25 FPS, while having stable FPS of 50–60 when it focuses away from the rendered object.

An Interactive Augmented Reality Volume Rendering Mobile Application

893

Fig. 3. Sample slices from the first CT dataset [1].

Fig. 4. (Left) An AR 3D object rendered by ray-tracing technique by the application, full opacity, no clipping. (Center) Same Object, but with reduced opacity to 0.04. (Right) Same object as in the center figure, but with clipping 45% of it along the x-axis, and 30% along the yaxis.

894

A. S. Mady and S. Abou El-Seoud

Fig. 5. Sample slices from the second used dataset [1].

Fig. 6. An AR 3D object rendered by ray-tracing technique of the second used dataset by the application, full opacity, no clipping. (Center) Same Object, but with reduced opacity to 0.04. (Right) Same object as in the center figure, but with clipping 40% of it along the x-axis.

3.2

Discussion

The proposed software will help in facilitating the process of medical images visualization, to save time and effort of surgeons and radiologist. It only required a couple of seconds to render a data-set of 200–400 images. This method has great potential to be used in many surgeries like minimally invasive procedures where the surgeon is required to view internal structures mapped with the patient body in real-time. Results indicate that using the proposed method can help in the rendering and 3D visualization of CT volumes in very short time that lead to exact recognition of different large size objects.

An Interactive Augmented Reality Volume Rendering Mobile Application

895

4 Related Work Back in 2010, a project was started by a group of scientists from the university of München, Germany. This project or study aimed to map the CT scans obtained from patients with their body. Objects in a scene are kept track of by an AR system that the scientists developed, their AR system consists of optical tracking and a video seethrough Head Mounted Device (HMD). The process of mapping data from the CT onto a human body is carried out by two distinct optical tracking systems, a four infrared (IR) cameras called “ARTtrack2” mounted to a room’s ceiling, obtaining an outside-in optical tracking system. While an IR camera mounted directly on the HMD is used as an inside-out optical tracking system. They have used video image as a context layer, while a focus layer is rendered via a volume rendering technique [11]. Images of their work are shown in Fig. 7.

Fig. 7. “(a) Illustration of the occlusion problem. (b, c, d) Render pipeline for correct occlusion handling, (b) video texture, (c) hit texture for the skin, (d) final composition of (b) and (c). (e) like (d) with in-body MPR. (f) Focus and Context rendering with shaded volume rendering for the focus layer (bone), virtual mirror and instrument” [11].

5 Conclusion and Future Work 5.1

Conclusion

In this work, discussed the in-developed system was discussed and previewed that could be used as a new method in visualization of medical images in medical practices.

896

A. S. Mady and S. Abou El-Seoud

The application delivers a suitable visualization to surgeons and radiologists, helping to create a better environment for surgeries. 5.2

Future Work

Developing a Mixed Reality volume rendering application, packed with a machine learning model to choose the most suitable transfer function for each medical imaging dataset to be rendered in RGB instead of grayscale, and detection of abnormalities in the user selected dataset.

References 1. Abou El-Seoud, S., Mady, A., Rashed, E.: An interactive mixed reality ray tracing rendering mobile application of medical data in minimally invasive surgeries. Int. J. Interact. Mobile Technol. (iJIM) 13(03), 29 (2019) 2. Vavra, P., et al.: Recent development of augmented reality in surgery: a review. J. Health Eng. 2017, 4574172 (2017) 3. De Paolis, L.T., Aloisio, G.: Augmented reality in minimally invasive surgery. In: Advances in Biomedical Sensing, Measurements, Instrumentation and Systems, pp. 305–320. Springer (2010) 4. Dicomstandard.org: DICOM Standard (2019). https://www.dicomstandard.org/. Accessed 14 Jul 2019 5. Google Developers: ARCore overview | ARCore | Google Developers (2019). https:// developers.google.com/ar/discover/. Accessed 14 Jul 2019 6. Udupa, J.K., Goncalves, R.J.: Medical image rendering. Am. J. Card. Imaging 7(3), 154–163 (1993) 7. What is a Volume Pixel (Volume Pixel or Voxel)? - Definition from Techopedia. https:// www.techopedia.com/ 8. Volume ray casting. En.wikipedia.org (2017). https://en.wikipedia.org/wiki/Volume_ray_ casting 9. Möller, T.: Direct Volume Rendering. University of Vienna 10. Komura, T.: Volume Rendering, Visualization – Lecture 10. The University of Edinburgh (2008) 11. Wieczorek, M., et al.: GPU-accelerated Rendering for Medical Augmented Reality in Minimally-invasive Procedures. Bildverarbeitung für die Medizin 574, 102–106 (2010)

Design of an Accessible Web Portal for the Labor Insertion of People with Blindness Javier Sánchez Guerrero1 , Julio Alfonso Mocha-Bonilla1(&) Esmeralda Zapata-Mocha2, and Sandra Carrillo Ríos3

,

1

2

Facultad de Ciencias Humanas y de la Educación, Unidad Operativa de Investigación y Desarrollo, Universidad Técnica de Ambato, UTA, 180103 Ambato, Ecuador {jsanchez,ja.mocha}@uta.edu.ec Facultad de Ciencias Humanas y de la Educación, Carrera de Pedagogía de la Actividad Física y Deporte, Universidad Técnica de Ambato, UTA, 180103 Ambato, Ecuador [email protected] 3 Facultad de Ingeniería en Sistemas, Electrónica e Industrial, Universidad Técnica de Ambato, UTA, 180103 Ambato, Ecuador [email protected]

Abstract. This work is part of the study carried out in the central zone of Ecuador, formed by the provinces of Tungurahua, Pastaza, Chimborazo, Bolivar and Cotopaxi, where people with blindness and their labor situation were initially detected, taking into account Ecuadorian legislation and the Constitution of the Republic of Ecuador in Art. 330 mentions that people with disabilities are guaranteed insertion and accessibility in equal conditions, guaranteeing paid work. Supported by this legislation, an Accessible Web Portal has been designed to support the group of people with total blindness, thus facilitating their labor insertion in the productive sectors. Our research aimed to categorize the skills and abilities of people who are blind. The methodology consists of two stages: The first was constituted by the identification of skills and abilities, while the second stage was the creation of a portal with accessibility patterns designed through HTML and CSS where the personal data of individuals with blindness are shown, that is to say, to make the job profiles visible to all employers. It is concluded that the use of the accessible web portal generates dynamics in the process of labor insertion, highlighting that blind people can access the portal without problems, finally the accessible portal must be considered a means of common use for people who need to be employed and are visually impaired. Keywords: Web portal insertion


Accessibility
Blindness
Employment
Labour

© Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 897–908, 2021. https://doi.org/10.1007/978-3-030-49932-7_83

898

J. Sánchez Guerrero et al.

1 Introduction In order to perceive a person’s visual disability, its typology must be analysed: low vision, partial blindness or total blindness. It is also necessary to know the biological, psychological and social characteristics in order to be able to understand the incident factors in the socio-familiar and personal impact of a blind person. The psychological impact, for example, in terms of development is particularly the same as in people without disabilities, but usually the psychic social impact is negative from different fronts [1]. Within employability, certain variables of labor market insertion must be considered for people who present blindness, as is the case of the skills, abilities and limitations presented by each person. However, for blind people there are no real efforts to support them to achieve the desired labor insertion within public or private companies. Therefore, it is fundamental to analyze the abilities, skills and strategies to include people who are blind, because it seems that the efforts are inefficient, in spite of the current laws [2, 3] which are only written on paper, waiting for the fulfillment of the proposed objectives. We are currently experiencing social exclusion in the workplace for people who are blind. One of the problems identified lies in the challenge of inclusion toward higher education, which generates a fair system for all participants with visual disabilities, where higher education promotes new educational approaches with curricula that solve classroom strategies [4] to break social barriers. In such virtue, the university challenge is posed to promote the active participation of students with diverse disabilities, when dealing with blindness in people must be motivated to socio-educational success, which generates academic opportunities, active, dynamic, responsible in terms of their skills and abilities, in itself to train blind people so that they can achieve a real insertion into the labor market. Visual impairment as a comorbid factor may be associated with various problems or disorders that may further hinder autonomous and professional development [5]. People who are blind set out to develop various types of goals: academic and learning, social reinforcement, achievement, or reward [6]; such goals arise when “achievement goals are diversified into two different ones: performance goals, which could be the motivation for short-term achievement, and achievement and reward goals that represent the desire for long-term achievement” (Jover, Navas, & Sampascual, 2009, pp. 208), it is there, where people with disabilities wish to formulate their own selfconcept, an academic-labor self-concept formulated from disability based on achievement skills. Among the factors that must be considered to generate a good development of the subject focused on labor insertion is the emotional self-concept, taking into account the factors proper to human evolution, which are directly inherent that recognize the social influence on the welfare and adaptation of the individual [7]. Another considerable factor is age, since visual problems change based on a person’s development and worsen into adulthood, for example in the case of low vision, problems may worsen until blindness is reached; for this reason it is necessary to generate treatment and rehabilitation programs as a prevention factor [8].

Design of an Accessible Web Portal for the Labor Insertion of People with Blindness

899

Labor market insertion is a very broad topic of great interest, in which at a global level all people should be immersed [9], specifically public and private companies. In order to understand what labor market insertion is in terms of social inclusion, it is necessary to understand the difficulties suffered by people who are blind, to know what the social and labor problems suffered by this type of people are, then functional problems must be understood; after an experiential analysis, strategies can be proposed to provide a viable solution [10]. The purpose of accessible Labor Insertion is to offer people who are in a situation of social and labor exclusion the intention of including them in a labor sector, considering that work allows the person to access an economic status within society, motivating access to social, political and economic participation, based on a productive environment [11].

2 State of the Art Today web portals have become substantial means of communication, through which you can find quality information, i.e. everything we are looking for thanks to computer skills, permanent and updated information, having access to information efficiently by so-called web means of accessibility [12]. Web portals have become one of the most widely used elements of modern times, as they are a source of communication between people, the complex communication and computing infrastructure behind the modern web service becomes increasingly affordable to meet the diverse requirements of people [13]. This is the case in the areas of health, education, training and information search, where mobile technologies are being used as an opportunity to improve the usability and acceptability of web portals [14]. In the case of medicine, the use of technology for disease prevention is being incorporated, the objective of which is to ascertain the state of health through so-called computerized diagnoses carried out by professionals who work with ICT [15], in whether web portals offer support to users to have access to a series of resources that can help and provide solutions to various problems, by means of which people can solve their needs and have access to information for everyone equally [16]. In short, portals or websites help countless users to solve various problems by providing information on countless topics that meet the needs of users [17]. Web portals have become of great impact on institutions, as they generate information management [18], but when it comes to governance indicators in relation to the perceived quality of public accessibility services, it is reported that these are not found on municipal websites in any country, In the case of Ecuador, this transactional level, characterized by UNESCO for two-way communication between the government and its citizens, has not yet been reached [19]. Online job portals collect vacancies on the web as a transcendent means for job demand, matching supply and the labor market [20]; the accessibility of a website means that everyone, including people with disabilities, can perceive, understand and interact with the site [21]. Several studies have been carried out on the accessibility of people with blindness: Regarding social inclusion, research has been carried out on automated accessibility for people with disabilities, which emphasizes the need to consider human skills as a

900

J. Sánchez Guerrero et al.

spectrum and must be addressed in order to use automated and accessible systems [22]. The modern computerized society with access to visual information becomes a necessity for all types of users, however, as blind people can access, namely reviewing the literature on Fitts Law and visual data accessibility solutions in blind users can be considered a first step towards an experimental evaluation of vibratory touch screen accessibility, towards an interface design adapted for blind people [23]; accessibility of visual information has become a necessity for all kinds of tasks and users, access to graphics and images on websites especially for blind users, allows the use of digital assistance tools, using adapted software, screen reader and adapted browser, whose vibration feedback with touch screen must be studied and fully understood before integrating other modalities such as sound, human speech or other forms of haptic feedback [24]; other studies on accessibility standards in websites are based on the principle of access to the same information, its application allows visual information through screen readers for the blind, however, to improve the usability of the website, it is necessary to take into account the specific needs of users and the satisfaction of participants [23]. Therefore, in our research, thanks to the contribution and use of web portals that serve as a channel of communication to access information on a given topic, we find accessibility barriers, very few sites are designed to provide accessibility to people who are blind; therefore we have seen fit to implement a friendly platform for people who are blind and need access.

3 Methodology According to the National Council on Disabilities (CONADIS), in the city of Ambato there are 906 people with visual impairment (low vision, partial blindness and total blindness), of which 499 people present total blindness, verified sample as affirmed [25] in relation to random sampling, by means of which each of the members of the population will have the same possibility of being included in the sample, this technique consists of assigning each member of the population a number and in this way any of them has the possibility of being a participant in the investigation, with which the taking of the sample and its origin is justified, contrasted on the basis of CONADIS support information. As it was a study carried out on people with total blindness, it was necessary to manage the permits with the relatives of the subjects involved, respecting at all times the protocols on human trials, as mentioned in the Declaration of Helsinki of 1994 updated in Seoul by the Assembly of the World Medical Association, in force since 2008. In this way, we base our study so that in the future it can be replicated in other contexts. Clubs, schools, associations of blind people from the city of Ambato, specifically from the central zone of Ecuador, were part of the study. All subjects were informed of the objectives and characteristics of the research, and family members or legal representatives agreed with the purpose and intent of the research.

Design of an Accessible Web Portal for the Labor Insertion of People with Blindness

901

In the field work, interviews were conducted with people who are blind, and who already work in the labor sector to learn about the activities they perform, as well as managers, directors and chiefs of staff of public and private companies, in order to obtain information on the labor fields of companies, categorizing and determining the labor activities in which people with blindness can perform. It was possible to classify the participants according to the degree of work performance, according to the skills and abilities of each subject. Specific indicators of job profiles were found which include: customer service, packaging of footwear and clothing, reception assistant, information assistant for user service in supermarkets and travel agencies, library assistant in Braille system, typist and typesetter. 3.1

Process

The design of the Web Portal with Accessibility Patterns was designed for the Labor Insertion of people with blindness, which responds to the need to have a friendly space on the web, which allows people with different degrees of visual disability (genetic or acquired) the opportunity to enter personal information, as well as their job profile in which the person with blindness can perform. In blind people, it is fundamental to develop positive social relationships, which play an important role within the work environments focused on: Socialize all pertinent information to achieve an efficient labor insertion in accordance with the valuation of their skills and abilities, in such a way that the skills developed for the performance of a specific job are valid and useful. To increase job opportunities, opening the labor market for workers who are blind, to the greatest number and types of tasks within organizations, the capacity and competence of workers is productive. With this, we avoid restricting their opportunities to certain tasks, multiplying their possibilities of personal and social development. To increase the number of people with blindness in the companies, so that not only the possible tasks to develop are increased but also the types of businesses with the presence of workers who present visual disability, a wider and wider range of possible jobs; so that the employers and the society in general assume that effectively the people with disability can perform in the labor and productive sector. Enable support in the world of work for people with visual impairment, so that organizations can build work facilities adapted to the needs of people with disabilities, develop jobs, by establishing selection processes and creating procedures for preparation or training for the performance of a particular job. Figure 1 shows the process followed for the development of the accessible website, taking into account accessibility patterns for blind people and their insertion into the labour market.

902

J. Sánchez Guerrero et al.

Accessible site design with patterns

Identify accessibility barriers

• Apply ichthrones Ihmig Apply recommendations of the W3C

• Recognize traditional designs Recommendations of the W3C

Evaluate the accessible website • Use different accessibility assessment tools Correct errors evaluated

Fig. 1. Accessible web portal methodology

As indicated by [26], referring to patterns as a way of “Redesign of a university web portal applying accessibility standards. Breaking down barriers for visually impaired users”, using web tools to evaluate the validity of HTML, WCAG 1.0 and the accessible web for automatic use [27], in this case the design of a website accessible to blind people. Our research was based on a doctoral thesis work based on the use of Web Accessibility patterns [28], which describes the patterns according to the following structure: • • • • • • •

Context: Description of the context in which the problem occurs. Problem: Brief description of the basic problem. Guidelines: Guidelines WCAG 1.0 relevant in this context. Derivation: Explanation of the solution, including optional justification. Solution: Summary of the solution to the problem. Example: Sample application of the pattern. Most relevant patterns: Other patterns whose use may also be applicable.

Table 1 presents the different accessibility standards with reference to the WCAG 1.0 recommendations.

Table 1. Web Accessibility patterns by Ihmig Simon. Accessibility patterns 1 HTML document 1.1 Meta-Data 1.2 Document Relations 1.3 Order of page elements

WCAG 1.0 13.2 13.9 9.4 (continued)

Design of an Accessible Web Portal for the Labor Insertion of People with Blindness

903

Table 1. (continued) Accessibility patterns 2 Navigation 2.1 Navigation menu 2.2 Chart-based navigation menu 2.3 Skip links 2.4 Keyboard shortcuts 2.5 Breadcrumbs 2.6 Site Map 2.7 Additional navigation 3 Content 3.1 Texts 3.1.1 Headers 3.1.2 Headlines based on Graphics 3.1.3 Reading sequence 3.1.4 Links 3.1.5 Quotes 3.1.6 Abbreviations 3.1.7 Change of language 3.2 Boards 3.3 Forms 3.4 Multimedia 3.4.1 Images and graphics 3.4.2 Image maps 3.4.3 Video 3.4.4 Audio 4 Graphic design 4.1 Color selector 4.2 Scalability 4.3 Keyboard Operability 4.4 Style Switcher 4.5 Hide content 4.6 Replace Image

WCAG 1.0 6.1, 9.4, 10.5, 12.3, 13.1, 13.4, 13.5, 13.6 3.5, 5.1, 5.2, 5.5, 5.6 13.6 9.5 13.4 13.3

3.5, 3.5 3.3, 2.1, 3.7 4.2, 4.1 5.1, 6.3,

14.1 3.6 6.3, 6.4, 13.1 14.1 5.2, 5.5, 5.6 9.4, 10.2, 10.4, 12.3, 12.4

1.1 1.1, 1.5, 9.1 1.4, 8.1 1.4, 8.1 2.2 3.4 9.4

3.1

The content presented by the website is structured because of two forms, one to be filled out by blind users and the other to be used by entrepreneurs for our case study of zone 3 of central Ecuador. In the form of the blind the personal and particular information of each individual with blindness is requested, in addition, of the information of a relative that serves as contact, the level of visual deficiency and the identification of the abilities and skills that each person possesses.

904

J. Sánchez Guerrero et al.

Fig. 2. Accessible Web Portal

Fig. 3. Accessible Web Portal

Design of an Accessible Web Portal for the Labor Insertion of People with Blindness

905

Figure 2 and 3 show the design of the accessible web portal for the labour insertion of people with blindness. In addition, the accessible website has information of interest, in the legal order, labor, attitude, support institutions and other elements that give a consultation space to blind people. As the website has the characteristic of being accessible, it does not imply that its design ceases to be attractive and functional for normal people, on the contrary, it offers a pleasant visualization to the general public.

4 Results Accessibility Assessment Tool The application of technology has become an important contribution in all areas of knowledge, however, is not always designing technology taking into account people with disabilities, which are increasingly with limitations in the accessible management of web pages. There are different tools that allow the automation of the evaluation process of a website, depending on the levels of conformity to the WCAG specifications, in such a way that they are based on practical methods automatically but are not replaced by the manual method. Our study used the Examinator tool, mainly for its ease of use and for being a free automatic tool that is on the web so that it has public access in the URL http:// examinator.ws/. The Accessible portal for the labour insertion of the present project can be found in the URL http://seguimientograduados.uta.edu.ec/discapacidades/ so we proceeded in the tool to generate the evaluation of the site. Figure 4 shows the result of the evaluation of accessibility of the site for labour insertion.

Fig. 4. Accessibility results

906

J. Sánchez Guerrero et al.

In the same, 12 tests have been performed, most of which determine an excellent rating, validating the usability of the site, specifically for the subjects of study and in the area where the fieldwork was carried out. Although it is true, there are negative results in the evaluation, this allows to generate a process of restructuring of the site in conformity with the observations that have been obtained, it is necessary to emphasize that our work and the results are preliminary, reason why it is given opening to future works related to the accessibility and the labor insertion of the people who present blindness, of equal form with the abilities, the qualification and the labor follow-up.

5 Conclusions As it is a project of social application, it seeks to benefit the group of people with visual disabilities, promoting a true social inclusion so that they can publicize their skills and abilities through a massive information medium (Accessible Web Portal) and have greater significance, in the subject of labor insertion. As preliminary results, it was obtained through different automatic evaluation tools, acceptable values in terms of accessible design patterns, which serve to verify the ease that visually impaired users can navigate and upload their data, offering employers the opportunity to learn their skills in the field. It has been possible to carry out the first tests in which blind people have successfully managed to manipulate the accessible web portal. In future works, data and results will be presented in relation to the job profiles found, taking into account the abilities and skills of people who are blind. It should be emphasized that during the use of the accessible platform it was possible to determine that there are people who have a high school education and higher level, which should give priority to their professional training in the workplace. Acknowledgements. We would like to express our gratitude to the visually impaired people of central Ecuador and their families, as well as to the Technical University of Ambato for their support in carrying out our research work.

References 1. Rojas, M.S., Cosme, S.R., Fernández, J.C., Álvarez, M.J., Restrepo, D.D., Correa, S.A., Rodríguez, C.R., Jiménez, L.P., Arismendy, S.R., Valencia, L.C.C., Valencia, S.A.: Caracterización de una población con discapacidad visual (baja visión y ceguera) atendida en dos Instituciones Prestadoras de Salud de Medellín. Med. UPB 34(1), 30–39 (2015) 2. Flores, D.R., Hernández, C.N.: Capacitación Profesional Docente: Realidades de la Educación Inclusiva. Cuaderno de Pedagogía Univ. 13(26), 15–25 (2016) 3. Vásquez Espitia, Uriel. I: Proyecto pedagógico de promoción de los derechos de los niños con discapacidad visual. Nómadas, (Col) (38), 268–270 (2013) 4. Gross, M.: Prácticas inclusivas para la población estudiantil en condición de discapacidad visual en el entorno universitario. Alteridad. Rev. de Educación 9(2), 108–117 (2014)

Design of an Accessible Web Portal for the Labor Insertion of People with Blindness

907

5. Masini, E.F., Giacomini, L., Dalva, R.O.S.A., Carvalho, T., Guimarães, V.: Atención del niño con discapacidad visual y problemas de aprendizaje. Int. J. Dev. Educ. Psychol. 3(1), 333–341 (2006) 6. Navas Martínez, L., Sampascual Maicas, G.: Un análisis exploratorio y predictivo sobre las orientaciones de meta y sobre el contenido de las metas de los estudiantes. Horizontes educacionales 12(1), 23–34 (2008) 7. Polo Sánchez, M.T., López-Justicia, M.D.: Autoconcepto de estudiantes universitarios con discapacidad visual, auditiva y motora. Rev. latinoamericana de psicología 44(2), 87–98 (2012) 8. Escudero, J.C.S.: Discapacidad visual y ceguera en el adulto: revisión de tema. Med. UPB 30(2), 170–180 (2011) 9. Rojkín, C.I.F.: La inserción de personas con discapacidad en el mercado laboral privado. Invenio 18(35), 85–104 (2015) 10. Ordóñez, C.: Breve análisis de la inserción laboral de personas con discapacidad en el Ecuador. Alteridad 6(2), 145–147 (2011) 11. Mercado-García, E., García-Vicente, L.M.: La inserción laboral de las personas con discapacidad: Una salida profesional para trabajadores sociales. Portularia 10(1), 51–60 (2010) 12. Ismail, A., Kuppusamy, K.S.: Accessibility of Indian universities’ homepages: an exploratory study. J. King Saud Univ.-Comput. Inf. Sci. 30(2), 268–278 (2018) 13. Xue, J.A., Wang, J., Liu, S., Ma, X., Wang, H.: Dissecting persistent instability of web service: a joint perspective of server schedule dynamics and path latency. Int. J. Commun Syst 31(7), e3441 (2018) 14. Coughlin, S.S., Stewart, J.L., Young, L., Heboyan, V., De Leo, G.: Health literacy and patient web portals. Int. J. Med. Inf. 113, 43–48 (2018) 15. Cerdan, J., Catalan-Matamoros, D., Berg, S.W.: Online communication in a rehabilitation setting: experiences of patients with chronic conditions using a web portal in Denmark. Patient Educ. Couns. 100(12), 2283–2289 (2017) 16. Lian, X., Zhang, L., Jiang, J.: Goss, W: An approach for optimized feature selection in largescale software product lines. J. Syst. Softw. 137, 636–651 (2018) 17. Sukhoroslov, O.: Building web-based services for practical exercises in parallel and distributed computing. J. Parallel Distrib. Comput. 118(1), 177–188 (2018) 18. Pinho, C., Franco, M., Mendes, L.: Web portals as tools to support information management in higher education institutions: a systematic literature review. Int. J. Inf. Manag. 41, 80–92 (2018) 19. Henríquez-Coronel, P., Bravo-Loor, J., Díaz-Barrera, E., Vélez-Romero, Y.: Open government and citizen participation in the web portals of Ecuador GADM. In: International Conference on Information Theoretic Security, vol. 721, pp. 1146–1155. Springer, Cham, January 2018 20. Lovaglio, P.G., Cesarini, M., Mercorio, F., Mezzanzanica, M.: Skills in demand for ICT and statistical occupations: evidence from web-based job vacancies. Stat. Anal. Data Min. ASA Data Sci. J. 11(2), 78–91 (2018) 21. İşeri, Eİ., Uyar, K., İlhan, Ü.: The accessibility of Cyprus Islands’ higher education institution websites. Procedia Comput. Sci. 120, 967–974 (2017) 22. Oostveen, A.M., Lehtonen, P.: The requirement of accessibility: European automated border control systems for persons with disabilities. Technol. Soc. 52, 60–69 (2018) 23. Giraud, S., Thérouanne, P., Steiner, D.D.: Web accessibility: filtering redundant and irrelevant information improves website usability for blind users. Int. J. Hum Comput Stud. 111, 23–35 (2018)

908

J. Sánchez Guerrero et al.

24. Tekli, J., Issa, Y.B., Chbeir, R.: Evaluating touch-screen vibration modality for blind users to access simple shapes and graphics. Int. J. Hum.-Comput. Stud. 110, 115–133 (2018) 25. Spiegel, M.: Statistical Panic: Cultural Politics and Poetics of the Emotions (2010) 26. Sosa, H., Gaetán, G., Martin, A.: Rediseño de un portal web universitario aplicando patrones de accesibilidad. Derribando barreras para usuarios con discapacidad visual (2016) 27. Akram, M., Sulaiman, R.: Comparative web accessibility evaluation of Saudi government websites for compliance with WCAG 1.0 and WCAG 2.0 using automatic web accessibility tools. J. Theor. Appl. Inf. Technol. 97(10), 2656–2668 (2019) 28. Ihmig, S.: Web Accessibility Patterns. Universität Hamburg, Department Informatik, Hamburg (2007)

Case Studies in Mobile Learning

MassiveLearning: Online Masterclass Course Ndeye Massata Ndiaye(&) and Cheikh Ahmadou Lamine Yakhine Diop(&) Virtual University of Senegal, Dakar, Senegal {ndeyemassata.ndiaye,yakhine.diop}@uvs.edu.sn

Abstract. Challenges related to the overcrowded colleges amphitheaters have been the subject of several conferences on the state of higher education in sub Saharan Africa. With the advent of information and communications technology (ICT), institutions of higher learning are undergoing a rapid shift. The traditional physical in-class learning is rapidly shifting to full-scale distance learning. The shift brings some challenges in teaching and learning methods as various media are used to suit different learning patterns. This article presents a distance learning module, (massive Learning), whose main objective is to showcase an offline distance-learning lecture model through the use of video and explanatory slides. Keywords: Masterclass


Mobile learning
ODL

1 Introduction The increase in student enrolment in public universities is a challenge for higher education authorities in Senegal. Each year, the number of high school graduates increases, while the welcoming capacity of the public and private universities remains largely steady. Authorities are thus facing the challenge of overcrowded amphitheaters which impact results negatively. Open and distance learning remains the best way to achieve a positive rating on quality indicators. Initiatives previously used in northern countries, such as virtual universities based on distance learning platforms, are emerging. A distance-learning uses information and communication technologies (ICT) for the benefit of education, and more specifically in pedagogy. The students, called learners, are supervised by a team of tutors who are under the supervision of a teacher, who is the architect of the subject taught. All courses resources and learning activities are accessible via the online platform. The main challenge for distance education actors remains the isolation of the learner. This is the new age of shared course videos on the Internet. Based on the same principle as computer tutorials, teachers produce videos of their courses. While distance learning platforms are a viable alternative to overcrowded amphitheaters, they remain a rather expensive solution for most sub Saharan countries. Access to these distance-learning platforms requires an internet connection, which is a luxury for many in sub saharan countries [1]. Digital media require a stable bandwidth. Also, the quality of the Internet connection must necessarily be sufficient in order for the learner to have a positive learning experience. © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 911–918, 2021. https://doi.org/10.1007/978-3-030-49932-7_84

912

N. M. Ndiaye and C. A. L. Y. Diop

Designing an offline module, adapted to mobile terminals, is a necessity in our sub saharan’s environment [2]: amphitheaters can no longer accommodate the large number of students. Our MassiveLearning module uses media and learning strategy in order to deliver a course. It consists of a video of the teacher giving the lecture, the screen is also populated with explanatory slides. A menu allows the student to browse through video sequences attached to a slide presentation. MassiveLearning uses a video compression algorithm that is user friendly and transferable through mobile devices. This article, present a distance learning methodology developed to help with the over crowdedness of amphitheaters using ICT.

2 College Level Lecture A lecture is the most common teaching method used in higher education. For several decades, it has become a means of transmitting knowledge in various fields of study. Disseminating new information to a large audience is the objective of the lecturer. During the presentation, the teacher provides an overview of a topic and thus arouses interest towards it. In practice, the teacher holds the floor throughout the session while the students’ listen and take note. Some professors allow exchanges with students. In addition to the oral presentation, a book or course document (handout) is made available to learners. The lecture system has some advantages in the education system. For a large group, it is the most appropriate way to transmit knowledge. From a financial standpoint, it is still less expensive to group students in an amphitheater than to divide them into smaller classroom. Administratively managing a group of students at the same level and in the same stream is easier. However, it is a method that is often criticized. The fact that students are mainly passive learners in this system makes it less interesting. To this we can add the following drawbacks: – the speed at which the teacher speaks can make note-taking difficult for students. – the lack of eloquence: the audience will find it difficult to follow and understand. In addition, lectures are different from other educational activities by the fact that they are not compulsory, which may increase the drop-out rate. The topics covered are often theoretical because the practical aspects are reserved for tutorials and practical classroom sessions. Several authors examine the issue of passivity in lectures. In Rabut article entitled “Do we still need to give lectures” [3], the answer is YES. But it proposes two methods to make students active. The first method consists in giving the course material to the students before the session. And during the course, learners are divided into small groups and work on the questions they have asked themselves prior to coming to class; the teacher goes through the groups to answer their question. In the second method, the course is transformed into a problem that highlights the concepts of the course. The first Rabut method resembles a “flipped classroom” that has been popular in recent years with the advent of the ICT for education. Flipped, because the educational activities reserved for learning at home are now done in the classroom. Indeed, the

MassiveLearning: Online Masterclass Course

913

course resources are put online and learners get familiar with them before attending the session. The latter becomes a “giant workshop” [4] where the teacher, who is a tutor by virtue of his role, answers questions and “unblocks” the students. Based on these findings, we looked at the development of a module, or learning tool, that would bring together the video of the lecture and the resources (used in learning as in the case of the reverse class). But, the question is whether MassiveLearning could be a solution to over crowdedness? The vast numbers of students in Senegalese universities has become a recurrent problem. The Cheikh Anta Diop University in Dakar has exceeded its capacity, even though students continue to be oriented there. Regional universities were thought at as a solution to the problem, but the development of these universities, especially in terms of infrastructure, did not follow. In this context, could we talk about the quality of teaching? The increase in the number of students creates a lot of problems in the learning process. If university budgets do not increase proportionately to the number of new high school graduates, the consequences will be felt throughout the system: – Lack of infrastructure: learning conditions, such as laboratories, TD/TP rooms, and amphitheaters. – Decrease in success rates – Shortage of teacher-researchers leads to a low supervision rate; – Difficult management of social works. In response to this situation, other solutions had to be put in place, in addition to the construction of new universities. ICTs are bringing a new wave of reforms. It is a question of setting up distance training; making educational resources accessible by installing WIFI within the university campus. It is in this same vein that we have designed the MassiveLearning module that will allow students to follow the course at any time. Distance learning systems make learners more autonomous, and considerably develop their cognitive skills.

3 The Use of ICT in High Learning The introduction of information and communication technologies (ICT) in education is perceived as an innovation in the education system, moving from a traditional training system (board, chalk; teacher, learner) to a fully mediatized training system (distance learning). Technological innovation is essential in the education system with the advent of ICT, we are now talking about ICT for Education. This reflection is resumed by Charlier [5] who said: “the choice to offer a training system integrating cyberspace is often linked to environmental pressure: changes in the resources offered by technologies and concomitant changes in training needs”. This pressure will lead to technological and pedagogical innovation. ICT for education will represent an important potential for pedagogical innovation with new practices for teachers, learners, in short for the entire education system. They have several advantages: – A new way of communicating: through the media (images, photos, diagrams, videos, animations, simulations);

914

N. M. Ndiaye and C. A. L. Y. Diop

– A new way of processing information: copying, pasting, formatting, searching for information; – A new way of teaching: coaching, explaining and demonstrating differently; – A new way of learning: knowledge appropriation, individualized learning. The integration of ICT into amphitheaters begins with the use of devices to facilitate communication with the public: retro projectors and latest models video projectors. Using these tools require prior preparation to be done by the teacher, such as preparing the presentation in a digital format. For teachers who are not familiar with computers, the preparation can be complex and almost impossible. In addition to the presentation of the digitized course, several technological tools are part of the learning material. In addition to the digitized lecture course, a paper or handout course material is designed for students.

4 MassiveLearning It is a distance-learning method that can be used as a standalone, as a complement to a traditional lecture, or inserted into a distance learning platform to replace the lecture given in classroom. The originality of the tool lies in its composition and size: – MassiveLearning is essentially composed of two media: a video of the teacher synchronized with slides. Each slide is explained in a sequence of the video. The latter is also accessible via a menu containing all the sections covered in the video as links. – MassiveLearning uses a data compression algorithm, including video, that allows it to be easily transferred over a mobile network. All components (video, slides, …) are embedded in a single module that can be downloaded and adapted to mobile devices and smartphones. Why a Video Synchronized with Images? The role of documents and the media in teaching/learning situations is increasingly being discussed in the science of education with the emergence of information and communication technologies (ICT). Media tools are an integral part of the process of knowledge transmission. Daniel Peraya in his article Visualizing the Image [6], proposes a table by Gerard and Rogiers showing the proportion of illustrations in a text: a mathematics textbook in secondary school should contain a minimum of illustrations equal to 30%. Visual representations are an important part of the learning support system. From kindergarten to higher education, textbooks are developed using visual materials, helping the teacher to set up their learning system. In the fields taught, whatever the type of training (face-to-face or distance learning), the trainer is obliged to use the media to transmit knowledge to the trainees. “A good diagram always presents a harmony between these three essential qualities: – Technical qualities to ensure optimal readability; – Accessibility qualities and explanation to meet cognitive requirements; – Aesthetic qualities to satisfy emotionality and aesthetic sensation” [6].

MassiveLearning: Online Masterclass Course

915

Visual representations can be classified into two categories: static and dynamic representations. In the static category, we can mention images, diagrams, photos, graphs, tables and verbal languages. The last category, dynamic, has emerged with the development of digital media. These are animations, videos and simulations. Our module, composed of videos and animated images, is a “cognitive artifact” [7] whose main objective is to mediate between the student and the teacher, but also between the student and the course, as shown in Poisson’s pedagogical triangle (Fig. 1) [8]. The pyramid shows 4 sides: – – – –

The The The The

self-training side via the NTICEs; pedagogical triangle; mediation side of training via the NTICEs; mediatization of knowledge.

The four sides show the whole process of self-learning: from the mediatization of knowledge to the transmission of knowledge between the student and the teacher. Although a video is a strong means of communication, it is not enough to address various subjects. Combined with a set of explanatory texts, the duo allows students to limit note taking thanks to the slides, but also to break the distance between the teacher and the students.

Fig. 1. Multimedia engineering pyramid

916

N. M. Ndiaye and C. A. L. Y. Diop

MassiveLearning demonstrates the intersection of cultural industry tools (use of videos and animated images) with learning scenario. A. Description of the tool and the model The course page is built on a model based on the main elements that characterize a course in higher education. In addition, there are all the actions that a student could carry out during his or her learning: consult the bibliography, web pages (useful links) or contact the teacher. On this last point, it is rather within the framework of our model that we have the contact of the faculty. The student wishing to ask questions during the learning process has the possibility to contact the teacher but also all the tutors who supervise the course: the question will be sent to all the course supervisors. The advantage is the reduction of response times: the first tutor who receives the question answers. MassiveLearning is composed of three main parts (Fig. 2): – Header: it contains the institution’s logo and the course title. – Resources: located in the middle of the page, they consist of the course video, slides and a menu to access the different sequences of the video. – The footer: it includes all the necessary information about the course: the author, the bibliography, the useful links and the contact of the course teaching team. The bibliography and links have the advantage of limiting the time spent on research by students. With the emergence of digital libraries, all references will be accessible via the bibliography section. Similarly, related articles or educational activities related to the topics covered in the course may be included in the useful links.

Fig. 2. MassiveLearning

MassiveLearning: Online Masterclass Course

917

Navigation buttons, synchronized with the video, are associated with the slides and allow the student to navigate between the slides and the video sequences. An additional window containing only the video and slides in full screen mode is inserted on the module page. B. Technical specifications 1) HTML5/Javascript/CSS3 The tool is developed in HTML5, Javascript and CSS3. It is an essential trio for developing offline web applications. One of the new features of HTML5 is the multimedia function with video, audio and canvas tags; even if all these features already existed with Flash. Synchronization Video-Slides-Menu: Using Javascript functions, the slides of the course are synchronized with the video. Each video sequence is associated with its explanatory slide, and in a spontaneous way. Similarly, a menu composed of links is synchronized with the video-slide pair. The learner has the possibility to navigate the menu to follow the parts of the course (video slide) that interest him/her. 2) Offline mode The module operates in offline mode. The learner does not need an internet connection to take the course. In sub-Saharan African countries, Internet access is expensive for some population groups. Even if some university spaces have free wifi, connection is a luxury for residences. It is to overcome these access problems that we have developed an “offline” module. The folder containing all the files of the module is downloadable and the video and slides can be played back in offline mode. 3) A responsive design MassiveLearning is responsive, it automatically adapts to the size of the multimedia support. Instead of developing a single interface adapted to a standard computer screen or an interface for each type of support, responsive design was chosen for the module. It can be adapted to all formats: computer screens, tablets and smartphones.

5 Conclusion While a lecture remains the most widely used teaching method in universities, it has its limitations in terms of increasing number of students which impacts. The quality of the lecture in an amphitheater. This leads to lower success rates. Distance education remains the best solution to overcome the problems of over crowdedness. With the advent of ICT, the new generation of students is familiar with use of digital tools. It then makes sense to use ICT to support teaching learning process. MassiveLearning, with its responsive design and compression algorithm, allows students to follow their course at a distance, on any multimedia medium. It can be a complement to the lecture, or replace it when it is used in a completely digitalized training format.

918

N. M. Ndiaye and C. A. L. Y. Diop

Prospects for the Future Our objective is to design a platform of online lectures. It will be a different platform from traditional learning management system (LMS) because it is composed of a simple interface that fits the university’s organizational chart. Each course will provide access to a MassiveLearning module of the course in question. This platform will represent an educational package with all the books and software useful for the learner. Always in the need to make it easily accessible, each course package must be able to be used in “offline” mode.

References 1. Dakouré, É.: TIC et développement en Afrique: approche critique d’initiatives et enjeux. Revue française des sciences de l’information et de la Commun. 4 (2014). http://journals. openedition.org/rfsic/939, https://doi.org/10.4000/rfsic.939 2. Béché, E.: Open and distance learning in French-speaking Sub-Saharan Africa: a literature review. Int. Rev. Res. Open Distrib. Learn. 19(3) (2018). https://doi.org/10.19173/irrodl. v19i3.3265 3. Rabut, C.: Faut-il encore faire des cours magistraux. Université de Toulouse (INSA, IREM, IMT) 4. www.classeinversee.com 5. Charlier, B., Bonamy, J, Saunders, M.: Apprivoiser l’innovation. In: Technologie et innovation en pédagogie. Dispositifs innovants de formation pour l’enseignement supérieur. De Boeck, Bruxelles (2002) 6. Peraya, D.: Visualiser l’image, Document RTF (1999) 7. Norma, D.A.: Cognitive artefacts. In: Carroll, J.M. (ed.) Designing Interaction: Psychology at the Human-Computer Interaction. Cambridge University Press, Cambridge (1991) 8. Poisson, D.: Modélisation des processus de médiation – médiatisation: vers une biodiversité pédagogique, CUEEP-USTL-TRIGONE

Touch Gesture Performance of Kindergarten Children in E-learning Applications: A Case Study in Sri Lanka Uthpala Samarakoon1(&) and Hakim Usoof2(&) 1

2

University of Colombo School of Computing, Colombo, Sri Lanka [email protected] Department of Statistics & Computer Science, University of Peradeniya, Kandy, Sri Lanka [email protected]

Abstract. With the advancement of technology, e-learning has become a major component of education. Hence, parents now show more interest in games and educational applications based on touch sensitive devices. Today, even very small children play with touch screen devices. Every day, many tablet-based educational apps are introduced to app stores with the objective of improving knowledge of small children and providing them with some sort of edutainment. But recent research shows that most of these apps are not successful due to usability issues with small children. The reason for this is the app developers’ lack of understanding of the abilities and limitations of children that vary with their age as well as their exposure to technology. Hence, most of the children find it difficult to use the apps due to their inability to perform the gestures or understand the concepts. Therefore, most of the children targeted apps are not as successful as anticipated. There are existing studies on gesture performance, but many of them focus on children in developed countries. The gesture performance of a small child may be affected by factors such as exposure to the latest technologies, socio-economic background of the child and native language. Hence, this research is focused on analyzing the gesture performance of children of ages 3 to 5 (Kindergarten level) in Sri Lanka. This paper also makes recommendations on criteria that will ensure better gesture performance by making some changes to interface design for children targeted applications. Keywords: Gesture performance learning


Kindergarten children
Tablet-mediated

1 Introduction According to Turril (2014), in American families 77% of children play entertaining games on tablets while 57% play educational games on tablets. As per Michael Robb (2017), the amount of screen media has not changed much from 2011 to 2017, but how children use media has changed significantly. As per the findings, children aged 8 and under spend an average of about two-and-a-quarter hours a day with screen media, almost the same amount they devoted to screens in 2011. But much more of that time is © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 919–929, 2021. https://doi.org/10.1007/978-3-030-49932-7_85

920

U. Samarakoon and H. Usoof

spent on mobile devices; the average amount of time spent with mobile devices each day has tripled, going from 5 min a day in 2011 to 15 min a day in 2013 to 48 min a day in 2017. As stated, there has been a five-fold increase in tablet usage among American families having children aged 3 to 8 over the years 2011 to 2013. Smartphone usage among children also increased to 63% in 2013 from 41% in 2011 (Study et al. 2013). As predicted by eMarketer (2018), the number of tablet users in the world will reach 171 million by 2018. These statistics indicate the popularity of touch screen devices among people across the world. Sri Lankan families are also now moving towards touch related technologies such as smart phones and tablets. Census department reports (Satharasinghe et al. 2007) show that the number of households that own a laptop or a desktop has fallen to 22.5%. This indicates the rise in tablet and smart phone usage within the country. According to Sri Lanka Telecom (Sri Lanka Telecom, Mobile and Broadband, Statistics and Analyses 2017), mobile penetration has risen from 96% in 2012 to 126% in 2017 in Sri Lanka. According to government data, smart phone sales have been on the rise for the country’s leading consumer durable retailers such as Abans PLC and Singer (Sri Lanka) lately. Singer has reported that tablet/media pad sales have increased by 105% (Wettasinghe 2017). Evidently, tablet application usage is increasing among Sri Lankan families. As even small children are attracted to these touch devices, they can access tablet-based educational applications. The main purpose of any tablet mediated application is to provide education to users. When the user is a child, the interface and design of application are very important. As per existing research on usability of e-learning applications by small children either through mobile or tablet, researchers have identified many usability issues. In tablet-based applications the correct use of gestures is important for effective interaction. A gesture is any physical contact movement that a digital system can sense and respond to without the aid of a traditional pointing device such as a mouse or stylus (Saffer 2009). Gesture performance of small children may also depend on fine motor skills, cognitive ability and familiarity with technology. Hiniker et al. (2015) point out that relatively little research work has been done to evaluate pre-schoolers’ and toddlers’ ability to perform touch screen gestures compared to school-age children. Rust et al. (2014) concur with this observation. According to Hourcade (2007), touch screens are always well received by young children, perhaps because they provide a more tangible means of selecting options on the screen, provided the icons are sized appropriately. He recognized this while working on the design of the “International Children’s Digital Library” at the University of Maryland, while collaborating with very young design partners who always preferred to work with touch screens in the laboratory rather than desktop computers, which required them to use a mouse. Moreover, the lack of standardized gestures for these young users makes the development of well-designed multi-touch interactions even more crucial (Ingram et al. 2012). Until recently, there have not been many research efforts that addressed multi-touch interaction with kindergarten children. The reason for this is possibly that age has been considered as a limiting factor for experimental studies because young children do not have the verbal and cognitive skills to express their preferences, have short attention spans and are easily distracted (Egloff 2004; Kremer 2012). However, according to

Touch Gesture Performance of Kindergarten Children

921

Piaget (1973), children from 2 years onwards can interpret images and symbols, and play with imaginary objects, which mean they could participate in research on multitouch technology. Further, most current commercial applications do not take full advantage of multi-touch technology and limit gestures to tap and drag in most cases (Nacher et al. 2015). Therefore, analyzing touch gesture performance of children is important for children targeted software development. Only a limited amount of research devoted to analyzing touch gesture performance of children exists. Even these researches were conducted in developed countries like UK and Spain and only one study was done in a developing country, namely Malaysia. As the aim of this research is to identify touch gesture performance of kindergarten children in Sri Lanka, it will prove very useful to developers working on design of effective interactive apps for children. Table 1 lists a summary of the studies conducted by several researchers in which the primary focus was on analyzing the touch gesture performance of children aged 3 to 5 (Aziz 2013; Aziz et al. 2013; Aziz et al. 2014; Nacher et al. 2014). Table 1. Summary of Experiments in which the Primary focus is on Touch Gesture Performance of Children Aziz (2013) Country Gesture/Age Group Tap Drag Flick Pinch Spread Long Press Drag & Drop Double Tap

United Kingdom Age Age 4–5 3–4 (%) (%) 100 100 100 100 100 100 0 100 0 100 – – 100 100 – –

Aziz et al. (2013) United Kingdom Age Age 4–5 3–4 (%) (%) 100 100 100 100 73 100 82 100 36 100 – – 100 100 – –

Aziz et al. (2014) Malaysia Age 3–4 (%) 100 100 100 71 64 – 100 –

Age 4–5 (%) 100 100 100 100 100 – 100 –

Nacher et al. (2014) Spain Age 3–4 (%) – – – – – 62 – 50

Age 4–5 (%) – – – – – – – –

According to these results, Children aged 3–4 show poor performance in gestures such as pinch, spread, flick and rotate. But they can perform tap, drag, and drag & drop gestures successfully. On the other hand, children aged 4–5 can perform almost all the gestures. Gestures like double tap and long press were not tested on children aged 4–5 but results show low success rates for children aged 3–4. Four of the five experiments were conducted in developed countries like UK and Spain while only one experiment was conducted in Malaysia. This highlights the lack of research in developing countries with respect to children and touch gestures.

922

U. Samarakoon and H. Usoof

Gesture performance of children may also be influenced by cultural factors, socioeconomic background and amount of exposure to technology. But there is no research evidence to verify these matters. Therefore, there is a pressing need for more research to discover whether there is any difference between the gesture performances of children in developing and developed countries. Existing research indicates that fine motor skills of children are influenced by their socio-economic status and behavior (Venetsanou and Kambas 2010). This research assesses the gesture performance of children aged 3–5 in Sri Lanka and makes recommendations for E-learning application development targeting them.

2 Materials and Methods Experimental research methodology was chosen for this study. This is a powerful technique for evaluating the cause-and-effect behavior of relationships. The purpose of this research was to identify the touch gestures that can be performed by pre-school (ages 3 to 5) children in Sri Lanka and pinpointing any marked gesture performance differences between them and children in developed countries. A literature review was conducted to identify the common gestures used in existing children’s applications. Then, the most common gestures mentioned in the literature were selected for use in the experimental applications. Gestures not included in the applications such as pinch, spread, and long press were experimented with by performing standard tablet operations like photo album and moving the icons. Animal Math, Preschool Math games for Kids and Math for Children aged 3–5 were chosen as the testing apps. The selected applications were then assessed by getting kindergarten children to use them. 2.1

Participants

35 Sri Lankan pre-school children aged 3 to 5 were selected for the experiment. Fifteen were of age 3–4 and the rest were of age 4–5. 12 were males and 23 were females. According to the code of ethics for research with children, a printed consent form was given to the teachers and the children’s guardians to get their permission. 2.2

Materials

Initially, the ten most common gestures were identified based on the Touch gesture reference guide (Kremer 2012). The selected gestures were Tap, Double tap, Drag, Flick, Pinch, Spread, Long press, Point & Click, Drag & Drop and Rotate. Fifty free educational Math applications with a content rating of 4.0 or above were downloaded from the Google Play store. Each app was then analyzed to verify the availability of the ten gestures. Table 2 shows the gesture usage as a percentage for each of the selected apps.

Touch Gesture Performance of Kindergarten Children

923

Table 2. Gesture usage of selected Apps Gestures Tap (T) Drag (D) Flick (F) Pinch (P) Spread (S) Long Press (LP) Point & Click (P&C) Drag & Drop (D&D) Double Tap (DT) Rotate (R)

Percentage of apps using the gesture 100% 30% 12.5% 0% 0% 0% 2.5% 42.5% 2.5% 0%

According to the findings, the most commonly used gesture is tap, followed by drag and drop, drag, and flick. Availability of double tap and point & click were very low, and gestures such as pinch, spread, long press and rotate were not used at all in any of the selected tablet-based applications. This research used three apps for the experiment. Many apps with content rating of 4.0 and above were selected initially and then shortlisted based on maximum gesture availability. Additionally, factors such as attractiveness were also considered for final selection. There were very few apps or none in the selected list to check for gestures such as pinch, spread, long press, double tap and rotate. Hence, the researchers used the operational gestures of the tablet’s built-in apps and desktop to check the performance of the rare or unavailable gestures. For example, children were asked to perform the pinch and spread gestures using the photo album and the long press gesture with movement of icons. As there were not many rotation gestures included in children’s apps or general tablet gestures, this research excluded same. Table 3 shows the apps selected for this study. The selection of apps was crucial since most of the free apps did not contain all the gestures. Table 3 lists the gestures used in each app from the nine selected gestures, which were identified after analyzing 50 math related applications. Table 3 also shows the Google App store’s recommended age rating of each app. Apps were selected based on the gestures available in each app and the app’s rating. Attractiveness of each app for children was also considered. Table 3. Selected apps for testing gesture availability Application Age Google name recommended app store rating

Gestures

Animal Math MathAge 3–5 Preschool math games

Under 5 years 4.3

x

Age 3–5

4

x

Age 3–5

4.1

x

Tap Double Drag Flick Pinch Spread Long Point Drag Tap Press & Click & Drop x x

x x x

924

2.3

U. Samarakoon and H. Usoof

Procedure

The main experiment investigated the ability of pre-school children (aged 3–5) to use the nine gestures. The study provided a comfortable environment for the children to use the tablet without other distractions. Samsung Tab 3 was used for the experiment with Android 4.3 Jelly Bean operating system. A video camera was used to record the gestures made by each child. Other information recorded included the child’s age, gestures they could use, finger movement while using each gesture, and problems faced by the children while using the gestures with apps. Each child was asked to perform the gestures on its own and allowed 3 attempts. The researcher provided instructions in the child’s native language, when English language was a barrier.

3 Results Results section explains the gestures that could be performed by children in Sri Lanka. Gestures that can be Performed by Sri Lankan Children The experiment used nine common gestures: Tap, Double tap, Drag, Flick, Pinch, Spread, Long press, Point & click, and Drag & drop. The video recorded during the experiment with 15 children aged 3–4 and 20 children aged 4–5 was analyzed. Figure 1 shows the results of the observations on gesture performance. The children who completed the task successfully within the allowed number of attempts were counted as passing the gesture performance while the others were counted as having failed the test.

120 100

100 87 94

95 89 80

80 60 40

55 49 40

95 80 89

75 60 70 45 40 43

40 33 37

60 47 54 15 7 11

20 0 Tap (T) Drag (D) Flick (F) Pinch (P) Age 3-4

Spread (S) Age 4-5

Long Point & Drag & Double press Click Drop Tap (DT) (LP) (P&C) (D&D) Overall

Fig. 1. Gesture performance of Children Aged 3 to 5

Touch Gesture Performance of Kindergarten Children

925

As per the results of Fig. 1, overall 94% of children succeeded in the Tap gesture while 6% completed it after a few attempts. Some children did not correctly position the tap at first or they just pressed instead of tapping. In addition, there was a 13% performance gap between the two age groups. Accordingly, children in the 3–4 age group showed lower performance (87%) than children in the 4–5 age group. The reason may be reduced motor control or inability to comprehend the gesture due to younger age. Drag gesture was tested based on the way children drag a figure on a given number template. Overall, 49% of children succeeded in this gesture while 51% failed to perform the drag gesture at all. Age-wise only 40% of age 3–4 children completed the task within the allowed attempts while it was less than 15% in the case of age 4–5 children. It was observed that many of them could not correctly identify the starting and ending points of the drag action. There was also a problem with some apps that did not show the path to be dragged or drawn, so the children could not figure out how to perform the gesture. 89% of children were able to perform the Flick gesture easily whereas11% failed to do so. As happened previously, age 3–4 children were not as successful in this gesture as age 4–5 children. The results show that overall only 43% of children succeeded in the Pinch gesture within the given attempts. 57% of children failed to perform the gesture. It was observed that children lacked the fine motor skills required to perform the pinch gesture. For this gesture too, the success rate of the 4–5 group was higher (by 5%) than the 3–4 age group. As per the results, children found it easier to perform the Spread gesture than the Pinch gesture. The overall success rate for Spread gesture was 70% while 30% failed. Long Press proved to be a difficult gesture for pre-school children. Overall, only 37% of children were successful. The fail rate for this gesture was around 63%. Only about half of all children were able to perform the Drag & drop gesture straight away. The success rate was 54%, with46% being unable to perform the gesture. Point and Click is an alternative gesture for Drag and Drop and the results show greater success rate for the former (89%) than the latter (54%). Only 11% of children totally failed in Point and Click, which is lower than the Drag and Drop failure rate (46%). Overall success rate was 40% for Double Tap gesture in the experiment. Majority (66%) failed to perform the gesture. Main observation was that most of the children failed to tap in quick succession so there was too long a pause between the two taps. This may be due to lack of fine motor control as age 4–5 children showed a somewhat higher success rate than those aged 3–4. When comparing the success percentages shown in Fig. 1, a notable observation is that age 4–5 children have a higher success rate for all the gestures than the age 3–4 group.

926

U. Samarakoon and H. Usoof

4 Discussion As the results show (Fig. 1), app designers can use Tap, Flick, Spread, and Point & Click/Drag & Drop gestures when designing for children ages 3 to 5 in the Sri Lankan context. It does appear that it would be better to use the Point & Click gesture rather than the Drag & Drop gesture in certain situations such as coloring, since the former has a higher success rate. This also suggests that children aged 3–5 struggle to perform gestures like Drag, Pinch, Long Press and Double Tap. Therefore, it is recommended to avoid such gestures in pre-school targeted applications. These results indicate that the selection of gestures needs to be age appropriate, especially when the design is for small children. One major finding of this research is that nine common gestures, as indicated in Table 4, have been clearly identified as either suitable or not suitable for use in apps aimed at children of ages 3 to 5 in the Sri Lankan context. Table 4. Gestures recommended and not recommended for children aged 3–5 Gestures recommended for Kindergarten applications Tap Flick Point & click Spread

Gestures not recommended for Kindergarten children Drag & drop Drag Pinch Long press Double tap

The results shown in the above table have been sorted in descending order in both columns. In the left column, the easiest gestures for children to perform are at the top, whereas in the right column the most difficult gestures are at the top. As per the findings of the study, the researchers suggest that designers and developers give priority to gestures that can be easily performed by children. The least effective gestures should be used only in special situations and with proper visual instructions. Furthermore, when comparing the results of this research (Fig. 1) with research done in developed countries (Table 1), it is clear there is a difference in gesture performance in all the categories. As per the results, gesture performance of Sri Lankan children was considerably lower than that in other countries. This could be due to the lack of access to technology and socio-economic factors. Moreover, there was no research that covered the point & click gesture in developed countries, but Sri Lankan children performed it with a higher success rate in comparison to the drag & drop gesture. It is also evident that there are some additional guidelines to follow to improve gesture usage by small children in apps, such as the use of natural gestures wherever possible. Natural gestures are the gestures that we use in day today life when performing activities. For example, when we move objects on a screen, by default we use drag and drop.

Touch Gesture Performance of Kindergarten Children

927

Further, the same gesture must be used throughout the application to perform similar tasks. Hence, it is better to stick to a unique and consistent gesture throughout the application. Suitable hints must be provided about the gestures. Therefore, it is recommended that application developers include a short demonstration of gesture usage for each activity. 4.1

Best Practices, Suggestions and Possibilities When Designing for Children

Additionally, some observations made during the experiment may prove useful to improve the gesture performance of small children. One was that children of age three to five always try to perform gestures according to the way they move objects naturally. They try to map their real-life experiences to their gesture performance. When they want to connect points to complete a picture, they use the drag and drop gesture, which is closer to the natural way of connecting things rather than the point and click gesture, which was implemented in the application. The next observation was that as the experiment proceeded, most of the small children tried to use the same gesture for similar situations. Therefore, when different applications use different gestures for similar actions, the children become confused. Another finding was that most of the children cannot guess what kind of gesture they must perform in some applications. Only a few applications do a demonstration for them. The other apps just explain the activity, but no clues are given regarding the gesture to be used. Language barrier is another problem faced by small children. Most of them could not understand the instructions given in English since their native language is ‘Sinhala’. Therefore, the researchers suggest that designers consider this point when designing for children. It was clear that most of the failures on drag and drop gesture was due to lack of cognitive skills at this age. Children who failed in the gesture would after starting the drag activity attempt to figure out the destination to drop and then release the finger after dragging part way.

5 Conclusion The primary focus of this study was to identify touch gesture performance of Sri Lankan children aged 3–5. There is a lack of research studies whose primary focus is on identifying touch gesture performance of pre-school level children. Summary percentages were calculated based on the results of the experiments. As per results the researchers can recommend gestures such as tap, flick, point & click and spread to the designers of children targeted applications. Gestures such as spread, pinch, drag, drag & drop, double tap and press should be avoided as much as possible and used only in special situations with proper visual guides. The research shows overall that the designers need to consider the age factor when designing for children. The interface design should also be considered carefully for this age group.

928

U. Samarakoon and H. Usoof

Acknowledgements. The researchers wish to express their gratitude to all the teachers, supporting staff and children who were involved in the study. We express our special thanks and appreciation to the parents of the children for giving their consent and allowing their children to participate in the study. The cooperation of all these people has helped greatly to make this study a success.

References Turril, D.: Shifts in Viewing The Cross-Platform Report, September 2014 eMarketer: US Digital Users: eMarketer’s Estimates for 2018. https://www.emarketer.com/ Report/US-Digital-Users-eMarketers-Estimates-2018/2002215 Hiniker, A., Sobel, K., Hong, S.R., Suh, H., Irish, I., Kim, D., Kientz, J.A.: Touchscreen prompts for preschoolers: designing developmentally appropriate techniques for teaching young children to perform gestures. In: Proceedings of the 14th International Conference on Interaction Design and Children, pp. 109–118 (2015). https://doi.org/10.1145/2771839. 2771851 Hourcade, J.P.: Interaction design and children. Found. Trends® Hum.-Comput. Interact. 1(4), 277–392 (2007). https://doi.org/10.1561/1100000006 Ingram, A., Wang, X., Ribarsky, W.: Towards the establishment of a framework for intuitive multi-touch interaction design, p. 66 (2012). https://doi.org/10.1145/2254556.2254571 Robb, M.: Kids’ Screen Time Shifts Dramatically Toward Phones and Tablets (2017). https:// www.commonsensemedia.org/blog/kids-screen-time-shifts-dramatically-toward-phones-andtablets Rust, K., Malu, M., Anthony, L., Findlater, L.: Understanding childdefined gestures and children’s mental models for touchscreen tabletop interaction, pp. 201–204 (2014). https:// doi.org/10.1145/2593968.2610452 Saffer, D.: Designing Gestural Interfaces: Touchscreens and Interactive Devices. O’Reilly Media Inc, Newton (2009) Satharasinghe, A.: Census Department Measures ICT Penetration into Households, pp. 1–6 (2007) Sri Lanka - Telecoms, Mobile and Broadband - Statistics and Analyses (2017). https://www. prnewswire.com/news-releases/sri-lanka—telecoms-mobile-and-broadband—statistics-andanalyses-300552420.html Study, A.C.S.M.R., Maldonado, R., Valverde, O., Blommaert, A., Fournié-Zaluski, M.-C, Roques, B.P.: Zero to eight children’s media use in America 2013. Ann. New York Acad. Sci. 713 (2013). https://doi.org/10.1111/j.1749-6632.1994.tb44091.x Wettasinghe, C.: Sri Lanka computer ownership falls as usage of smartphones, tabs gains (2017). http://www.dailymirror.lk/article/Sri-Lanka-computer-ownership-falls-as-usage-ofsmartphones-tabs-gains–126727.html Egloff, T.H.: Edutainment: a case study of interactive cd-rom playsets. Comput. Entertainment 2(1), 13 (2004) Kremer, K.E.: Conducting game user experience research with preschoolers. In: Workshop on Games User Research: Practice, Methods, and Applications (Collocated to CHI’12) (2012) Piaget, J.: The Child and Reality: Problems of Genetic Psychology. (Trans. Arnold Rosin). Grossman (1973) Nacher, V., Jaen, J., Navarro, E., Catala, A., González, P.: Multi-touch gestures for prekindergarten children. Int. J. Hum.-Comput. Stud. 73, 37–51 (2015)

Touch Gesture Performance of Kindergarten Children

929

Aziz, N.A.A., Batmaz, F., Stone, R., Chung, P.W.H.: Selection of touch gestures for children’s applications. In: Science and Information Conference (SAI 2013), pp. 721-726. IEEE, October 2013 Aziz, N.A.A., Mat, N.S., Batmaz, F., Stone, R., Paul, C.: Selection of touch gestures for children’s applications: repeated experiment to increase reliability. Int. J. Adv. Comput. Sci. Appl. 5(4), 97–102 (2014) Nacher, V., Jaen, J., Catala, A., Navarro, E., Gonzalez, P.: Improving pre-kindergarten touch performance. In: Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces, pp. 163–166. ACM, November 2014 Aziz, N.A.A.: Children’s interaction with tablet applications: gestures and interface design. Children 2(3), 447–450 (2013) Venetsanou, F., Kambas, A.: Environmental factors affecting preschoolers’ motor development. Early Child. Educ. J. 37(4), 319–327 (2010)

Learning Diaries—A Valuable Companion of Mobile Learning for Higher Education in Software Engineering Sigrid Schefer-Wenzl(&) and Igor Miladinovic University of Applied Sciences Campus Vienna, Vienna, Austria {sigrid.schefer-wenzl, igor.miladinovic}@fh-campuswien.ac.at

Abstract. Over the past two decades, mobile learning has become a popular part of educational technology in higher education. Mobile learning has the potential to individualize, personalize, and optimize learning processes. It enables students to learn, collaborate, and share ideas among each other anywhere and anytime with the aid of mobile devices. This requires a high degree of students’ autonomy and self-organization and may lead to a significant fragmentation of the learning process. Learning diaries can serve as a didactic tool to support, structure, and consolidate students’ learning progress in distance learning phases. In this paper, we present our experiences with integrating reflective learning diaries into eight Bachelor courses taught in mobile learning settings. We introduce different course designs combining mobile learning and learning diaries, and conclude with our lessons learnt on how to effectively integrate learning diaries into mobile learning settings. Especially in software engineering education, learning diaries offer an effective learning tool to develop students’ autonomy and reflection capabilities. Keywords: Higher education engineering education


Learning diaries
Mobile learning
Software

1 Introduction Mobile devices are essential learning tools in higher education to provide educational access anytime, anywhere (Wagner and Wilson 2005). Additionally, in the area of teaching and learning in higher education, mobile computing technologies have become a very important research topic. The usage of mobile devices for educational purposes has led to what is known as mobile learning, providing new opportunities for active learning by utilizing the advantages of mobility and wireless technology activities (Al-Emran et al. 2016; Briz-Ponce et al. 2017). With the use of mobile devices, learners are able to create their own learning environment anytime, anywhere, and either alone or in social collaborative learning settings (Heflin et al. 2017). Through collaborative learning settings, learners are able to interact with one another in a way that produces more learning than if they worked alone (Sivian 1986). © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 930–937, 2021. https://doi.org/10.1007/978-3-030-49932-7_86

Learning Diaries—A Valuable Companion of Mobile Learning

931

In the context of software engineering education, traditional-style lectures are still much more common than mobile learning settings (Gary et al. 2013; Ghezzi and Mandrioli 2006; Tillmann et al. 2012). This usually leads to high dropout rates, low satisfaction and limited student self-reliance (Berkling and Thomas 2013; Combéfis et al. 2016). However, the acceptance of new teaching methods depends on the selection of the proper tools, such as learning diaries. The purpose of these tools should be communicated in advance to mitigate the risk of non-acceptance (Huismann and Wallenius 2018). Learning diaries can serve as a didactic tool to support, structure, and consolidate students’ learning progress in distance learning phases. They contain students’ written elaboration and reflection of their learning experience and progress, also including their emotions, opinions and attitudes as part of their understanding. Thereby, learning diaries facilitate the reflection in action and on action as well as increase the level of feedback between students and teachers in both directions. This paper presents our experiences with integrating reflective learning diaries into eight different Bachelor courses of our Bachelor program “Computer Science and Digital Communications”. The courses took place over the past two years teaching both, full-time and part-time students in software engineering education. In Sect. 2, we discuss some findings on learning diaries as an educational tool. In Sect. 3, we introduce different course designs combining mobile learning and learning diaries, with a particular focus on the goals of learning diaries. Subsequently, we present our lessons learnt on how to effectively integrate learning diaries into mobile learning settings. Finally, Sect. 5 concludes the paper.

2 Learning Diaries Learning diaries are educational instruments to document students’ written reflections of their learning progress (Nückles et al. 2004). In educational settings, the terms learning diary or learning journal are often used simultaneously. Structured learning diaries are often suggested in order to register sequences of states of self-regulated learning in terms of time series (see, e.g. Nett et al. 2012; Schmitz and Wiese 2006) and usually pursue the goal of reflection. In higher education, the main purpose of learning diaries is to facilitate students’ comprehension and retention of complex issues in university courses (Connor-Greene 2000) or to encourage critical reflection on learning content (Campbell et al. 1999). It usually consists of several entries, also called learning protocols that can vary in extensiveness or in degree of structure. It is one of the few educational methods that can also be used to capture students’ emotions, opinions and attitudes as part of their learning process (Munezero et al. 2013). The main benefit for the lecturer is that a learning diary may provide valuable insights into the different learning processes and outcomes of participants in a course and that they help to diagnose possible misunderstandings (Nückles et al. 2004; Connor-Greene 2000). In addition, students are motivated to reflect on their learning activities and to take responsibility for their own learning process (Huismann and Wallenius 2018). The role of feedback as well as

932

S. Schefer-Wenzl and I. Miladinovic

informing students of the benefits of self-regulated learning seems to be very important in order to leverage the full benefits of learning diaries (Fabriz 2014). Learning diaries are important tools to support students’ self-regulated learning. This implies that students are able to plan, organize, and monitor their learning process. Hereby, learning diaries particularly support self-monitoring, which includes the systematic and critical observation and evaluation of one’s own learning process (Zimmerman and Paulsen 1995).

3 Course Designs In this paper, we describe our experiences with two different mobile learning-based course designs integrating learning diaries as an important didactic element. Both courses were applied in two full-time and two part-time classes, so that we can report findings of eight different courses that took place over the last two years in our Bachelor program “Computer Science and Digital Communications”. In addition, we introduce the role of learning diaries in a new course, which we are currently working on. This course is expected to start in September 2019. 3.1

Learning Diaries for Wrap-Up

We applied the first course design on the course Software Engineering in the 3rd semester of our Bachelor study program. At this time the students are supposed to have basic programming skills. The main goal of this lecture is to teach structured methods for software development.

Fig. 1. Learning diaries for wrap-up

Learning Diaries—A Valuable Companion of Mobile Learning

933

In the first course design, new topics are introduced during the in-class hours, as illustrated in Fig. 1. We used different warm-up exercises based on an example project, which went through the complete lifecycle of a software project during the course. The major part of this course is done in distance learning phases, which are designed for mobile learning. To leverage the benefits of mobile learning, learning material is provided in different kinds of formats (audio, video, and text) and adapted for small displays. In addition, learning snippets need to be made available that students can consume easily and in a short time. In this setting mobile learning is used for deepening the knowledge of the in-class hours. In addition, small software projects are implemented in groups. The role of learning diaries in this course setting is to weekly wrap up the findings from the mobile learning activities. This should help to reflect the learning progress after each topic, and to make the newly obtained knowledge sustainable by means of a structured repetition of the new material. In the learning diaries students answer the following questions: “What did I learn? What was new to me? Was there something that changed my views and why? Focus on and analyze the themes important to you.” and “What did I not understand? What went against my own ideas and why? What was less comprehensible and why? Focus on and analyze questions that left you puzzled.” The lecturers provide individual feedback for each learning diary entry and directly answer open questions. 3.2

Learning Diaries for Warm-Up

The second course design was applied in a course called Mobile App Development in the 4th semester of our Bachelor study (Schefer-Wenzl and Miladinovic 2018). At this time, students already have a basic understanding of software engineering methods as well as solid programming skills. The main goal of this course is to teach the specifics of mobile apps and to further deepen software engineering and programming skills (Fig. 2).

Fig. 2. Learning diaries for warm-up

934

S. Schefer-Wenzl and I. Miladinovic

In contrast to the course design described in Sect. 3.1, in this setting new topics are introduced in the mobile distance learning phase. Thus, self-study material for all topics is provided in advance to in-class hours and software projects are implemented in virtual teams. Learning diaries are used for reflecting the introductory material and to formulate questions which are discussed in-class. As the course offers very limited in-class hours, this time is valuable and needs to be spent carefully. The structure of the learning diaries is the same as stated in Sect. 3.1. However, the purpose of the learning diaries is different, as students get to know new topics on their own and reflect on what they understood, what was unclear, and how they acquired further knowledge on these topics. In this setting, the in-class hours are directly influenced by the input given in the learning diaries. In the in-class hours, we clarify questions and further deepen topics that are frequently stated in the learning diaries for further explanation. The remainder of the time is used for new exercises and coaching of software projects. 3.3

Learning Diaries for Long-Time Documentation

This course concept is still under development and will be applied in the course Scientific Technical Writing in the 3rd semester of our Bachelor study program. The main goal of the course is to introduce students to writing scientific technical papers. This knowledge is the basis for several seminar papers during the studies, as well as for the bachelor thesis in the last two semesters. Figure 3 shows the concept of the course. Similar to Sect. 3.1, we plan to start with in-class hours introducing new topics with warm-up exercises. In the mobile learning phases, the students deepen these topics in self-study. After approximately 1/3 of the course, students start to write a seminar paper in the mobile learning phases. This activity is supported by coaching units during the in-class hours. At the end of the course, students have written a full paper on a technical subject and have basic knowledge about scientific writing.

Fig. 3. Learning diaries for long-time documentation

Learning Diaries—A Valuable Companion of Mobile Learning

935

Here, learning diaries serve a different purpose than in Sects. 3.1 and 3.2. Besides the reflections of the obtained knowledge, the students weekly document the rules and recommendations for technical writing, as well as the findings during both phases, i.e. in-class hours and mobile learning. This documentation should help the students during the rest of the studies on scientific writing. In addition, the learning diary is used as a communication tool with the lecturers. Students can ask specific questions regarding their paper, as well as propose topics for the in-class hours.

4 Lessons Learned In the learning diaries, students have the opportunity to raise further questions autonomously for an extended inquiry in the in-class hours. In-class hours can then be adapted accordingly. However, in most course settings mainly part-time students took advantage of asking questions. This might be linked to their increased self-confidence due to work experience and limited time resources for further study. We also found that providing templates and examples of learning diary entries has a positive effect as students get a deeper knowledge on how to reflect and assess themselves. According to (Huismann and Wallenius 2018) this may be due to students feeling much more confident and engaged in the way they reflect their own learning achievements when templates are provided. Also, our students benefited much more from the advantages of this educational tool right from the beginning. The role of immediate and individualized feedback as well as informing students of the benefits of self-regulated learning seems to be very important in order to leverage the full benefits of learning diaries. Students also integrated emotional information into the entries; they significantly lose motivation, if they do not feel that the lecturers carefully read their entries. In addition, it is important to deepen and discuss topics that were not well understood in the upcoming in-class hours. The main difference between the course designs in Sects. 3.1 and 3.2 is the way to present new topics. In the concept presented in Sect. 3.1, learning diaries are used as a wrap-up tool to reflect on each topic introduced in class and further deepened via mobile distance learning. From our experiences, this setup is especially suitable for students completely new to a topic, i.e. software engineering in our case. In an advanced course, learning diaries can already be used at an earlier stage, as students are able to combine new with already existing knowledge. In the distance learning phase, students should reach a certain knowledge level on a specific topic and reflect on their learning progress, which is then discussed and further developed in class, as in the concept presented in Sect. 3.2. In both cases, we experienced that the quality of learning diary entries should have an influence on the final grade. This encourages students to devote enough time to mobile learning phases. We also found out that learning diaries improve the response rate of students to mobile learning tasks.

936

S. Schefer-Wenzl and I. Miladinovic

5 Conclusions Learning diaries are a well-known tool for reflecting and tracking the learning progress. We have already integrated this tool into two different courses in the software engineering context and plan to extend it to one additional course. In this paper we presented the design of these three courses and discussed the different objectives of the learning diaries. We found that especially in software engineering education, which is often characterized by very diverse levels of students’ knowledge, a high degree of group work, and lots of informal learning from sources such as forums, videos or reused code, learning diaries offer an effective learning tool to develop students’ autonomy and reflection capabilities. However, the potential of learning diaries has to be harnessed by appropriate integration into the course design, leveraging benefits of mobile learning for both, students and lecturers. In our future work we will present detailed results on evaluations of the described courses and also the final concept of the new course.

References Wagner, E., Wilson, P.: Why learning professionals need to care about mobile learning. Am. Soc. Train. Dev. 59(12), 40–43 (2005) Gary, K., Lindquist, T., Bansal, S., Ghazarian, A.: A project spine for software engineering curricular design. In: Proceedings of the 26th International Conference on Software Engineering Education and Training (2013) Ghezzi, C., Mandrioli, D.: The challenges of software engineering education, pp. 115–127. Springer, Heidelberg (2006) Tillmann, N., Moskal, M., de Halleux, J., Fahndrich, M., Bishop, J., Samuel, A., Xie, T.: The future of teaching programming is on mobile devices. In: Proceedings of the 17th ACM Annual Conference on Innovation and Technology in Computer Science Education, New York, NY, USA (2012) Al-Emran, M., Elsherif, H.M., Shaalan, K.: Investigating attitudes towards the use of mobile learning in higher education. Comput. Hum. Behav. 56, 93–102 (2016) Briz-Ponce, L., Pereira, A., Carvalho, L., Juanes-Méndez, J.A., García-Peñalvo, F.J.: Learning with mobile technologies – students’ behavior. Comput. Hum. Behav. 72, 612–620 (2017) Heflin, H., Shewmaker, J., Nguyen, J.: Impact of mobile technology on student attitudes, engagement, and learning. Comput. Educ. 107, 91–99 (2017) Sivian, E.: Motivation in social constructivist theory. Educ. Psychol. 21(3), 209 (1986) Nückles, M., Schwonke, R., Berthold, K., Renkl, A.: The use of public learning diaries in blended learning. J. Educ. Media 29(1), 49–66 (2004) Connor-Greene, P.A.: Making connections: evaluating the effectiveness of journal writing in enhancing student learning. Teach. Psychol. 27, 44–46 (2000) Campbell, C.M., Parboosingh, J., Gondocz, T., Babitskaya, G., Pham, B.: Study of the factors influencing the stimulus to learning recorded by physicians keeping a learning portfolio. J. Contin. Educ. Health Prof. 19, 16–24 (1999) Munezero, M., Montero, C.S., Mozgovoy, M., Sutinen, E.: Exploiting sentiment analysis to track emotions in students’ learning diaries. In: Proceedings of the 13th Koli Calling International Conference on Computing Education Research, Koli, Finland, November 2013 (2013)

Learning Diaries—A Valuable Companion of Mobile Learning

937

Huismann, J., Wallenius, L.: Empowerment on-line collaborations: learning diaries as a sustainable learning tool. In: Proceedings of the 8th International Conference on the Future of Education, Florence, Italy, June 2018 (2018) Fabriz, S.: Fostering self-monitoring of university students by means of a standardized learning journal—a longitudinal study with process analyses. Eur. J. Psychol. Educ. 29(2), 239–255 (2014) Zimmerman, B.J., Paulsen, A.S.: Self-monitoring during collegiate studying: an invaluable tool for academic self-regulation. New Dir. Teach. Learn. 63, 13–27 (1995) Nett, U.E., Götz, T., Hall, N.C., Frenzel, A.C.: Metacognitive strategies and test performance: an experience sampling analysis of students learning behavior. Educ. Res. Int. 2012, 1–16 (2012) Schmitz, B., Wiese, B.S.: New perspectives for the evaluation of training sessions in selfregulated learning: time-series analyses of diary data. Contemp. Educ. Psychol. 31, 64–96 (2006) Berkling, K., Thomas, C.: Gamification of a Software Engineering Course and a detailed analysis of the factors that lead to its failure. In: Proceedings of the International Conference on Interactive Collaborative Learning, Kazan, Russia, pp. 525–530. IEEE Press (2013) Combéfis, S., Beresnevicius, G., Dagiene, V.: Learning programming through games and contests: overview, characterisation and discussion. In: International Olympiad in Informatics, Vilnius, Lithuania, vol. 10, pp. 39–60 (2016) Schefer-Wenzl, S., Miladinovic, I.: Leveraging collaborative mobile learning for sustained software development skills. In: Proceedings of the International Conference on Interactive Mobile Communication, Technologies and Learning, Kos, Greece (2018)

A ‘Small and Thick’ Portrait of Kabelo’s Digital Play Shafika Isaacs(&) University of Johannesburg, Johannesburg, South Africa [email protected]

Abstract. This article analyses the part of a broader ethnographic narrative portrait of a boy called Kabelo that deals with his everyday digital play. Kabelo lives and schools in Soweto, South Africa and was followed over 16 months. His portrait is a response to the dearth of sociological knowledge on the everyday digital lives of African children. A social justice orientation combined with critical ethnography, narrative inquiry and portraiture provided the conceptual and methodological framework and guided the use of a range of qualitative data collection strategies. Through activity systems analysis (ASA) (Engeström 2015) the interrelationships and tensions within his world of digital play are anaysed. The study concludes that Kabelo’s relative access to mobile digital technologies illuminates his mobilities within and between his world of digital play and enables his voice, agency and capabilities. However, tensions exist based on the misrecognition of the subject’s agency and capabilities in policy. The study recommends the socially-just recognition of digital play capabilities as part of a list of children’s play capabilities for inclusion in policy and formal learning systems. Attention to the production of small and thick digital ethnographies to complement big and broad data is also recommended. Keywords: m-Learning
Digital ethnography, narrative, portraiture, South Africa
Soweto
Boys
Digital play
Children
Capabilities
Play
Social justice

1 Introduction This article is a deliberate attempt at an in-depth gaze into the everyday digital play, a ten-year-old African township1 boy-child, pseudo-named Kabelo. It is a response to the absence of voice and participation of African children in global conversations about them, including on mobile learning (m-learning) and more recently, on emerging ‘frontier technologies’ for sustainable development (Ramalingam et al. 2016). It is also a response to the preponderance of big and broad data, the disruptions in traditional qualitative research methods (Burrows and Savage 2016), and the accompanying dearth

1

A township in South Africa, refers to physically-bounded geographical spaces that were reserved for black people during Apartheid (1948–1994). Today they are spaces where mostly impoverished, predominantly black, working-class and under-class communities reside.

© Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 938–949, 2021. https://doi.org/10.1007/978-3-030-49932-7_87

A ‘Small and Thick’ Portrait of Kabelo’s Digital Play

939

of thick data (Wang 2013) such as those offered by ethnographic and sociological knowledge on educational technologies (Selwyn and Facer 2014; Selwyn 2019). Academic scholarship and practice knowledge on m-learning has been limited to systematic reviews (Crompton et al. 2017); theoretical modelling (Traxler 2016; Sharples et al. 2007), landscape reviews (Isaacs 2012; West 2012); and limited evaluations of m-learning interventions (Isaacs et al. 2019). Ethnographic studies remain sparse. Even fewer offer critical ethnographic (Carspecken 1996) perspectives. This paper aligns with the quest for more and better ethnographic deep-dive studies on mobile, digital lives associated with the emergence of digital ethnography (Pink et al. 2016). To this, it adds the need for a social justice perspective, particularly in the face of persistent structural inequalities such as that experienced in South Africa (Badat and Sayed 2014). It builds on the few studies that have illuminated children’s subjectivities and digital diffusion in naturalistic settings in the UK (Livingstone and Sefton-Green 2016; Sharpe et al. 2016). Ethnographic studies by Sefton-Green and Erstad (2009) of teenagers in Norway; Lemphane and Prinsloo (2014) and Walton and Pallitt’s (2012) on children in Khayelitsha2 offer further examples. While these studies provide rich, nuanced insight into the way an encroaching digital world manifests in the everyday lives of children in naturalistic settings, they do not occupy the mainstream of knowledge production that prioritizes quantitative methodologies. This article contends that knowing the child’s digital world contributes to a richer understanding of the way large scale changes are experienced in daily life.

2 The Study: School Performance vs Everyday Play in the Life of a Township Boy The study on which this article is based, was prompted by two significant influences on the learning landscape for children: an emerging pattern of boy academic underperformance in South Africa (van Broekhuizen and Spaull 2017) and worldwide (Mullis et al. 2016) and dramatic shifts in learning catalyzed by mobile digital technologies (Traxler 2016) amidst an unfolding neoliberal globalization (Ball 2013). The former lends credibility to a dominant deficit narrative about a systemic learning crisis (World Bank 2017) while the latter has led to debates about the value of m-learning and ‘frontier technologies’ in education (UNESCO 2019). However, absent are the voices, views and perspectives of the children and an understanding of the way systemic changes are experienced by individual children. The study’s purpose was to make visible the storied perspectives of the township boy-child and his experience with academic performance and everyday play and within this, his digital play. It was based on a purposefully selected sample of four boys who were followed for 12 months, one of whom, Kabelo, became a choice for further in-depth first-person study. Kabelo was followed for a further four months. Kabelo was chosen because his underperformance was framed as a learner with special educational needs (LSEN), his move to an LSEN

2

Khayelitsha is one of Cape Town’s largest and fastest growing black townships.

940

S. Isaacs

school and the nature of his access and use of digital technologies. Due to space constraints, this article only illuminates the everyday digital play dimensions of the Kabelo’s portrait.

3 Background Theorists of human play agree that through play, children explore, make sense of, and create much of their world (Nilsson et al. 2018; Vygotsky 1967; Piaget 2013; Bruner et al. 1976). The study of Kabelo foregrounds children’s lived experience (van Manen 2016) of the world, as a lived play experience. To this, Huizinga (1949) adds that play is integral to the culture of being human. Thus, from the child’s viewpoint, play is not always joyful; nor trivial; nor an end in itself; nor is it about just playing games, and it is not always free. Moreover, in the 21st century, the child’s playworld is continually being infused with a digital world (Albrecht and Tabone 2017). This digital world involves situated engagement with digital cultures mediated by socially-constructed digital tools, signs and symbols, the design, production and consumption of which are embedded and embodied in culture and disparate social relations (Feenberg 2017). Digital play is not only a reference to digital gameplay but manifests as the complex entanglement of everyday life with the rapid diffusion of digital technologies; the influence of the digital media industry and the associated commercialization and marketisation of play, popular media culture and digital gameplay (Lester and Russell 2014). Digital play also includes m-learning, which involves “learning across multiple contexts, through social and content interactions, using personal electronic devices” (Crompton 2013:4). That Kabelo lives in a structurally-polarizing South Africa, necessitates a social justice orientation. Fraser’s (2008) three-dimensional model of social justice with an emphasis on misrecognition and recognition is combined with the capabilities approach (CA) (Sen 2005; Nussbaum 2002). Recognition relates to whose attributes and capabilities are regarded by society as valuable and the way an individual is seen and acknowledged reciprocally as an equal by other individuals, institutions and systems. It is linked to the concept of misrecognition which highlights the explicit and implicit, overt and subtle, external and internalized ways in which power-marginalized individuals experience invisibility, inequality, disadvantage, exclusion, marginalization and powerlessness within institutions and systems (Fraser, ibid). Sen (2005) and Nussbaum’s (2002) complement Fraser (ibid) by illuminating the quality of life that individuals are capable of choosing. Sen (ibid) defines capability as the individual’s freedom, opportunities and choices to do what she or he considers to be valuable. Sen (ibid) and Nussbaum’s (ibid) reference is to the human capability to fulfil the essential quality of life functions in which agency and the freedom to choose are crucial dimensions.

A ‘Small and Thick’ Portrait of Kabelo’s Digital Play

941

4 Methodology Wang (2013) highlights the value of ‘thick data’ in relation to big data. In support, albeit critically, this article analyses a ‘small and thick’ ethnographic portrait, based on critical ethnography (Carspecken 1996; Geertz 1973) narrative inquiry (Connelly and Clandinin 2006) and portraiture. Portraiture, as a form of narrative inquiry, listens for a story and illuminates the relationship between researcher and research subject (Lawrence-lightfoot and Davis 1997). The research questions ask pertinently, what are Kabelo’s stories of his everyday exposure to digital technologies, and how does he experience his everyday digital play? 4.1

Methods

Data collection strategies included direct observation, interviews; focus group discussions with four boys and separately, with their mothers (Patton 2015); and digital story-telling (Lambert 2010). The latter involved the use of mobile phones given to each of the boys to support the storytelling about their daily lives based on storytelling workshops held with them. Data was captured systematically through audio recordings of interviews and focus group discussions; photographs and video recordings of direct observations during site visits, and the use of field notes. Audio and video recordings were transcribed and captured manually and were consolidated into a spreadsheet which enabled the systematic, iterative coding, categorizing and thematizing of chunks of data as they emerged over the 16 months. The research limitations included that the researcher was not conversant with the African languages that Kabelo, his friends and family speak and understand. While the researcher was supported by a translator, he was not present on some occasions. In these cases, Kabelo was encouraged to speak in his preferred language, and the audio recorded conversations would be translated subsequently. 4.2

Data Analysis: From Data to Portrait

Charmaz’s (2014) constructivist grounded theory methodology was applied premised on theory construction derived from iterative qualitative data collection and simultaneous, progressive data analysis. Over time, 1254 codes were developed from the data, and through an iterative process of interpretation, inference and meaning-making, 38 categories were developed, from which ten themes emerged. Five of these themes were related to play: digital play, pretend-play, game-play, play as making, violent play and play and care. From these, overarching concepts of play capabilities, recognition and misrecognition emerged, linked to social justice. To analyze the inter-relationship between emerging concepts, the study employed activity systems analysis (ASA) (Engeström 2015) as a heuristic. Within his digital play activity system, Kabelo is the subject who mediates his world through the use of digital tools and signs to make meaning, under given conditions and rules, with particular communities in which varying relations of power reside through a division of labour (Fig. 1).

942

S. Isaacs

Fig. 1. The structure of Kabelo’s digital play as an activity system Engeström (2015)

5 Findings and Analysis 5.1

Subject

Kabelo’s portrait reveals he is a playful, digitally-engaged, multilingual child who is capable of making choices, judgements and decisions, and who can act with agency. His confidence, assertiveness, and agency are supported by his appropriation of a range of digital artefacts within a nurturing stable, supportive, church-going community and family. He is also an aspirant child with an imagination about a desired future. As reflected in his mobile games, he aspires to be a truck driver, a fast car owner, an engineer and he wants to go to the university in Soweto. 5.2

Tools and Signs

I apply Warschauer and Matuchniak (2010) concept of digital access by which they mean not just the physical availability of internet-enabled computing devices but also the factors that support or constrain access, mainly in the home. Phyfer et al. (2016) found that 46 per cent of South African children were able to access the Internet whenever they wanted while Lemphane and Prinsloo (2014:15) found that the children in Khayelitsha had no computer access at all and were allowed only limited access to their parents’ mobile phones. In Kabelo’s parents’ home, however, they had a radio; a digital satellite television (DSTV) subscription with wide range of local and global broadcast channels, an Explora that enables television-on-demand; a DVD player; a handheld Leap-pad game player, a toy laptop, and Kabelo had a TV in his bedroom in both his parents’ and grandparents’ homes. He also has access to his mother’s smartphone and later, to a tablet which he shares with his mother. On his tablet, were a few neatly displayed applications ranging from Facebook, two bible apps, YouTube, English language and mathematics apps as well as ShareIt, a popular offline sharing app. This was evident when WhatsApp messages would be sent to him, and his mother and they would respond a few days later. “I didn’t have datas”. Kabelo’s mother downloads free apps, but she “pays in data which costs a lot of money.” Thus while Kabelo has widespread access to digital devices, he faces challenges with unaffordable data (Gillwald et al. 2018).

A ‘Small and Thick’ Portrait of Kabelo’s Digital Play

5.3

943

Object

Kabelo’s portrait reveals that the object of his digital play is for meaning-making and self-making by occupying a range of offline and some online virtual spaces, moving in and between these spaces and creating local virtual networks. Much of his various forms of digital play take place in the home often simultaneously, alongside mundane activities such as eating while the TV plays local or news channels, often as background noise. He uses a mobile device while he eats at home; when he is in a car or on public transport, and he watches TV or videos in the hour before bedtime. Kabelo’s digital play is mainly as a digital consumer. He watches downloaded movies such as Fast and Furious3, and at times, online videos. “Me I am taking a selfie. I feel happy about taking photos because I like photos because it’s nice and it is fun, and it makes me happy”. Kabelo’s mother’s phone and the phone he was given, contain numerous photos, many of which were ‘selfies’, of him smiling, pouting, in a ‘gangsta’ pose, posing with his school uniform, with his sister, his mother and his grandfather. He creates videos of life in Soweto which include his Sunday church attendance, swimming at the local public swimming pool. His self-made video clips contain recordings of him dancing to local music or musical scenes of local artists performing on TV. They further include clips from Cartoon Network or Msanzi TV4, or of him singing gospel songs in Sesotho, the language he speaks most often. He recorded significant chunks of the funeral of Winnie Mandela on his phone while it was showing on television. All of his self-produced videos are conducted in English, and his recordings of rap music and Mzansi TV channel programmes are in Sesotho with the latter showing English subtitles. While he produces selfies and videos, these are not shared with others. He has never played video games on a console player or a social game online with other children, in any of his homes, which resonates with the findings of Lemphane and Prinsloo (2014) on children in Khayelitsha. For learning, he ‘reads’ from a bible app that includes an English voice-over, for 20 min on Fridays and Saturdays. He also plays with an arithmetic app and an English vocabulary app when his mother asks him to. When asked which learning apps he likes most, Kabelo refers to the truck driving game app called Euro Trucks. He demonstrated how he uses the control icons to steer the vehicle in motion and to park it in the designated parking bay. He also downloaded a car driving game where he beat other cars in a race. “I love this game because I can drive fast”, he tells me while playing the game. Kabelo uses Google searches when doing his homework, and communicates mainly with family via WhatsApp. He does not subscribe to popular social media such as Facebook or Instagram but uses ShareIt often to swop videos and photos with friends offline. Thus, Kabelo’s digital play not only involves gameplay. It reflects the presence of the popular media culture in his daily life and his use of mobile digital spaces that are more offline than online, and he moves between these depending on data access. His 3

4

Fast and Furious is an American media franchise owned by Universal Pictures that based on a series of action films mainly concerned with illegal street racing, heists and spies. A reference to a suite of local TV channels.

944

S. Isaacs

digital consumption is mainly for entertainment, some learning and engagement with friends through offline networks. Much of the literature refers to selfie culture based on the public sharing of selfies and videos on social networks such as among youth in India (Dutta et al. 2016). For Kabelo producing selfies are about self-expression and self-making in the way Bruner (1991) explains as being “all in the constructing” of the self and of a culture, in the making of the text or, in this case, the photograph or video (1991:27). 5.4

Rules

The rules in his digital play activity system include global and national laws and policies related to children’s formal rights, their right to play and their rights to privacy and security in their use of digital media. Third et al. (2014) provide ten key findings from their focus group discussions with children around the world, one of which is that child-centred definitions of risks and opportunities with social media are needed and that policy-makers need to listen to children and not assume that they know what is best for them. In South Africa, the digital rights of children are under-researched and underrepresented in policy, as shown by Byrne and Burton (2017) on South African children’s use of digital technologies. In this respect, Kabelo’s presence in policy on children and digital rights are limited. Kabelo’s digital play, is also governed by ‘informal’ rules such as which programmes he is allowed to watch on TV and the games he is allowed to play on his tablet and his mother’s mobile phone and the amount of screen time he is allowed when he is at home. There is evidence of his mother controlling his screen time and encouragement for him to play outside in the garage area. 5.5

Community

In Kabelo’s play-world activity system, the members of his community are his parents, relatives and friends who share the object of meaning-making and self-making. Community and friendships are mediated through ShareIt, which is widely used by Kabelo and his friends to share music, games, videos and photos offline. Thus, community networks are created albeit offline, through such sharing. It also that Kabelo balances his gameplay with his friends outside and his digital friends with whom he shares and swops music and video via ShareIt. This differs markedly from Lemphane and Prinsloo’s (2014) reference to the online social gaming of a more privileged group of children living in South African suburbs. 5.6

Division of Labour

The division of labour in Kabelo’s digital play is evident from his role as an ardent consumer of global digital media products, reflecting the presence of global, commercialized digital media corporations in his daily life. There are also moments of vertical relations with the adults in his home. On one home visit, Kabelo was playing with WordSearch, an app on his mother’s mobile phone. When asked whether he downloaded the app, he said that he did not and pointed to his mother. They argued

A ‘Small and Thick’ Portrait of Kabelo’s Digital Play

945

over who downloaded it. He insisted that it was her. This scene demonstrates the ways in which Kabelo asserts his digital play preferences, reflecting his sense of agency in relationship with the adults in his life. 5.7

Outcome

The outcome of Kabelo’s digital play activity system suggests that he develops digital play capabilities while at the same time, his digital play is imbued the consumption of a mix of commercialized local and global digital cultures. He believes that he learns to cook from watching the cooking TV channel because he tries out recipes in the kitchen; he develops political opinions from watching the news also exemplified by his articulated high regard for Winnie Mandela; he claims to be streetwise from watching Mzansi TV and listening to rap songs from popular Soweto artists. His mother also believes that the truck game provides him with knowledge about how to drive and park a car or a truck. “When I drive my car, he tells me mummy you must put the car into neutral now; when I am at a robot, and now you must put the car in first gear and drive mummy when the robot turns green.” The findings also reflect his creativity, design and making capability in the production of selfies that he enhances with patterns, frames and captions and his self-made videos about his daily life in Soweto. While it reflects some digital capabilities, it also reveals limited digital citizenship (Hollandsworth et al. 2011) insofar as Kabelo lacks awareness of Internet and digital safety. These emergent digital play capabilities open up the prospect for expanding conversations on lists of children’s capabilities as developed by Biggeri et al. (2011) through further development of their play capabilities and digital literacies (Nascimbeni and Vosloo 2019). 5.8

Contradictions

Engeström (2015: 137) explains that contradictions drive the activity system as sources of change and development. There appear to be tensions at different levels in Kabelo’s digital play activity system. One of these tensions exists between the subject and rules of the system, more specifically, the subject as an agential and capable child for whom digital play is integral to his being as per Huizinga (1949). There is a tension between the subject and how he assumes agency through his digital play and how this remains misrecognised (Fraser 2008) in current, limited rights-oriented policies and laws, as shown by (Phyfer et al. 2016). Nthontho’s (2017) review of children’s rights literature found that policy strides were made to legitimize children’s rights in terms of their protection, participation and resource provision. However, there are several ways in which children are not conceptualised as equal partners and stakeholders. She provides the example of the depiction of children as future adults, which shifts attention away from their conditions in the present. To resolve these contradictions will require further attention to children’s digital rights and citizenship in policy.

946

S. Isaacs

6 Conclusions and Implications Thus, in answering the question of how Kabelo experiences his everyday digital play, his portrait reveals how the digital and play are infused in his everyday lifeworld as a township boy. They show that everyday digital play assumes different modalities both online and offline and as movements within and between these modalities. They further highlight the emergence of ‘informal’ learning spaces and networks that can influence his formal academic learning spaces. The study shows further that there are emergent tensions between existing policy on Kabelo’s everyday digital play practices and that digital play capabilities emerge as part of his everyday knowledge while his digital citizenship remains limited. This tension assumes the form of misrecognition. For this to be remedied will require the socially-just recognition of Kabelo’s digital play capabilities as a crucial part of his agency. The implications are that a shift in policy and practice there will need to be a stronger interplay between his everyday digital play and the capabilities that emerge from this, with formal academic knowledge, in curriculum change and the culture of learning and schooling, as well as in national policy on digital citizenship for children. Further research on valorizing children’s digital play capabilities will also be required in the context of a universal, albeit flexible, contextually-relevant list of children’s capabilities such as those developed by Biggeri and Mehrotra (2011) and Walker (2006) as part of a social justice enterprise. Attention to the production of small and thick digital ethnographies to complement big and broad data is also recommended. Acknowledgement. This research was supported by the National Research Fund (NRF) grant number 98573. It was undertaken as part of a doctoral study at in the Faculty of Education at the University of Johannesburg. Opinions and conclusions are those of the author and are not attributable to the NRF. Professor Elizabeth Henning (South African Research Chair on the ‘Integration of Mathematics, Science and Languages in the Primary School’), provided the supervision support and enabling environment for this research.

References Albrecht, R., Tabone, C.: Exploring alternative pathways: children, pedagogy and play in the age of digital technology. J. Theor. Res. Educ. 12(2), 89–111 (2017) Badat, S., Sayed, Y.: Post-1994 South African education: the challenge of social justice. Ann. Am. Acad. Polit. Soc. Sci. 652, 127–148 (2014). https://doi.org/10.1177/0002716213511188 Ball, S.J.: Global Education Inc.: New Policy Networks and the Neoliberal Imaginary (2013). https://doi.org/10.4324/9780203803301 Biggeri, M., Ballet, J., Comim, F. (eds.): Children and the Capability Approach. London: Palgrave Macmillan UK (2011). https://doi.org/10.1057/9780230308374 Biggeri, M., Mehrotra, S.: Child poverty as capability deprivation: How to choose domains of child well-being and poverty. Children Capabil. Approach (2011). https://doi.org/10.1057/ 9780230308374 van Broekhuizen, H., Spaull, N.: The ‘Martha Effect’ : The Compounding Female Advantage in South African Higher Education (Econpapers). Stellenbosch (2017). https://doi.org/RePEc: sza:wpaper:wpapers290

A ‘Small and Thick’ Portrait of Kabelo’s Digital Play

947

Bruner, J.: Self-making and world-making. J. Aesthet. Educ. 25(1), 67–78 (1991). https://doi.org/ 10.2307/3333092 Bruner, J., Jolly, S., Sylva, K. (eds.) Play–Its Role in Development and Evolution (Book), 1st edn. Penguin, London (1976) Burrows, R., Savage, M.: After the Crisis? Big Data and the Methodological Challenges of Empirical Sociology. Sotsiologicheskie Issledovaniya (2016) Byrne, J., Burton, P.: Children as internet users: how can evidence better inform policy debate? J. Cyber Policy 8871, 1–14 (2017). https://doi.org/10.1080/23738871.2017.1291698 Carspecken, P.F.: Critical Ethnography in Educational Research: A Theoretical and Practical Guide (Kindle). Routl, London (1996). https://doi.org/10.5860/choice.34-1057 Charmaz, K.: Constructing Grounded Theory. Sage Publications, Los Angeles (2014) Connelly, F.M., Clandinin, D.J.: Narrative inquiry. Handbook Complement. Methods Educ. Res. 103(1994), 477–487 (2006). https://doi.org/10.1080/00220670903323354 Crompton, H., Burke, D., Gregory, K.H.: The use of mobile learning in PK-12 education: A systematic review. Comput. Educ. 110, 51–63 (2017). https://doi.org/10.1016/j.compedu. 2017.03.013 Crompton, H.: A historical overview of mobile learning: toward learner-centered education. In: Berge, Z.L., Muilenburg, L.Y. (eds.) Handbook of Mobile Learning, 1st edn., pp. 3–14. Routledge, New York (2013) Dutta, E., et al.: Attitudes toward selfie taking in school-going adolescents: An exploratory study. Indian J. Psychol. Med. 38(3), 242–245 (2016). https://doi.org/10.4103/0253-7176.183094 Engeström, Y.: Learning by Expanding: An Activity Theoretical Approach to Developmental Research, 2nd edn. Cambridge University Press, New York (2015) Feenberg, A.: A critical theory of technology. In: Felt, U., Rayvon, F., Miller, C.A., Smith-Doerr, L. (eds.) Handbook of Science and Technology Studies, 4th edn., pp. 635–663. MIT Press, Boston (2017) Fraser, N.: Scales of Justice. Reimagining Political Space in a Globalizing World, 1st edn. Polity Press, Cambridge (2008) Geertz, C.: Thick description: toward an interpretative theory of culture. The Interpretation of Cultures, pp. 3–32. Basic Books, New York (1973) Gillwald, A., Mothobi, O., Rademan, B.: The State of ICT in South Africa. Cape Town (2018). https://researchictafrica.net/wp/wp-content/uploads/2018/10/after-access-south-africa-stateof-ict-2017-south-africa-report_04.pdf. Accessed Hollandsworth, R., Dowdy, L., Donovan, J.: Digital citizenship in K-12: it takes a village. TechTrends 55(4), 37–47 (2011). https://doi.org/10.1007/s11528-011-0510-z Huizinga, J.: Homo Ludens. A Study in the Play Elements of Culture, 1st edn. Routledge & Kegan Paul Ltd., London (1949) Isaacs, S.: Mobile Learning for Teachers in Africa and the Middle East. UNESCO Working Paper Series on Mobile Learning, pp. 1–33 (2012) Isaacs, S., Roberts, N., Spencer-Smith, G.: Learning with mobile devices: a comparison of four mobile learning pilots in Africa. South Afr. J. Educ. 39(3), 1–13 (2019) Lambert, J.: Digital Storytelling Cookbook. Digital Storytelling Cookbook. Digital Diner Press, Berkely (2010) Lawrence-lightfoot, S., Davis, J.H.: The Art and Science of Portraiture, 1st edn. Jossey-Bass Inc., Ed., San Francisco (1997) Lemphane, P., Prinsloo, M.: Children’s digital literacy practices in unequal South African settings. J. Multilingual Multicultural Devel. 35(7), 738–753 (2014). https://doi.org/10.1080/ 01434632.2014.908894

948

S. Isaacs

Lester, S., Russell, W.: Children’s right to play. In: Brooker, L., Blaise, M., Edwards, S. (eds.) The SAGE Handbook of Play and Learning in Early Childhood, pp. 294–305. SAGE Publications Ltd., London (2014). https://doi.org/10.4135/9781473907850.n28 Livingstone, S., Sefton-Green, J.: The Class: Living and Learning in the Digital Age, Kindle edn. New York University Press, New York (2016) Mullis, I.V.S., Martin, M.O., Foy, P., Hooper, M.: International Results in Reading, Boston (2016) Nascimbeni, F., Vosloo, S.: Digital Literacy for Children. Exploring Definitions and Frameworks. UNICEF, New York (2019) Nilsson, M., Ferholt, B., Lecusay, R.: ‘The playing-exploring child’: reconceptualizing the relationship between play and learning in early childhood education. Contemp. Issues Early Childhood 19(3), 231–245 (2018). https://doi.org/10.1177/1463949117710800 Nthontho, M.: Children as stakeholders in education: does their voice matter? South Afr. J. Childhood Educ. 7(1), 1–7 (2017). https://doi.org/10.4102/sajce.v7i1.434 Nussbaum, M.: Capabilities and social justice. Int. Stud. Rev. 4(2), 123–135 (2002). https://doi. org/10.1111/1521-9488.00258 Patton, M.Q.: Qualitative Evaluation and Research Methods. Sage, Thousand Oaks (2015) Phyfer, J., Burton, P., Leoshut, L.: South African Kids Online: Barriers, Opportunities and Risks. A Glimpse into South African Children’s Internet Use and Online Activities, Cape Town (2016). http://www.cjcp.org.za/uploads/2/7/8/4/27845461/south_african_kids_online_technic al_report.pdf. Accessed Piaget, J.: Play, Dreams and Imitation in Childhood. Routledge, London (2013). https://doi.org/ 10.4324/9781315009698 Pink, S., Horst, H., Postill, J., Hjorth, L., Lewis, T., Tacchi, J.: Digital Ethnography: Principles and Practice, 1st edn. SAGE Publications Ltd., London (2016) Ramalingam, B., Hernandez, K., Prieto Martin, P., Faith, B.: Ten Frontier Technologies for International Development. Brighton (2016). https://www.gla.ac.uk/media/media_524607_ en.pdf Sefton-Green, J., Erstad, O.: Identity, Community, and Learning Lives in the Digital Age. Cambridge University Press, New York (2009). https://doi.org/10.1017/CBO978113902623 9.001 Selwyn, N.: What is Digital Sociology?, Kindle edn. Polity Press, Cambridge (2019) Selwyn, N., Facer, K.: The sociology of education and digital technology: past, present and future. Oxford Rev. Educ. 40(4), 482–496 (2014). https://doi.org/10.1080/03054985.2014. 933005 Sen, A.: Human rights and capabilities. J. Hum. Devel. 6(2), 151–166 (2005). https://doi.org/10. 1080/14649880500120491 Sharpe, R., Beetham, H., de Freitas, S., Conole, G.: An introduction to rethinking learning for a digital age. In: Sharpe, R., Beetham, H., de Freitas, S. (eds.) Rethinking Learning for a Digital Age: How Learners are Shaping their Own Experiences, Kindle edn., p. 234. Taylor & Francis Kindle Edition, London (2016) Sharples, M., Taylor, J., Vavoula, G.: A theory of learning for the mobile age. In: The SAGE Handbook of E-Learning Research, pp. 221–247 (2007). https://doi.org/10.1007/978-3-53192133-4_6 Third, A., Bellerose, D., Dawkins, U., Keltie, E., Pihl, K.: Children’s Rights in the Digital Age (2014) Traxler, J.: Mobile learning research: the focus for policy-makers what is mobile learning? J. Learn. Devel. 3(2), 7–25 (2016). http://files.eric.ed.gov/fulltext/EJ1108183.pdf. Accessed UNESCO. Beijing Consensus on Artificial Intellligence in Education (2019)

A ‘Small and Thick’ Portrait of Kabelo’s Digital Play

949

Van Manen, M.: Researching Lived Experience: Human Science for an Action Sensitive Pedagogy, 2nd edn. (2016). https://doi.org/10.4324/9781315421056 Vygotsky, L.S.: Play and its role in the mental development of the child. Soviet Psychology 5(3), 6–18 (1967). https://doi.org/10.2753/RPO1061-040505036 Walker, M.: Towards a capability-based theory of social justice for education policy-making. J. Educ. Policy 21(2), 163–185 (2006). https://doi.org/10.1080/02680930500500245 Walton, M., Pallitt, N.: “Grand Theft South Africa”: games, literacy and inequality in consumer childhoods. Lang. Educ. 26(4), 347–361 (2012). https://doi.org/10.1080/09500782.2012. 691516 Wang, T.: Why big data needs thick data. Ethnography Matters (2013) Warschauer, M., Matuchniak, T.: Chapter 6: New technology and digital worlds: analyzing evidence of equity in access, use, and outcomes. Rev. Res. Educ. 34, 179–225 (2010). https:// doi.org/10.3102/0091732X09349791 West, M.: Mobile Learning For Teachers: Global Themes. UNESCO Working Paper Series on Mobile Learning (2012) World Bank: World Development Report 2018: Learning to Realize Education’s Promise. The World Bank, Washington, DC (2017). https://doi.org/10.1596/978-1-4648-1096-1

Use of the Fractal Analysis of Non-stationary Time Series in Mobile Foreign Exchange Trading for M-Learning A. Kuchansky1, A. Biloshchytskyi1, S. Bronin1(&), S. Biloshchytska2, and Yu. Andrashko3 1

2

Taras Shevchenko National University of Kyiv, Kiev, Ukraine [email protected], [email protected] Kyiv National University of Construction and Architecture, Kiev, Ukraine 3 State University “Uzhhorod National University”, Uzhhorod, Ukraine

Abstract. Mobile foreign exchange trading system for m-learning is proposed. It’s used for time series analysis skills learning. Method of pre-forecasting fractal R/S analysis of non-stationary time series is integrated in system. This method includes: persistence, anti-persistence and random level determination based on the calculation of the Hurst exponent. To calculate the average value of the nonperiodic cycle of time series, as well as to establish the potential profitable of assets that are represented by financial time series. A criterion for determination of the average length of non-periodic cycles based on the smoothing of V-statistics with simple moving average and Kaufman’s adaptive moving average is proposed. It has been confirmed that most financial time series are more or less persistent and endowed with long-term memory of their initial conditions using computer simulation. Time series of course pairs are close to random. Using fractal analysis in m-learning mobile foreign exchange trading systems for smartphones based on iOS or Android operating systems is suggested. The system is characterized by visualization and description of all stages, which has to be executed for time series analysis. Practical use of this system has shown high efficiency for time series analysis skills learning. Keywords: M-learning
Mobile foreign exchange trading system analysis
Hurst exponent
Time series


R/S-

1 Introduction 1.1

Introduction to Time-Series Analysis for M-Learning Tasks

The most part of economic, physical, technical and natural processes are nonstationary. Time series representing these processes are the following components’ complex: trend’s function component, cyclic components with different periods, fluctuations, etc. While forecasting such time series certain difficulties may be faced. Thus, learning system development is up to date and top notch ad it allows to train financial market traders and analysts. Trained specialists will obtain skills in time series analysis, profitability of assets assessment, forecasting tools effectiveness, etc. © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 950–961, 2021. https://doi.org/10.1007/978-3-030-49932-7_88

Use of the Fractal Analysis of Non-stationary Time Series

951

The important trader’s skill is ability to use time series pre-forecasting analysis methods. It can be done on the basis of fractal analysis. Fractal analysis of time series was proposed by B. Mandelbrot and R. Hudson [1, 2] and developed by E. Peters and E. Feder [3, 4]. As an integral part of discrete nonlinear dynamics methods, it is designed to study nonlinearities in dynamics of time series, including financial ones. Fractal analysis as an important element of pre-forecasting analysis can be integrated into a multi-tasking analytical decision support system for mobile foreign exchange trading. It may contain detailed market analysis, graphically outputs real-time information, analyzes expert forecasts and addition to training of traders, etc. This analytical system is presented as an application for a mobile phone supported by iOS and Android. The main functions of such systems are: broadcast video market experts, trade signals execution, display of current exchange rates, recommendation of trading strategies, information on events, news and forecasts and also trader training to work on the foreign exchange market. An important part of such a mobile application is the fractal time series analysis, which provides extensive information about the statistical characteristics of the process. The disadvantage of fractal market analysis application is the presence of empirical parameters and the need for the analysis of some statistical characteristics visually. Authors propose a method for pre-forecasting fractal analysis of financial time series where disadvantages are automated. Investors and analysts in the financial markets may apply above mentioned method for identification in the long-term time series, determining the average length of non-periodic cycles. This method is integrated to the mobile foreign exchange trading system. The educational functionality of this system allows to train traders effectively in time series analysis skills. 1.2

Review

M-learning it’s a new stage of e-learning systems’ development and recently is often enough used for different areas specialists’ trainings. Integration of m-learning systems and analysis as well as processing systems of financial indicators’ time series enhances the training of traders and financial analysts and provide the possibility to obtain skills in real time mode. A general architecture for m-learning systems is described in [5]. The method described in this research is based on the algorithm of the Hurst exponent calculation. There are several methods for calculating the Hurst exponent, where the main is the R/S analysis [3] and the detrended fluctuation analysis (DFA) [6– 8]. Approaches applied to the calculation of the Hurst exponent based on the R/Sanalysis procedure are described in [3, 4, 9] in details. The empirical rules and guidelines for R/S analysis, as well as the visual analysis of V-statistics for determining the average length of the non-periodic cycle, are described in [3]. The effect of longterm memory in time series is described in [1, 3, 9]. Application of forecasting methods and pre-forecasting analysis of time series in various applications are described in [10–30]. In particular the forecasting power of the method of cluster analysis using the same behavior of the time series of the stock market, as well as the use of this method for efficient forecasting of stock prices, were investigated in [10]. The method for modeling samples is proposed for the task of short-term forecasting of time series is proposed in [11]. The method of selective

952

A. Kuchansky et al.

comparison with the sample for the problem of constructing combined forecasting models for signs of increments of time series with an unstable nature of oscillations, considering the identification of similarities or indexation in [12]. The method of forecasting the increments of time series in conditions of uncertainty using the trend model of moving averages is described in [13]. The method of the nearest neighbor for this problem is considered in [14, 15]. The method for indexing time series based on the identification of similarities between them was described in [16]. The clustering method for finding similarities in multidimensional spaces that are represented by multidimensional time series is described in [17]. Application of time series analysis methods is also used in education in order to predict the potential of scientific directions development [18]. The values of time series in this case are the ratio of results evaluations of scientist’s research activities for different periods of time. The method can be used for identifying promising areas of research formed in a scientific environment. The method of classifying scholars in research, based on the identification of similarities in time series, is described in [19]. The method of constructing evaluations of the results of scientist’s scientific research activity based on the analysis of publications citations is described in [20]. The articles [21, 22] describes a parametric model for assessing and predicting the quality of educational institutions, which uses the approaches to the comparison of assessments. The task of identifying similarities and analyzing time series is also used to identify incomplete duplicates in textual information [23]. In [24], a conceptual model of the system for finding incomplete duplicates using the identification of similarities in electronic documents is described. In [25] describes the approaches of intellectual decision-making methods in business using classical time series forecasting models. Adaptive models of short-term forecasting of time series, methods of constructing combined methods are described in [26]. In [27] we consider models and methods of prediction of time series using intelligent data analysis: neural network, genetic algorithms, fuzzy analysis, etc. Both classical forecasting models and forecasting models using similarity identification are often used as components of complex forecasting, modeling and decisionmaking systems. In [28] an analysis of the design features of information-analytical systems for forecasting of time series with the use of expert evaluation was carried out. In [29], in particular, the method of fuzzy cluster analysis, which uses the identification of similarities, is considered. The use of expert estimation to forecast time series in information systems is also described in [30]. [31] deals with the management of project configuration management in the development of distributed systems, which can use the identification of similarities between projects on the set of indicators that these projects characterize [32, 33]. 1.3

Aims and Objectives

The aim of the research is to develop and describe the method of pre-forecasting fractal analysis of time series for use in mobile foreign exchange trading and traders mlearning. Also, to establish criteria in this method for estimating the average length of non-periodic cycles, identifying series with long-term memory, etc. Visualization of the

Use of the Fractal Analysis of Non-stationary Time Series

953

fractal analysis methods’ work results should be laid in mobile foreign exchange trading systems for m-learning. It’s needed for traders and financial analysts’ effective preparation.

2 Fractal Analysis of Time-Series Data Sets 2.1

Fractal R/S Analysis

According to the principles of fractal analysis, time series have a fractal dimension 1\D\2, endowed with properties of scale invariance (self-similarity) and memory of their initial conditions. It is believed that the time series that reflect the development of economic processes have a fractal structure. The fractal dimension indicates the degree of “spine” of the time series. In practice, the fractal dimension is replaced by the Hurst exponent, on the basis of which the degree of smoothness of the time series is determined [3, 34]. The H exponent is determined on the basis of the fractal dimension by the formula H ¼ 2  D, where is 0  H  1. Hurst exponent characterizes the ratio of the component of the trend function to white noise and can be used for the classification of time series: the establishment of non-random time series with a stable trend and random rows (including non-Gaussian ones). The calculation of the Hurst exponent can be based on the R/S analysis procedure. In [35] justified the applicability of this method for the study of financial and economic processes. There are three classifications for time series depending on the value of the Hurst exponent. If 0 < H\0:5 or 0\H\E, then there isn`t correlation between retrospective and predictive values in the time series. This time series is antipersistence. If E\H\1, then the time series is characterized by persistence behavior. If H ¼ E or H ¼ 0:5, then the time series is random and has no memory of its initial conditions. The value of E is calculated according to the formula described in the work [36]: rffiffiffiffiffiffiffiffiffiffi
 sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi X s1 Rs 2 si E ; ¼ pðs  1Þ i¼1 i Ss for [ 300. In this article, for the calculation of the Hurst exponent we will apply the methodology proposed in [3]. Let a series of n observations are given Z ¼ fzi gni¼1 . For each of the initial segments of this time series fzi gsi¼1 with length s ¼ 3; 4; . . .n calculate the P mean values by the formula zs ¼ 1s si¼1 zi , accumulated deviations find the formula Ps xs;t ¼ i¼1 ðzi  zs Þ, for s ¼ 3; 4; . . .n, where Rs ¼ max xs;t  min xs;t . Then the 1ts 1ts qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffi P 1 standard deviation for each of the segments determined by Ss ¼ s si¼1 ðzi  zi Þ2 , for s ¼ 3; 4; . . .n. The velocity of the accumulated deviation is normalized by dividing   by the mean square deviation for each segment s and the dependency schedule lg

Rs Ss

by

lgðsÞ is built also known as R/S trajectory. Then the linear regression equation is constructed based on the least squares’ method; the coefficient with an independent variable will be the Hurst exponent (a ¼ const):

954

A. Kuchansky et al.


 Rs lg ¼ lgðaÞ þ H lgðsÞ: Ss The Hurst exponent can also be considered as a function of s [3]: H ðsÞ ¼

lgðRSss Þ lgð2sÞ

.

Behavior constructed on  the basis of this function of the H trajectory or function dependence H ðsÞ of lg 2s , as well as R/S trajectories, can be used to detect such properties of the time series as: intervals of long-term and short-term dependence; the presence of cyclic components and the average length of the nonperiodic cycle. Typically, a visual analysis of trends in the V-statistics is used to find the length of a cycle. It consists in identifying the points of trend change that can signal the end of the cycle, as well as the intervals of growth, stabilization and decline of the curve, which, with increasing number of observations, determines the attraction of the process to a persistent or random. The growth of V-statistics with an increase in the number of observations indicates the persistence of the current section of the series, and stabilization - on the predominance of white noise. V-statistics is calculated by: Vs ¼ pRffissSs , where Rs is variance and Ss is standard deviation for s ¼ 3; 4; . . .n. It is confirmed in [3] that the moment of change of the trend of the V-statistics graph, which is expressed by the dependence Vs of lgðsÞ indicates the length of both the non-periodic cycle. 2.2

Method of Pre-forecasting Fractal Analysis of Time Series

The method of pre-forecasting analysis of non-stationary time series involves the calculation of the Hurst exponent based on the R/S analysis of the time series; definition of the average length of the nonperiodic cycle and the identification of “long memory” in the time series; selection of assets for trading. Here is a description of these algorithms, illustrating calculations for a specific task. Analysis stages description and visualization in system is necessary component of traders’ learning of time series analysis skills. Let’s set the time series of prices for petrol for the period from 2016 to 2019, daily data on closing prices, the length of the

þ1 series is 775. Let’s denote it by Z ¼ fzi gni¼1 ¼ zðt1 Þ ; zðt2 Þ;...;zðtn Þ , ti – discrete moments of time. Perform the following steps: Step 1. We visualize the given time series, that is, we will build a price chart. Step 2. We implement the procedure of R/S analysis. Calculate the Hurst exponent. To find the Hurst exponent we will consider the time series Z ¼ fzi gni¼1 where zi ¼ lgðzi Þ lgðzi1 Þ

for i ¼ 1; 2; . . .n. This requirement, described in [3] (see Fig. 1). The next step is to calculate the scale Rs and standard deviation Ss for each of the segments fzi gsi¼1 of length s ¼ 3; 4; . . .n. Let’s introduce the designation X ¼   ðx3 ; x4 ; . . .; xn Þ where xs ¼ lgðsÞ and Y ¼ ðy3 ; y4 ; . . .; yn Þ where ys ¼ lg RSss for s ¼ 3; 4; . . .n and suppose that between the factors X and Y there is a linear dependence, i.e. Y ¼ a þ bX. Identify the values of the coefficients a and b from the condition of

Use of the Fractal Analysis of Non-stationary Time Series

955

Fig. 1. Time series of prices for petrol (NYMEX) after conversion.

Pn 2 minimizing the function s¼3 ðys  ða þ bxs ÞÞ ! min by the least squares method [27]. As a result, for the time series Z received the following estimates: ^ a ¼ 0:549, ^b ¼ 0:783. The estimation of the coefficient b will be the Hurst exponent of the time series Z, i.e. H ¼ 0:783. Determination factor is R2 ¼ 0:9198. For a series Z, we construct on one graph R/S- and H-trajectories and the regression line Y ¼ 0:549 þ 0:783X (see Fig. 2).

Fig. 2. R/S (black) and H (red) trajectory for the time series Z.

Step 3. Check the hypothesis about the  significance of the H exponent for the Z Rs series. For the series Z, the indicator E Ss ¼ 0:5452. So, the Hurst exponent for this series H ¼ 0:783, then the hypothesis about his accident is rejected. The value of the Hurst exponent indicates that the input time series Z is persistent, and the process described by this time series is characterized by the presence of long-term memory and has a trend-resistant non-periodic cycle. 2.3

Criteria for Determining the Length of a Non-periodic Cycle

The criterion is based on smoothing the V-statistics curve and identifying the moments of changing the initial trend of the curve, taking into account the smoothed values.

956

A. Kuchansky et al.

Smooth a series Vs for s ¼ 3; 4; . . .; n with the help of a simple moving average with period p according to the formula: ss þ p ¼

1 Xp1 V ; s ¼ 3; 4; . . .; n  p j¼0 s þ pj p

and the Kaufman’s adaptive moving average by the formula: at ¼ ct Vt þ ð1  ct Þat1 ; where ct ¼ ðEt ðf  sÞ þ sÞ2 , Et ¼ Pr1Vt Vtr i¼0

jVti Vti1 j

are the coefficient of efficiency as the

ratio of the total price movement to the sum of the absolute values of the noise market movement for the period r, t ¼ s þ r; n, f , s are fast and slow smoothing factors, f ¼ p1 2þ 1, s ¼ p2 2þ 1 and p1 \p2 (see Fig. 3). The length of the non-periodic cycle is k if the conditions are fulfilled at the time k: • from the moment k the Kaufman index falls for at least two subsequent points, that is ak [ ak þ 1 [ ak þ 2 . it is significant that the Kaufman index should increase monotonously to the k-th point, which is explained by the behavior of the V-statistics; • the moving average flow at this moment does not exceed the value of the Kaufman index, i.e. sk \ak , sk þ 1 \ak þ 1 . • there is a sharp change in the trend of V-growth statistics on the fall: Vk1 \Vk , Vk [ Vk þ 1 , at the same time, the value of V-statistics at the kth point reaches the local maximum, exceeding the values of the simple moving average and the Kaufman index: Vk [ ak [ sk . The choice of a Kaufman flow filter is due to the adaptive nature of its coefficients. Thus, the system, subject to the use of this criterion, can calculate the value of a nonperiodic cycle without human intervention, that is, in automatic mode. It should be noted that the abscissa axis for the plot on which the V-statistics and the mean averages are constructed is the logarithmic value of s, that is, after obtaining the moment for which the specified conditions are fulfilled: the point from the abscissa axis xk you need to use the formula k ¼ 10lgðxk Þ . According to the graph V-statistics and moving averages for the studied series, one can see that these conditions are fulfilled for xk ¼ 1:255 that is, the value of a nonperiodic cycle for a given series 10lgð1:255Þ  18. It should be noted that the verification of conditions must be carried out starting from the point k ¼ 10, xk ¼ 1, as well as the construction of a regression line to determine the Hurst exponent. This empirical rule is formulated in [3]. Let’s consider other definition method of the length of a non-periodic cycle. It is known that the moment of a fracture or abrupt change in the initial trend of the H trajectory, as a rule, from the rising to a decreasing, provided that the R/S trajectory has previously changed its initial trend, indicates the length of the cycle.

Use of the Fractal Analysis of Non-stationary Time Series

957

Fig. 3. Plot V-statistics and moving averages, p ¼ 7, r ¼ 4, p1 ¼ 3, p2 ¼ 10

Let’s denote through F ðZ Þ a family of series of fixed length m, each of which is constructed from the input time series Z by a flow window method, that are the series mþ1 n fz i gm i¼1 , fzi gi¼2 ,. . ., fzi gi¼nm þ 1 . For each of these series, apply the R/S analysis procedure, construct the corresponding H and R/S trajectories, and define the length of the cycles kj , j ¼ 1; n  m þ 1 from condition that the point kj match the length of the þ j1 cycle for time series fzi gm if H is trajectory at the point kj þ 1 or kj þ 2 crosses the i¼j R/S trajectory, while both trajectories change the previous downward trend starting from the point kj so Hkj þ 1 \Hkj i ðR=SÞkj þ 1 \ðR=SÞkj , ðR=SÞkj þ 2 [ Hkj þ 2 and the R/S trajectory is not in the anti-persistence zone, i.e.ðR=SÞkj [ 0:5. Next, we will construct a histogram of the length distribution of cycles kj for þ j1 families of time series fzi gm , j ¼ 1; n  m þ 1. On the basis of the histogram, you i¼j can estimate the average length of the cycle. By the histogram for the input row (see Fig. 4), the median is 18, and the average value is 18.81. Since the data on time series reflects the daily prices of series petrol, it is possible to conclude from the analysis that the average value of the cycle (quasicycle) is almost a month (18 working days). Each of the described criteria shows the high efficiency of setting the value of cycles when working in automatic mode. Next step is to analyze the behavior of the Hurst exponent in dynamics. In order to provide a more detailed time series analysis, it is proposed to investigate the behavior of the Hurst exponent in dynamics. The results of this study can be used to break down a number of sites by their level of persistence. This allows you to follow the current development of the process and forecast it for the future. The flux index is a function which is constructed from the Hurst exponent for the family of series F (Z), which are formed from the studied time series Z by the flow window method. For the time series that is being investigated, the graph of the change in the indicator shows that the time series is persistent both in general and on local segments (m = 500). Next step is determination of the potential profitable of the asset represented by the financial time series. The purpose of the R/S-based formulation of the R/S-analysis of asset selection criteria presented by time series for trading may also be from the point

958

A. Kuchansky et al.

Fig. 4. Histogram of the distribution of the lengths of the cycles for the family of series F(Z)

of view of risk assessment of financial investment in asset data. This task can be used to construct asset selection criteria for an investment portfolio. Let’s put the task of assessing the risk of financial investment in assets of a certain type. The task of assessing the effectiveness of investing is a complex procedure for the sequential analysis of various economic and mathematical indicators, the calculation of which depends on the type of investment. We will consider investments that represent the flow of payments of primary payments and subsequent revenues when investing in commodity assets and performing speculative operations on the market. The task which is presented to the subject of risk, in this case, the investor, consists in choosing from a set of possible assets such, investing financial resources in which would provide the maximum economic effect in accordance with the requirements of the subject, taking into account financial risks. Under financial risks we will understand the risks associated with transactions with financial assets. Estimation of alternative investment options according to the requirements of the subject of risk can be realized both on the basis of analytical and calculation methods, and with the help of fractal analysis of the market. Let each asset meet retrospective information in the form of time series of prices for this asset. The procedure for assessing the effectiveness of investing consists in the implementation of the main stages of the complex fractal analysis described above: the implementation of the procedure of successive R/S-analysis, the definition of the Hurst exponent, the construction of V-statistics, identification of the process with long-term memory, determination of the average length of the cycle. The final step is to check whether the criteria for selecting assets for trading is satisfied by the characteristics found. An asset represented by a financial time series, is potential profitable, if: • the financial time series is near long-term memory; • the behavior of the Hurst exponent in the dynamics demonstrates stable persistence in the area of the series, which precedes the input of the investor on the market;

Use of the Fractal Analysis of Non-stationary Time Series

959

• at high values of the Hurst exponent, and subject to the presence of short cycles, it can be argued that the market is increasing, and vice versa, if the Hurst exponent is low, then in the case of long cycles, it can be argued that the market is falling. The ability to identify these characteristics is an important trader’s skill. Therefore, this mobile foreign exchange trading system provides the opportunity to train traders with this skill. Traders need to analyze the time series stored in the system base. Then they get feedback about identification results of the described characteristics.

3 Conclusions The article proposes a method for forecasting fractal analysis of time series, formulates the criteria for determining the average length of non-periodic cycles and other fractal characteristics. The results of the analysis can be used by investors and analysts to select the most potential profitable, assets represented by financial time series. This method is integrated to mobile foreign exchange trading systems. The system is characterized by the detailed visualization of all stages of time series’ pre-forecasting analysis in the charts’ form. The special feature of the developed mobile foreign exchange trading system is integrated m-learning tools for training traders’ skills of analysis & processing of financial indicators’ time series. The usage of system allows to enhance the training of traders’ and financial analysts’ effectiveness with the following features: 1. Development of the necessary skills for work with time series based on real data and in real time mode. 2. Detailed visualization and description of all stages of time series’ analysis. Also, potential profitableness justification considering the calculated indicators.

References 1. Mandelbrot, B.: Statistical methodology for non-periodic cycles: from the covariance to R/S analysis. Ann. Econ. Soc. Measur. 1, 259–290 (1972) 2. Mandelbrot, B.B., Hudson, R.L.: The (Mis)Behavior of Markets: A Fractal View of Risk, Ruin and Reward. Basic Books, New York (2004) 3. Peters, E.E.: Fractal Market Analysis: Applying Chaos Theory to Investment and Economics. Wiley, New York (1994) 4. Feder, J.: Fractals. Springer US, New York (1988). https://doi.org/10.1007/978-1-48992124-6 5. Georgiev, T., Georgieva, E., Smrikarov, A.: M-learning – a new stage of e-learning. In: International Conference on Computer Systems and Technologies, p. 28 (2004) 6. Peng, C.-K., Buldyrev, S.V., Havtin, S., Simons, M., Stanley, H.E., Goldberger, A.L.: Mosaic organization of DNA nucleotides. Phys. Rev. E 49(2), 1685–1689 (1994) 7. Peng, C.-K., Havlin, S., Stanley, H.E., Goldberger, A.L.: Quantification of scaling exponents and crossover phenomena in nonstationary heartbeat time series. Chaos 5, 82–87 (1995). https://doi.org/10.1063/1.166141

960

A. Kuchansky et al.

8. Kantelhardt, J.W.: Fractal and multifractal time series. In: Meyers, R. (eds.) Mathematics of Complexity and Dynamical Systems, pp. 463–487. Springer, New York (2012). https://doi. org/10.1007/978-1-4614-1806-1_30 9. Kyrychenko, L., Deineko, Z.: Estimating the self-similarity of a stochastic time series using the wavelet analysis method. Radioelectron. Comput. Syst. 4(38), 99–105 (2009) 10. Parzen, E.: Long memory of statistical time series modeling. In: NBER/NSF Time Series Conference, p. 10. Texas A&M University (2004) 11. Nayak, R., te Braak, P.: Temporal pattern matching for the prediction of stock prices. In: 2nd International Workshop on Integrating Artificial Intelligence and Data Mining (AIDM 2007), Australia, pp. 95–103 (2007) 12. Singh, S.: Pattern modeling in time-series forecasting. Cybern. Syst. 31(1), 49–65 (2000). https://doi.org/10.1080/019697200124919 13. Kuchansky, A., Biloshchytskyi, A.: Selective pattern matching method for time-series forecasting. Eastern-Eur. J. Enterpr. Technol. 6(4(78)), 13–18 (2015). https://doi.org/10. 15587/1729-4061.2015.54812 14. Perlin, M.S.: Nearest neighbor method. Rev. Eletrôn. de Administr. 13(2), 15 (2007) 15. Fernández-Rodríguez, F., Sosvilla-Rivero, S., Andrada-Félix, J.: Nearest-neighbour predictions in foreign exchange markets. FEDEA Working Paper No. 2002-05 (2002). https://doi. org/10.2139/ssrn.300404 16. Kahveci, T., Singh, A.: Variable length queries for time series data. In: Proceedings of the 17th International Conference on Data Engineering, pp. 273–282 (2001). https://doi.org/10. 1109/icde.2001.914838 17. Li, C., Chang, E., Garcia-Molina, H., Wiederhold, G.: Clustering for approximate similarity search in high-dimensional spaces. IEEE Trans. Knowl. Data Eng. 14(4), 792–808 (2002). https://doi.org/10.1109/tkde.2002.1019214 18. Biloshchytskyi, A., Kuchansky, A., Andrashko, Yu., Biloshchytska, S., Dubnytska, A., Vatskel, V.: The method of the scientific directions potential forecasting in infocommunication systems of an assessment of the research activity results. In: 2017 IEEE International Conference « Problems of Infocommunications. Science and Technology » (PIC S&T), pp. 69–72 (2017). https://doi.org/10.1109/infocommst.2017.8246352 19. Biloshchytskyi, A., et al.: A method for the identification of scientists’ research areas based on a cluster analysis of scientific publications. Eastern-Eur. J. Enterpr. Technol. 5(2(89)), 4– 10 (2017). https://doi.org/10.15587/1729-4061.2017.112323 20. Biloshchytskyi, A., Kuchansky, A., Andrashko, Yu., Biloshchytska, S., Kuzka, O., Terentyev, O.: Evaluation methods of the results of scientific research activity of scientists based on the analysis of publication citations. Eastern-Eur. J. Enterpr. Technol. 3(2(87)), 4– 10 (2017). https://doi.org/10.15587/1729-4061.2017.103651 21. Otradskaya, T., Gogunskii, V., Antoshchuk, S., Kolesnikov, O.: Development of parametric model of prediction and evaluation of the quality level of educational institutions. EasternEur. J. Enterpr. Technol. 5(3(83)), 12–21 (2016). https://doi.org/10.15587/1729-4061.2016. 80790 22. Biloshchytskyi, A., et al.: A method to evaluate the scientific activity quality of HEIs based on a scientometric subjects presentation model. Eastern-Eur. J. Enterpr. Technol. 6(2(90)), 16–22 (2017). https://doi.org/10.15587/1729-4061.2017.118377 23. Lizunov, P., Biloshchytskyi, A., Kuchansky, A., Biloshchytska, S., Chala, L.: Detection of near duplicates in tables based on the locality-sensitive hashing method and the nearest neighbor method. Eastern-Eur. J. Enterpr. Technol. 6(4(84)), 4–10 (2016). https://doi.org/10. 15587/1729-4061.2016.86243

Use of the Fractal Analysis of Non-stationary Time Series

961

24. Biloshchytskyi, A., Kuchansky, A., Biloshchytska, S., Dubnytska, A.: Conceptual model of automatic system of near duplicates detection in electronic documents. In: 2017 IEEE International Conference « The Experience of Designing and Application of CAD Systems in Microelectronics » (CADSM), pp. 381–384 (2017). https://doi.org/10.1109/cadsm.2017. 7916155 25. Vercellis, C.: Business Intelligence: Data Mining and Optimization for Decision Making, p. 417. Wiley, Cornwall (2009). https://doi.org/10.1002/9780470753866 26. Lukashin, Yu.: Adaptive Methods of Near-Term Time Series Forecasting, p. 416. Finance and Statistics, Moscow (2003) 27. Snytyuk, V.E.: Forecasting Models Methods Algorithms, p. 364. Maklaut, Kyiv (2008) 28. Mulesa, O., Geche, F., Batyuk, A., Buchok, V.: Development of combined information technology for time series prediction. In: Shakhovska, N., Stepashko, V. (eds.) CSIT 2017. AISC, vol. 689, pp. 361–373. Springer, Cham (2018). https://doi.org/10.1007/978-3-31970581-1_26 29. Mulesa, O., Geche, F., Batyuk, A.: Information technology for determining structure of social group based on fuzzy c-means. In: 2015 Xth International Scientific and Technical Conference “Computer Sciences and Information Technologies” (CSIT), pp. 60–62 (2015). https://doi.org/10.1109/stc-csit.2015.7325431 30. Mulesa, O., Geche, F.: Designing fuzzy expert methods of numeric evaluation of an object for the problems of forecasting. Eastern-Eur. J. Enterpr. Technol. 3(4(81)), 37–43 (2016). https://doi.org/10.15587/1729-4061.2016.70515 31. Morozov, V., Kalnichenko, O., Liubyma, I.: Managing projects configuration in development distributed information systems. In: 2017 2nd IEEE International Conference on Advances Information and Communication Technologies (AICT), pp. 154–157 (2017). https://doi.org/10.1109/aiact.2017.8020088 32. Kuchansky, A., Biloshchytskyi, A., Andrashko, Yu., Biloshchytska, S., Shabala, Ye., Myronov, O.: Development of adaptive combined models for predicting time series based on similarity identification. Eastern-Eur. J. Enterpr. Technol. 1(4(91)), 32–42 (2018). https:// doi.org/10.15587/1729-4061.2018.121620 33. Kuchansky, A., et al.: Combined models for forecasting the air pollution level in infocommunication systems for the environment state monitoring. In: 2018 IEEE International Conferences on Intelligent Data Acquisition and Advanced Computing Systems, pp. 125–130 (2018). https://doi.org/10.1109/idaacs-sws.2018.8525608 34. Hurst, H.E.: Long-term storage capacity of reservoirs. Trans. Am. Soc. Civ. Eng. 116, 770– 799 (1951) 35. Mandelbrot, B.B.: When can price be arbitraged efficiently? A limit to the validity of the random walk and martingale. Models Rev. Econ. Stat. 53(3), 225–236 (1971) 36. Anis, A., Lloyd, E.: The expected value of the adjusted rescaled Hurst Range of independent normal summands. Biometrika 63, 111–116 (1976)

Work-in-Progress: SMART-WATER, a Novel Telemetry and Remote Control System Infrastructure for the Management of Water Consumption in Thessaloniki Christos Mourtzios1(&), Dimitrios Kourtesis1, Nikolaos Papadimitriou1, Gerasimos Antzoulatos2, Ioannis-Omiros Kouloglou2, Stefanos Vrochidis2, and Kompatsiaris Ioannis2 1

2

Research and Development Department, APIFON S.A. Telecommunications, Thessaloniki, Greece {c.mourtzios,dk,n.papadimitriou}@apifon.com Information Technologies Institute (ITI), Centre for Research and Technology Hellas (CERTH), Thessaloniki, Greece {gantzoulatos,kouloglou,stefanos,ikom}@iti.gr

Abstract. Water scarcity and water stress issues pose a serious threat to the global population. Climate change, drought, population growth and consolidation in urban centres have all been increasing the pressure on water service providers to deploy more sustainable approaches in urban water management. Real-time monitoring and control of water consumption are key ingredients for a smart water management system which will raise consumers’ environmental awareness and reduce costs. This paper introduces a smart infrastructure system which enables remote telemetry and control of water consumption via a web application. The bidirectional and reliable communication between terminal devices (smart meters and valves) and the end-user (consumers and water utility operators) is realized through a fixed hybrid network which uses multiple telecommunication protocols. The metering data is collected and further processed through modern data analytics tools to give a deeper insight on water consumption and behavioral patterns. Keywords: Internet of Things
Smart cities
Smart metering Actuation
LPWAN
Predictive big data analytics


Telemetry


1 Introduction According to the United Nations Department of Economic and Social Affairs, 1.8 billion people will be living in countries or regions with absolute water scarcity by 2025, and two thirds of the world’s population could be living under water stressed conditions [1]. Water utility companies are increasingly investing in automated water consumption telemetry infrastructures to create new enhanced services. These technological infrastructures are

© Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 962–970, 2021. https://doi.org/10.1007/978-3-030-49932-7_89

Work-in-Progress: SMART-WATER, a Novel Telemetry

963

also known as Advanced Metering Infrastructures (AMI) [2, 3] and have evolved out of Automated Metering Reading (AMR) technologies [4, 5] which first appeared in the 1980s. AMR involves the automated transmission of recorded consumption data, typically via public (e.g. GSM) or private radio systems, to servers for storage and further processing by the utility company and/or third parties [6]. Usually, this involves the use of existing compatible “dumb” meters [7], which are modified/upgraded resulting in “smart enabled meters”. AMR technologies result in improved accuracy and proliferation of metering data but only offer a slight increase in data density, e.g., one read per month, although higher frequencies have been used [8]. In addition to what is offered by AMR technologies, AMI also allows for continuous bidirectional communication between the meter and the end-user (utility or consumer) and significantly higher data density. AMI offers a vast volume of data that enables real-time monitoring and data analysis. When applied in water management systems, AMI technologies enable near real-time monitoring of water consumption, immediate detection of water loss which may be due to leakages or unauthorized usage, environmental protection via water resources’ splurge reduction and the potential for significantly improved customer service. Water management infrastructures typically utilize wireless data collection methods commonly known as “walk-by” and “drive-by” [9, 10]. These methods allow for wireless collection of metering data from water meters through a portable short-range radio frequency reader carried by the staff in charge. The water metering staff will approach the point of installation of the meter at close distance (tens of meters), either on foot (‘walk-by’) or on a vehicle (‘drive-by’), in order to be within range of reception. Obviously, these methods do not allow for frequent data reception and therefore cannot constitute the foundation of intelligent infrastructures which collect and process the measured data in near real time. Furthermore, these technologies cannot support bidirectional communication between terminal devices and a water management infrastructure enabling remote control exercise. For the aforementioned reasons, research on smart water management systems has turned to the development of fixed communication networks, which allow bidirectional connectivity of terminal meter devices to a remote server in real-time. Recent research in Greece has revealed that the wireless reception of metering data from water meters which are located inside buildings and underground installation points comes with high risk of data loss and important reliability issues [11, 12]. The challenges involved in radio transmission of water metering data in Greek cities are attributed to the dense urban environment, the reinforced structural materials and the limitations of legacy RF technologies. Nevertheless, a new generation of Machine to Machine communication (M2M) technologies, commonly referred to as Low Power Wide Area Network (LPWAN) technologies [13, 14], promise to alleviate current challenges in water meter device connectivity.

964

C. Mourtzios et al.

2 Related Work The introduction of modern Information & Communication Technology in water supply management systems aims to create new solutions for measuring water consumption and reducing unnecessary wastage, contributing to raising water consumption awareness among consumers and promoting sustainable use of natural resources. One such example is the 2015 pilot project i-WIDGET (Improved Water Efficiency through ICT Technologies for Integrated Supply-Demand Management). This was a pilot project emphasizing the design and development of a web-based platform under the collaboration of Southern Water (UK), the water company of Barcelos (Portugal) and a number of other specialists [15, 16]. Another significant pilot project is “SmartH2O”, which focused on water demand and how it is affected by the use of smart meters. The application of the methodology developed by the project was shown to have achieved a consumption reduction of 10% in the pilot area tested in Switzerland and 20% in Spain [17]. Studies have also been conducted to visualize household consumption data and to develop a platform for mobile phones, including digital educational games [18]. Other examples of pilot projects are “WATERNOMICS” [19] and “WISDOM” [20], which focused on the use of ICT technologies to improve water resource management.

3 Purpose SMART-WATER is a collaborative pilot project run by the Thessaloniki Water Supply and Sewerage Company S.A. (EYATH); APIFON Telecommunications; and the Information Technology Institute (ITI) of the Centre of Research and Technology Hellas (CERTH). The project puts forward a novel design of an infrastructure that utilises modern telemetry and remote-control technologies to provide innovative services to consumers and water utility operators. The end-user web applications that are part of the smart water management infrastructure deliver novel services to consumers while raising environmental awareness. The integrated system infrastructure will be tested in the urban area of Thessaloniki, the second largest city in Greece.

4 Approach 4.1

Gateways

As mentioned earlier, the key to an intelligent water metering infrastructure is the deployment of a fixed telecommunication network comprising of wireless communication gateways deployed across locations within a coverage area. The gateway installation locations need to be selected on the basis of telecommunication range requirements and cost constraints. In the case of SMART-WATER the gateways are hybrid telecommunication network nodes allowing for bidirectional connectivity via multiple telecommunication protocols (LoRaWAN, wM-bus, GSM, NB-IoT) between terminal devices and a protocol-agnostic network server. Their use is crucial since none

Work-in-Progress: SMART-WATER, a Novel Telemetry

965

of the terminal devices use wireless internet protocols to communicate directly with the central server where device and data management takes place. The key role of the gateways is data transmission from wireless terminal devices, via an Internet protocol, to the central server as well as routing of downlink commands in the opposite direction (from the server to the terminal devices). The hybrid gateway is a special-purpose electronic device developed to communicate with the terminal devices through the LoRa and wM-Bus wireless protocols and connect them to the cloudbased server over NB-IoT and GSM cellular protocols. The gateway device uses a Raspberry Pi Zero (1 GHz, single-core CPU, 512 MB RAM) as a host Fig. 1. Custom gateway and its electronic components computer power unit and peripheral components for the wireless interfaces. The components are: iC880A IMST LoRaWAN Concentrator, iM871A IMST wMBus adapter, SODAQ SARA N211 NB-IoT node, MultiConnect mDot LoRa node and Huawei E3372 4G (Fig. 1). The host system operates several packet forwarding algorithms developed using Python, Go and C++ programming languages to establish the bidirectional connection between the terminal devices and the server. Given that gateways serve as the main building block of the proposed system, they must be controlled and managed remotely so that the network administrators can have full-scale network supervision and monitoring. All gateways run a software enabling remote control and actions like SVN repository updates and firmware upgrade over the air, log extraction, configuration, etc., at any time, via a cloud-based network management platform which provides a graphical user interface environment. The communication between the gateways and the platform is achieved through the MQTT (Message Queuing Telemetry Transport) messaging protocol. A messaging broker service (MQTT broker) runs at both sides (gateways and platform) to create communication channels (‘topics’) and route requests and responses between them. 4.2

Field Testing

Based on the RF technologies (LoRa, WM-bus, GSM, NB-IoT) used by the terminal devices and the custom gateways, field tests in real conditions were realized inside the building blocks of the candidate consumers at the urban area of Thessaloniki to draw conclusions on the practical capabilities and limitations of the above technologies as

966

C. Mourtzios et al.

well as the possible solutions for the implementation of the fixed wireless network. The testing procedure was realized through a custom ‘tester’ device which executed consecutive field tests on wM-Bus, LoRa, NB-IoT and GSM coverage at specific locations. The results of the tests are projected in a HTML-based GUI where the user added metadata such as the exact location and address where the test took place. The metadata of the test campaigns were locally saved and then forwarded and stored to a cloudbased database (InfluxDB) for further processing.

5 SMART-WATER Architecture The SMART-WATER novel system is analyzed into 5 interactive layers that encompass the basic system processes (Fig. 2) and specified as follows:

Fig. 2. Basic components of SMART-WATER system architecture

1. End Device Layer: SMART-WATER metering data sources are divided into two categories, namely the endogenous sources of terminal devices (smart meters, smart valves) and external resources which consist of other interacting systems with the SMART-WATER ecosystem environment, for example the ERP/CRM systems of the Water Utility company (e.g. EYATH S.A.). 2. Device Connectivity Layer: The key role of this layer is the bidirectional transmission of the metering data, from the smart terminal devices to the central server using wireless telecommunication protocols, as well as the forwarding of commands in the opposite direction. The bidirectional hybrid fixed network is deployed via special-purpose multi-protocol indoor/outdoor gateways which operate as intermediate telecommunication nodes.

Work-in-Progress: SMART-WATER, a Novel Telemetry

967

3. Device and Data Management Layer: The Data Management Services component receives, decodes, decrypts and stores the data sent by the smart meters. Furthermore, it manages the actuation commands from the control center to the smart valves. The Device Management Services component is responsible for the status of terminal devices and their remote control as well as for the supervision of telecommunication nodes. Storing and sharing the vast amount of data which flows from smart terminal devices to the system is made possible through the Apache Kafka platform, which provides high-performance solutions for data storage and distribution. 4. Data processing and analytics layer: The purpose of this layer is to directly process and analyze the metering data as well as to forward the data and analysis results for storage into the Data Storage Services component. Specifically, tailored state-of-the-art methods and techniques from the field of statistics, predictive data analytics and Machine Learning are applied. Initially, the streaming data passes through a pre-processing step using Kafka Streams. 5. User Interaction Layer: SMART-WATER end-users are water utility company operators and consumers. Each group has different needs, requirements and perspectives using the system. Specifically, the User Interface of the water utility company encompasses functionalities for direct monitoring of water consumption of all or selected consumers, advanced reporting and comparative spatio-temporal analysis of the obtained consumption streaming data. Furthermore, it enables EYATH’s personnel to control the status of all telemetering and remote-control devices installed in the field, as well as to manage the devices (Fig. 3, Fig. 4). Similarly, the consumer’s web interface will provide consumers with tools to monitor their water consumption, to benchmark their consumption levels against the average consumption in their area and to control the flow of water by sending actuation commands to the water valve installed on their premises, in near real-time.

Fig. 3. SMART-WATER’s dashboard – overview picture of water consumption

968

C. Mourtzios et al.

Fig. 4. SMART-WATER’s dashboard – picture of a single water meter (customer)

6 Conclusions The research being conducted as part of the SMART-WATER project will give the water utility company of Thessaloniki (EYATH) a sound understanding of consumers’ behavioural patterns rendering cost reduction, better resource allocation and modernization of consumer services a viable option. Telecommunication-wise, evidence from field tests suggests that reliable and uninterrupted bidirectional data transmission can only be feasible via a hybrid fixed communication network utilizing modern IoT and conventional RF protocols to achieve wide area coverage and deep in-building penetration. The comparison between LoRaWAN and wM-Bus technologies, both in laboratory testing conditions and in real field test conditions, has shown that LoRaWAN significantly outperforms wM-Bus technology in terms of coverage and penetration. However most renowned water meter manufacturers prefer the mature and widely tested wM-Bus technology. In addition, the fact that terminal devices (water meters, valves) use more than one communication protocol (LoRaWAN and wM-Bus) makes it necessary to create a hybrid multi-protocol network that will provide bidirectional connectivity between terminal devices and the central server. Implementing such a network that reliably support many different connectivity scenarios requiring the use of multi-protocol gateways. Given the limited range of the wM-Bus terminal devices, the gateways must be located close to them and at the same time remain connected to the central infrastructure via GSM or NB-IoT. Acknowledgements. This research has been co‐financed by the European Union and Greek national funds through the Operational Program Competitiveness, Entrepreneurship and Innovation, under the call RESEARCH – CREATE – INNOVATE (project code: T1EDK- 04337).

Work-in-Progress: SMART-WATER, a Novel Telemetry

969

References 1. Sierra Wireless, Unlock the Potential of Smart Water Metering with Cellular Communications. White Paper, © 2014 Sierra Wireless Inc. 2. McHenry, M.P.: Technical and governance considerations for advanced metering infrastructure/smart meters: technology, security, uncertainty, costs, benefits, and risks. Energy Policy 59, 834–842 (2013) 3. Turner, A., Retemal, M., White, S., Palfreeman, L., Panikkar, A.: Third Party Evaluation of Wide Bay Water Corporation Smart Metering. Institute for Sustainable Futures, University of Technology, Sydney: Sydney, Australia; Snowy Mountains Engineering Corporation: North Sydney, Australia, pp. 1–62 (2010) 4. Qingyang, L., Bingzhen, Z., Wang, Y., Hu, J.: Experience of AMR systems based on BPL in China. In: Proceedings of the IEEE International Symposium on Power Line Communications and Its Applications (ISPLC), 29 March ± 1 April 2009, pp. 280–284 (2009) 5. Sridharan, K., Schulz, N.N.: Outage management through AMR systems using an intelligent data filter. IEEE Trans. Power Deliv. 16(4), 669–675 (2001) 6. Readdy, A.: Overview of automatic meter reading for the water industry. In: Proceedings of 31st Annual Qld Water Industry Workshop—Operations Skills, Rockhampton, Australia, 4– 6 July 2006, pp. 1–7 (2006) 7. Hauber-Davidson, G., Idris, E.: Smart water metering. Water 33, 38–41 (2006) 8. Hill, T.: The smart grid for water. Water Effic., 50–51 (2011) 9. Levy, R.G., Karen Wilson, J.: Unlocking the potential for efficiency and demand response through advanced metering. Lawrence Berkeley National Laboratory, 30 June 2004 10. Harney, A.: Smart metering technology promotes energy efficiency for a greener world. In: Analog Dialogue, vol. 43, no. 1. ©Analog Devices, Inc. (2010). ISSN 0161-3626 11. Mylopoulos, Y., Kolokytha, E., Mentes, A., Vagiona, D.: Urban water demand management — the city of Thessaloniki-Greece case study. In: Maksimovic, Č., Butler, D., Memon, F.S. (eds.) Advances in Water Supply Management. © 2003 Swets & Zeitlinger, Lisse (2003). ISBN 90 5809 608 4 12. Tsavdaridou, A.D.: Measurement of urban water consumption by the adoption of new technologies: an application in the city of Thessaloniki. MSc thesis, Civil Engineering Department, Engineering School, Aristotle University of Thessaloniki (2011) 13. Ikpehai, A., Adebisi, B., Rabie, K.M., Anoh, K., Ande, R.E., Hammoudeh, M., Gacanin, H., Mbanaso, U.M.: Low-power wide area network technologies for Internet-of-Things: a comparative review. IEEE Internet Things J. 6(2), 2225–2240 (2019) 14. Keith, E.N., Guibene, W., Mark, Y.K.: An evaluation of low power wide area network technologies for the Internet of Things. In: 2016 International Wireless Communications and Mobile Computing Conference (IWCMC), 5–9 September 2016 15. Kossierisa, P., Panayiotakisb, A., Tzoukaa, K., Gerakopouloua, P., Rozosa, E., Makropoulosa, C.: An eLearning approach for improving household water efficiency. Procedia Eng. 89, 1113–1119 (2014). 16th Conference on Water Distribution System Analysis, WDSA 2014 16. Kossierisa, P., Kozanisa, S., Hashmib, A., Katsiric, E., Vamvakeridou-Lyroudiab, L.S., Farmanib, R., Makropoulosa, C., Savicb, D.: A web-based platform for water efficient households. Procedia Eng. 89, 1128–1135 (2014). 16th Conference on Water Distribution System Analysis, WDSA 2014 17. Rizzoli, A., Castelletti, A., Fraternali, P., Novak, J.: Demo Abstract: SmartH2O, demonstrating the impact of gamification technologies for saving water. Comput. Sci. Res. Dev. 33(1-2), 275–276 (2017). https://doi.org/10.1007/s00450-017-0380-5

970

C. Mourtzios et al.

18. Froehlich, J., Findlater, L., Ostergren, M., Ramanathan, S., Peterson, S., Wragg, L., Landay, J.A.: The design and evaluation of prototype eco-feedback displays for fixture-level water usage data. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, New York, NY, USA, pp. 2367–2376 (2012) 19. Galli, L., Fraternali, P., Pasini, C., Baroffio, G., Dos Santos, A., Acerbis, R., Riva, V.: A gamification framework for customer engagement and sustainable water usage promotion. In: E-Proceedings of the 36th IAHR World Congress, The Hague (2015) 20. Clifford, E.C., Van Andel, S.J., Van de Giesen, N., Kouroupetroglou, N., Mink, S., Smit, J. S.: Interactive water services: the WATERNOMICS approach. Procedia Eng. 89, 1058–1065 (2014). https://doi.org/10.1016/j.proeng.2014.11.225

Educational Robotics for Creating “Tangible Simulations”: A Mixed Reality Space for Learning the Day/Night Cycle Stefanos Xefteris, George Palaigeorgiou(&), and Helen Zoumpourtikoudi University of Western Macedonia, Florina, Greece [email protected], [email protected], [email protected]

Abstract. Human cognition is guided and shaped by the physical manifestation of objects around us- acting and thinking are intertwined. By integrating digital interactions to the physical world, we improve the potential to perceive and understand objects, spaces and notions and we facilitate the creation of mental representations since these are directly influenced by the way we bodily interact with objects. The integration of educational robotics, mixed reality and tangible interfaces provides us with a fertile ground on which to develop new learning scenarios. Astronomical phenomena, from the simplest to the most obscure include difficult to grasp concepts for which learners usually develop strong misconceptions. In this paper, we propose the use of educational robotics as a tangible simulation tool of the Earth’s movements, in a mixed reality space for learning the day/night cycle. In order to evaluate the proposed environment, a pilot study was conducted with sixteen elementary students. Data were collected through pre/post tests, an attitudes questionnaire and semi-formal group interviews. Students’ answers indicated that they were excited with and impressed by the proposed intervention while the gained invaluable knowledge about the day/night cycle. Keywords: Educational robotics


Mixed reality
Tangibles
Astronomy

1 Introduction One of the major issues for science educators is how children perceive natural phenomena, more so regarding astronomical ones. The “big world” cannot be intuitively understood by learners, with consolidated preconceptions and folklore knowledge creating hard-to-beat misconceptions [1]. For this reason, learners are frequently unable to conceptualize and explain the phenomenon of the day and night cycle. Although routine, the phenomenon induces complicated concepts since it addresses a cosmic scale larger than our every-day routine perceptions and is hugely affected by our earthgrounded point of view [2]. Teaching astronomical phenomena poses significant challenges. The solar system and our neighboring celestial bodies are the first contact of children with the universe. And while the universe around us affects and shapes our lives, teaching about it proves © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 971–982, 2021. https://doi.org/10.1007/978-3-030-49932-7_90

972

S. Xefteris et al.

hard. Especially in young ages, the geocentric model of our solar system seems more intuitive and acceptable rather than the heliocentric model, whose explanation seems unnecessarily complicated [3]. Thus, even if we have the necessary information and models, even if astronomy teaching is by now integrated in modern curricula [4, 5] and novel interventions acquaint elementary school students more with astronomical phenomena, research indicates that still, science educators have trouble negating children’s misconceptions [6, 7]. Regarding specifically the day/night cycle, research shows that children may develop many different mental representations. Advances in ICT have greatly improved the teaching of astronomical phenomena. From 3D representations and virtual models [8], mixed reality and augmented reality applications [9] to complicated setups integrating multiple sensors [10], new technologies have been a great tool for leveraging conceptual change [11]. Although ICT has shown not only great potential, but also great results, its applications usually lack a vital ingredient: tangibility. Tangible learning makes use of physical manipulatives – things the learner can touch, feel and use. Tangible agents merge the digital representations with the physical world by natural interactions, facilitating the creation of conceptual anchors on which children can build new knowledge. The HumanComputer Interface is not anymore an interface, with the user naturally manipulating physical representations of digital objects [12]. Robotics aids in transforming mixed reality applications to tangible mixed environments: In a mixed reality setting, the robot functions as a double agent -being a digital entity with a physical presence. The robot becomes itself a tangible interface to the digital representation of the simulated world. Integration of robotics in learning environments usually targets computational thinking [13] and fosters the acquisition of higher order mental skills. In this article, we present a mixed reality tangible environment that uses a robotic agent as a natural phenomenon simulator that aims to provide a novel experience of learning astronomical phenomena and specifically the day/night cycle. This proposal tries to introduce a novel approach by transforming the robot from a means to induce computational thinking to a tangible simulation environment where computational thinking is coupled with aspects of the scientific methodology. Here students observe, hypothesize and test their assumptions by transforming them to programs and executing them on the robot to check their results. The learning environment consists of a floor based augmented space where the Earth (the robot simulator) can perform all the range of its movements and a tangible answering pad where students pick their answers in multiple choice questions as the intervention progresses. The goal of the study was to explore the learning effects of this tangible environment. The study examined both cognitive and affective aspects of the intervention, testing variables such as learning efficacy, usability and motivation (ease of use, autotelic experience, user stimulation etc.). The present paper gives an outline of the relevant literature in Sect. 2, describes the proposed platform in Sect. 3, while in Sects. 4 and 5 we discuss and analyze our methodology and findings.

Educational Robotics for Creating “Tangible Simulations”

973

2 Literature Review Physical interaction with natural objects facilitates learning through involvement of the user at multiple levels [14]. Conceptual processing motivates our sensorimotor systems and thus the important aspects of cognition are embodied – interaction shapes of our representations [15]. Hence, by transforming abstract processes and relationships or macroscopically unobservable phenomena to physicalized representations that learners can act upon, we facilitate retention of new knowledge and create new conceptual anchors that help shape perception [16]. Consequently, the introduction of new interaction technologies paves the way for the deployment of embodied learning interventions. New environments foster much more -mentally- attainable representations of abstractions, turning them into concrete examples and induce a physicalization of interaction that facilitates the expression of intellectually challenging content. Tangible interfaces intertwined with mixed reality representations and robotics shape a distinct framework for designing learning interventions [17, 12] that is yet largely unexplored in current research; a framework that presents an opportunity for highly innovative activities and creation of complex environments. Regarding the students’ perceptions and misconceptions about astronomical phenomena, research has shown ever-present problems. In such a survey [18], exploring day/night cycle mental models among students eighteen (18) different models emerged. Another study counted up to 9 different mental models of the solar system among children 9–11 years old [19]. Basic knowledge on the solar system seems to be hindered mainly by naïve preconceptions, folklore and modern media (especially films) [20]. The unattainable of macroscopic real observation makes it easier to accept intuitively better explanations (such as the geocentric model). The advent of virtual and augmented reality has shown promising results. These technologies provide an experience that immerses students and create an improved perspective. Exploring the Sun and Moon through 3D imagery/tours with virtual reality glasses had positive effects on the learning of related phenomena in second-class students [21]. Studying mobile app simulations of phenomena revealed that short exposure to simulations outweighed traditional teaching [22]. Recently, the transition from “old” three-dimensional simulations on a screen, to augmented reality educational applications that have tangible aspects [9, 23] has shown promising results. Educational robotics offers new teaching opportunities for educators. Usually educational robotics is considered a subject related to the development of computational thinking [24]. And although educational robotics have been used both as simulators of physical phenomena [25, 26] and for teaching orbital engineering [27], there is an empty space in literature concerning the use of robotics in a teaching scenario as simulators of an astronomical phenomenon -like the day/night cycle- itself. We should note here that there is an abundance of orreries constructed with Lego Mindstorms components [28], but it seems that neither of them has been exploited in a teaching scenario-not more than any other ready-made orreries. Robotics affordances have been used in scenarios concerning history and geography [12] along drones [29] and inside mixed reality/augmented reality environments along wearables. There is a certain trend for development of multimodal scenarios that combine multiple interactive and

974

S. Xefteris et al.

embodied modalities that aim to create new contexts of learning, transforming the process to a more experiential and active one [30]. Mixed reality applications transform students to the protagonists of the system to be familiarized with, allowing them to fully control and manipulate it. Students become actors in an exciting and immersive story with vivid visual and audio stimulation. The combination of mixed reality with educational robotics can become in the future a framework of reference under which other technologies can be integrated, in order to create powerful learning opportunities.

3 The Learning Environment In this study we tried to create a mixed reality learning scenario for the night/day cycle which is based on the following precepts: 1. Exploitation of embodied affordances via tangible objects 2. Creation of a mixed reality environment in an immersive context where astronomy is interleaved with problem solving activities and scientific thinking 3. Transformation of the role of the robot to that of a tangible simulation agent. The learning environment is based on an augmented floor that initially depicts the space between Earth and Sun. On the floor is placed a natural model of the Sun and a robotic model of Earth that is capable of performing four movements using five motors: Rotation (around itself), circumnavigation (around the Sun), tilting of the axis as well as an independent move that keeps the axis stable with respect to the sun. During the scenario, students in groups of 2 are asked to answer questions via a tangible interface and perform programming tasks. The tasks aim to expose pre-existing knowledge and drive students to form hypotheses based on their previous responses. The hypotheses the students make are tested “experimentally” by programming the robotic model of the Earth to move. The whole framework of application aims to help children recall previous knowledge as well as reveal and clear misconceptions such as “the night comes when clouds come in front of the Sun and block it”. To construct the augmented space, a simple 170  170 cm print canvas was used. Then, using a vertically mounted projector, we augmented our floor space with a space background as shown in Fig. 1.

Fig. 1. The Earth movement simulator (EV3), the model sun and the MR augmented ground

Educational Robotics for Creating “Tangible Simulations”

975

The game was implemented in Scratch 3.0, and the children’s answers to multiple choice questions were delivered through tangible answering board implemented with a Makey-Makey. The Earth’s motion simulation robot was built from 2 sets of Mindstorms EV3 - and it required 2 Smart Brick units to control the 4 movements required for its operation. In Fig. 2 we see the Earth in two different positions (Summer-Winter).

Fig. 2. Earth simulator in Summer (up) and Winter (down) position

Students alternated between activities, from answering multiple choice questions through the Makey-Makey interface, to programming the robot to simulate these answers and to check their correctness, as shown in Fig. 3.

Fig. 3. The answering interface, the programming station and checking of results

The game session begins at the augmented floor. After a short introduction to the phenomenon, the scenario unfolds in twelve (12) increasing difficulty questions. All aspects of the day/night cycle are presented to the students, from simple ones, such as the direction of the earth’s rotation to more complex, such as time zones, and the height of the Sun at noon in different seasons. Students are asked to program the Earth

976

S. Xefteris et al.

movement simulator to perform the exact movement they hypothesized was the response to the last answered question and see if they were correct. The scenario exploits the robotic model as a simulator, but at the same time enables students to perform usually hard for them mathematical calculations such as representing fractions of the 24 h cycle with decimals and linking them to absolute hours duration as well as fractions of the earth’s rotation around itself (i.e. 12 h = half a day = half a rotation = 0.5 rotations to put into the programming block that controls the simulator). In order to make programming tasks easier for the students, and transfer the weight of the scenario more to the simulation and less to the programming tasks, we implemented “My Block” blocks which reduced the complexity of the necessary programming. Thus, students after performing the necessary mental calculations (i.e. if 24 h are one rotation how many rotations are 6 h?) they programmed the simulator robot with their exact answer without the need to calculate EV3 parameters such as direction and speed.

4 The Study 4.1

Participants

In order to evaluate the proposed scenario a pilot study was conducted with sixteen (16) elementary school students and took place in the premises of the Department of Elementary Education. Eight (8) of the students where sixth grade students and eight (8) fifth grade students. The students played in the scenario in eight (8) groups of two (2) students of the same class, with each session lasting approximately 60 min. 4.2

Procedure

Before starting the intervention, a short introduction was made to the students, to familiarize them with the available interfaces. During the intervention the researcher provided students with additional help whenever it was deemed necessary. The students completed online pre- and post-knowledge tests right before and immediately after the intervention. After the end of their respective sessions students also completed online questionnaires to evaluate the whole experience. Additionally, the researchers conducted group interviews with informal questions after each session. 4.3

Research Instruments

The intervention was evaluated using pre/post knowledge tests, an attitudes questionnaire examining how students perceived the environments usability and design and finally a semi-formal group interview. The pre and post cognitive tests were identical and consisted of twenty nine (29) multiple choice questions; The questions aimed at recalling previous knowledge concerning the Earth’s distinctive movements, the causes and effects of the day/night cycle (with questions concerning our time and older civilizations), the parts of the day, seasonal changes in day/night duration, differences among time zones, orientation on the cardinal points of the horizon and the properties of Earth’s axis. The questionnaire concerning the students’ attitudes consisted of

Educational Robotics for Creating “Tangible Simulations”

977

twenty-one (21) 5-point Likert scale questions (1 corresponding to total disagreement, 5 to total agreement) and focused on the usability and attractiveness of the tangible environment. The attitudes questionnaire contained questions from the AttrakDiff [31] questionnaire and the Flow State Scale [32]. The former tries to ascertain how the attractiveness of an interactive environment is experienced by its users, in terms of its usability and appearance, while the latter monitors if the experience is genuinely satisfying. The following variables were evaluated: • Ease of Use (3 questions): concerns how easy to use the system is and its learnability • Autotelic experience (3 questions): concerns the extent to which the system offers internal user satisfaction • Perceived learning (3 questions): concerns students’ perceptions on the educational value of the system • User Focus (3 questions): concerns the concentration during the use of the system • Pragmatic Quality (4 questions): concerns the extent to which the system allows a user to achieve his goals • Hedonic Quality-Stimulation (3 questions): concerns the extent to which the system meets the user’s need for innovation and whether it is of interest • Hedonic Quality-Identity (3 questions): concerns the extent to which the system allows the user to identify with it. Our findings indicate that variables where consistent with a satisfactory Cronbach’s a (>0.70) as seen in Table 2. The researchers conducted semi-formal interviews immediately after the end of each session aiming to record each student’s qualitative assessment of the intervention, allowing them to express in their own words their personal opinion of their experience with the robotic simulator in the virtual space. The recorded interviews where transcribed, encoded and compared within and between cases. In the end, the research team collaborated in order to consensually identify the salient arising issues.

5 Findings The students’ answers indicated that they were excited with and impressed by immersive teaching scenarios and that they didn’t face difficulties in the actual use of the offered modalities –which would hinder the learning process. In the interviews, students insisted that these methods are more effective and more interesting that traditional teaching and claimed that their understanding of the provided knowledge was increased – a claim supported by the analysis of the post- knowledge tests. Students also claimed that as a side-benefit they acquired knowledge outside of the central scope of our intervention, such as that their home country’s size in relevance to the planet and the celestial neighborhood.

978

5.1

S. Xefteris et al.

Pre-post Test Results

Both pre- and post-result samples were tested for normality by application of the ShapiroWilk test and both variables were found following the normal distribution (p > 0.005). Each answer was assigned a zero (0) if it was false and a one (1) if it was correct. After calculating each students’ mean performance score, we applied a paired samples t-test on the two variables to test if there was a significant statistical difference among the two scores. As seen in Table 1 our intervention allowed students to show a highly improved performance after its end with significant difference (t = −6.674, p < 0.001). Hence, we conclude that this intervention had an actual and significant learning impact. Table 1. Pre- and post-test results Mean SD Pre-test 0.56 0.18 Post-test 0.83 0.11

5.2

Students’ Attitudes Towards the Environment

Students’ answers to the attitudes’ questionnaire (see Table 2) indicate that the proposed environment was highly successful in facilitating the students’ engagement with the astronomical learning subject. The students enjoyed their interactions and were highly activated and focused in achieving the set objectives. Most of the students supported that they would like to use similar environments often (M = 4.06, SD = .0.92), that the environment was easy to use (M = 4.43, SD = 1), that they were totally focused in the required tasks (M = 4.43, SD = .89) and that their experience was enormously satisfying (M = 4.68, SD = .60). Moreover, as also seen in Table 2, students provided positive feedback regarding the learning efficiency of the environment. Students indicated that with this approach they would learn more and better than at school (M = 4.43, SD = .0.71) and in shorter time (M = 4.5, SD = .89). Answers also show that students preferred using similar environments at school too (M = 4.18, SD = .98). Students, as expected, were highly attracted to an educational approach that provides immersive and rich learning experiences. Moreover, in this case, students didn’t exhibit strong insecurities concerning the use of an environment they are not familiar with, being motivated more by curiosity. Table 2. Attitudes questionnaire answers Easiness Focus Enjoyment Perceived learning Pragmatic quality Hedonic identity Hedonic stimulation

Minimum 2.33 3.00 3.50 3.33 3.75 3.00 3.00

Maximum 5.00 5.00 5.00 5.00 5.00 5.00 5.00

Mean 4.29 4.54 4.81 4.43 4.65 4.27 4.70

SD .80 .64 .40 .60 .51 .78 .55

Cronbach’s a 0.77 0.85 0.90 0.80 0.94 0.94 0.70

Educational Robotics for Creating “Tangible Simulations”

979

From the AttrakDiff questions, it is apparent that the students considered the environment provided them with the appropriate functions for understanding the presented material (pragmatic quality). The hedonic quality variable, measuring acquired pleasure (originality, engaging and fun experience) and avoidance of boredom had high values. Students also indicated that they identified with the environment (Hedonic Quality-Identity variable) and perceived the experience as inspiring and novel (Hedonic Quality-Stimulation). The Hedonic Quality variables scores indicate that such immersive digital simulation environments integrating a gamified learning context can greatly facilitate the development of empathy for the learning process among students. 5.3

Interview Results

The observations already presented were also validated after the analysis of the students’ interviews. The students’ answers indicated that the proposed educational space can transform the student to an active part of the learning process. The students underlined their enjoyment, stressing out that the environment was very entertaining and effective, gaining their interest with its creative technological elements but also by activating them in exploring and trying to solve problems that might be less interesting in the normal context of a classroom. The proposed activities and interaction style, coupled with the live-experiment aspect of the environment offered a playful and attractive experience. Mixing multiple technologies addressed the student’s technological expectations offering a vivid and authentic learning environment, where technology, while ubiquitous, is not dominating the foreground. “It was interesting and it was something we cannot always do, very special…”. “It seemed very interesting to me (the intervention). It could easily attract our interest and was very creative and it was not boring, and I think that this pedagogical method is very good to teach a child.”

Regarding the comparison of the intervention to the traditional teaching methods, most students argued that the new proposal is better, more effective and that they would prefer their school lessons to be like it. “It would be more fun, you would learn more and would be interesting.” “I do not like to just open a book and learn something by heart, because you learn more things when you see it and you understand it (the learning object) than when you read it.” “It is more fun this way because we use robots and computers… We do not use robots at school”.

Most children were impressed by the use of Educational Robotics in a teaching intervention and enjoyed programming the robotic simulator. In addition, the findings showed that students did not appear to have significant difficulties with any aspect of the intervention. “[I was impressed by] Everything. The Lego Robot, by programming a little more” “The [task concerning the] degrees of the Earth’s axis were difficult to grasp. We haven’t talked about it at school much.” “I liked the man [Lego figure] we put on the Earth to demonstrate the directions (cardinal points of the horizon). I was a little bit troubled by the [pre-test] questions at the beginning.”

980

S. Xefteris et al.

Finally, the students thought that they learnt a variety of information that they did not know and addressed some important questions they had formed earlier and which might have been wrongly formulated in their minds. “I thought Greece was much bigger than I saw on the Earth model, in relation to the rest [of the planet]. It is like the other countries around, but now I have confirmed it and told myself that it is not as big as I thought before.” “I learned that the Earth spins around herself.” “I learned about the Earth’s axis tilt” “I thought that in the North pole it is 12 months of day and in the South Pole 12 months of night and now I clarified it in my head and know what is right”

6 Conclusions Overall, in this study we aimed to assess if and how a tangible simulation-mixed reality context can become a fruitful canvas for learning about astronomical phenomena. Although offering new perspectives in a short time is a challenging task, students answers indicated that tangible interactions, educational robotics and mixed reality can positively affect their performance. Spatial relations in the universe and how they affect localized perception of phenomena is a complicated task at young ages, but considering the resulting outcomes, we assess that a game with tangible simulations can become a solid background for conveying effectively new knowledge and re-assessing old knowledge. Of course, pilot studies with a wider user base will be needed for validating our research hypotheses. Using low cost hardware and by mixing reality with virtual objects and robots, seems an interesting new trend for learning technology. Researchers, teachers and students can create and replicate such environments with easiness and effectiveness.

References 1. Özsoy, S.: Is the earth flat or round? Primary school children’s understandings of the planet earth: the case of Turkish children. Int. Electron. J. Elem. Educ. 4(2), 407–415 (2017) 2. Turkmen, H.: After almost half-century landing on the moon and still countering basic astronomy conceptions. Eur. J. Phys. Educ. 6(2), 1–17 (2017) 3. Chalkia, K.: The solar system in the universe. Heraklion (2006) 4. Department for Education and Skills: Handbook for Secondary Teachers in England - The National Curriculum (2004) 5. National Science Education Standards. National Academies Press, Washington, D.C. (1996) 6. Hannust, T., Kikas, E.: Young children’s acquisition of knowledge about the earth: a longitudinal study. J. Exp. Child Psychol. 107(2), 164–180 (2010) 7. Ravanis, K., Christidou, V., Hatzinikita, V.: Enhancing conceptual change in preschool children’s representations of light: a sociocognitive approach. Res. Sci. Educ. 43(6), 2257– 2276 (2013) 8. Sun, K.-T., Lin, C.-L., Wang, S.-M.: A 3D virtual reality model of the sun and the moon for e-learning at elementary schools. Int. J. Sci. Math. Educ. 8(4), 689–710 (2010)

Educational Robotics for Creating “Tangible Simulations”

981

9. Fleck, S., Hachet, M., Bastien, J.M.C.: Marker-based augmented reality. In: Proceedings of the 14th International Conference on Interaction Design and Children - IDC 2015, pp. 21–28 (2015) 10. Chen, C.-C., Lin, P.-H.: Development and evaluation of a context-aware ubiquitous learning environment for astronomy education. Interact. Learn. Environ. 24(3), 644–661 (2016) 11. Bonito, T., Almeida, A.: The role of ICT to change misconceptions of some astronomy concepts in children of primary school. In: Electronic Proceedings of the ESERA 2015 Conference. Science Education Research: Engaging Learners for a Sustainable Future, Part/Strand (2016) 12. Xefteris, S., Palaigeorgiou, G.: Mixing educational robotics, tangibles and mixed reality environments for the interdisciplinary learning of geography and history. Int. J. Eng. Pedagog. 9(2), 82–98 (2019) 13. Eguchi, A.: RoboCupJunior for promoting STEM education, 21st century skills, and technological advancement through robotics competition. Rob. Auton. Syst. 75, 692–699 (2016) 14. Anderson, M.L.: Embodied cognition: a field guide. Artif. Intell. 149(1), 91–130 (2003) 15. Mahon, B.Z., Caramazza, A.: A critical look at the embodied cognition hypothesis and a new proposal for grounding conceptual content. J. Physiol. 102(1–3), 59–70 (2008) 16. Lindgren, R., Tscholl, M., Wang, S., Johnson, E.: Enhancing learning and engagement through embodied interaction within a mixed reality simulation. Comput. Educ. 95, 174–187 (2016) 17. Palaigeorgiou, G., Tsapkini, D., Bratitsis, T., Xefteris, S.: Embodied learning about time with tangible clocks, pp. 477–486 (2018) 18. Chiras, A.: Day/night cycle: mental models of primary school children. Sci. Educ. Int. 19(1), 65–83 (2008) 19. Sharp, J.G., Kuerbis, P.: Children’s ideas about the solar system and the chaos in learning science. Sci. Educ. 90(1), 124–147 (2006) 20. Xakjiά, K.: The Solar System Within the Universe: The Path from Scientific Knowledge to School Knowledge. Crete University Press, Heraklion (2006) 21. Isik-Ercan, Z., Kim, B., Nowak, J.: 3D visualization in elementary education astronomy: teaching urban second graders about the sun, earth, and moon. In: World Summit on Knowledge Society, pp. 500–505 (2012) 22. Schneps, M.H., Ruel, J., Sonnert, G., Dussault, M., Griffin, M., Sadler, P.M.: Conceptualizing astronomical scale: virtual simulations on handheld tablet computers reverse misconceptions. Comput. Educ. 70, 269–280 (2014) 23. Antoniou, P.E., Mpaka, M., Dratsiou, I., Aggeioplasti, K., Tsitouridou, M., Bamidis, P.D.: Scoping the window to the universe; design considerations and expert evaluation of an augmented reality mobile application for astronomy education, pp. 409–420. Springer, Cham (2018) 24. Khanlari, A.: Teachers’ perceptions of the benefits and the challenges of integrating educational robots into primary/elementary curricula. Eur. J. Eng. Educ. 41(3), 320–330 (2016) 25. Stoppa, M.: Study of Galileo’s plane using educational robotics. Int. J. Innov. Educ. Res. 6(10), 226–241 (2018) 26. Souza, M.A.M., Duarte, J.R.R.: Low-cost educational robotics applied to physics teaching in Brazil. Phys. Educ. 50(4), 482–488 (2015) 27. Christofi, N., Talevi, M., Holt, J., Wormnes, K., Paraskevas, I., Papadopoulos, E.: Orbital robotics: a new frontier in education. In: 6th International Conference on Robotics in Education (2015)

982

S. Xefteris et al.

28. LEGO MOC-4477 Earth, Moon and Sun Orrery (Technic 2016) | Rebrickable-Build with LEGO (2016). https://rebrickable.com/mocs/MOC-4477/JKBrickworks/earth-moon-andsun-orrery/#comments. Accessed 07 July 2019 29. Palaigeorgiou, G., Malandrakis, G., Tsolopani, C.: Learning with drones: flying windows for classroom virtual field trips. In: 2017 IEEE 17th International Conference on Advanced Learning Technologies (ICALT), pp. 338–342 (2017) 30. Honig, W., Milanes, C., Scaria, L., Phan, T., Bolas, M., Ayanian, N.: Mixed reality for robotics. In: IEEE International Conference on Intelligent Robots and Systems, pp. 5382– 5387, December 2015 31. Hassenzahl, M., Monk, A.: The inference of perceived usability from beauty. Hum. Comput. Interact. 25(3), 235–260 (2010) 32. Jackson, S.A., Marsh, H.W.: Development and validation of a scale to measure optimal experience: the flow state scale. J. Sport Exerc. Psychol. 18(1), 17–35 (1996)

TimeTracker App: Facilitating Migrants’ Engagement in Their Second Language Learning Olga Viberg1(&), Mohammad Khalil2, and Gustav Bergman1 1

KTH Royal Institute of Technology, Stockholm, Sweden {oviberg,gubergman}@kth.se 2 University of Bergen, Bergen, Norway [email protected]

Abstract. With globally increased migration and mobility across countries, it has become critical for many people to acquire a language of the host country. This study focuses on the case of adult migrant workers who are i) enrolled in second language education and ii) faced to a challenge of balancing between their professional and private lives and their studies of the target second language, needed for their successful integration into the host society. This explorative case study aims to support migrant learners in their second language acquisition across learning settings. It introduces and evaluates the use of specifically designed tool, TimeTracker, aimed at helping learners to keep track on how much time they spend on studying the target language influences their engagement and motivation to continue to study the host language. Eight participants from Sweden and Norway used the tool during the period of one to two weeks. Overall, the results show that using TimeTracker to log and keep track of the time spent on studying a host language inspired the learners to i) allocate more time on their second language learning, and ii) become more engaged in it through raising awareness and monitoring of their learning activities. Keywords: Second language learning
Engagement
Motivation
Time management
Learning analytics
Mobile learning
Migrant learners

1 Introduction Migrants’ ability and skills to communicate in the language(s) of a host society is crucial for their successful integration into it [1]. Their motivations and circumstances for studying a host second language differ. When arriving in a new country, some of them study the target second language full time (e.g., newly arrived asylum seeking adult migrants in Sweden or Norway), whereas others (e.g., migrant workers) have to study the host language in parallel with their full-time work. In this study, we target the latter mentioned group of migrant learners who have to balance between their full-time work, their private life and the study of the host language. Balancing between work and learning a second language is a highly challenging task for learners [2], suggesting the learners focused in this study often have a limited time to devote to studying the host language and largely need to do it in their spare time. For example, in these ‘spare’ time © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 983–994, 2021. https://doi.org/10.1007/978-3-030-49932-7_91

984

O. Viberg et al.

slots, migrants can: i) study a course textbook while commuting to or from a workplace, ii) use a language learning app for e.g., vocabulary acquisition during a break at work, or iii) watch movies in the targeted language at night. These are some learning activities but set in different contexts and on different platforms. With these, often timely limited, study sessions available, it becomes challenging for learners to understand how much time they allocate to their language learning. This is important since “the amount of time students actually spent on learning has been identified as one of the central constructs affecting learning success” [3, p. 82]. Hence, time spent on various learning tasks is recognized as one of the key characteristics that constitutes learner engagement. Learner engagement refers to the level of involvement with a learning activity [4] and is a multidimensional construct consisting of several qualities of students’ participation, ranging from enthusiastic, focused, and an emotionally positive attitude towards a learning activity to a complete lack of interest [5]. Considering the time spent on learning activities as one of the key constructs of learner engagement, it becomes critical for learners to keep track of the time spent on language learning activities. This study aims at supporting migrant learners in this task. To help individuals track their engagement with the language learning activities, there are two key aspects to be considered: i) tracking the time spent on learning activities has to be measured in some way, and ii) the user data collected should be presented back to the students to support them in monitoring and reflecting upon their own learning process which would facilitate improvements in their learning outcomes. However, since learners continuously move between formal and informal learning contexts and often use different kinds of technologies, including both hardware and software for their language learning, valuing and measuring time spent on their multifaceted language learning activities becomes a challenge. Taking into account all the scattered learning moments that a migrant learner takes part of throughout the day, a solution for collecting data about students’ time spent on language learning activities is an approach that can be further developed to enable an easy access to key moments throughout the second language learning. Collecting those moments from the server backend side and analyse them to enhance the learning experience is called Learning Analytics (LA). Applying LA methods to analyse learners’ data is a pivotal asset in digital learning systems. One of the key objectives of LA is to visualize students’ stored actions via user/learning dashboards that support learner awareness, reflection and sense-making [6]. Such dashboards can be accessed through various technologies. A technology available independently of location is a mobile device that is “probably the only artifact co-existing with the learner in all scattered learning moments and learning contexts throughout the day” [2]. Learners’ own mobile devices can be used to both log and receive information about time spent on learning activities. A way to present learner’s data back to her/him is to visualise time devoted to language learning across learning settings. This can help learners to develop their time management strategies, skills and knowledge needed for their learning success [7] in their second language acquisition (SLA). Overall, a considerable body of research on mobileassisted language learning testifies its efficacy in a wide range of contexts [e.g., 8, 9]. Still, there is a lack of studies pertaining to the specific needs and capabilities [8] of migrant learners, and this research study aims to fill this gap.

TimeTracker App: Facilitating Migrants’ Engagement

985

With the overall goal to facilitate migrant learners’ engagement in their language learning, focusing on the understanding of- and visualising time spent on language learning activities, this study aims to answer the following research questions: i. How can we support migrant learners’ engagement in their second language learning across learning settings? ii. How does a specifically designed tool aimed at aiding migrant learners to keep track of the time they spend on studying the target language influence their engagement in their second language learning?

2 Background 2.1

Engagement in Second Language Acquisition

Engagement and motivation are two key factors that characterise good language learners: while motivation is about why one would put effort into something, engagement is about to engage someone in action and stay in it as well as it is more about attentiveness or willingness to participate in an activity [10]. Disengaged behaviour has been found to have a negative effect on individual’s learning [11]. Among factors that influence learner engagement, self-assessment (i.e., when a learner compares her progress with herself and monitors it) has been emphasized to be critical for one’s learning progress [12]. Earlier research offers several relevant suggestions for facilitating learner engagement, including the provision of challenging materials for students and encouraging interaction between them [see e.g., 11, 13–15]. When it comes to SLA, scholars stress that language students wish: i) to be engaged in interactive classroom activities, ii) to have a positive learning environment with friendly classmates and teachers, and iii) to engage in learning activities connected to real life tasks (e.g., talking with native speakers outside the class) [10]. Yet, most of the suggested so far solutions for facilitating language students’ engagement refer largely to classroom settings as opposed to less formal learning settings, in which migrants often engage in their studies of a host second language/s. This study thus focuses on helping migrant learners to facilitate their engagement to study the target language across formal and informal learning settings. 2.2

Time Management

The concept of time measurement in educational research has been used for a long time. Already in 1963, Carrol proposed a learning model centred around time, where learning was defined as a function dependent on the effort spent related to the effort needed [16]. Since time effectively spent on learning and total time passed is not always equal, she separated these two. Stallings emphasized that student learning is dependent on how time is used, not how much time is allocated [17]. Kuh stresses that the time students spend preparing outside of class is typically half of what is expected [18]. Additionally, he highlights: “without knowing how students spend their time, it’s almost impossible to link student learning outcomes to the educational activities and processes associated with them” [p. 15]. Others investigated the relationship between systematic observation of time-on-task and student engagement and found correlations

986

O. Viberg et al.

between time on task and students’ self-reported engagement [19]. One of the challenges with measuring the aspects of time-on-task in learning is that while students might study for a period of time, they might still be distracted throughout that period. A more recent research has shown that given the easy access to technology, learning today is plagued by high levels of distraction and multitasking, which leads to negative effects on students’ learning [20]. Another challenge regards the fact language learners study across learning environments and use different resources for it. A way to enable students to gain insights about their studying efforts is to visualize the time-on-task data generated by the logs [7], deriving from the use of technologies-in-use. 2.3

Learning Analytics and Visualizations

Learning analytics (LA) is defined as “the measurement, collection, analysis and reporting of data about learners and their contexts for purposes of understanding and optimizing learning and the environments in which it occurs” [21]. LA is driven by collection and analysis of data, commonly traces of learning behaviour [22]. LA provides a means of analysing the learner’s behaviour to gain new insights about teaching, learning and decision making. LA tools are used to help understand and optimize the learning process. Scholars [23] used LA to investigate whether time-ontask and the number of learning sessions had an impact on completing Massive Open Online Courses, and found the time dedicated to online tasks matters. The study uncovers not all students with shorter and larger number of learning sessions lead to study success, but the time spent on tasks varies and depends on the learning context. One of the key elements of LA is the information presented back to teachers and/or students in order to optimize their teaching activities and/or learning behaviour [21]. This can be achieved by developing learning dashboards displaying the relevant information back to the user. Learning dashboards visualizing self-regulated learning processes, including time management skills, have proved valuable [24]. Visualizations of time-on-task were also found to improve learner awareness and self-reflection [25]. Santos et al., developed a visualization tool for activity tracking to enable students’ self-reflection on their learning activities and comparison with peers [2]. The Student Activity Meter is another example of visualization tools aimed at students for tracking their learning progress [26]; it uses time-on-task together with such variables as resource use and forum view and post actions to improve learners’ awareness of how they spend time on learning activities; students liked using the tool because of the insight and motivation it provided.

3 Methodology 3.1

Case Study Settings

This study focuses on adult migrant learners participating in second language education. The target group of migrants were employees who have been living in Sweden and Norway for up to two years. Organizations in which learners work facilitate their integration into the local societies by encouraging them to enroll in second language

TimeTracker App: Facilitating Migrants’ Engagement

987

education and by offering them language courses - often much limited in time - to be taken during their working hours. Participants from Sweden enrolled in two hours’ Swedish lessons twice a week. Participants from Norway enrolled in three hours Norwegian lessons similarly twice a week. All the lessons are sessioned in a face-toface instruction where a teacher leads the class. To pass the courses, students are expected to study outside the classes. A total of four working migrants studying Swedish and four studying Norwegian participated in this study for one to two weeks. To facilitate this study, we asked the participants to log their time spent on language learning activities outside of the classroom in a web-based mobile application designed for the purposes of this study, called the TimeTracker App. 3.2

Design of the TimeTracker App

Responsive web app [7] was designed to carry out this study. We aimed at allowing learners to log the time spent on their language learning activities from anywhere and from any device in a simple process. TimeTracker has two main screens, time log screen (see Fig. 1), and dashboard screen (see Fig. 2). The app back-end core was built using PHP, CSS, HTML and JavaScript, and it features account creation together with login/logout functionality to separate each user. It uses a MySQL database to store data. Charts and visualizations are built using the JavaScript library Chart.js. All the plots in TimeTracker are tended to be interactive making it easier and more dynamic for students to demonstrate different inputs.

Fig. 1. TimeTracker log screen

Fig. 2. TimeTracker dashboard screen

Fig. 3. TimeTracker dashboard with two plots

Time Log Screen. The aim of a learning scenario presented at the time log screen (Fig. 1) is to enable students to log time spent on different language learning activities. The screen features a form where students could choose one of the LSRW skills (Listening, Speaking, Reading, and Writing) and ‘other’ (i.e., activities that they could

988

O. Viberg et al.

not categorise according to the LSRW scale), a name of a learning activity, a date (prefilled to current date), and time in minutes spent on one activity. When tapping the ‘Log’ button at the bottom of the screen, the log is stored into the database and the user will be directed redirected to the now updated home. Dashboard Screen. The dashboard screen features a panel with two types of visualizations. The first one is a pie chart exhibiting how much time one spent on learning activities with each segment representing one category (Fig. 2). The second visualization represents a bar chart (Fig. 3) displaying two interactive bars: a) how much time a student logs in one week, and b) a daily pre-set learning goal. 3.3

Instruction and Setup

At the start of the study, the participants were informed that the study’s purpose was to help them to keep track of time spent on language learning activities outside of the scheduled weekly learning sessions. They were then introduced to the TimeTracker and instructed how to sign up and use it. They used the app over a period of one to two weeks and could access it either through their own mobile devices or laptops/desktops. 3.4

Data Collection

To understand how the participants used TimeTracker and how it affected their engagement and motivation for studying Swedish and Norwegian, eight semistructured interviews were conducted. The interviews were conducted online using either Skype or Zoom software. All interviews were recorded with permission from the interviewees; followed a protocol and took about 15–20 min. The purpose of the interview was to understand: i) what studying habits the participants had before using TimeTracker, ii) how the app-in-use affected their engagement and motivation to continue studying the host language and iii) what their attitudes towards the use of the application for their SLA are. To collect user data from the app, quantitative data of user logs were exported from the app’s database in CSV format. 3.5

Data Analysis

The data exported from the app’s database were processed using Python. The logs were grouped by each user in order to extract values such as number of logs, popularity of categories, total minutes logged per user and average time spent per log per user. All the interviews were carried out by two authors and participants’ verbal consent was confirmed. Every interview was transcribed and analysed using a conventional content analysis [27], which is often used when trying to describe a phenomenon. It revolves around the researcher immersing themselves in the data, finding codes in the data and allowing new insights to emerge [27]. In this study, the interview transcripts were read repeatedly in order to achieve immersion and to obtain a sense of the whole. They were then read word by word and those describing key codes were highlighted. The authors then made notes of their first thoughts and initial analysis. The codes were then sorted into categories. Lastly, descriptions for each category were developed.

TimeTracker App: Facilitating Migrants’ Engagement

989

The method of this study has two limitations: i) the presented results are based on the data generated and reported by eight migrant learners, and ii) the study has been carried out over a short period of time. To be able to validate this study’s findings, a further research that would involve more participants using the tool for a longer is needed. This would minimize the risk of the participants having for example, a very busy week, which could influence the results.

4 Results and Discussion 4.1

TimeTracker-in-Use: Usage Data

We used LA log files in a simple and direct way. In total, the participants registered a total of 49 logs over the duration of the study. We compared the user-input log files with the timestamp of the logs to understand whether the users used TimeTracker after hand or directly. It was quite clear that most of the participants logged their studying times directly. With respect to the most popular LSRW category, the listening and “other” at 11 logs, followed by writing at 10 log, then speaking at 9 logs, and finally reading at 8 logs. Descriptive statistics were collected (see Table 1). The total time logged by the eight participants was 2444 min. The mean is 49.87 min and the median is 22.50 min. Respondent 5 recorded the highest number of logged minutes with 1270 min and 11 logs. Respondent 6 was noted to record the lowest logged minutes with only 1 log and 10 min in total. Table 1. TimeTracker usage data # of users # of logs Total logged (mins) Mean (mins) Median (mins) 8 49 2444 49.87 22.5

4.2

Participants’ Pre-study Language Learning Practices

The majority of the participants stressed that it is hard to balance between their professional and private lives and learning of the host second language. R1 argued it was hard to motivate himself to study the host language after coming home from a long working day. Similarly, R5 emphasized: “Because I have a lot of duties, I have a family and a lot of work, so learning a new language is challenging to me. It’s, of course, difficult because I am working, it’s really hard.” As a result of not having that much time to study the host language due to work and private life, most of the respondents found it hard to estimate how much time they dedicate to their language learning outside the classroom. The findings show that three participants (R1, R7 & R8) used to plan a set of time to study each day. For instance, Respondent 1 tried to spend at least 20 min per day at her language learning, but at the same time, reported it was hard to find time to study every day. Another one (R8) stated that she aims to dedicate 10–20 min a day to her second language learning. Also, the participants expressed that they rarely set up their

990

O. Viberg et al.

own daily language learning goals. Respondent 4, for example stated that, “as a daily learning goal, I didn’t really set one because I didn’t know how much time I could dedicate outside of the classroom. But I planned to do some studying outside of my work time. I usually set up one hour or one and a half if I could at the evening time”. R6 explained that she used to do it before, but was “too busy now,” suggesting that there is a lack of consistency in her language learning daily goal setting. However, one R7 stressed that he used to set his daily language learning goals. None of the participants actively logged any of their second language learning activities before the study. Respondents 3 and 4 sometimes used language learning apps (e.g., Duolingo) that keep track of the learning activities they perform within them; yet, none of them considered the automatic logs generated by those apps. 4.3

TimeTracker-in-Use: Opportunities

The participants’ attitudes towards TimeTracker before using it in the study were overall positive; when introduced to it all of them liked the concept of logging and keeping track of the time spent on studying Swedish or Norwegian. When using the app the respondents 2, 4 and 6 chose to use their computers and the rest - their smartphones. The desktop users explained they already used their computer for work, so it was easier for them to just use the computer. The majority of the respondents perceived the app easy to use; it was easy to keep track of how much time they spent on their language learning activities. When asked if they logged the activities directly in connection with the relevant study sessions, 60% of the learners responded that they tried to log them directly. Still, 40% found it hard to remember to log time directly, and instead added the time studied later. R5 logged all the time after one or two days of studying. R4 logged both directly and indirectly: “One time I did it right after, and other times I logged some things that I did it after, because I had not studied that much.” The findings also show that few participants used the app feature to set a daily learning goal. There were two key reasons for that. The first relates to the fact that 60% did not know that the functionality existed in the app, and hence did not use it. Only three respondents (R3, 4, & R6) knew the feature existed. The other relates to the fact that they did not have a daily goal set, often because they did not have time to study every day. Respondent 3 argued it would be better to set a learning goal per week instead, since a weekly goal would be easier to plan ahead for a busy schedule. The results reveal all the respondents became more aware of how much time they spent on their language studies compared to before using TimeTracker. R5 stated: “Yes, when I studied without logging my studied time it was hard for me to realize how much time was spent,” whereas R1 stressed she became more aware of how little time she spent on her language learning when using the app. All the respondents stated the appin-use encouraged them to spend more time on their language learning studies. R1 emphasized it was “very useful” to log the time because she became “far more motivated to study” when doing so; R2 argued that he used the app through a home screen of his smartphone, which remind him to study once he started to use her device for other purposes. Also, R4 stressed, “the app-in-use makes you more inspired to spend time because there is something that is keeping track. Maybe you think ‘yeah I’ve

TimeTracker App: Facilitating Migrants’ Engagement

991

studied’ but when you look back, maybe you could have studied more, for example when riding the train or something.” The use of the setting a daily-learning-goal scenario was found to be motivating for facilitating participants’ learning: “Yes. It motivated me since I can see how many minutes, I need to split my time across all the activities” (R6). The integrated learning dashboard in TimeTracker helped the learners to visualise their language learning process (e.g., R3, R5, R7 & R8). As stated by R5, “when I looked at the visualizations I realized maybe I had to spend more time on listening or speaking. So yeah, it made me think I may need to spend extra time on a particular language learning activities [e.g., speaking]”. In summary, the majority of the participants expressed that they would like to continue using TimeTracker in their future institutionalised second language studies as it facilitates their engagement in the target language learning. 4.4

TimeTracker-in-Use: Challenges

Several limitations of the TimeTracker app have been stressed by the participants. The most common one concerns the app’s time logging functionality. In particular, it relates to the categories users could choose when logging their time spent on various language learning activities. 40% of the participants expressed the difficulty to find a suitable category for their logs. R1 used for example, other language learning software to study and felt the option “other” was a little bit too broad. R2 remarked on learning activities spanning across multiple categories, which made him confused, whereas R5 studied grammar a lot, preferring this category to be offered separately. Moreover, R6 stated: “There is one little confusing issue which is about the activity part and also speaking or listening, because we can combine both of these options and currently in the app we need to pick only one option”. Yet, since every participant used the app differently, finding relevant categories that will suit everyone may be a challenging task. This could be achieved by offering different types of categorizations. One type would focus on the key language learning skills. Another one - on how learners have studied, e.g., if they’ve used language learning software, books or other resources. Adding different types of categories would also open up for more visualizations (e.g., a new chart illustrating the distribution of logs over the new type of categories) with the aim to further i) increase learner awareness and ii) to facilitate their engagement. Another limitation regards the offered time slots options in the app. R4 argued the overview showing either only one week or all weeks at the same time was limiting since being able to see e.g., two weeks at the same time would allow for some easier comparison between the weeks. Designing for comparison between different weeks might improve the learners’ sense of progression. To achieve this, users should be able to compare their learning activities across different weeks side by side. This might help with self-assessment, as the learner can use her past weeks as an indication for how well s/he is doing in terms of time spent studying. Another design limitation relates to the issue of navigation in the app. R7 found setting a daily learning goal tricky to discover, whereas R8 needed some time to understand how the app works: “the first time usage was challenging. Honestly I was totally unaware about what I have to do and why and how.” It is thus suggested to have a brief tutorial for new users.

992

O. Viberg et al.

5 Implications and Conclusion The overall goal of this study was to facilitate migrant learners’ engagement in their language learning, focusing on the understanding of- and visualising time spent on language learning activities. 5.1

Increased Engagement

The results suggest that migrant learners become more motivated and engaged in their second language studies when they use TimeTracker. Since they all had very busy schedules and not that much time to plan their studies in advance, it was hard for them to keep track of the time spent on language learning. The use of TimeTracker raised their awareness of how much time is spent outside the classes; in particular, on the time spent on exercising different language skills (i.e., LSRW). This matches with earlier findings [26], showing that visualizations of several variables, including time-on-task increases learner motivation. Some parallels exist between this study’s findings and the correlations between time-on-task and self-reported engagement found by [19]. Even if only a few respondents set their daily learning goals, most of them stressed that seeing how little time they spent on studying the host language inspired them to allocate more time, and to become more motivated and engaged in their language learning. 5.2

Design Implications

Since the learners need to log their time spent on language activities manually and they are very busy, any extra obstacle might hinder them to use the app. The results of both the qualitative and quantitative data analysis show that the respondents used TimeTracker differently, indicating that the tool has to allow for each learner to use it in a personalised way to be usable enough for facilitating one’s engagement in her/his second language learning. Despite the fact that the majority of the participants found TimeTracker an easy to use application, several design implications that would improve the usability aspect of this or similar tools have to be considered. Such tools need to: • allow learners several alternative options to select and categorise language learning activities • provide an option to compare the time logged in the past days, weeks and months and be reflected on the user dashboard • offer more options for setting a learning goal task, including daily, weekly and even longer terms language learning goals to be able to reflect upon student learning progress • develop and integrate push notifications that would remind learners to log their learning activities • integrate a short user tutorial for beginners • improve the responsive design of the application for mobiles and desktop computers.

TimeTracker App: Facilitating Migrants’ Engagement

993

This study contributes to both the fields of practice and research. For practitioners, the results provide valuable insights into how one can facilitate migrant learners’ engagement in their second language studies by offering specific tools that support them in visualising time spent on different language learning tasks. For researchers, the findings offer knowledge in terms of how we can employ students’ own technologies in combination with LA - with a focus on measuring time spent on language learning activities - and relevant visualisation to support their SLA. All in all, the results show that using a tool to log and keep track of the time spent on studying a host language outside classroom inspired migrant learners to i) allocate more time on their second language learning, and ii) become more engaged in it through raising awareness and monitoring of their learning activities. The use of visualisations helped the learners to keep track of which areas of language learning they devoted more or less time to, thus, increasing their awareness of the possible learning gaps to be filled.

References 1. Dervin, F.: Exploring ‘new’ interculturality online. Lang. Intercult. Commun. 14(2), 191– 206 (2014) 2. Tabuenca, B., Kalz, M., Drachsler, H., Specht, M.: Time will tell: the role of mobile learning analytics in self-regulated learning. Comput. Educ. 89, 53–74 (2015) 3. Kovanovic, K., Gaševic, V., Dawson, D., Joksimovic, S., Baker, R., Hatala, S.: Does timeon-task estimation matter? Implications for the validity of learning analytics findings. J. Learn. Anal. 2(3), 81–116 (2015) 4. Robinson, C.: Student engagement. J. Appl. Res. High. Educ. 4(2), 94–108 (2012) 5. Skinner, E., Kindermann, T., Furrer, C.: A motivational perspective on engagement and disaffection. Educ. Psychol. Measur. 69(3), 493–525 (2009) 6. Khalil, M., Taraghi, B., Ebner, M.: Engaging learning analytics in MOOCS: the good, the bad, and the ugly. In: Proceedings of the International Conference on Education and New Developments, Ljubljana, Slovenia, pp. 3–7 (2016) 7. Verbert, K., Govaerts, S., Duval, E., Santos, J., Van Assche, F., Parra, G., Klerkx, J.: Learning dashboards: an overview and future research opportunities. Pers. Ubiquit. Comput. 18(6), 1499–1514 (2013) 8. Kukulska-Hulme, A., Viberg, O.: Mobile collaborative language learning: state of the art. Br. J. Edu. Technol. 49(2), 207–218 (2018) 9. Viberg, O., Grönlund, Å.: Cross-cultural analysis of users’ attitudes toward the use of mobile devices in second and foreign language learning in higher education: a case from Sweden and China. Comput. Educ. 69, 169–180 (2013) 10. Bass-Dolivan, D.: Students’ engagement with second language learning: a sociocultural approach. University of Wollongong Thesis Collection, 1954–2016 (2011). https://ro.uow. edu.au/theses/3357 11. Vermeulen, A.: Improving student engagement through visualization of course activities. A Doctoral Dissertation (2014) 12. Cocea, M., Weibelzahl, S.: Can log files analysis estimate learners’ level of motivation? In: Proceedings of the 12th Workshop on Knowledge Discovery, Data Mining, and Machine Learning (2006) 13. Meece, J., Anderman, E., Anderman, L.: Classroom goal structure, student motivation, and academic achievement. Annu. Rev. Psychol. 57(1), 487–503 (2006)

994

O. Viberg et al.

14. Said, S., Al-Homoud, M.: Increasing student motivation. In: Proceedings of the Discussion Form on Faculty and Student Motivation (2004) 15. Zepke, N., Leach, L.: Improving student engagement: ten proposals for action. Act. Learn. High. Educ. 11(3), 167–177 (2010) 16. Carroll, J.: A model of school learning. Teachers Coll. Rec. 64(8), 723–733 (1963) 17. Stallings, J.: Allocated academic learning time revisited, or beyond time on task. Educ. Res. 9(11), 11–16 (1980) 18. Kuh, J.: Assessing what really matters to student learning - inside the national survey of student engagement. Change Mag. High. Learn. 33(3), 10–17 (2001) 19. Spanjers, D., Burns, M., Wagner, A.: Systematic direct observation of time on task as a measure of student engagement. Assess. Effective Interv. 33(2), 120–126 (2008) 20. Bowman, B., Waite, B., Levine, L.: Multitasking and attention. In: The Wiley Handbook of Psychology, Technology, and Society, pp. 388–403 (2015) 21. Lester, J., Klein, C., Rangwala, H., Johri, A.: Learning analytics in higher education. ASHE High. Educ. Rep. 43(5), 9–135 (2017) 22. Greller, W., Drachsler, H.: Translating learning into numbers: a generic framework for learning analytics. Educ. Technol. Soc. 15(3), 42–57 (2012) 23. Khalil, M., Wong, J.: 2018 Learning with MOOCS (2018). https://doi.org/10.1109/lwmoocs. 2018.8534681 24. Sedrakyan, G., Malmberg, J., Verbert, K., Järvelä, S., Kirschner, P.: Linking learning behavior analytics and learning science concepts: designing a learning analytics dashboard for feedback to support learning regulation. Comput. Hum. Behav. (2018). https://doi.org/10. 1016/j.chb.2018.05.004 25. Govaerts, S., Verbert, K., Klerkx, J., Duval, E.: Visualizing Activities for Self-reflection and Awareness (2010). https://doi.org/10.1007/978-3-642-17407-0_10 26. Govaerts, S., Verbert, K., Duval, E., Pardo, A.: The student activity meter for awareness and self-reflection. In: Proceedings of the 2012 ACM Annual Conference Extended Abstracts on Human Factors in Computing Systems Extended Abstracts - CHI EA 2012, p. 869 (2012). https://doi.org/10.1145/2212776.2212860 27. Hsieh, H.-F., Shannon, S.: Three approaches to qualitative content analysis. Qual. Health Res. 15(9), 1277–1288 (2005)

“School – University – Industry” Cooperation “Cypress – National Instruments CLUB” Doru Ursutiu1(&), Cornel Samoila2, Patrick Kane3, Magdalena Ciurea4, Mircea Stremtan5, and Cristian Ravariu6 “Transilvania” University – AOSR, Brasov, Romania [email protected] 2 “Transilvania” University – ASTR, Brasov, Romania 3 Cypress Semiconductor University Alliance, Austin, USA 4 National Institute of Materials Physics, Măgurele, Romania 5 National Instruments Romania, Cluj-Napoca, Romania Faculty Electronics, University “Politechnica” of Bucharest, Bucharest, Romania 1

6

Abstract. In this paper we introduce a model of cooperation between High School-University-Industry sustained by activities and results obtained in “Cypress – National Instruments Club” started at “Transilvania” University of Brasov – Romania. This strategic collaboration between High SchoolUniversity-Industry was become necessary because technology is evolving at a rate that is quickly outpacing traditional engineering teaching methods. New methods such as hands-on learning labs, industry-centric curriculum, permanent updating of facilities, and using new adaptive learning software are being adopted. “Transilvania” University is targeting education at the industries of the future rather than industries of the past and for this reason we must apply the same strategies inside the high school in order to well prepare the children’s, future students and the new engineers. Keywords: LabVIEW


Programmable System-on-Chip (PSoC®)
Creativity

1 Introduction 1.1

Partners

Starting from 2000 in “Transilvania” University of Brasov Romania we started to observe the new tendencies in Electronic Engineering (EE) and we understand that the next 20 years will be dominated by mobile wireless technologies and we have guessed that Remote Engineering (RE) will play a dominant role. For these reasons we are collaborating with University of Applied Sciences Villach – Austria and discuss one strategic partnership to organize one international conference in this field. Because the both universities have well established collaborations with the common partner National Instruments (NI) from USA and we use in our laboratories LabVIEW (Laboratory Virtual Instrumentation Engineering Workbench) – one revolutionary software in connection with electronics but also well adapted to any © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 995–1001, 2021. https://doi.org/10.1007/978-3-030-49932-7_92

996

D. Ursutiu et al.

educational applications and development, we decided to use Virtual Instrumentation (VI) as the main software support for our Remote Engineering developments in Research and Education. These initial steps and discussions led to the creation of the Remote Engineering and Virtual Instrumentation (REV) conferences: the first one in 2004 in Austria and the second one in 2005 in Romania. From start of this series of conferences we put together the idea of a triangle of cooperation, Industry – Education – Development, and we started to invite partners from these fields and to cover all the new and modern developments and decided to do the conferences every year in a different country. From 2005 in Romania we decided to use NI ELVIS technology [1] together with LabVIEW [2] and we applied for a National Instruments academic grant. “Transilvania” University was in 2005 the second university in Europe who implemented and used still now this modern technology in education. Year by year we added new systems (myDAQ, myRIO, etc.) to be able to sustain our ideas connected to improve engineering education and better attract children to the fields of electronics, remote engineering, and virtual instrumentation. 1.2

Programmable System-on-Chip PSoC®

The Research and Education (R&E) group was founded in 1998 in the frame of TEMPUS-S-JEP-12536/97 – Project, CVTC “Centre for Valorization and Transfer of Competence”, with external founders: INSA – Lyon France, UPC – Barcelona Spain, TEUCHOS – Versailles France, “TRANSILVANIA” University Brasov – Romania “Lucian Blaga” University Sibiu – Romania. Inside CVTC we started a Creativity Laboratory (CL) intended especially for development and to increase the University and Industry collaboration with the ultimate goal of adapting our engineering education to align with the actual trends in electronics engineering market and to be more adaptable to the actual tendencies. In next steps we started to select for students and industrial applications done inside the Creativity Laboratory of the CVTC some flexible devices and with great future in new development, the embedded System-on-Chip (SoC). PSoC is a true programmable embedded SoC integrating configurable analog and digital peripheral functions, memory, and a microcontroller on a single chip [3]. With an extremely flexible visual embedded design methodology that includes preconfigured, user-defined peripherals and hierarchical schematic entry, you can easily program and reprogram it. The Cypress PSoC Creator software (graphical oriented similar to the National Instruments LabVIEW) was the best solution to assist us in Creativity Laboratory together with National Instruments technologies and software in all our future research and education activities. An important number of Master, PhD, and POSTDOC students succeeded finishing their thesis’ using this infrastructure. Currently, CVTC has a Thin Films and Nanosystems “Radu Grigorovich” Laboratory, a Noise and Fluctuation Laboratory, a CISCO Academy, and a LabVIEW Academy. In addition to all this initiatives in 2014, in cooperation with industry partners, we just launched the “CYPRESS – NATIONAL INSTRUMENTS CLUB (CNIC)”. In Fig. 1 some images at the opening

“School – University – Industry” Cooperation

997

of the CNIC club done in collaboration with Cypress University Alliances (CUA), Brasov High schools children’s and “Transilvania” University students.

Fig. 1. Launching with CUA the Cypress – National Instruments Club in “Transilvania” University of Brasov

2 Collaboration with High Schools Inside the CVTC “Cypress – National Instruments Club” we started to develop training activities and organize events with children from the most important High Schools from Brasov. We combined the activities of High School students with students from first two years of Applied Electronics and at the same time extended invitations to people from Brasov industry to teach them. We like to introduce some of most important activities developed in this environment and results of this triple interaction: High School – University – Industry.

(a)

(b)

(c)

Fig. 2. Club activities: (a) Club presentation to first year students, (b) High School students learn LabVIEW and (c) Workshop on VOLTERA printer for participant’s at ROBOTOR Contest

Last year we organized LabVIEW training for high school students and first year students. As you can see in Fig. 2. Every year first year students (Fig. 2a) come inside the club and receive information’s about all the new software and hardware technologies offered for their use inside the CNIC Club. After this initial contact they can work together with the high school students (Fig. 2b) in LabVIEW ACADEMY and to do training activities with

998

D. Ursutiu et al.

all the developments boards (Cypress, National Instruments, developed by other students with VOLTERA printer, etc.). At the level of University the CVTC Center tries to be involved also in external activities and here we present our involvement in organizing Robotor 2018/2019 Contest (http://robotor.ro). The organization of the Robotor 2018/2019 is supported by a group of local companies and institutions involved in the development of the skills and competences of the following generations: Miele, Siemens, Benchmark, Continental, Kronwell, Brasov Community Foundation, “Transilvania” University Braşov (represented by CVTC) and County School Inspectorate Brasov and from this year CYPRESS semiconductor by CYPRESS – National Instruments CLUB. In 2018 participants at the Robotor 2018 was invited to one interactive workshop (Fig. 2c) in connection with use of Voltera printer. The Voltera V-One printer, present in CNIC club, help students and children’s to create fast and easy two layer prototype circuit boards. Gerber files go in, printed circuit boards come out. The dispenser lays down a silver-based conductive ink to print student’s circuit’s right before their eyes. Assembling traditional and additive boards is easy with the V-One’s solder paste dispensing and reflow features. Simply mount the new prototype board on the print bed and import the Gerber file into Voltera’s software. The V-One can already be found at many of the leading research institutes and universities across the world and is an essential tools for anyone that cannot afford to wait weeks (like in our CNIC club) for a PCB prototype to be printed at a 3rd party board house before moving to the next iteration of prototyping.

3 Recent Results in Training High School and University Students In recent years, the Center for Valorization and Transfer of Competences through the Creativity Lab and the Cypress National Instruments Club has been involved in the collaboration of HIGH SCHOOL - UNIVERSITY - INDUSTRY. It was envisaged an extension/improvement of the curricular system in the school and the university, through common actions and training within the laboratory, connected with the evolution for the need of specialists requested by the Romanian electronics enterprises. Considering the evolution of communication systems, we have chosen to develop from simple to complex applications on Li-Fi (Light-Fidelity) communication as a light mediated communication, replacing, widespread Wi-Fi systems. Li-Fi is a high-speed wireless network technology that uses the electromagnetic waves in the visible spectrum for data transmission. The principle of Li-Fi systems is based on coding and transmitting data by modulating the amplitude of light sources according to welldefined and standardized protocols. Li-Fi technology has developed as a result of radio spectrum clustering in unlicensed Wi-Fi (2.4 GHz and 5 GHz) bandwidth due to high device density, resulting in undesirable interference, occupying most of the Wi-Fi band, which leads to a dramatic decrease in transfer speed. Using a common LabVIEW (Laboratory Virtual Instrumentation Engineering Workbench) software and a modern NI ELVIS (National Instruments Engineering Laboratory Virtual Instrumentation Suite) system, children in the 7th and 9th grades (see the image in Fig. 2b), in collaboration with the first year university students have

“School – University – Industry” Cooperation

999

developed Li-Fi applications, from simple ones developed by children to the most complex, developed in collaboration with students of the first year in Electronics Faculty, which they will present at the National Physics Symposium in Fagaras Romania (Fig. 3). At the end of the presentation, the participants were presented to them a commercial Li-Fi equipment (from the University’s Creativity Lab - CVTC), produced by PURE Li-Fi, a company founded by Prof. Harald Haas and Dr. Mostafa Afgani in 2012 as the spin-out of the University of Edinburgh to create OEM components, including LiFi drivers and receivers. In Fig. 3 we present these three steps, from simple to more complex, asa demonstration of Li-Fi communication for physics laboratory experiments – easily adapted to the high school environment [4].

(a)

(b) Fig. 3. Li-FI simple Physics laboratory demonstrator: (a) schema bloc and (b) realized experiments on NI ELVIS LED panel direct modulated and LED amplified

For the industrial application, a functional model and a prototype of lighting adjustment were developed using a matrix light sensor with germanium (GEnc) realized by the National Institute of Physics of the Materials on the basis of quantum dots technology. The basic idea is automatic regulation of the room light using the wireless system in dependence with the intensity of the light received from outside, so that on the working tables always be assured the same illumination. Wi-Fi control for 6 of the 12 LED lamps arranged in the CVTC Creativity Laboratory was made using IQRF technology. The module developed by IQRF (Fig. 4) is ordered inside the 6 lamps and allows for an on/off control and dimming of the illumination.

1000

D. Ursutiu et al.

Fig. 4. Light control one IQRF card

In the ceiling will be mounted the matrix mentioned above for which coordination has developed a LabVIEW application (Fig. 5) for control and visualization of adjustable lighting areas (a 2  3 lighting control matrix is expected on the working table with the maintaining of a steady level without light variations to do disturbing of the user).

Fig. 5. LabVIEW application monitoring of “areas of interest” as illumination.

“School – University – Industry” Cooperation

1001

4 Conclusions In conclusion the structures created at the “Transilvania” University of Brasov: CVTC, Creativity Laboratory and Cypress National Instruments Club were the best solutions able to support this kind of connectivity: HIGH SCHOOL - UNIVERSITY – INDUSTRY. Recent publications, activities and especially the new generation of students who enrolled for the specialization in Applied Electronics support harmonious collaboration. Industrial partners from region of Brasov and also internationally are very satisfied with our last generations of students. Every year we receive more and more high school candidates from central part of Romania and many of them were already trained by us in high school module of our “Cypress – National Instruments Club” and “Creativity Laboratory” of CVTC, proof that the module convinced them to continue their education in this direction. These students know already LabVIEW, PSoC systems, PSoC Creator and rapid prototyping in electronics (using Voltera PCB printing system) and thus they had a faster and better adjustment to faculty of electronics curricula. This CNIC Club contributes in the last years also to improving and updating the knowledge of high school teachers and we succeed to train in CVTC many engineers from industry. All this structures created in “Transilvania” University of Brasov clearly contributed at one sustained SCHOOL – UNIVERSITY - INDUSTRY advantageous collaboration in both directions of Education and Research. Acknowledgment. The paper was developed under the authority of CVTC (Center for Valorization and Transfer of Competence) of “Transylvania” University of Brasov – Romania and partially funded from the budget of the National Grant PN III –“New directions of technological development and utilization of advanced nanocomposite materials” contract no 47 PCCDI/2018, Program I – “Development of the National Research System”, Project 2: “Oxidic nanocompozites functionalized for sensors applications”.

References 1. NI ELVIS manual web link. https://mil.ufl.edu/3111/docs/ELVIS/ELVIS_I_hardware_users_ manual.pdf 2. Essick, J.: Hands-On Introduction to LabVIEW for Scientists and Engineers, 4th edn. Oxford University Press, Oxford (2018). ISBN-13: 978-0190853068, ISBN-10: 0190853069, 720 p. 3. Currie, E.H., Van Ess, D.: PSoC3/5 Reference Book, 29 March 2010. Web link. http://web. mit.edu/6.115/www/document/psoc_book.pdf 4. Schwartz, M., Manickum, O.: Programming Arduino with LabVIEW. Packt Publishing Ltd., Birmingham (2015). Computers – 102p.

PerFECt: A Performative Framework to Establish and Sustain Onlife Communities and Its Use to Design a Mobile App to Extend a Digital Storytelling Platform with New Capabilities Nektarios Moumoutzis1,2(&), Alexandros Koukis1, Marios Christoulakis1, Ioannis Maragkoudakis1, Stavros Christodoulakis1, and Desislava Paneva-Marinova2 1

Laboratory of Distributed Multimedia Information Systems and Applications, School of Electrical and Computer Engineering, Technical University of Crete, 73100 Chania, Crete, Greece {nektar,christoulakis,imarag,stavros}@ced.tuc.gr, [email protected] 2 Institute of Mathematics and Informatics, Bulgarian Academy of Sciences, 1113 Sofia, Bulgaria [email protected]

Abstract. This paper presents how the concept of virtual community can be further elaborated to account for the use of modern digital technologies in a way that promotes the establishment of rich social contexts within which human creativity is exercised and learning happens. The new reality brought about by digital technologies is characterized by new qualities, new opportunities for action, new community affordances, that are captures by the term onlife. This term was initially proposed in the Onlife Manifesto and used in this paper to signify certain qualities that are important in understanding modern digital platforms and applications as well as guide the efforts for effectively extending to provide new capabilities to their users. Specific design principles are presented exemplified by the presentation of the design of a mobile app addressing creativity and learning in the field of cultural heritage to showcase how these principles are put in action. This mobile app enables the users of an existing digital storytelling platform inspired by traditional Shadow Theatre, develop their own digital puppets in a way that incorporates the creative process of developing traditional shadow theatre puppets. The use of the mobile app is compared to the need to use external image processing tools and the pros and cons of each approach are presented and linked to the design principles initially presented. Keywords: Digital storytelling communities


Digital puppets
Shadow theatre
Onlife

© Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 1002–1014, 2021. https://doi.org/10.1007/978-3-030-49932-7_93

PerFECt: A Performative Framework to Establish and Sustain Onlife Communities

1003

1 Introduction People learn and create within certain social contexts, in environments, physical or virtual, within which other people are engaged as well possibly with different roles but common goals and expectations to create or learn. People are connected to each other in this process of creating or learning. They even connect to other people that are not present such as creators of artifacts used, creators of the knowledge they employ or even with people that will exist in the future and will use the products being created. In this respect, the central concept of community is employed to signify the social context within which human creativity is exercised or learning happens. The goal is to provide a comprehensive approach for supporting Onlife Communities by employing digital technologies within an overarching framework that is informed by current trends in re-conceptualizing and re-thinking about our societies facing the so called “hyperconnected era”. This is reflected in the term “onlife”, which has been employed in The Onlife Manifesto [1]. This term stresses the fact that the deployment of information and communication technologies and their uptake by society radically affect the human condition, modifying human relationships as well as relationships of humans to the world. The term onlife is a neologism introduced by Prof. Luciano Floridi claiming that “we are neither onlife nor offline, but onlife”. Its use in the Onlife Manifesto is related to the fact that the ever increasing pervasiveness of digital technologies lead to a blurring of the distinction between reality and virtuality as well as between human, machine and nature along with a reversal from information scarcity to information abundance and a shift from the primacy of entities to the primacy of interactions. To elaborate a framework for the establishment and support of onlife communities with the aim to empower their members to control how digital technologies support their capabilities to create and learn, there is a need to depart from established engineering practices that are based on monolithic designs done by technology experts. A new conceptual framework is needed that is based on the hypothesis that digital systems can be realized by the composition of elementary components with limited initial design and be put to work by end users, eventually facilitated by IT engineers that play the role of catalysts of change and evolution of those systems towards directions that could not be initially foreseen [2]. Elaborating on this design approach, this paper proposes a framework to establish and sustain of onlife communities, i.e. communities of creators using digital tools in a certain domain, emphasizing creativity and learning. This framework is presented in Sect. 2. Section 3 presents eShadow, a platform that promotes creativity within a digital storytelling approach that is inspired by traditional shadow theatre. Section 4 presents how the framework presented in Sect. 2 is applied and how it has been used to interpret how users understand and use the eShadow in order to develop a mobile app that further enhances their creativity and learning potential. Section 5 concludes and presents directions for future work.

1004

N. Moumoutzis et al.

2 The PerFECt Framework Technology in general and digital technologies in specific is a catalyst for establishing and sustaining certain social structures as Cabitza et al. [3] underline and exemplify. They emphasize the fact that end users are becoming “producers” of contents and functionalities. On the other hand, the term expert user is suggested to signify an expert in a particular domain with main goal to develop the technological capabilities available on that domain. An expert user engages in creative/authoring activities without being a professional software developer. Usually the role of end user and that of an expert user are played by different people that may also belong to different communities. Furthermore, Cabitza et al. [3] suggest the role of meta-designer to describe the work done by professionals who create the socio-technical conditions for empowering end users in acting as active contributors of contents and functionalities. A metadesigner creates open systems that can be further developed by their users acting as codesigners. However, apart from the technical conditions necessary to set up such environments, there is a need to effectively create the social conditions that will allow expert users to build and adapt the artifacts to be used by end users. In respond to this need, a special user role is specified: maieuta-designers. A maieuta-designer creates the necessary preconditions for facilitating expert users appropriate the design culture and technical notions necessary for the meta-task of artifact development and involving as many end users as possible in the process of continuous refinement of the artifact, by improving participation. The user of the term “maieuta” directly references the Socratic method of getting people acquire notions, motivations and self-confidence to undertake challenging tasks. End users, expert users, meta-designers and maieuta-designers engage in certain interactions with each other as well as with the digital artifacts and tools causing the emergence of a co-evolution phenomenon. Meta-designers focus on designing and providing the most effective tools that may sustain the co-evolution between end users and expert users. Maieuta-designers facilitate the transition from the role of end user to the role of expert user thus empowering people to appropriate and contribute to their digital artifacts. If certain end users are not interested or fail to move towards the role of expert user, maieuta-designers may facilitate system evolution by systematizing the reporting of opportunities or shortcomings, as identified by end users, and proposing solutions handled by expert users or even suggest further technological contributions from meta-designers. As a final result of all these interactions, two co-evolution processes emerge: • The first cycle addresses the interactions between end users and the system. It refers to the use of software devoted to the end users. It is shown in Fig. 1 (left) with three homocentric cycles of arrows that represent the action-interpretation cycle at the lower level, the task-object cycle at the middle level and community-technology cycle at the upper level.

PerFECt: A Performative Framework to Establish and Sustain Onlife Communities

1005

• The second cycle is analogous and it is shown in Fig. 1 (right). It addresses the use of the technological environment and the corresponding software components as building blocks of the system in continuous evolution. Three homocentric cycles (levels) are present here as well: action-interpretation, task-object, and communitytechnology.

Fig. 1. The main components of PerFECt framework

There is a clear symmetry between the two co-evolution processes in terms of their constituent interaction cycles. In particular: 1. The inner interaction cycle in each co-evolution process addresses actions initiated by the corresponding user role or software that are interpreted by the other party, software or user role respectively. 2. The middle cycle refers to the co-evolution of the user task and the related digital artifact within the boundaries of the system. 3. The outer cycle captures the idea that the overall environment within which a user is working (community), co-evolves with the technology that supports the operation of the environment. All user roles (end users, expert users, maieuta-designers and meta-designers) together with the digital artifacts, tools and even underlying physical objects used to embed the technologies used (e.g. within the so called Internet of Things or other settings that enhance physical objects with new affordance that exploit the capabilities of digital technologies) are considered as a whole: an Onlife Community. This term is used to emphasize the fact that all user roles, via their interactions along with the two co-evolution processes, create an aggregation of humans that engage with other humans as well as with machines and natural entities in mindful interactions in a way that generalizes the notion of online communities offering new opportunities for creative expression and learning. Within this framework an important notion is used to describe blends of machines and physical objects that generalize the notion of software or tool: The notion of

1006

N. Moumoutzis et al.

universality. In both centers of the two co-evolution cycles in Fig. 1 this notion is explicitly used to characterize the artifacts employed (universal objects on the left and universalizing assemblies on the right). Universality addresses the issue of causality in digital representations. A universal object is essentially an object that exhibits a behavior that can be understood by humans because it is based on causal relationships with other objects or processes. A universalizing assembly, on the other hand, is considered as certain assembly of digital artefacts that creates a new type of artifact that can be understood and manipulated as a universal object. The term universality is adopted from Brenda Laurel’s seminal book “Computers as Theatre” where it is considered as in important aspect of all representational worlds that goes back to Aristotle’s Poetics where the idea of universal actions was introduced: “… an action is universal if everybody can understand it, regardless of cultural and other differences among individuals.… Aristotle posits that any action can be “universalized” simply by revealing its cause; that is, understanding the cause is sufficient for understanding the action, even if it is something alien to one’s culture, back-ground, or personal ‘reality’.” [4], p. 94.

Within the PerFECt framework, the meta-task of expert users is to enable the universalization of certain objects by exploiting available tools in the form of performative artifacts (pArtifacts) to account for the incorporation of the idea of performativity in digital technologies. Performativity describes the relationship between humans and the artifacts they create that is triggered by social interaction. Such interactions continuously recreate the bonds that keep the society as a whole. Niedderer [5] emphasizes that performative objects are designed to facilitate mindful awareness of the physical and symbolic social actions and their consequences within which they are used. The term performative artifacts (pArtifacts) used here, captures the idea of intentional design for social interaction, to create and sustain social bonds and call for symbolic social actions that re-create the social contexts within which we live in. Consequently pArtifacts offered by meta-designers are essentially the catalysts for the establishment and sustenance of onlife communities supporting the bonding of their members as well as the bonding between their members and the artifacts they create and use. To make the above concepts more clear and understand how the PerFECt framework can be used to better understand how digital technologies are actually used and what kind of social interactions they facilitate or promote, a particular case will be presented next. The specific software studied is a digital storytelling platform that is inspired by traditional shadow theatre. The next section presents its main features.

3 eShadow Shadow theatre is a storytelling tradition of many countries Far East and Middle East using flat articulated puppets which are held between a light source and a translucent screen or scrim. It is a medium with significant educational value within the wider context of drama and performance arts [6]. This is due to its ability to engage people and promote their creativity. In particular children and adults find their own ways to act

PerFECt: A Performative Framework to Establish and Sustain Onlife Communities

1007

and imitate, create dialogues, get inspired and convey their own messages, direct, become stage designers, sing, strengthen their self-confidence giving life to the puppets, improvise and create their own stories. Thereby they cultivate their oral speech skills and develop in multiple modes their intelligence (multiple intelligences) in an entertaining manner. eShadow (http://www.eshadow.gr) is the digital version of shadow theatre [7]. It enriches traditional features with digital technology elements to offer a new way of dramatized and personalized digital storytelling. It enables the production of rich multimedia content interactively using innovative input devices and supports online collaboration. It offers an intuitive way of setting up scenes and enacting them. The user can select the desired scenery objects and digital puppets and then move them with mouse drag operations. All movements can be easily recorded along with the voice of the user. These recordings can be exported in appropriate file formats to be further edited with external video processing tools. eShadow gives emphasis in the realistic motion simulation of shadow theatre’s puppets (Fig. 3) that is based on a physics engine. Realistic movement provides an explanation for the popularity of our platform among Greek teachers and students as revealed during the field trials as well as during testing with actual professional performers [8]. Furthermore, realistic movement of digital puppets creates an atmosphere of playful interaction where the users are very easily engaged in theatrical improvisations that can be very important to develop communication skills related to oral expression and interpretation of body language. eShadow is used in many schools to promote project-based learning combining arts with a wide range of school topics ranging from language learning, history and humanities to mathematics, physical sciences and computer science [7–9]. The creation of such artworks with eShadow corresponds to a project-based process with three distinct stages that is generic and refers to any media artwork such as movies, digital games, interactive animations etc. In particular, these stages are the following: Preparatory actions (pre-production), subsequent development (production) of the main materials of the artwork, and final assembly (post-production). One of the most engaging phase in this creativity workflow refers to the development of the puppets that will be used in a digital story. Note here that eShadow offers several readymade digital puppets to facilitate rapid development of digital stories. In many cases, however, the themes of the stories or the educational objectives of the teachers open up the opportunity to engage in the creative process of developing new puppets. To facilitate this process a digital puppet authoring tool was initially developed for end users and used in combination with eShadow [8]. Furthermore, an open architecture was adopted in the representation of the computer files corresponding to digital puppets so that external image processing tools could be used [7] by expert users that wish to create digital puppets that do not conform to predefined forms. A new need emerged during learning interventions undertaken in pre-primary education. In particular, teachers participating in these learning interventions, during their interactions with maieuta-designers during training workshops asked for a way to facilitate digitization of paper-made shadow puppets using a simple mobile app. The whole process was organized on the basis of the PerFECt framework (see Sect. 4).

1008

N. Moumoutzis et al.

Before elaborating more on this app and the corresponding learning interventions, it is necessary to provide some details regarding the structures of eShadow puppets and their representation. Their structure is very similar to the structure of traditional shadow theater puppets. In particular, they consist of two or more parts which are joined together by joints as shown in Fig. 2.

Fig. 2. Examples of shadow theatre articulated puppets. Red circles represent joints that connect puppet parts. The puppet on the left has two-parts with one joint while the puppet on the right is more complex having four parts and three joints that connect them.

The female puppet on the left of Fig. 2 consists of two parts: the top and bottom of its body which are connected with only one joint. The male puppet to the right of Fig. 2 consists of four parts: The main body, waist and two legs. There are three joints: one that connects the main one body with waist and two that connect the waist with left and right legs respectively. These two typical cases (with four or two parts) capture the structure of most puppets of traditional Greek shadow theater. To synthesize such puppets, one has to develop their parts in certain material, cut them and join them together at the correct joint positions. The female puppet consists, as mentioned, of two parts which are associated with one joint. These two parts are illustrated separately in Fig. 3 as well as their connecting points (joint anchors) to form the joint of the puppet. The selection of connection points, especially for the bottom part affects not only the aesthetics of the puppet but also its behavior during handling due to the force of gravity. Wrong point selection will have as a result the lower part of the puppet to be unbalanced. Consequently, puppets are tested before the position of joints is fixed.

PerFECt: A Performative Framework to Establish and Sustain Onlife Communities

1009

Fig. 3. Synthesis of shadow theatre articulated puppets. On the left, an example of a two-part puppet is shown with its constituent parts and the corresponding joint anchors (red circles) in each part. On the right, an example of a four-part puppet is shown. In both cases the red rectangles represent the boundaries of areas used to place each constituent part.

4 Using the PerFECt Framework to Understand How eShadow can be Effectively Used and Extended Employing the PerFECt framework, its concepts and user roles, to better understand the use of tools such as eShadow can be put within a wider context that accounts for the rich social interactions that could be promoted towards the establishment of onlife communities. In particular, eShadow can be considered as a representative tool on how a community can be established (in the field of cultural heritage and learning) that brings together: • Software developers supporting the software and providing further enhancements to address the needs of the users. • Puppet creators that prepare materials, such as puppets, that can be used to support other creators, those that use the platform to develop animations and stories on various themes. • Storytellers that use materials offered by creators to develop digital stories. Using the user roles’ terminology introduced in Sect. 2, the above categories of participants in an eShadow-based community can be reframed as follows: • Software developers that support eShadow and implement further enhancements are the meta-designers of the PerFECt framework. As meta-designers, they are expected to offer an open system that can evolve by its users as co-designers. To enable this, eShadow offers an open representation based on json files to enable the creation of content such as digital puppets.

1010

N. Moumoutzis et al.

• Puppet creators that develop digital puppets are what the PerFECt framework describes as expert users. They address the needs of end users using the open system capabilities offered by meta-designers to develop new reusable materials. • Storytellers that use eShadow to develop digital stories are what the PerFECt framework calls end users. They essentially use the creations of expert users in the form of universal objects, i.e. digital artifacts that exhibit a certain behavior that simulates the behavior of traditional puppets with all the corresponding causality stemming from the presence of gravitational forces. Apart from the above mentioned roles, which are directly related to eShadow as a tool simulating and extending a traditional creative environment, the PerFECt framework introduces yet another (fourth) user role: maieuta-designers. This role has a significant contribution in framing and supporting an onlife community addressing the social conditions for supporting the meta-task of expert users and the transition from the end user role to the role of expert user. This transition and support of expert users’ tasks are essentially a learning process that takes place within a social context (i.e. the community of users). In the case of eShadow, a typical function of maieuta-designers is to organize workshops where eShadow users can be trained on how to understand and use the representations of digital puppets, how they can use existing image processing tools to develop their own digital puppets [9] or remix existing puppets [8] to meet their particular needs. After successfully implementing this approach in many primary and secondary schools, with very interesting results showcasing the educational potential of such technologies and their importance in reconnecting young generation with their cultural heritage, a new project was initiated addressing pre-primary education. A certain need was identified to develop traditional puppets before digitizing them. This need came from certain pedagogical objectives. Furthermore, there was a need to support users without skills and available time to engage in the process for preparing digital puppets using image processing software. Using the terminology of the PerFECt framework, this was a situation when maieuta-designers collaborating with end users, revealed a need to communicate with meta-designers to ask for the development of a new generic component that will allow end users adopt a new way of work in order to become expert users, i.e. puppet creators. The result of this intervention was the design of ePuppet, a mobile application for facilitating the digitization of two- and four-part puppets. Assuming that a traditional puppet has already been created, its parts can be put on a flat surface with a constant color background so that the ePuppet app can be used to take a photo of the parts after appropriate aligning them with predefined templates, one for two-part puppets and one for four-part puppets (Fig. 4).

PerFECt: A Performative Framework to Establish and Sustain Onlife Communities

1011

Fig. 4. Digitization of shadow theatre puppets using ePuppet. The parts of the puppet are put on a flat surface with a constant color background (e.g. on a sheet of paper). The parts and the mobile device are moved as needed so that the parts are framed within the red boundaries shown in screen overlay and the joint anchors (red circles) are positioned correctly. Then, a photograph is taken that is transformed by ePuppet into a correct digital puppet representation.

The detailed steps followed by a user using ePuppet are shown as a sequence of screenshots in Fig. 5. In particular, initially the user provides a name for the new puppet and selects the corresponding template (if it will be a two- or four-part puppet). Next, the user activates the camera of the mobile devices and overlays a guiding template as a slide that is placed over the camera’s image. Moving parts of the puppet as well as the devices left to right, up and down and back and forth full alignment of the points is achieved connecting the joint anchors (red dots) to coincide with the connection points of the real puppet parts. After aligning the parts, the user selects the

Fig. 5. To create a digital puppet with ePuppet mobile app, the user follows four steps: (a) gives a name to the puppet and selects a template to use; (b) takes a photo of the physical parts of the puppet; (c) adjusts background color levels to automatically remove the background; and (d) stores the files created.

1012

N. Moumoutzis et al.

appropriate control to make a photo of the puppet. In the next screen, the user is able to adjust the background color levels to enable its automatic removal to clear the surrounding area of the puppet parts. After that, the application creates all necessary files so that the digital version of the puppet can be used in eShadow and lists the new puppet in the mobile app folder area. The final output of this process, is the digital representation of the puppet. In particular, two files are created: • An image file (sprite sheet) with all the constituent parts of the puppet. • A json file with the information about the placement of the constituent parts in the sprite sheet as well as the position of joint anchors. Figure 6 below depicts the json file and the corresponding sprite sheet for the twopart puppet presented above. The representation of the four-part puppet is analogous.

Fig. 6. Digital puppet representation. On the left, the json file describing how the puppet is composed of different parts, where these parts can be found in the corresponding sprite sheet and where the joint anchors are placed. On the right, the corresponding sprite sheet is depicted.

5 Conclusions and Future Work The concept of Onlife Communities as exemplified by the PerFECt framework addresses issues related to the establishment and support of rich sociotechnical contexts where engaging learning experiences can take place and human creativity can be effectively expressed. As a concrete example of how this framework can be put in action, this paper presents eShadow, a platform inspired by traditional shadow theatre to offer a digital storytelling environment to support creativity and learning. The PerFECt framework helps in better understanding how eShadow promotes creativity and learning and how it could be further enhanced and repurposed to promote further rich social interactions. Interesting workflows are identified that address the transition of eShadow users from the end user role to the expert user role. Within these workflows, external tools can be used to facilitate this transition and help users develop

PerFECt: A Performative Framework to Establish and Sustain Onlife Communities

1013

their digital media authoring skills. Furthermore, new tools can be designed, such as the ePuppet mobile app, to offer new creativity and learning opportunities. Future work will evaluate the learning effectiveness of using ePuppet in preprimary schools. Extensive usability testing is also foreseen as well employing standard experimentation tools including emotional response evaluation to document learner engagement potential. The overall evaluation approach employs design thinking for young children [10] addressing a complete creativity workflow including scenario development, creation of shadow theatre puppets in paper, digitization of the puppets using the ePuppet app and development of digital stories using eShadow. Finally, the applicability of the PerFECt framework will be explored in the context of facilitating the use of digital technologies in schools in general both for supporting learning processes and facilitating content creation and community management [11]. Acknowledgments. The development of the PerFECt framework presented in this paper is a work partly supported by the Bulgarian NSF under the research project №. DN02/06/15.12.2016 “Concepts and Models for Innovation Ecosystems of Digital Cultural Assets”. ePuppet has been used in pilot activities with schools, within the context of the Erasmus+ projects DISCOVER (2017-1-BG01-KA202-036327), TIM (2018-1-IT02-KA201-048139) and MUSILIB (2018-1-FI01-KA201-047196). Puppets depicted in Fig. 2, Fig. 3, Fig. 4, and Fig. 6 are digital versions of original creations by Mr. Nikos Mplazakis, a professional Greek Shadow Theatre puppeteer.

References 1. Onlife Initiative: The Onlife Manifesto. In: Floridi, L. (ed.) The Onlife Manifesto, pp. 7–13. Springer-Verlag GmbH, Cham (2015) 2. Cabitza, F., Simone, C.: Building socially embedded technologies: implications about design. In: Designing Socially Embedded Technologies in the Real-world, pp. 217–270. Springer, London (2015) 3. Cabitza, F., Fogli, D., Piccinno, A.: Cultivating a culture of participation for the co-evolution of users and systems. In: CoPDA@AVI, pp. 1–6 (2014) 4. Laurel, B.: Computers as Theatre. Addison-Wesley, Boston (2013) 5. Niedderer, K.: Designing mindful interaction: the category of performative object. Des. Issues 23(1), 3–17 (2007) 6. Hatzigianni, M., Miller, M.G., Quiñones, G.: Karagiozis in Australia: exploring principles of social justice in the arts for young children. Int. J. Educ. Arts 17(25), 1–20 (2016) 7. Moumoutzis, N., Christoulakis, M., Christodoulakis, S., Paneva-Marinova, D.: Renovating the cultural heritage of traditional shadow theatre with eShadow. Design, implementation, evaluation and use in formal and informal learning. In: DiPP 2018 Conference on Digital Presentation and Preservation of Cultural and Scientific Heritage, Institute of Mathematics and Informatics–BAS, Sofia, Bulgaria, vol. 8, pp. 51–70 (2018). ISSN 1314-4006 (Print), eISSN 2535-0366 (Online) 8. Moraiti, A., Moumoutzis, N., Christoulakis, M., Pitsiladis, A., Stylianakis, G., Sifakis, Y., Maragoudakis, I., Christodoulakis, S.: Playful creation of digital stories with eShadow. In: 11th International Workshop on Semantic and Social Media Adaptation and Personalization (SMAP), Thessaloniki, pp. 139–144 (2016)

1014

N. Moumoutzis et al.

9. Moumoutzis, N., Gioldasis, N., Anestis, G., Christoulakis, M., Stylianakis, G., Christodoulakis, S.: Employing theatrical interactions and audience engagement to enable creative learning experiences in formal and informal learning. In: Interactive Mobile Communication, Technologies and Learning, pp. 142–154. Springer, Cham (2017) 10. Stevenson, M., Bower, M., Falloon, G., Forbes, A., Hatzigianni, M.: By design: professional learning ecologies to develop primary school teachers’ makerspaces pedagogical capabilities. Br. J. Educ. Technol. 50(3), 1260–1274 (2019) 11. Yoshinov, R., Kotseva, M.: Vision for the engagement of the e-Facilitator in school in the inspiring science education environment. Serdica J. Comput. 9(3–4), 241–256 (2016). Institute of Mathematics and Informatics, BAS, ISSN 1312-6555

Exploring Impact of Olfactory Stimuli on User Performance on Mobile Platforms Sergio Caro-Alvaro1, Anas Ali Alkasasbeh2, Eva García-López1, Antonio García-Cabot1, Gregor Rozinaj3(&), and Gheorghita Ghinea2 1 2

Department of Computer Science, Alcala University, Alcala, Spain Department of Computer Science, Brunel University, Uxbridge, UK 3 Faculty of Electrical Engineering and Information Technology, Slovak University of Technology, Bratislava, Slovakia [email protected]

Abstract. In today’s modern age, it is almost impossible to exist without the Information Communication Technologies that makes our life easier. In addition to other areas, Information Communication Technologies is also needed in education. There are a number of ways to contribute to the development of the learning process, which we refer to in this article - student centric model, gamification, virtual and augmented reality (using Gear VR), remote and virtual labs. We will also discuss the educational application focused on the basics of geographical knowledge of Slovakia. Keywords: E–learning reality


Student centred learning
Gamification
Virtual

1 Introduction In this article we will talk about how modern communications technology affects each and every one of us. The ICT has already been used in many areas, and one of them is education that is inevitable for each of us. The offer is just making mostly important and learning information in a playful form. As regards education, the students can educate themselves anytime and anywhere thanks to the ICT and Internet. The need for ICT in education is certainly inevitable. The information we need to acquire is more and more, and teachers are not given the opportunity to devote themselves to each student individually as much as they need. It´s the one of the benefits ICT, where student can learn a lot of things in a playful way through a simple application. An example of such an application, where we are creating a blind map of Slovakia, will also be discussed in this article. Since the development of the learning process on all levels of learning determines the student centric model, gamification, virtual and augmented reality (using Gear VR), remote and virtual labs, student experience measurement, referral system, etc. we will discuss them here in more detail.

© Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 1015–1023, 2021. https://doi.org/10.1007/978-3-030-49932-7_94

1016

S. Caro-Alvaro et al.

2 Virtual, Augmented and Mixed Reality When we talk about these three types of realities, there are certainly a lot of people who see all these types as one and the virtual reality. But this is a mistake because they are very different from each other. We can further develop and compare them for better understanding (Fig. 1).

Fig. 1. The difference between Virtual, Augmented and Mixed reality [1]

2.1

Virtual Reality

The concept of virtual reality is well known today. When we hear the VR, we will literally introduce ourselves to escape from reality. But it’s just a computer-generated scenario that makes us feel something real. And that is exactly what today’s technologies allow us to do, to advance towards to reality maximally, seeing that technology is moving through incredible steps. Unlike mixed reality, which is more useful, virtual reality has been developed for entertainment purposes. In order to be fully immersed in a virtual world, a user needs a special headset. Without this headset would be the VR virtually unusable. We will describe the 2 main VR devices below. 1. Oculus Rift The Oculus Rift is one such VR system that works as a wearable headset capable of letting the users experience a virtual environment. The specifications of the Oculus Rift are [2]: • Head-mounted display • OLED display technology

Exploring Impact of Olfactory Stimuli on User Performance

• • • •

1017

2160  1200 resolution (1800  1200 per eye) 90 Hz refresh rate 110° or greater field of view (nominal) Compatible with Microsoft Windows (plans for OS X and Linux compatibility)

The Oculus Rift is designed to give a feeling of presence and provide a lifelike experience with its specialized design and software. It is customizable and adaptable to various uses. One common application is to enhance the gaming experience. The Oculus Rift comprises an integrated audio VR, thus providing a 3-D audio effect. It also includes rotational and positional tracking accomplished with the help of infrared sensors. Therefore, it can be used while users are sitting, standing or walking around the room. 2. HTC Vive Pro VIVE Pro is the most capable and fully featured virtual reality system VIVE has ever made. Designed to meet the needs of today’s most demanding VR users - from expansive office environments and crowded arcades, to the comfort of your living room. VIVE Pro surpasses the bar-setting experience to realize the most immersive virtual reality experience. Headset Specifications [3] • • • • • • • • 2.2

Screen: Dual AMOLED 3.5” diagonal. Resolution: 1440 1600 pixels per eye (2880  1600 pixels combined). Refresh Rate: 90 Hz. Field Of View: 110°. Audio: Hi-Res certified and high impedance headphones (removable). Input: Dual integrated microphones. Connections: USB-C 3.0, DisplayPort 1.2, Bluetooth. Sensors: SteamVR Tracking, G-sensor, gyroscope, proximity, IPD sensor. Augmented Reality

In addition to virtual reality, we also know the augmented reality and mixed reality. As regards to Augmented reality, Augmented reality (AR) is an interactive experience of a real-world environment where the objects that reside in the real-world are “augmented” bycomputer-generated perceptual information, sometimes across multiple sensory modalities, including visual, auditory, haptic, somatosensory, and olfactory [4, 5]. Enhanced Reality uses existing environments and overlaps virtual objects and information. This type of technology proves to be very useful in our daily lives. For a better understanding of what actually augmented reality means, we can give some examples of devices that use AR. 1. Hololens As a standalone headworn computer capable of projecting images that appear to be in the real world, the HoloLens is the first of its kind. With a collection of sensors and

1018

S. Caro-Alvaro et al.

cameras, the HoloLens is constantly scanning the space around the user and updating a 3D mesh of that area in what is referred to as a spatial map. Another feature of this spatial map is occlusion. So if you have your home mapped correctly, and you bounce a virtual ball off the wall, it can, for example, disappear as it rolls into your kitchen, just as it would in real life [6]. 2. Meta 2 The Meta 2 is a tethered, head-mounted device. That means it requires a connection to a traditional PC to function and is designed to be used in a stationary location, with no ability to wander around your real world environment. Crafted around the idea of being a replacement for your 2D screen, Meta 2 plugs into your computer’s video port, with sensor input and image processing being handled by the connected PC’s processors [6] (Table 1). Table 1. Hololens and Meta 2 comparsion [6]

2.3

Mixed Reality

Another notion associated with virtual reality is a mixed reality. What this type of reality offers us, is essentially a combination of augmented and virtual reality. It includes both physical elements and virtual elements. Offering the ideal blend of powerful hardware and distinctive features, Samsung’s HMD Odyssey is without a doubt, the best Windows Mixed Reality (WMR) headset on the market today. The HMD Odyssey has two 3.5-inch AMOLED displays, each with a resolution of 1440  1600 pixels. The displays provide a rich and unparalleled visual experience, further enhanced with an immersive 110-degree field of view. The headset also supports 360-degree spatial sound, which is made possible by premium built-in AKG headphones (Fig. 2).

Exploring Impact of Olfactory Stimuli on User Performance

1019

Fig. 2. Smsung HMD Odyssey

3 Student Centred Learning Learning models were long focused on keeping things standard to make the lives of teachers and institutional administrators simpler. This focus has shifted in recent years to a more student-focused learning model. One powerful tool within this model is adaptive learning, and research shows its advantages over traditional teaching. It is because of these benefits that teachers, colleges and universities should consider shifting their focus. This educational method uses computers to cater to each student’s unique learning needs. This dynamic process adapts teaching methods based on responses to questions, interactive experiences and tasks. It’s nearly impossible for instructors to focus on each student individually. This makes online learning integral in teaching with these methods. Because of this, eLearning solutions are essential in adaptive learning [8]. This method of student centric method has many real benefits –,blended learning opportunities, students learn faster new things and it´s likely to engage students more than a classroom.

4 Gamification Gamification is the process of taking something that already exists – a website, an enterprise application, an online community – and integrating game mechanics into it to motivate participation, engagement, and loyalty. Gamification takes the data-driven techniques that game designers use to engage players, and applies them to non-game experiences to motivate actions that add value to your business [9] (Table 2).

1020

S. Caro-Alvaro et al. Table 2. 10 primary game mechanics [10]

5 Virtual and Remote Laboratories 5.1

Virtual Laboratories

These laboratories are very helpful, specially for scientists of Biology, Physics & Chemistry, who are working on their experiments. These types of laboratories allows them to cooperate on same project no matter where they currently are located (Fig. 3).

Fig. 3. Virtual laboratory

5.2

Remote Laboratories

Unlike the traditional laboratories, where people have to do everything manually, for example pressing buttons, Remote laboratories allow them to perform it, over the Internet.

Exploring Impact of Olfactory Stimuli on User Performance

1021

Benefits Convenience - A remote laboratory can be accessed 24 h per day, 7 days a week from any Internet connected location, without requiring staff supervision. Cost Efficiency - Due to more efficient sharing of resources, educational institutions can satisfy laboratory demand with less equipment. Alternatively, expensive and bulky equipment can be shared and utilised more effectively. Enables Distance Education Courses that traditionally had to be taught on-campus in their entirety may now offer opportunities for distance learning. Students who work and study part time may also find remote laboratories helpful in balancing their commitments. Equity Through the use of automated management systems, all laboratory users can be assigned time limits during busy periods and queued to ensure that everyone has their ‘fair share’ of time to carry out experiments. Security As public access to the laboratory location is not required, remote laboratories are highly immune to wear and tear, theft and vandalism. Safety - Many experiments can pose fire, electrical, radiation, biological and hazards to personal safety. Educational institutions may benefit from reduced OH&S compliance costs and reduced insurance premiums [11] (Fig. 4).

Fig. 4. Remote laboratory

6 Application Some of the abovementioned principles have been implemented to our application which is practical part of this project. This application’s theme is Slovakia and most famous natural and historical tourist places. Its main goal is to help users get to know

1022

S. Caro-Alvaro et al.

geographical position of our biggest towns and most known castles, lakes and caves on the map. The game engine we have used is UNITY, which is multiplatform and can be used for several types of applications such as mobile games, PC games, web, console games, etc. As programming language, it has been used C# and we have used programming environment Visual Studio Community 2012, which is not default while using UNITY. Platform used for this application is Cardboard because it is supported for all brands of mobile phones which support virtual reality. However, the game can only be played using touchpad or external controller. The application consists of main menu with short overview what the game is about, what is the goal of the game, who is target audience and how particular games should be played. The application consists of 3 main parts: 6.1

Learning Mode

This mode consists of 4 maps of Slovakia – – – –

map map map map

containing containing containing containing

50 20 20 10

most most most most

populous towns of Slovakia known Slovak caves known Slovak castles known Slovak lakes

This scene also includes one display window which shows name of the object (town, cave, etc.) after pointer click on the object. 6.2

Game Mode

• 1st level The same maps as in the learning mode are present, but there are some differences. This level is more interactive. In the beginning the random object (town, cave, etc.)

Fig. 5. Scene example

Exploring Impact of Olfactory Stimuli on User Performance

1023

name is displayed on the display window and user is supposed to find the object on the map and choose it. Based on if it is correct user earns or loses some points. There are also short sounds included based on this (Fig. 5). • 2nd level This scene uses similar principle as the abovementioned game except for the objects are invisible and therefore user has to identify the position where it should be placed on the map. In order to not decrease user s motivation when he does not identify all (especially the small ones and the less known) cities, castles, etc. correctly, we have decided to divide the towns to five groups (according to the population), the caves, lakes and castles to two groups. The smaller and less known objects will have bigger tolerance of their position on the map. 6.3

Quiz

This theme includes short quiz including mainly basic questions regarding Slovak nature, towns and buildings. There will be some extra not basic but very interesting questions to keep the player s attention. User gets points and badge as reward. Acknowledgement. Several methods and techniques of using virtual reality in education field are discussed. The main goal of the application is to increase student’s motivation. Some of them have been successfully implemented in the application with geographical theme aimed to help students get to know Slovakia a little more. This research was done with support of projects VEGA 1/0800/16 INOMET and H2020N Ref. No.: 688503.

References 1. 2. 3. 4.

5. 6. 7. 8. 9. 10.

https://www.bunchball.com/gamification/game-mechanics/ https://www.techopedia.com/definition/31556/oculus-rift/ https://www.vive.com/us/vive-pro-vr/ The Lengthy History of Augmented Reality”. Huffington Post. 15 May 2016. Schueffel, Patrick (2017). The Concise Fintech Compendium. Fribourg: School of Management Fribourg/Switzerland https://next.reality.news/news/whats-difference-between-hololens-meta-2-magic-leap0181804/ https://www.lifewire.com/best-windows-mixed-reality-headsets-4173017/ https://www.focuseduvation.com/student-centric-learning-why-adaptive-learning-isimportant/ https://www.bunchball.com/gamification/ https://www.bunchball.com/gamification/game-mechanics/ https://seiler.it/remote-and-virtual-labs/

Building a Virtualized Cybersecurity Lab Using Industry Support, Academic Programs and Open Source Solution for Setting-Up a Virtualized Cybersecurity Lab Titus Bălan(&), Dan Robu, Florin Sandu, and Alexandra Bălan “Transilvania” University of Braşov, Braşov, Romania {titus.balan,dan.robu,sandu, alexandra.balan}@unitbv.ro

Abstract. The domain of cyber security has gained an increased importance in the industry that is naturally reflected in the universities curricula that have included security courses and dedicated programs in their offer. The scope of this paper is to describe the requirements of a cybersecurity virtual platform, the basic mandatory tools to be included and the methods for building a flexible and extensible cyber security laboratory, taking benefit of industry support, academic programs and open source implementations. Keywords: Cybersecurity
Virtualized lab
Security industry
VDI
Remote laboratory
GNS3

1 Introduction 1.1

Increase in Industry Demand for Cybersecurity Specialists

As every company must take into consideration aspect of the IT security and data protection, the need of security specialists on the market is increased. Cyber security as specialization also evolved and has developed several sub-domains. Best method for cyber specialists’ development is via a hands-on approach, in the lab. With the advances of the Cloud technologies and with the proliferation of “as-aService” resources, with focus on Infrastructure-as-a-service and Network-as-a-Service, evolving towards Network Function Virtualization (NFV), it became easier to integrate and deploy ad-hoc cybersecurity infrastructure elements and complex topologies in a virtualized environment. 1.2

Empirical Observations While Building a Cybersecurity Lab

In this paper we will describe the necessities and methods that are suitable for preparing a cybersecurity laboratory environment. As case study, based on empirical observations, we will focus on the virtualized laboratory implemented as part of the infrastructure of “Transilvania” University of Brasov.

© Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 1024–1032, 2021. https://doi.org/10.1007/978-3-030-49932-7_95

Building a Virtualized Cybersecurity Lab

1025

We would like to share our experience in related to the architecture of the system, some cost related considerations but also some integration difficulties and encountered issues, observations from our empirical approach, that can bring information to other universities that are preparing dedicated laboratories. We would like to emphasize in our description the list of resources and the industry support via sponsorship, academic programs and open-source resources that can be used for different cybersecurity subjects.

2 Requirements for a Cybersecurity Virtual Learning Platform 2.1

Defining the “User Needs” and Related Work

We have studied several related papers related to building a cybersecurity lab, from indication provided by SANS Institute [1], to other [2–5]. However, none of them present methods for flexibility and versatility of network topologies that Cloud orchestration, doubled by recent GNS3 [6] integration brings. As the domain had a blooming development, there are also commercial laboratories that offer on-demand and prepared lab facilities, and here we can mention Cisco VIRL [7], Cybrary [8] or Immersive labs [9]. In this paper, by user we refer to the teachers of the dedicated subjects that should be able to teach different subjects of the broad subject of cybersecurity: – The students should be able to Remote usage via VDI (desktop virtualization) – Each student should have its own environment, thus there are no conflicts and running conditions, but there is a big demand in resources (specially processing, DRAM and HDD). – Due to security reasons, the systems should be isolated from the rest of the university infrastructure, thus any security tests and experiments do not affect the operational network – The system components (virtual machines) should be instantiated on demand based on the subject and lab specific configuration, so an orchestration method for IaaS should be present – Possibility for students to edit configurations, add new elements, new software components, new Docker virtual container deployments – Possibility for cleaning the system, thus a new student starts with a fresh environment, after the work performed by another student – Access to resources from the university intranet and from external connections via VPN 2.2

Cybersecurity Topics that Need a Virtual Lab

We have analyzed the software and virtualized network elements needed for our laboratories that should serve more disciplines:

1026

T. Bălan et al.

– For network security and perimeter defense topics we had the need to integrate different firewall elements and VPN endpoints (pfSense [10], IP Fire [11], Cisco ASAv, Cisco Firepower Thread Defence, Cisco Firepower Management Center), Intrusion Detection Systems/ Intrusion Prevention Systems, virtualized routers and switches (topics related to mitigation of different L2 security threats). – For ethical hacking there are some elements that should be integrated like: Kali Linux [12], Metaspoitable framework [13], DVWA (Damn Vulnerable Web Applications) [14], different tools for vulnerability scanning, other attackable machines (Windows, Linux, Android systems, like for example preinstalled virtual machines of different distributions, like the ones from provided by OSBoxes [15]) – For management of cybersecurity incidents, we can recommend the Security Onion distribution [16], including different NIDS (Snort, Suricata, Bro) and HIDS elements, and the ELK Stack Suite (Elastic Search, Logstash, Kibana) but also other options like the AlienVault OSSIM (Open Source SIEM) [17] or commercial SIEM (Security information and event management) solutions with academic programs that offer one year subscriptions (e.g. Splunk SIEM) [18]. – For data mining and Big Data topics there are Hadoop/Cloudera and Apache Spark dedicated images. – For generic laboratories in cryptography and network security the set of laboratories from SEED labs are of great support. – Deployment of infected machines for IT forensics and malware analysis are also necessary, taking into consideration the restriction of the complete lab isolation of the cybersecurity sandbox from the university operational network; For this domain there are also dedicated distributions that provide pre-integrated virtual machines like Remnux [19] or SANS DFIR [20]

3 Architecture of a Dedicated Lab System The problem is how to integrate all the above-mentioned elements in functional topologies, how to deploy them on demand based on the scenario and interest is a certain lab (Fig. 1). 3.1

Virtualization Layer

The hardware elements consist of three servers, each having at least 192 GB of DRAM and Nvidia graphic boards for GPU acceleration. We will not detail the complete configuration, as this is not relevant, can be extended and might not be relevant over time. When it comes to choosing a hypervisor one of the elements to consider are costs. For a home lab, even a nested hypervisor might be suitable, but for complex topologies, a normal PC or laptop cannot be used, so students need a dedicated virtualized environment. There are many appliances that request 16 GB of RAM, but most of the time there is a need of simultaneous elements running same time, each needing 8 or 16 GB of RAM.

Building a Virtualized Cybersecurity Lab

1027

From cost point of view and because we also needed to fulfill the VDI condition and the Citrix Xen Desktop [21] is a robust VDI solution, we have decided to use the Xen Server hypervisor, that are free of charge on the acquisition of the XEN Desktop licenses. The XEN Desktop licenses are perpetual, so we have opted for this solution, though other options were taken into consideration (VmWare vSphere).

Fig. 1. Virtualized desktop VDI via XEN Desktop can connect to resources not only university intranet users, but also external users

Another element to be virtualized is the video board, thus the applications can benefit of a GPU acceleration, mandatory for certain applications (e.g. Blockchain based applications). NVIDIA has a very good academic program [22], offering discounted licenses that are also perpetual, so the investment is only initial. 3.2

Lab Topology

As most of these machines emulate attacks and threats or network elements that can run in an enterprise environment, our laboratory is structured centered on a firewall that separated between security zones: a LAN environment (inside), a WAN environment (outside) and a demilitarize zone (DMZ).

1028

T. Bălan et al.

Fig. 2. The environment for one student of the cybersecurity lab. Different virtual machines can be started in each area or topology can be extended in GNS3, depending on the lab scenario

Each of the virtual machines was created in XEN Server like a reference image (“golden image”) that is than instantiated for each user on demand. Such images include besides the Windows 10 initial connection image also other images like: a pfSense firewall, Kali Linux distribution, a Metasploitable, DVWA, etc. Each student has his own LAN-WAN-DMZ environment. The environment for each student is isolated from the environment of another student. This is done by the usage of VLANs (Virtual LANs), thus, each student profile is running in a dedicated VLAN separated area. For the initial lab setup 16 student profiles were created, but the plan is to extend the hardware resources, so additional students could use the platform simultaneously. Teachers/tutors have dedicated accounts. For administrative purpose of the user roles, and Active Directory server was developed as part of the global topology. Each student environment has connectivity to Internet on the WAN site (Fig. 2), thus simulating a real scenario. We have observed some problems in connecting to Kali Linux from XEN Desktop, so we have also used the open source alternative No Machine (probably the source of this problem was the Desktop Environment running in our Kali distribution). The teacher or lab admin can decide on his own will the virtual machines that will be started or stopped, depending on the lab work to be performed. This is done via the XEN Orchestra, as seen in Fig. 3.

Building a Virtualized Cybersecurity Lab

1029

Fig. 3. XEN Orchestra, the method for teachers or lab admins to prepare the lab scenario, stopping the unnecessary VMs, thus saving resources

3.3

GNS3 Integration

GNS3 has evolved from a Cisco IOS emulator, to a complete network orchestrator that can integrate network elements form multiple vendors, virtual machines (QEMU, VirtualBox, VmWare), Docker virtual containers. The appliance list of GNS3 is impressive and continue expanding and additional GNS3 offers a GNS3 VM (Virtual Machine) that can run on top of a hypervisor with much better performance. This is why we have decided to integrate GNS3 because of the flexibility and possibility for the student to experiment with new elements. We should mention the Cisco ASAv next generation firewall image that can run in this environment, or the Security Onion distribution that can also run embedded in GNS3. The great advantage of the GNS3 VM is the possibility to connect the GNS3 topology to external elements, thus, each element can be connected in the LAN, WAN or DMZ part of the topology. Figure 4 presents a GNS3 topology, including a Cisco ASAv element that is connected via the “Cloud” element to WAN network (where a DHCP server is active, as visible in Fig. 2, and also connection to Internet is provided). As an example, a Site-to-site VPN can be established between the ASAv and the pfSense firewall, even if the pfSense is not part of the GNS3 topology. There are some limitations in relation to these Cloud elements, as there are actually Layer 2 bridges that can create L2 loops in the environment, so additional testing is needed when setting up the GNS3 integration.

1030

T. Bălan et al.

Fig. 4. GNS3 topology can be integrated with the rest of the virtualized environment. Cloud-1 element makes the connection of the Cisco ASAv to the WAN network

Depside the versatility introduced by GNS3, we have to mention some problems that we have seen when running on top of XEN Server. Many of the elements in GNS3 run in a virtualized environment (for example the Cisco ASAv is running on top of QEMU). Thus, in order to have ASAv working in GNS3 VM we had to enable nested virtualization in XEN Server. Apparently, there is a problem on running nested virtualization in GNS3 on top of Xen Server that results in a complete system crash. Thus, even if the complete solution was based on XEN Server, we had to move one of the three physical servers from XEN Server to VmWare vSphere (installed only on one machine, so we could use the free version). This machine was dedicated only to running GNS3 VMs.

4 Support from Industry and Usage of Open Source The hardware elements (the 3 servers) are Atos Bull Sequana servers, offered by Atos Romania, the Cisco images for the ASA, FTD, FMC are demo versions received courtesy of Cisco Romania. For most of the equipment or software elements there are academic programs (like for example the Citrix Academic program), the Nvidia academic program with very big discounts on the perpetual licenses for video card virtualization, other programs where universities can apply (like for example the Splunk SIEM academic program or the Cloudera academic program). In general, motivation of industry/vendor support in education is related to promotion of own products that are used and learned by the students and furthermore possible recruitment of graduates that are already trained to use the vendor technology. However, in case on the cybersecurity topic, the industry responded with enthusiasm for cooperation with universities, as vendors would like to make their name known in an area with exponential increase of market demand. Also, most of the open source tools have also a commercial version that could be further used in university projects, once the know-how is built.

Building a Virtualized Cybersecurity Lab

1031

5 Conclusion All the benefits of Cloud technologies are reflected also in the academic laboratory organization: flexibility, reuse of resources, scalability (the HW elements can be multiplied or upgraded), low investment on the desktop elements (thin clients) and use of perpetual licenses (for VDI Desktop virtualization and video board virtualization) are means for limiting the costs to one-time investment. We have illustrated a case study for a cyber laboratory integration. The paper is focused on the topology that is needed to be pre-deployed, thus the integration work that each student must perform is limited. Following the “classical” enterprise security zone separation (LAN, WAN, DMZ), plus the integration with GNS3 VM that increases the number of scenarios to be deployed, we have demonstrated a method for fast lab service orchestration, based on teacher requirements. This was made possible by also using the orchestration methods provided by XEN but can be further automated. Though problematic to be integrated with XEN Server and even with Hyper-V in the case of nested virtualization, the GNS3 VM offer the possibility to connect network components via the “Cloud” elements to any region of the topology, thus complex virtualized elements can be easily connected (Cisco ASAv or Security Onion) to other existing elements, or ad-hoc Docker containers can be easily installed. As next steps we would like to develop a scheduling and reservation system [23, 24] for the lab resources via a web dedicated interface [25], integrated with the local Active Directory server and with the Single Sign-On possibilities [26] (even extended to EDUROAM, already part of the university infrastructure). Thus, student will be able to reserve timeslots for their lab work and access remote the resources. Acknowledgment. We would like to show our gratitude to Atos Romania for their support in proving the hardware platform as sponsorship and integration efforts, Cisco Romania for providing demo version of networking software and other partners that have granted resources via academic programs.

References 1. Hart, B.: Building a Security Lab with Virtual Machines, (GSEC V1.4 Option 1), GIAC SANS. https://www.giac.org/paper/gsec/2186/building-security-lab-virtual-machines/ 103719 2. Izzat, M., Alsmadi, M.N.A.-K., Abu-Shanab, E.: Requirements and challenges for building a national open security lab. In: First Summit on Countering Cyber Crimes, Riyadh, KSA, 2729/10/2015, pp. 1-15 (2015) 3. Niyaz, Q., Sun, W., Xu, R., Alam, M.: LightVN: a light-weight testbed for network and security experiments. In: 12th International Conference on Information Technology - New Generations, pp. 459–464. IEEE (2015) 4. Bardas, A.G., Xinming, O.: Setting Up and using a cyber security lab for education purposes. Consortium Comput. Sci. Colleges, JCSC 28(5), 191–197 (2013)

1032

T. Bălan et al.

5. Mirkovic, J., Benzel, T.V., Faber, T., Braden, R., Wroclawski, J.T., Schwab, S.: The DETER project: advancing the science of cyber security experimentation and test. In: IEEE International Conference on Technologies for Homeland Security (HST), 8–10 November, IEEE, pp. 1–7 6. https://www.gns3.com/ 7. http://virl.cisco.com/ 8. https://www.cybrary.it/ 9. https://www.immersivelabs.com/ 10. https://www.pfsense.org/ 11. https://www.ipfire.org/ 12. https://www.kali.org/ 13. https://www.metasploit.com/download 14. http://www.dvwa.co.uk/ 15. https://www.osboxes.org/ 16. https://securityonion.net/ 17. https://www.alienvault.com/products/ossim 18. https://www.splunk.com/en_us/solutions/industries/higher-education/academic-licenses. html 19. https://remnux.org/ 20. https://digital-forensics.sans.org/ 21. Citrix Educational License program. https://www.citrix.com/buy/licensing/programs/ education.html 22. NVIDIA GPU Grant program. https://developer.nvidia.com/academic_gpu_seeding 23. Li, Y., Esche, S.K., Chassapis, C.: A scheduling system for shared online laboratory resources. In: 2008 38th Annual Frontiers in Education Conference, Saratoga Springs, NY, pp. T2B-1–T2B-6 (2008) 24. Wang, N., Chen, X., Song, G., Parsaei, H.: Develop a scheduler and federated authentication for remote laboratory access. In: Proceedings of 2015 12th International Conference on Remote Engineering and Virtual Instrumentation (REV), Bangkok, pp. 214–219 (2015) 25. Stelea, G.A., Fernoaga, V., Gavrila, C., Popescu, V., Murroni, M.: Mobile accessible rich internet web application enhanced with AMP publishing technology. Rev. Air Force Acad. 1, 71–78 (2019) 26. Hu, J., Sun, Q., Chen, H.: Application of Single sign-on (SSO) in digital campus. In: 2010 3rd IEEE International Conference on Broadband Network and Multimedia Technology (ICBNMT), Beijing, pp. 725–727 (2010)

Work-in-Progress: Developing a Master Programme for Specialists in Industry 4.0 How an International Partnership of Universities Developed the Curricula for a Master Programme for Specialists in Industry 4.0 Tom Savu(&) and Andrei Dumitrescu Manufacturing Engineering Department, POLITEHNICA University of Bucharest, Bucharest, Romania [email protected], [email protected]

Abstract. Industry 4.0 is storming the world and irreversibly changes it. If Industry 4.0’s physical components develop at an accelerated rate, the human component tends to keep pace. Considering this, it is the responsibility of universities to renew their curricula as soon as possible for the new industrial reality. The MSIE 4.0 project has brought together universities from Thailand, Poland, Portugal and Romania to develop a master programme oriented to Industry 4.0. After analysing existing curricula from various universities and performing a gap analysis between the skills required by industry and those offered by universities, a new curriculum was drawn up, consisting of 16 courses. Keywords: Industry 4.0


Master programme
Curriculum development

1 Introduction In the last decades, spectacular advances have been made in many areas. The internet, local networks, wireless connections, and powerful computers have made information of any kind retrieved, transmitted, and processed in huge quantities and at great speeds. The industry not only took over the advances in information technology, but has moved from mass production to small-scale production and even personalized production (in some areas). The development of products and services has outgrown the routine of processes associated with large market segments and is now addressing niches. And the list of progresses could continue. Industry’s development faces a new paradigm, called Industry 4.0. This name was used because, according to industry historians, it is a fourth industrial revolution, after the reference moments of: use of steam and mechanization; the use of electricity and assembly lines and, respectively, automation based on the first generations of computers. Industry 4.0 uses the cyber-physical systems [1], cloud computing process, the internet of things, the industrial internet of things [2], cognitive computations and artificial intelligence, and nanotechnologies [3], biotechnologies, additive technologies, etc. [4]. © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 1033–1038, 2021. https://doi.org/10.1007/978-3-030-49932-7_96

1034

T. Savu and A. Dumitrescu

Industry 4.0 works osmotically with new production strategies such as Agile Manufacturing and Mass Customization. Factories that until recently operated individually and isolated are now integrated into networks, and their autonomous systems exchange information. Moreover, virtualization of the manufacturing process and supply chain warrants lean inter-company operations offering real-time access to relevant product/service and production information for all collaborating parties [5]. Because the current society is increasingly interconnected, a change in an area of human activity affects all other areas, and if the change is major as the fourth industrial revolution, then many other areas will experience major changes. One example is the Smart City [6]. Inevitably, the human component of Industry 4.0, and it is not the only one, is undergoing transformation. But as in all revolutions, industrial or of other nature, the human component manifests a certain inertia. A cause of this inertia is the reluctance with which most professionals accept major changes. Another cause, perhaps more important, is the traditionalism of higher education institutions that hampers the process of dynamic change of study programs. But the process of adapting the academia to the new economic and social realities has begun. Many professionals in education and training are working to innovate the training of human resource for Industry 4.0. Research education [7], training of industry specialists 4.0 [8–10], advanced training methods and tools [11] are targeted and more others.

2 MSIE 4.0 Project The overall aim of MSIE 4.0 (“Curriculum Development of Master’s Degree Program in Industrial Engineering for Thailand Sustainable Smart Industry”) project is to enhance the capacity and ability of universities in Thailand (and not only) for the delivery of a high quality competence-based curriculum for Master’s degree in industrial engineering that supports sustainable smart industry (Industry 4.0), conforms to European Qualifications Framework (EQF) and is applicable to EU partner universities. The project will be specifically focused on [12]: 1. Modernization of the education of industrial engineering discipline in Thailand by the development of a curriculum for Master’s degree in industrial engineering to support sustainable smart industry; 2. Development of courses, learning and teaching tools, delivery processes and platforms for student-centred learning of the curriculum; 3. Implementation of modern ICT tools and methodologies for effective studentcentred learning of the curriculum; 4. Introduction of quality assurance and of the EQF approach for the delivery of the curriculum meeting international-accepted education requirements; 5. Establishment and continuation of partnerships among partner universities.

Work-in-Progress: Developing a Master Programme for Specialists in Industry

1035

The project’s partnership consists of prestigious universities from Thailand, Poland, Portugal and Romania: • • • • • • • • •

Asian Institute of Technology (AIT); Chiang Mai University (CMU); Czestochowa University of Technology (CUT); Khon Kaen University (KKU); King Mongkut’s University of Technology North Bangkok (KMUTNB); Prince of Songkla University (PSU); Thammasat University (TU); University of Minho (UMinho); University POLITEHNICA of Bucharest (UPB).

3 Developing the Curricula At the beginning, the partnership focused on examining several curricula of existing master programmes in the four countries. Useful information was obtained and a list of key competencies of the Industry 4.0 specialist could be written. In order to have a structured and verified approach, the analysis on industry and student needs used as references the study “Industry 4.0 Readiness” [13] and the webtool “Industry 4.0 Self-Assessment” of PricewaterhouseCoopers [14]. A special attention was given to correlation of the industry needs, student needs and the current achievement of academia. A gap analysis was performed and it resulted the Industry 4.0 applications on which a real competitive master curriculum should focus: Quality Management; Flexible Production Planning and Scheduling for Demand Changes and Customization; Maintenance Management; Data Distribution; Logistic and supply chain management; Inventory Management; Trend analysis; Forecasting; Real Time Process Control; Data Analytics etc. [15]. For all of the above, there were determined the role of master programme graduates and the required competence level. Before establishing the courses of the master programme, the partnership set by consensus, after several discussions, a series of principles: synchronization of competences with disciplines to be taught; targeting high levels in Bloom’s taxonomy; reducing the transfer of skills by teaching from the chair in favour of project-based learning, etc. Regarding the courses, each partner submitted a list of courses (name, structure, etc.) that it considered suitable for the specificity of the master programme. The final selection was made by the Project Executive Committee (PEC), body consisting of one representative from each partner. For each course, PEC established a team of few partners who will work on the course development. After elaboration, each course will be validated by the entire partnership. European and national legislation, as well as the regulations specific to each university, did not allow the design of a unique master programme that would be valid for absolutely all universities. It was decided that 6 of the 16 courses would be considered “core” and mandatory for the entire partnership. To these 6 courses will be added others (in variable number) depending on the objective interests of each university.

1036

T. Savu and A. Dumitrescu

Also, the number of teaching hours, the number of credits, etc. will be established within each partner university. At the moment of writing of this paper, the curriculum of the new programme was established and the teams are developing the 16 courses. Approximately 25–30% of each course has been completed, and in October 2019 the first modules will be tested. Then, as they will be ready, the new modules will be tested. The modules will not be tested by all partners, but according to a graph agreed by agreement. The approved courses are briefly described in the following paragraphs [16]. Course 1: Enterprise Management in Digital Economy. This course has the objective to provide the students with competences on using integrated and systemic solutions in helping the management to meet the requirements of Digital Economy. Students will learn on how to adapt management, its strategies and functions to smart and sustainable solutions that Industry 4.0 has brought to companies. Course 2: Project Management for Industry 4.0. This course has the purpose to prepare graduates to activate in and manage complex projects and multidisciplinary teams in the new highly agile digitized and challenging smart industries. There will be used digital platforms to fulfil personalised project services for individual customers time and time again. Course 3: Smart Operations Management. This course is aimed to build competences on development and implementation of continuous and efficient operations while creating a digital copy of the end-to-end process. It will use real world data to understand, assess, and simulate the end-to-end operation in order to improve and manage all operations efficiently. Course 4: Quality Management for Extended Enterprise. This course has the purpose to develop competences in implementing quality procedures of control and monitoring systems that cover manufacturing operations, supply chain operations, etc. Course 5: Sustainable Supply Chain Management. This course will enable students to acquire competences in supply chain management, i.e. key issues like: supply network design, inventory control and management, delivery contract, information exchange, distribution strategy, strategic alliance, purchasing strategies, pricing strategies, etc. Course 6: Digital Factory. The course is aimed to enable students to the use of information technology in order to configure, model, simulate, evaluate, and operate a manufacturing process. Course 7: Advanced Optimization: Techniques and Industrial Applications. The course has the objective to transfer to students the competence of using various optimization techniques aimed to improve the decision-making process for practical problems in industries. Methods like linear, integer, nonlinear, and dynamic programming as well as network models are used. Course 8: Intelligent Decision Support Systems. This course is aimed to provide students with competences related to advanced decision-making concepts, process, strategies, and technologies. The decision-making process will be combined with the concepts of agile manufacturing and Industry 4.0. Course 9: Applied Data Analytics. The course is aimed to transfer competences on statistical methods and techniques required for data analysis, and also different data

Work-in-Progress: Developing a Master Programme for Specialists in Industry

1037

mining methods, techniques and algorithms used in practical problems involving the processing of big data for decision-making purposes. Course 10: Cyber-Physical Industrial Systems. The course has the objective of offering students competences about: the basic characteristics of the Cyber-Physical Systems, their application fields, components selection rules, programming methodology, specific aspects related to different measured physical parameters, and data storage, reporting and communications. Course 11: Collaborative Manufacturing Systems. The course will create competences related to future emerging sustainable smart manufacturing systems. It will present advanced approaches to manufacturing processes, including manufacturing strategies, high-speed machining, flexible tooling, tool‐less assembly, generative numerical control, adaptive and predictive process control, embedded sensors, data and simulation, and nanotechnology. Course 12: Additive Manufacturing for Industry 4.0. This course has the objective to build student competence in additive manufacturing and related technology. Also, aspects of reverse engineering will be approached. The applications will be in manufacturing, medical, etc. sectors. Course 13: Innovative Product Design and Development. This course will transfer to students competences related to creative thinking techniques, searching optimum solutions, entrepreneurial actions, market analysis, identifying customer needs, new product development, building and testing prototypes, marketing the innovative products etc. Course 14: Human-Centric Design for Operator 4.0. This course is aimed to build student competence in solving problems related to the incompatibilities of human capacities and limitations with products, services, systems and environments. The focus will be on the human being in the work process associated with Industry 4.0. Course 15: Customer Experience-Driven Design. This course has the objective to build student competence in creating a positive customer experience based on the concept of customer experience management and on a systematic approach for an experience-driven design process. The customer perception, customer involvement, and customer experience will be envisaged also by this course. Course 16: Leadership Communication and People Development in Digital Era. This course is aimed to transfer to students the competence of communication to different cultures and to different types of people in order to create a competitive advantage.

4 Conclusions The MSIE 4.0 project was conceived to develop a master programme oriented to the current and future realities of Industry 4.0. It united universities from Thailand, Poland, Portugal and Romania. The existing curricula from various universities was analysed in order to gather information regarding the needed competences and the formal requirements of a master programme in industrial engineering.

1038

T. Savu and A. Dumitrescu

A gap analysis between the skills required by the industry and those offered by universities revealed the needed competences in all their complexity. This allowed the development of the new curriculum, which is composed of 16 courses. Acknowledgment. The “Curriculum Development of Master’s Degree Program in Industrial Engineering for Thailand Sustainable Smart Industry” (MSIE 4.0) project (586137-EPP-1-20171-TH-EPPKA2-CBHE-JP) has been funded with the support of the Erasmus + programme of the European Union.

References 1. Lee, J., Bagheri, B., Kao, H.A.: A cyber-physical systems architecture for industry 4.0-based manufacturing systems. Manuf. Lett. 3, 18–23 (2015) 2. Gilchrist, A.: Industry 4.0: The Industrial Internet of Things. Apress, New York (2016) 3. Miyazaki, K., Islam, N.: Nanotechnology systems of innovation—an analysis of industry and academia research activities. Technovation 27(11), 661–675 (2007) 4. Schwab, K.: The Fourth Industrial Revolution. Crown Business, New York (2017) 5. Brettel, M., Friederichsen, N., Keller, M., Rosenberg, M.: How virtualization, decentralization and network building change the manufacturing landscape: an Industry 40 Perspective. Int. J. Mech. Ind. Sci. Eng. 8(1), 37–44 (2014) 6. Nam, T., Pardo, T.A.: Conceptualizing smart city with dimensions of technology, people, and institutions. In: Proceedings of the 12th Annual International Digital Government Research Conference: Digital Government Innovation in Challenging Times, pp. 282–291. ACM (2011) 7. Lima, R.M., de Graaff, E., Mesquita, D., Aquere, A.L.: Engineering education (research) in higher education institutions. In: PAEE/ALE 2018, 10th International Symposium on Project Approaches in Engineering Education (PAEE) and 15th Active Learning in Engineering Education Workshop (ALE) (2018) 8. Benešová, A., Tupa, J.: Requirements for education and qualification of people in industry 4.0. Procedia Manuf. 11, 2195–2202 (2017) 9. Baygin, M., Yetis, H., Karakose, M., Akin, E.: An effect analysis of industry 4.0 to higher education. In: 2016 15th International Conference on Information Technology Based Higher Education and Training (ITHET), pp. 1–4. IEEE (2016) 10. Pfeiffer, S.: Effects of industry 4.0 on vocational education and training. Institute of Technology Assessment, Vienna (2015) 11. Schuster, K., Groß, K., Vossen, R., Richert, A., Jeschke, S.: Preparing for industry 4.0– collaborative virtual learning environments in engineering education. In: Engineering Education 4.0, pp. 477–487. Springer, Cham (2016) 12. https://msie4.ait.ac.th. Accessed 12 June 2019 13. Lichtblau, K.: Industrie 4.0-Readiness. Impuls-Stiftung (2015) 14. Industry 4.0 Self-Assessment. https://i40-self-assessment.pwc.de/i40/landing/. Accessed 10 May 2018 15. Dumitrescu, A., Lima, R., Chattinnawat, W., Savu, T.: Industry 4.0 competencies’ gap analysis. Industry 4.0 4(4), 153–156 (2019). ISSN (print) 2534-8582, ISSN (web) 2534997X 16. Working internal document of the MSIE 4.0 partnership (2019)

Teachers’ Perceptions Towards the Use of Mobile Augmented Reality The Case of Greek Educators Christina Pasalidou and Nikolaos Fachantidis(&) University of Macedonia, Thessaloniki, Greece [email protected], [email protected]

Abstract. This paper examines the perceptions of Greek Primary School teachers about the educational use of mobile augmented reality (MAR). The research of this study includes the design and development of an educational application which involves augmented reality and mobile learning. Additional information and digital features, following the school curriculum, were superimposed to the user’s real world through the utilization of mobile devices. Primary school teachers were familiarized with the application and formed their opinions regarding mobile augmented reality in the teaching process after using it. Then, they filled out a questionnaire based on the technology acceptance model (TAM). The analysis of the data that were collected showed significant correlations between the variables. In particular, it was found that teachers’ intention to use the mobile AR application that was designed is related both to perceived usefulness and to perceived ease of use. Teachers enjoyed the mobile augmented reality experience and considered the app as a useful and easy to use tool, which has a lot to offer to their teaching. Keywords: Mobile learning TAM


Augmented reality
Teachers’ perceptions


1 Introduction – Defining the Technology of Mobile Augmented Reality (MAR) Augmented Reality (AR) refers to any case where the real environment is augmented with virtual objects and computer-generated graphics [1]. According to Azuma [2], AR combines the real with the virtual world, supplementing the existing reality and offering real-time interaction. The increased use of augmented reality has enabled mobile learning [3]. Mobile devices, such as tablets and smartphones, are combined with AR, allowing users to interact with virtual content that is now part of their physical environment [4]. According to Henderson and Yeow [5], primary school teachers consider mobile devices, such as iPads, as useful learning tools. The great features and benefits that mobile devices offer have caused their use for educational purposes [6]. Moreover, the advantages of mobile technologies have lead researchers examine their use within formal and informal learning environments [7]. © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 1039–1050, 2021. https://doi.org/10.1007/978-3-030-49932-7_97

1040

C. Pasalidou and N. Fachantidis

The powerfulness and affordability of mobile devices have made them a suitable platform for AR, providing more authentic and engaging learning experiences [3]. Hwang, Wu, Chen and Tu’s research [8] on elementary school students showed that game based learning through mobile augmented reality contributes to students’ positive attitude towards learning and improves their performance. In addition, another positive aspect of mobile AR in education has been the creation of motivating learning environments for students [9]. Apart from that, the use of mobile devices implies quick and easy access to additional information, with students being able to collaborate and learn in different ways [10]. In particular, the ability of mobile devices to interact with users wherever and whenever they desire seems useful in teaching [11]. Through mobile computing, the information people get from their senses is combined with the augmented features in the real environment that they receive from mobile devices [12]. Their small size and variety of capabilities allow mobile devices to show augmented reality content, while their widespread use leads to the social acceptance of MAR [13]. The development of AR technology and mobile computing has resulted in their combination with pedagogical approaches, changing the existing conditions in the field of Education [14]. Mobile augmented reality applications for educational purposes are constantly being developed [15]. In the context of mobile and ubiquitous learning, students interact with the environment and have enriched rather than static lessons [16]. However, these technologies may cause students to be overwhelmed and confused with the amount of material and complexity of tasks [17].

2 Teachers and MAR Use in Education Teachers’ perceptions about ICT in education influence their adoption in the school classroom [18]. Similarly, it is expected that this will also be accurate about the integration of mobile augmented reality in education. Thus, this research studies Greek teachers’ perceptions concerning mobile augmented reality technology. However, many researchers have pointed their interest at this direction. Kamarainen and her colleagues [19] conducted a survey regarding the integration of mobile augmented reality in an educational excursion outside of the school. The project was called EcoMOBILE and through its implementation both students’ and teachers’ views on the usefulness and teaching value of the MAR technology were examined. Teachers found that this way of teaching seemed to be more effective and that the interaction between students and the environment was promoted. In addition, Kerawalla et al. [20], in their comparative study used an AR interface, as well as traditional methods of teaching physics and came to the conclusion that flexibility, guided exploration and adaptation of the additional virtual content to the curriculum are necessary for effective teaching. Another Augmented Reality system for learning was designed by Liarokapis and Anderson [21]. Through 3D models, images, texts, videos, and sounds students were able to gain a deep understanding of complex learning concepts and phenomena. The findings of their study showed that AR is a useful tool for learning with its impressive properties [21].

Teachers’ Perceptions Towards the Use of Mobile Augmented Reality

1041

Mobile augmented reality can also enhance analog games with 3D objects, animations and other virtual content [22]. In this research, teachers filled out a questionnaire based on the TAM model. The findings showed that teachers seemed to maintain a positive attitude and consider the augmented educational games useful for the teaching practice [22]. In Huang, Li, and Fong’s research [23], principals and teachers were asked to express their opinions about educational activities that utilized mobile devises and augmented reality. Participants were positive, interested and willing to integrate MAR in education, although they had some concerns [23]. The educational use of mobile devices concerned Zaranis, Kalogiannakis and Papadakis [24], who found them effective and valuable tools for teachers. The researchers examined the use of mobile devices to teach Mathematics to 4-6 year-old students. Additionally, Bozkurt [25] found that AR, emerging from ubiquitous technologies and mobile devices, provides enriched and unique learning experiences. From the review of the literature it has been clear that mobile learning and AR offer new perspectives in the teaching practice [26]. According to Wu, Lee, Chang, and Liang, mobile augmented reality offers a variety of opportunities to support teaching and learning [27]. Given that, the integration of MAR in the classroom seems to be really close. For this reason, it is essential to further study the perceptions and aspects that define Greek primary school teachers’ intentions and plans to improve their teaching through this relatively new technology.

3 Methodology 3.1

Participants

In this quantitative research, a sample of 206 Primary School teachers has participated (79 men, 127 women), coming from different age groups in order the sample to be more representative in terms of the teaching experience and familiarity of the participants with new technologies. 3.2

The Instrument

For the purpose of this study a questionnaire has been used, based on Davis’ Technology Acceptance Model [28]. The questions were closed-ended on a seven-point Likert scale (1: Strongly Disagree, 7: Strongly Agree). The questionnaire was translated to Greek by two certified translators and reviewed by qualified members of the University staff with expertise in the use of ICT in Education. A pilot study was conducted on 30 teachers. The questionnaire that was used consisted of two parts. The first part included general questions, such as the participant’s gender, age, years of experience, familiarity with mobile devices and the existence of previous experience with mobile augmented reality applications for educational purposes. The second part of the questionnaire had the Likert-scaled questions and examined three main variables; Perceived Usefulness, Perceived Ease of Use and Behavioral Intention. According to Davis [29], perceived

1042

C. Pasalidou and N. Fachantidis

usefulness refers to “the degree to which a person believes that using a particular system would enhance his or her job performance”, meaning that if using the new system shows positive outcomes on the person’s work, it will be perceived as useful. On the other hand, if using the system occurs with no difficulties and is free of effort, it is then perceived by the user as easy to use [29]. Those two factors are deemed to be of great importance, having an effect on the person’s behavioral intention and willingness to use the system which is being studied. The role of the acceptance of a technology that is introduced is determining for its future utilization. Most researchers have chosen the TAM model or extended versions of it in order to study the attitude which people hold towards a new technological system [30]. The validity of this widely used model has been examined by a variety of researchers [31–34]. In this study we also proceeded in reliability and validity tests (Cronbach’s a, Principal Component Analysis – PCA). 3.3 3.3.1

Research Procedure and Data Processing

Design and Development of an Educational Experience Combining Mobile Learning and Augmented Reality In this study, an educational mobile augmented reality application was designed, especially for the Greek primary school teachers. The app is about the near and far side of the moon, its rotation and the orbit around the earth. To view its augmented digital content, users scan the pictures that are found in 6th grade Geography school book. Each picture constitutes a marker that triggers the app to superimpose the visual objects to the real world (Fig. 1). Users need a mobile device (either smartphone or tablet) to have this experience. Its use provides quick access to the additional information and digital content. Taking advantage of mobile learning features, the educational experience about the moon can be implemented in any place, at any time, providing great flexibility. The augmented reality platform which was used to design the MAR application was BlippAR (www.blippar.com). The application works both for IOS and Android system. Teachers could and still can use it anywhere, anytime. The only requirements are; the markers of 6th grade Geography school book, internet connection and also to have downloaded the BlippAR app on the mobile device. The application has been designed and developed in order to be suitable for this research. First of all, teachers were familiar with the content and educational material, as the additional digital information of the application was based on the school curriculum. Consequently, there was not any cognitive load added to the teachers. Also, teachers’ experience, training and pedagogical knowledge made the assessment of the mobile AR app easy. In this way, teachers could form an opinion about the capabilities and perspectives of mobile augmented reality in their teaching about the moon. Finally, the mobile AR app which was designed met the teachers’ needs concerning the teaching of earth and sun relations.

Teachers’ Perceptions Towards the Use of Mobile Augmented Reality

1043

Fig. 1. Snapshots from the mobile AR app about the moon

3.3.2 Procedure Primary school teachers who participated in this study to one of the researcher’s place of work in order to take part in activities based on the use of educational technologies. Firstly, teachers were familiarized with the concept, while the mobile augmented reality application, which was designed especially for them, was explained. Questions that aroused were answered both at the beginning and during the use of the mobile AR app, as well. Secondly, a mobile device (tablet) was provided to the participants along with the augmented part of the Geography school book. Teachers were called to interact with the mobile AR app and the digital features that were superimposed into their physical environment. Teachers scanned all the markers so that they successfully complete the AR experience. Finally, after using the mobile AR application about the moon, participants were asked to fill out the questionnaire, concerning their perceptions toward their experience on mobile learning and augmented reality. The duration of the procedure was approximately 20 to 25 min for each participant. The data were collected during the academic year 2017–2018. 3.3.3 Data Analysis Data of this study were collected through the questionnaire and analyzed using the Statistical Package for the Social Sciences - SPSS Statistics 21.0. To conduct this research, both descriptive and inferential statistics were used in order to extract results and draw more accurate conclusions. More specifically, after conducting a Kolmogorov-Smirnov normality test in SPSS to determine the distribution of the data, nonparametric Spearman’s correlation coefficient was used. In addition, reliability analysis (Cronbach’s a) and factor analysis for convergent and discriminant validity were implemented.

1044

C. Pasalidou and N. Fachantidis

4 Results According to the statistical analysis of the data, there are some results worth to mention. First of all, the participants of this study considered themselves quite (47.1%) or extremely (37.9%) familiar with mobile devices and adequate to use them in their everyday lives. Additionally, 67% of the Greek Primary School teachers who participated in the study stated that they had not had a similar experience with mobile learning and augmented reality for educational purposes in the past. The majority of the participants believed that the educational mobile AR app which was designed enhanced their teaching performance, effectiveness and productivity. To be more specific, 98% of the teachers perceived the educational mobile AR app about the moon as useful for their teaching practice, while three (3) participants out of 206 disagreed with this statement and only one of them was uncertain about the usefulness of a tool such as the mobile AR app that was designed. A percentage of 90.8% of the participants felt that learning to operate the mobile AR app is an easy task. The same percentage found that the interaction with this educational application is not difficult to understand. Actually, 185 teachers of the sample (n = 206) considered the interaction to be quite clear, while 91.1% answered that the mobile AR app about the moon was easy to use. The intention to utilize mobile learning and augmented reality application in the classroom was prominent, given that only 3 out of 206 participants held a negative attitude towards the use of the designed AR app in their teaching. A percentage of 3% seemed hesitant, while the majority showed that they intended (95.6%), would try (95.1%) and planned (89.7%) to integrate this mobile technology into their teaching in the future. 4.1

Reliability Analysis - Internal Consistency Reliability Test

The internal consistency of this scale was evaluated by Cronbach’s alpha. The value of the coefficient in each case was above 0.7, securing the desired reliability in order to continue into further analysis of the data (Table 1). Table 1. Reliability analysis Cronbach’s Alpha No. of items Perceived usefulness 0.920 4 Perceived ease of use 0.932 4 Behavioral intention 0.918 3

4.2

Convergent and Discriminant Validity

In order to test the validity of the scale, Principal Component Analysis was conducted. Three main components were found. Each component showed high loadings only with the items which, according to the questionnaire, refer to it and low loadings with the rest items (Table 2).

Teachers’ Perceptions Towards the Use of Mobile Augmented Reality

1045

Table 2. Validity test – Rotated component matrix

Item 1: The use of the mobile AR app can enhance my teaching performance Item 2: The use of the mobile AR app can make my teaching more effective Item 3: The use of the mobile AR app can enhance the productivity of my teaching Item 4: Generally, I consider that the mobile AR app can be useful in my work Item 5: Learning to use the mobile AR app in the classroom would be easy for me Item 6: I find it easy to interact with the mobile AR app that was designed Item 7: The interaction with the mobile AR app is clear and easy for me to understand Item 8: Generally, I consider that the mobile AR app is easy to use Item 9: I intend to use the mobile AR app that was designed in my classroom Item 10: I will try to use the mobile AR app that was designed in the classroom Item 11: I plan to use the mobile AR app that was designed in my teaching practice Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization.

4.3

Component 1 2 0.821 0.251

3 0.274

0.845

0.221

0.305

0.854

0.240

0.212

0.760

0.280

0.293

0.139

0.855

0.285

0.385

0.790

0.244

0.330

0.799

0.303

0.238

0.860

0.257

0.307

0.349

0.791

0.307

0.306

0.828

0.366

0.315

0.796

Descriptive Statistics

As shown to Table 3 below, the mean of the variables being studied is around 6, referring to the participants’ high level of agreement on usefulness, ease of use and possible integration of mobile learning and augmented reality application in Greek schools. Table 3. Mean and standard deviation Mean Perceived usefulness 6.00 Perceived ease of use 5.80 Behavioral intention 5.75

Std. deviation 0.74 0.89 0.83

1046

4.4

C. Pasalidou and N. Fachantidis

Inferential Statistics

After conducting the Kolmogorov-Smirnov test of normality in SPSS, it was found that the data were not normally distributed. As a result, nonparametric Spearman’s correlation coefficient was used. Spearman’s rho showed that all the variables being studied in this research were correlated with each other. The correlations were significant at the 0.01 level (Table 4).

Table 4. Spearman’s correlations Perceived Usefulness (PU) Spearman’s rho

PU

PEU

a

Correlation Coefficient Sig. (2-tailed) Correlation Coefficient Sig. (2-tailed)

N Correlation is significant at the 0.01 level (2-tailed)

Perceived Ease of Use (PEU) 0.459a

Behavioral Intention (BI) 0.585a

0.000

0.000 0.563a 0.000 206

Those correlations are consistent with Davis’ TAM model [28, 35]. Both perceived usefulness (0.585) and perceived ease of use (0.563) indicated a positive and statistically significant correlation with Greek Primary School teachers’ intention to take advantage of the great possibilities that the mobile AR application about the moon had to offer to their teaching. In addition, teachers’ perceptions concerning the ease of use of the educational AR app seemed to correlate significantly with their opinions about its usefulness (0.459).

5 Conclusions The present study used TAM to examine the factors that influence Greek teachers of Primary Education and their general behavioral intention to integrate the mobile AR app about the moon, which was designed to facilitate and enhance their teaching, in the classroom. The findings of this study showed that the participants were familiar with mobile devices and used them in their everyday lives. This finding is in agreement with Goggin’s research [36], where it was stated that mobile devices and wireless networks play an important role in people’s daily lives. The widespread use and accessibility of mobile devices to the majority of the population, along with their new, developed capabilities, caused great potentials in their exploitation in mobile learning and in the

Teachers’ Perceptions Towards the Use of Mobile Augmented Reality

1047

support of Augmented Reality applications, as well [26]. Nowadays, mobile technologies are also applied in educational settings [25], changing the way students are being taught. In the case of the Greek primary school teachers, although they considered themselves familiar with mobile technologies, they were not informed about educational applications which combine mobile learning, augmented reality and additional content, based on the curricular. During this study, teachers had the chance to interact with a mobile augmented reality app that was designed for primary education. The digital content which was superimposed to the user’s real environment through a mobile device (compatible for both IOS and Android smartphones or tablets) was about the moon and its relation with the earth. After interacting with the mobile AR app, teachers were asked to express their opinions through a written questionnaire. The questionnaire was based on the TAM model [29]. According to this model, there are two main aspects which can influence behavioral intention to use a new technology. Those are perceived usefulness and perceived ease of use [28]. Findings of this study confirmed Davis’ theory on the acceptance of technology, emphasizing on the factors that will probably have a crucial role in the future utilization of mobile learning and augmented reality educational applications, such as the one shown in this study. Similarly, Koutromanos, Styliaras and Christodoulou, found that the components of TAM predicted in-service teachers’ intention to use a spatial, hypermedia environment in their teaching [37]. According to Venkatesh, Morris, Davis & Davis [38], people tend to be more positive and willing to use a new technological means in their job when they believe it is easy to operate and at the same time it fulfills the purpose it is meant to accomplish [38]. In this study, the mobile AR app was found to be useful in teaching of this specific lesson about the earth and the moon. Moreover, Huang, Li, and Fong [23] came to the conclusion that teachers consider the technology of mobile augmented reality as a useful educational means which can increase teaching effectiveness, especially when it involves abstract or difficult-to-understand concepts [23]. Particularly, 3D virtual objects, superimposed via mobile augmented reality technologies, can have a positive outcome regarding students’ learning and attention, enhancing the efficiency and effectiveness of teaching [39]. Recognizing those advantages, teachers become enthusiastic and impressed by the usefulness of mobile AR systems, providing additional information to the students in a different way [20]. The mobile AR app that was designed was characterized as easy to use by the majority of Greek primary school teachers. This fact increases the possibilities of it being used in education, since the easy manipulation and interaction with 3D virtual objects [40], along with the use of mobile technologies [41], can provide profitable learning experiences [15, 42]. In this study, teachers seemed positive, having the intention to utilize the educational AR application about the moon in the future. According to Alkhattabi [43], there is great willingness and acceptance of mobile augmented reality technology for education [43]. This acceptance causes teachers’ desire to use it for educational purposes [22], making it a part of their teaching practice. Further research should be made, examining other factors that could affect teachers’ willingness to use mobile and augmented reality technologies in their classroom.

1048

C. Pasalidou and N. Fachantidis

Another suggestion would be to focus on the aspects which discourage and create hesitations to teachers concerning the integration of those technologies in education, such as the lack of technological knowledge, the anxiety during their use and the long hours of preparation. Finding those aspects could generate solutions.

References 1. Milgram, P., Kishino, F.: A taxonomy of mixed reality visual displays. IEICE Trans. Inf. Syst. 77(12), 1321–1329 (1994) 2. Azuma, R.T.: A survey of augmented reality. Presence Teleoperators Virtual Environ. 6(4), 355–385 (1997) 3. Alexander, B., Ashford-Rowe, K., Barajas-Murph, N., Dobbin, G., Knott, J., McCormack, M., … & Weber, N.: EDUCAUSE Horizon Report 2019 Higher Education Edition, pp. 3– 41. EDU19 (2019) 4. Dunleavy, M.: Design principles for augmented reality learning. TechTrends 58(1), 28–34 (2014) 5. Henderson, S., Yeow, J.: iPad in education: a case study of iPad adoption and use in a primary school. In: 45th Hawaii International Conference on System Science, pp. 78–87. IEEE (2012) 6. Squire, K., Klopfer, E.: Augmented reality simulations on handheld computers. J. Learn. Sci. 16(3), 371–413 (2007) 7. Koutromanos, G., Avraamidou, L.: The use of mobile games in formal and informal learning environments: a review of the literature. Educ. Media Int. 51(1), 49–65 (2014) 8. Hwang, G.J., Wu, P.H., Chen, C.C., Tu, N.T.: Effects of an augmented reality-based educational game on students’ learning achievements and attitudes in real-world observations. Interact. Learn. Environ. 24(8), 1895–1906 (2016) 9. Jamali, S.S., Shiratuddin, M.F., Wong, K.W.: A review of augmented reality (AR) and mobile-augmented reality (mAR) technology: learning in tertiary education. Learn. High. Educ. 20(2), 37–54 (2013) 10. Gikas, J., Grant, M.M.: Mobile computing devices in higher education: Student perspectives on learning with cellphones, smartphones & social media. Internet High. Educ. 19, 18–26 (2013) 11. Ardito, C., Buono, P., Costabile, M.F., Lanzilotti, R., Piccinno, A.: Enabling interactive exploration of cultural heritage: an experience of designing systems for mobile devices. Knowl. Technol. Policy 22, 79–86 (2009) 12. Huang, Z., Li, W., Hui, P., Peylo, C.: CloudRidAR: a cloud-based architecture for mobile augmented reality. In: Proceedings of the 2014 Workshop on Mobile Augmented Reality and Robotic Technology-Based Systems, pp. 29–34. ACM (2014) 13. Papagiannakis, G., Singh, G., Magnenat-Thalmann, N.: A survey of mobile and wireless technologies for augmented reality systems. Comput. Animation Virtual Worlds 19(1), 3–22 (2008) 14. Nincarean, D., Alia, M.B., Halim, N.D.A., Rahman, M.H.A.: Mobile augmented reality: the potential for education. Procedia-Social Behav. Sci. 103, 657–664 (2013) 15. Billinghurst, M., Henrysson, A.: Research directions in handheld AR. IJVR 5(2), 51–58 (2006) 16. Chin, K.Y., Chen, Y.L.: A mobile learning support system for ubiquitous learning environments. Procedia-Social Behav. Sci. 73, 14–21 (2013)

Teachers’ Perceptions Towards the Use of Mobile Augmented Reality

1049

17. Dunleavy, M., Dede, C., Mitchell, R.: Affordances and limitations of immersive participatory augmented reality simulations for teaching and learning. J. Sci. Educ. Technol. 18(1), 7–22 (2009) 18. Jimoyiannis, A., Komis, V.: Examining teachers’ beliefs about ICT in education: implications of a teacher preparation programme. Teacher Dev. 11(2), 149–173 (2007) 19. Kamarainen, A.M., Metcalf, S., Grotzer, T., Browne, A., Mazzuca, D., Tutwiler, M.S., Dede, C.: EcoMOBILE: Integrating augmented reality and probeware with environmental education field trips. Comput. Educ. 68, 545–556 (2013) 20. Kerawalla, L., Luckin, R., Seljeflot, S., Woolard, A.: “Making it real”: exploring the potential of augmented reality for teaching primary school science. Virtual Reality 10(3–4), 163–174 (2006) 21. Liarokapis, F., Anderson, E.F.: Using augmented reality as a medium to assist teaching in higher education. In: Proceedings of the 31st Annual Conference of the European Association for Computer Graphics (Eurographics 2010), pp. 9–16 (2010) 22. Yilmaz, R.M.: Educational magic toys developed with augmented reality technology for early childhood education. Comput. Hum. Behav. 54, 240–248 (2016) 23. Huang, Y., Li, H., Fong, R.: Using augmented reality in early art education: a case study in Hong Kong kindergarten. Early Child Dev. and Care 186(6), 879–894 (2016) 24. Zaranis, N., Kalogiannakis, M., Papadakis, S.: Using mobile devices for teaching realistic mathematics in kindergarten education. Creative Educ. 4(7), 1–10 (2013) 25. Bozkurt, A.: Augmented reality with mobile and ubiquitous learning: immersive, enriched, situated, and seamless learning experiences. In: Digital Tools for Seamless Learning, pp. 27– 41. IGI Global (2017) 26. Sural, I.: Mobile augmented reality applications in education. In: Mobile Technologies and Augmented Reality in Open Education, pp. 200–214. IGI Global (2017) 27. Wu, H.K., Lee, S.W.Y., Chang, H.Y., Liang, J.C.: Current status, opportunities and challenges of augmented reality in education. Comput. Educ. 62, 41–49 (2013) 28. Davis, F.D.: User acceptance of information technology: system characteristics, user perceptions and behavioral impacts. Int. J. Man Mach. Stud. 38(3), 475–487 (1993) 29. Davis, F.D.: Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 13, 319–340 (1989) 30. King, W.R., He, J.: A meta-analysis of the technology acceptance model. Inf. Manag. 43(6), 740–755 (2006) 31. Wong, K.-T., Osman, R.-B.T., Goh, P.S.C., Rahmat, M.K.: Understanding student teachers’ behavioural intention to use technology: technology Acceptance Model (TAM) validation and testing. Int. J. Instr. 6(1), 89–104 (2013) 32. Wu, K., Zhao, Y., Zhu, Q., Tan, X., Zheng, H.: A meta-analysis of the impact of trust on technology acceptance model: investigation of moderating influence of subject and context type. Int. J. Inf. Manag. 31(6), 572–575 (2011) 33. Šumak, B., Hericko, M., Pušnik, M.: A meta-analysis of e-learning technology acceptance: the role of user types and e-learning technology types. Comput. Hum. Behav. 27, 2067–2077 (2011) 34. Schepers, J., Wetzels, M.: A meta-analysis of technology acceptance model: investigating subjective norm and moderation effect. Inf. Manag. 44, 90–103 (2007) 35. Venkatesh, V., Davis, F.D.: A theoretical extension of the technology acceptance model: Four longitudinal field studies. Manag. Sci. 46(2), 186–204 (2000) 36. Goggin, G.: Cell Phone Culture: Mobile Technology in Everyday Life. Routledge, New York (2012)

1050

C. Pasalidou and N. Fachantidis

37. Koutromanos, G., Styliaras, G., Christodoulou, S.: Student and in-service teachers’ acceptance of spatial hypermedia in their teaching: the case of HyperSea. Educ. Inf. Technol. 20(3), 559–578 (2014). https://doi.org/10.1007/s10639-013-9302-8 38. Venkatesh, V., Morris, M.G., Davis, G.B., Davis, F.D.: User acceptance of information technology: toward a unified view. MIS Q 27, 425–478 (2003) 39. Chang, Y.J., Chen, C.H., Huang, W.T., Huang, W.S.: Investigating students’ perceived satisfaction, behavioral intention, and effectiveness of English learning using augmented reality. In: 2011 IEEE International Conference on Multimedia and Ex (2011) 40. Shelton, B.E., Hedley, N.R.: Using augmented reality for teaching earth-sun relationships to undergraduate geography students. In: Augmented Reality Toolkit, The First IEEE International Workshop, pp. 1–8. IEEE (2002) 41. Schmalstieg, D., Wagner, D.: Mobile phones as a platform for augmented reality. Connections 1, 1–2 (2009) 42. Billinghurst, M., Kato, H., Myojin, S.: Advanced interaction techniques for augmented reality applications. In: International Conference on Virtual and Mixed Reality, pp. 13–22. Springer, Heidelberg (2009) 43. Alkhattabi, M.: Augmented reality as e-learning tool in primary schools’ education: barriers to teachers’ adoption. Int. J. Emerging Technol. Learn. (iJET) 12(2), 91–100 (2017)

NavMusApp: Exploring the Instrumental Continuum A Mobile App for Classifying Musical Instruments in a Global Perspective Dionysios Politis1(&), Veljko Aleksić2, Gregory-Telemachos Stamkopoulos3, and Georgios Kyriafinis4 1

School of Informatics, Aristotle University of Thessaloniki, Thessaloniki, Greece [email protected] 2 Faculty of Technical Sciences, University of Kragujevac, Čačak, Serbia [email protected] 3 University Ecclesiastical Academy of Thessaloniki, Thessaloniki, Greece [email protected] 4 Cochlear Implantation Center, AHEPA University Hospital, Thessaloniki, Greece [email protected]

Abstract. A systematic scientific taxonomy designated upon the musicological and physical sound properties of Musical Instruments dates since 1914; it is the legendary Hornbostel-Sachs classification scheme. However, lacking, then, the communicative ability to contemplate profoundly and at length the tremendous multiplicity of worldwide contemporary musical subtle differences in sound expression and nuances of national identities, it provided an “ontology”-based methodology for classifying the organological discourse in depth at a later time. Even further, as digitally mastered sounds have prevailed recently, mobile computing provides suitable instrumentation meant to anticipate, measure and scale hundreds of new or “marginalized” instruments that seek bibliographic classification. The multimedia potential, both for input and output of an App serving as a measuring device is figuratively presented. Keywords: Musicology


Mobile social networks
Interactive learning

1 Introduction Before the Multimedia Revolution through Internet (social) networks and similar ongoing narratives in the blogosphere, comprising a series of events occurring with an increasingly fast pace since the first years of this millennium, the global awareness of instrumentalism as far as its inhibiting effects over musicality are concerned, was more or less localized. Indeed, within the cultural domain of each civilization in perspective, certain instruments flourished, following a long evolutionary course. However, many of these © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 1051–1057, 2021. https://doi.org/10.1007/978-3-030-49932-7_98

1052

D. Politis et al.

were not conductive to mainstream orchestral performances of Western world musical paradigms [1]. For example, in the famous 2000 AD epic drama historical film “Gladiator”, the Academy Award winner composers H. Zimmer and L. Gerrard combine in their score, originating from distinctive Western Music masterpieces and enhanced by digital effects, a memorable woodwind instrument performance, that of duduk. Although this instrument was well perceived in the Balkans, Black Sea countries and Middle East for millenniums, it was not that well known in what we call the Western world nor had an accurate place in the endemics of Common Music Notation. It is not surprising, thus, that in most such classification schemes, duduk was though something like an “Armenian oboe” [2]. However, in the mentality of the people in South Eastern Europe, Caucasus, Asia Minor and Middle East such an unsettling perception of something very common to them [3], for centuries, provokes reasonable abstruseness (Fig. 1).

Fig. 1. Left, the distinguished performer who contributed to the duduk melodies of the “Gladiator” film soundtrack. Right, a 16th century AD fresco from the Valaam monastery, Meteora, Greece, thematizing a 10th century BC event that took place in Jerusalem and depicting the instruments used within this diachrony.

Historical evidence of any kind is vital to reconstruct the lineage of each instrument, when pre-16th century instruments are concerned. Although somewhat “old styled”, sources from the Bible (especially the Psalms), the Greek musicological literature and the Roman Empire counterparts [4], along with some archaeological findings are critical for reconstructing not only the global synchrony but also the diachrony of this continuum. Indeed, as seen in Fig. 2, this continuum seems to have unprecedented depth in historical terms, and a more brilliant future. More or less, the taxonomic systems primarily classify instruments into four or more main groups [5]. The principal mode for categorizing them relies on the way they produce sound. What makes this diversion more extensive, in recent times, is the way contemporary digital instrument manufacturers cope with the wind instruments or idiophones, i.e. organs that radically alter the fabric for instrument classification, in comparison to Western world schemes.

NavMusApp: Exploring the Instrumental Continuum

1053

Fig. 2. An exemplar woodwind instruments collection depicting specimens extensively used in the Balkans, exerted from the personal assembly of Professor for Musicology I. Kaimakis, as presented in a fair at the Vafopouleion Cultural Center of Thessaloniki in 2013.

Even further, a prevalent trend in the Computer World, as it is applied within the Computer Music industry is to create synthesizer-based instruments, with no clear affiliation to the way that the sound is produced (Fig. 3). In terms of classification, it seems that in the electronic world the way a sound is produced is not the primary concern but, rather, the way the acoustic signal is perceived.

Fig. 3. Left, Professor I. Kaimakis performing a non-standardized asymmetrical two-piped woodwind instrument. Center and Right, the classification of some 700 synthesized instruments along with the Interfacing Multi Pad of a widespread YamahaTM keyboard-synthesizer used in this research.

2 Problem Formulation: Classifying an Extending Continuum The use of electronic editions for a huge variety of instruments, along with the ability to enhance the collection of a synthesizer with professional-sounding tracks has the big advantage of enabling musicians to experiment with variants of existing, well received instruments, and thus enhancing their performance style, but, the same time, it

1054

D. Politis et al.

encompasses the risk of creating chaotic assemblies of non-documented instruments that are not compatible with the commercial MIDI arrangement [6]. Indeed, performers may enhance their staging potential, may play chords and other sequence-based MIDI effects, may vary the expression altering velocity settings, producing very good results, but, the yielded riff from different phrasing cannot generate a Composer Set Documentation, i.e. an assembly of classified MIDI-style instruments, globally accepted. This situation produces latency in the educational settings of musicology, since digital effect banks and programs in electronic music do not serve as mere assemblies of VST instruments, but rather as creativity systems optimizing system performance. Controllers are destined to configure issues of faster, easier access to edited effect parameters modulating phasing effects that assign performance contours with programming parameters. As seen in Fig. 4, the sound character of the performing instruments, the tonal response, especially in non-Western Music scales and the sonic adjustments heavily influence the staged rendition of the artistic ensemble [7].

Fig. 4. Left, the “Mousiko Polytropo” ensemble performing non-Western Music tunes, under the guidance of their leader, Professor I. Kaimakis. Right, the approximation of the performing two-piped woodwind instrument with a closely affiliated popular instrument from Asia Minor.

As musicians clear out, what makes the performance seen in Fig. 4 “different” from what a usual keyboard-synthesizer would reproduce, is the scale predominantly used, clearly unalike to any major or minor scales reproduced by Common Music Notation (aka CMN) schemes and tools. Even further, the dancers may step into the stage with complex rhythmic styles, like 7/8 (Kalamatianos), 6/8 (Syrtos) or “Tsifteteli” like, which are not, though, a problem for the synthesizer to accurately produce, provided the user has the right Music Cartridge to put in (Fig. 3, center). However, the ability to differentiate between closely ascertained styles for non Western Music patterns is a matter of ongoing Machine Learning research, now just having overcome the essentials [8].

NavMusApp: Exploring the Instrumental Continuum

1055

For example, the brilliant work of Hornbostel-Sachs [9] gives the schemes seen in Fig. 5 for classifying the instruments played by “Mousiko Polytropo” members. On the upper left corner, an abstract of the original 1914 survey is presented, classifying the percussion instrument, while below it lays a contemporary description, in terms of Physical properties [10], of the instrument recited by Professor Kaimakis.

Fig. 5. Left, up, an abstract from the 1914 Hornbostel-Sachs treatise describing, in terms of Physics, the percussion instrument of the recital. Left, down, a contemporary Hornbostel-Sachslike description of the two-reed aerophone instrument in perspective. Right, a comparison, in terms of microtonal correspondences, of the scales involved in such recitals.

Nevertheless, an Eastern Mediterranean based musicologist would argue that, although detailed, the Hornbostel-Sachs style description of the performing musickinetic recital, fails short in giving to the non-expert any nuance of what the staged event is about. Indeed, the scales used for this small-concert performance are having an amazing microtonal distribution, which would be difficult to conceptually describe in Western music nomenclature, let alone the CMN semantics [11]. Actually, in this point lays the strength of Mobile Communication and Learning. As contemporary events are usually video recorded using portable devices, if within a social network or blogosphere the small footage of the recital was distributed, even the reviewers unaware of musicological scholarly theories would acknowledge that this oriental performance has more to do with a different “touch and feel” of the modes and scales used, rather than the instruments themselves (which in antiquity were equivocally received in East and West Mediterranean [4, 12]). To solve this lack of data insight, a multimedia App was developed upon the Hornbostel-Sachs ontological scheme. The operator uses his smartphone as a navigator, attempting to find the closest existing instrument with the one actually in target, either on grounds of constructional similarity or in terms of sonic affiliation, i.e. out of the perceived sounds in exemplar melodies associated with it. The application provides a basic starting point centered on the 4 basic HornbostelSachs categories. Although not very proficient for electronic music, still the distinction of Membranophones, Chordophones, Aerophones and Idiophones seems to be a very promising ontology to deal with [10].

1056

D. Politis et al.

3 Navigating in the Unclassified World of Music The NavMuseApp user will commence his journey into the world of 100 + , thus far, instruments recorded within its database. A starting point may be the basic ontological scheme for classifying instruments, described in previous sections and figures of this research. In Fig. 6 details of this Interfacing methodology are depicted.

Fig. 6. Navigating with NavMusApp. One option for perspective users is to commence with the Hornbostel-Sachs ontologies, left, or alternatively, to have a visual search on the Data Base records, center. Videos, music samples and photos (right) are an asset in addition.

The user, may, of course, search for of the instruments supported in this App using the search algorithm compiled; as seen in Fig. 7, a part of its name, if not all the wording, would suffice to bring to surface the whereabouts of the organology involved. In practice, the operator of this application is inherently using Learning Models having to do with sound characteristics attested to a worldwide gamut of musical instruments. Musicality, as a science, apart from being an art, combines “natural” scales, for each instrument or group of instruments with musical styles and techniques. This App aims to ameliorate the performance of its users by improving the performance the computer system involved. Not only uncategorized instruments may be added to an enhanced model of the Hornbostel-Sachs ontology, resident within the application, but, furthermore by deciphering the frequency response of an instrument, based on its pitch, “color” and modal distributions, a more universal approach to the substrate of global musicality is achieved.

NavMusApp: Exploring the Instrumental Continuum

1057

Fig. 7. Searching instruments by their (English) name. Acknowledgement. The authors would like to express their gratitude to Professor of Musicology I. Kaimakis, along with his “Mousiko Polytropo” ensemble members, for providing the multimedia material upon which this research was based.

References 1. Montagu, J.: Origins and Development of Musical Instruments. Scarecrow Press, Lanham (2007) 2. Picken, L.: Folk Musical Instruments of Turkey. Oxford University Press, London (1975) 3. Barsky, V.: Chromaticism. Harwood Academic Publishers, Amsterdam (1996) 4. Pöhlman, E., West, M.L.: Documents of Ancient Greek Music. Oxford University Press, Oxford (2001) 5. Kartomi, J.: The classification of musical instruments: changing trends in research from the late nineteenth century, with special reference to the 1990s. Ethnomusicol. J. Soc. Ethnomusicol. 45(2), 283–314 (2001) 6. Selfridge-Field, E. (ed.): Beyond MIDI: The Handbook of Musical Codes. MIT Press, Boston (1997) 7. Newson, L., Richerson, P.J., Boyd, R.: Cultural evolution and the shaping of cultural diversity. In: Kitayama, S., Cohen, D. (eds.) Handbook of Cultural Psychology, pp. 454– 476. Guilford, New York (2007) 8. Pennanen, P.R.: Lost in scales: Balkan Folk Music Research and the Ottoman Legacy. Myзикoлoгиja 8, 127–147 (2008) 9. von Hornbostel, E., Sachs, C.: Systematik der Musikinstrumente. Ein Versuch Zeitschrift für Ethnologie xlvi, 553–590 (1914) 10. Spyridis, H.: Physics and Music Acoustics. Grapholine, Thessaloniki (2005) 11. Giannelos, D.: La Musique Byzantine, p. 1996. L’Harmattan, Paris (1996) 12. Mavroedis, M.: The Musical Modes in Eastern Mediteranean Sea – The Byzantine Echos – The Arabic Maqam – The Turkish Maqam (in Greek). Fagotto Editions, Athens (1999)

Developing Communities of Practice to Maximize the Usability and Impact of Clean Sport Education in Europe: IMPACT Project Lambros Lazuras1, Antonia Ypsilanti1, Vassilis Barkoukis2, Panagiotis Stylianidis2, Nikolaos Politopoulos2, and Thrasyvoulos Tsiatsos2(&) 1 Sheffield Hallam University, Sheffield, UK {L.Lazuras,A.Ypsilanti}@shu.ac.uk 2 Aristotle University of Thessaloniki, Thessaloniki, Greece [email protected], {pastylia,npolitop,tsiatsos}@csd.auth.gr

Abstract. Over the last decade, the European Commission, the World AntiDoping Agency and the International Olympics Committee have made significant investments for the development of evidence-based anti-doping education. Although this is an important step towards eliminating doping in amateur and professional, elite sport, still most of the existing projects operate in a fragmented manner and there is a considerable lack of synergies. Consequently, there has been little consideration of effective ways to manage and sustain the knowledge created in this area in ways that will further maximize and sustain the usability and impact of existing anti-doping education initiatives. For this purpose, project IMPACT was developed in order to deliver, for the first time, multi-stakeholder Communities of Practice for Clean Sport Education across Europe. Through the implementation of face-to-face and web-based communities of practice, IMPACT will provide a knowledge management framework that will transform existing anti-doping educational resources into meaningful, timely, and sustainable tools, and stimulate the generation of innovative antidoping educational solutions. This will be achieved through a strategic partnership between academic experts in anti-doping research and online education, and international and national sport stakeholders (i.e., sport associations; antidoping agencies; organizations for doping prevention in amateur sports). Keywords: VCoPs


Anti-doping
Sports

1 Introduction The use of prohibited performance-and-appearance enhancing drugs (PAEDs) is the most common doping method in both competitive/elite and amateur athletes. The extant research has shown that self-reported PAEDs use in elite athletes can range between 14% and 39%, [1], whereas higher prevalence, 43.6% to 57.1%, has been demonstrated in studies using indirect questioning methods, such as the random response technique. The use of PAEDs is prolific in young amateur athletes and gym © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 1058–1064, 2021. https://doi.org/10.1007/978-3-030-49932-7_99

Developing Communities of Practice to Maximize the Usability

1059

goers too. A study in five European countries showed that approximately 1 in 5 amateur athletes and exercisers aged between 16 and 25 years had used PAEDs at least once, and a recent review of the extant research showed that PAEDs use can be initiated as early as the age of ten years [2]. These figures suggest that a large population of young people engaged in sport and exercise are at-risk for, or have already initiated the use of PAEDs - a class of substances, such as anabolic steroids, that have been associated with both short and long-term health effects; mood changes and mental health difficulties; suicidal thoughts and attempts; and even early death [3–8]. One way to tackle the use of PAEDs use in young people involved in sport and exercise is through concerted and evidence-based education. This is even more important in the context of amateur and grassroots sport where anti-doping education and related preventive measures are scarce, as compared to elite and organized sport [9, 10]. In recognition of this problem, different sport stakeholders, governing bodies and regulatory authorities have invested in the development evidence-based (online, offline and blended) education against the use of PAEDs, and more than 30 research projects have been funded over the last 5 years. Furthermore, the World anti-Doping Agency (WADA) recently issued the International Standard of Education (ISE) that mandates that regional and national anti-doping organizations develop and implement antidoping education initiatives targeting both athletes and coaches. While this advance in anti-doping education is welcomed, still, the following challenges are faced: a) firstly, there appears to be a considerable fragmentation of the generated knowledge mainly because the different projects/stakeholders work on an isolated mode and synergies with other project teams across the globe are scarce; b) competing and overlapping projects exist and there is little effort in disentangling “what works” and identifying best practices in anti-doping education; c) most projects endure for 2 to 3 years and their findings tend to phase out after the lifetime of the project mainly due to lack of sustained funding. Therefore, there is a need to develop an approach that will enable synergies among different stakeholders and end-users of anti-doping education (e.g., educators, physical education teachers, personal trainers, coaches, athletes), facilitate the recognition of best practices and attendant knowledge transfer among interested parties, and foster the sustainability and long-term use of existing anti-doping education by introducing relevant projects to different populations (e.g., outside the consortium of a given project) and for longer periods of time (i.e., beyond the project’s timelines). The main aim of this paper is to propose an innovative approach that will address these issues through the application of new learning technologies, namely Virtual Communities of Practice (VCoPs). The paper is structured as follows. The next section presents the idea of CoPs and why they are important for anti-doping education, and the subsequent sections present how physical/face-to-face and virtual communities of practice for clean sport education will be realized through project IMPACT, an international project that has been funded for two years by the Erasmus+ Sport Program of the European Commission.

1060

L. Lazuras et al.

2 Communities of Practice According to [11], communities of practice represent “groups of people who share a concern or a passion for something they do and learn how to do it better as they interact regularly”. The guiding principle of a CoP is that learning is not necessarily a top-down process, with knowledge flows from an expert (e.g., educator) to recipients without prior knowledge on the subject matter (i.e., pupils; students; trainees). Rather, learning is “situated”, knowledge flows are bidirectional, and knowledge is shaped and used dynamically as the members of a learning community share ideas, experiences and practices relevant to the subject matter they’re interested in [12]. According to this perspective learning happens through social interactions and the active involvement of “learners” in the initiation, development, and sustainability of the community of practice. According to [13], the three key contentions of communities of practice are: a) Knowledge resides in the “act of knowing”: Expertise is the result of “static” knowledge, and of accumulated experience that has been gained through practice. Communities of practice serve as a dynamic and living repository of accumulated knowledge. b) Knowledge is tacit as well as explicit: In addition to formal and explicit knowledge that can be delivered through manuals and formal training, tacit (or implicit) knowledge represents the accumulated knowledge that is embodied in experiences and represents a deep understanding of complex and context-specific problems. Tacit knowledge is valuable and irreplaceable and should be treated as equally (if not more) important to explicit knowledge. c) Knowledge is social and dynamic: Although the subjective experience of knowing a subject matter can be largely individual, knowledge itself is a social asset that we get to earn through social interactions with others with more expertise and different practices - in fact, the involvement of people/actors with different perspectives, knowledge assets and practices is often used to stimulate innovative solutions. Knowledge is also changing at a fast pace and individual actors (e.g., individual employees, individual organizations) cannot follow the fast-pacing knowledge generation. Communities of practice provide the framework where knowledge can be captured, contextualized, and re-used in ways that serve the individual and collective learning needs of involved parties. There are three main reasons why Communities of Practice are important for clean sport education in both amateur and elite sport. First of all, situated learning is learning based on current practices and relevant training needs and policy objectives [12]. So defined, anti-doping education that is meant to make a significant impact in behavior change (i.e., reduce the prevalence and risk for doping use across levels of sports) and policy-making (i.e., enable the development of polices that will gradually eradicate doping from sports) cannot be abstracted from the very context it takes places, or the needs of actors and sport stakeholders that are meant to tackle doping in sports. Situated learning and more specifically, communities of practice, can provide a useful alternative to existing approaches to anti-doping education that have been largely

Developing Communities of Practice to Maximize the Usability

1061

guided my more “traditional” learning styles that involve top-down knowledge flows (e.g., from academic experts to a limited number of sport stakeholders or athletes), and do not allow for meaningful social interactions among sport stakeholders that are directly involved in anti-doping practices as part of their regular operations. Secondly, although a large number of anti-doping education projects have been funded by the European Commission, the International Olympics Committee, and the World Anti-Doping Agency in the last 5 years, there have been little synergies and collaboration between them. It appears that there has been an overemphasis on antidoping knowledge creation at the expense of knowledge management. Unless knowledge is further utilized beyond existing project’s lifetime, it is rendered obsolete and its impact is undermined; unless knowledge is stored and integrated in diverse contexts (e.g., cultural, organizational); and unless knowledge on anti-doping is contextualized to different training and policy needs, then it becomes abstracted from (and quite possibly irrelevant to) the social contexts that are in need of it. A large body of research in diverse settings, ranging from secondary education, to medical education, and organizational innovation has demonstrated that communities of practice significantly improve knowledge management processes and can enable the creation of innovative solutions - this is a really important asset in today’s knowledgedriven economy [14–16]. Communities of Practice have also been applied to help translating innovative research findings into actual clinical practice in the healthcare context [17]. Importantly, communities of practice yield such benefits in organizational knowledge management and utilization in a cost-effective (i.e., without bearing the costs of formal education and training) and efficient manner (i.e., without hierarchy concerns and bureaucracy; [18]). Our contention is that Communities of Practice for Clean Sport Education can provide the infrastructure for effective and sustainable management of anti-doping education knowledge. So far, the concept of communities of practice has never been applied in this area.

3 Aims and Objectives of Project IMPACT Project IMPACT will utilize the concept of Communities of Practice for the first time in the context of clean sport/anti-doping education, through the strategic partnership between key academic experts who possess a large intellectual capital on developing and evaluating anti-doping education, and international and national stakeholders and key actors from the word of sports (sport associations; anti-doping agencies; organizations for doping prevention in amateur sports). Essentially, project IMPACT applies the principles of the Triple Helix Model [19] which enables the synergies between different stakeholders, such as academics, governmental authorities and end-users/civic society. IMPACT’s approach will further enable the social dialogue on combatting doping in sports at a larger and pan-European scale and will also help in translating knowledge exchange and synergies into more effective, reflective and impactful joint anti-doping ventures. The specific objectives of project IMPACT are to: • Identify and utilize relevant empirical evidence to guide the development of the first community of practice for clean sport education in Europe.

1062

L. Lazuras et al.

• Develop face-to-face and virtual communities of practice for clean sport education to enable synergies and collaborative learning about state-of-art clean sport education among key sport stakeholders, anti-doping organizations, and expert academics. • Enable knowledge sharing and the emergence of best practices with an emphasis on protecting clean athletes from doping use; minimizing risk and harmful practices in athletes involved in or at-risk for doping use; identifying new areas for policy intervention; and developing strong anti-doping social norms through whistleblowing promotion across levels of sports, ethnic minority groups, and types of sport. • Increase awareness of anti-doping organizations and sport stakeholders across Europe about the benefits of using communities of practice for clean sport education, policy-making, and research, and for stimulating future innovative anti-doping education practices.

4 Future Work The main goal is the design and implementation of the digital infrastructure that will support the virtual community of practice of project IMPACT, and this includes: (a) designing and developing the virtual community of practice platform based on an open source content Management System; (b) designing and creating the content to be included in the virtual community of practice; and (c) evaluating the structure, content, and functionality of the virtual community of practice. This work process will include the following activities: • Virtual community of practice design and development: This activity focuses on transforming the needs into implementations of a virtual community of practice that aims to support Clean Sport Education along the dimensions/areas that will emerge from the offline communities of practice. • Creation of educational content: This activity aims to transform and create educational content that will be used in order to be included in the virtual community of practice. • Evaluation of the virtual community of practice: This activity pertains to the evaluation of the virtual community of practice by the sport stakeholders participating in IMPACT’s consortium (i.e., iNADO, Dopinglinkki, CyADA, and SS Lazio Cycling), and by members of our national communities of practice in Finland, Italy, Greece, Cyprus and the UK. Acknowledgement. This project has been funded with support from the European Commission’s 2018 Erasmus+ Sport Program. This publication reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein. The authors of this research would like to thank IMPACT team who generously shared their time, experience, and materials for the purposes of this project.

Developing Communities of Practice to Maximize the Usability

1063

References 1. de Hon, O., Kuipers, H., van Bottenburg, M.: Prevalence of doping use in elite sports: a review of numbers and methods. Sports Med. 45(1), 57–69 (2014). https://doi.org/10.1007/ s40279-014-0247-x 2. Nicholls, A.R., Madigan, D.J., Backhouse, S.H., Levy, A.: Personality traits and performance enhancing drugs: the dark triad and doping attitudes among competitive athletes. Pers. Individ. Differ/ 112, 113–116 (2017). https://doi.org/10.1016/j.paid.2017.02. 062 3. Hartgens, F., Kuipers, H.: Effects of androgenic-anabolic steroids in athletes. Sports Med. 34 (8), 513–554 (2004) 4. Darke, S., Torok, M., Duflou, J.: Sudden or unnatural deaths involving anabolic-androgenic steroids. J. Forensic Sci. 59(4), 1025–1028 (2014) 5. Frati, P., Busardo, P.F., Cipolloni, L., De Dominicis, E., Fineschi, V.: Anabolic androgenic steroid (AAS) related deaths: autoptic, histopathological and toxicological findings. Curr. Neuropharmacol. 13(1), 146–159 (2015) 6. Lindqvist, A.S., Moberg, T., Eriksson, B.O., Ehrnborg, C., Rosén, T., Fahlke, C.: A retrospective 30-year follow-up study of former Swedish-elite male athletes in power sports with a past anabolic androgenic steroids use: a focus on mental health. Br. J. Sports Med. 47 (15), 965–969 (2013) 7. Pope Jr., H.G., Kanayama, G., Athey, A., Ryan, E., Hudson, J.I., Baggish, A.: The lifetime prevalence of anabolic-androgenic steroid use and dependence in Americans: current best estimates. Am. J. Addict. 23(4), 371–377 (2014) 8. Quaglio, G., Fornasiero, A., Mezzelani, P., Moreschini, S., Lugoboni, F., Lechi, A.: Anabolic steroids: dependence and complications of chronic use. Intern. Emerg. Med. 4(4), 289–296 (2009) 9. Barkoukis, V.: Moving away from penalization: the role of education-based campaigns. In: Barkoukis, V., Lazuras, L., Tsorbatzoudis, H. (eds.) The Psychology of Doping in Sport, pp. 215–229. Routledge, Abingdon (2015) 10. Lazuras, L., Barkoukis, V.: Performance and appearance enhancing drug use in sports: a psychological perspective. In: Tenenbaum, G., Ecklund, R.C. (eds.) Handbook of Sport Psychology, 4th edn. Wiley Blackwell, New York (2018) 11. Wenger, E.: Communities of practice. Communities 22(5), 1–5 (2009) 12. Wenger, E.: Communities of practice and social learning systems: the career of a concept. In: Social Learning Systems and Communities of Practice, pp. 179–198. Springer, London (2010) 13. Wenger, E., McDermott, R.A., Snyder, W.: Cultivating Communities of Practice: A Guide to Managing Knowledge. Harvard Business Press, Cambridge (2002) 14. Cruess, R.L., Cruess, S.R., Steinert, Y.: Medicine as a community of practice: implications for medical education. Acad. Med. 93(2), 185–191 (2018) 15. Goodyear, V.A., Casey, A.: Innovation with change: developing a community of practice to help teachers move beyond the ‘honeymoon’of pedagogical renovation. Phys. Educ. Sport Pedagogy 20(2), 186–203 (2015) 16. Pattinson, S., Preece, D.: Communities of practice, knowledge acquisition and innovation: a case study of science-based SMEs. J. Knowl. Manag. 18(1), 107–120 (2014)

1064

L. Lazuras et al.

17. Thomson, L., Schneider, J., Wright, N.: Developing communities of practice to support the implementation of research into clinical practice. Leadersh. Health Serv. 26(1), 20–33 (2013) 18. Ji, H., Sui, Y.T., Suo, L.L.: Understanding innovation mechanism through the lens of communities of practice (COP). Technol. Forecast. Soc. Chang. 118, 205–212 (2017) 19. Etzkowitz, H., Leydesdorff, L.: The dynamics of innovation: from National Systems and “Mode 2” to a Triple Helix of university–industry–government relations. Res. Policy 29(2), 109–123 (2000)

5G Network Infrastructure

A Fiber Wireless A-RoF/IFoF Uplink Transmission of up to 0.6 Gb/s User Data Rate Over a 32-Element 60 GHz Beam-Steering Antenna for 5G Fronthaul Networks Eugenio Ruggeri1(&), Apostolos Tsakyridis1, Christos Vagionas1, George Kalfas1, Amalia Miliou1, Nikos Pleros1, and Yigal Leiba2 1

Department of Informatics, Center for Interdisciplinary Research and Innovation, Aristotle University of Thessaloniki, Thessaloniki, Greece [email protected] 2 Siklu Communication Ltd., Petach Tikva, Israel [email protected]

Abstract. We experimentally demonstrate an uplink transmission of a Fiber Wireless A-RoF/IFoF fronthaul link for 5G networks using a 32-element 60 GHz phased array antenna in the 1-m long wireless channel and 10 km of SMF spool for the optical link. At first, we characterize the link frequency response using a single tone measurement and perform a dynamic characterization using an 100Mbd QPSK, 8PSK and 16-QAM signal, resulting in 200 Mb/s, 300 Mb/s and 400 Mb/s user data rate, meeting the uplink user-rate KPI-requirements for 5G mm wave networks. Beam-steering capabilities for the Fiber Wireless link are shown for a 300 Mbd QPSK transmission and 15° degrees incident angle, showing flexibility for 5G mm Wave networks. Keywords: Fi-Wi
5G networks
mm-wave
Analog intermediate – frequency – over – fiber
Optical fronthaul
60 GHz beam-steering antenna

1 Introduction Future 5G Fiber Wireless (FiWi) networks will stimulate a large range of new services and applications scenarios, such as enhanced Mobile Broadband (eMBB) services, Fixed Wireless Access (FWA), Machine-to-Machine (M2M) communications, and Internet of Things (IoT) [1]. 5G Key Performance Indicators (KPIs) are driving stringent requirements in latency down to 1 ms and peak traffic as high as 10 Gb/s [2], while the first commercial 5G roll outs are aiming to serve eMBB and FWA [3, 4] to various users residing in rural areas or hotspot environments, e.g. at malls, airports etc. [5–9], where Uplink (UL) user rates are scaling up to 50 Mb/s for hotspot scenarios and 500 Mb/s for indoor FWAs [2]. In order to realize such capabilities for the mobile network, millimeter Wave (mmWave) frequencies are actively being investigated due to their larger unallocated spectrum and channel bandwidth supported [10], with massive Multiple In Multiple Out (MIMO) and phased array antenna systems with © Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 1067–1075, 2021. https://doi.org/10.1007/978-3-030-49932-7_100

1068

E. Ruggeri et al.

Fig. 1. The envisioned 5G Fiber Wireless network for future Fixed Wireless Access (FWA) services, using mmWave.

multiple radiating elements of up to 256 count supporting beamforming and beamsteering for spatial reusing a certain frequency [11]. Boosting the capacity of the wireless channels is then going to place a significant load on the optical Mobile FrontHaul (MFH) that has to transport the data between the Remote Radio Head (RRH) antenna and the Baseband Unit (BBU) of the Central Office. This has been so far performed by means of the Common Public Radio Interface (CPRI) protocol using Digital Radio over Fiber (D-RoF) techniques and is known to excess a high bandwidth penalty. On the contrary, Analog-Radio over Fiber (A-RoF) solutions have been proposed as more spectrally efficient transport techniques [12–14], driving global research activities towards truly converged FiWi networks with unified FiWi links that load the native radio signal on optical Intermediate Frequency over Fiber (IFoF) carriers. By using these A-RoF/IFoF transport schemes, capacities as high as 4.56 Gb/s have been demonstrated for the 28 GHz frequency range [15] up to 24 Gb/s for the frequency band of 60 GHz [16, 17] and up to 45 Gb/s for the 98 GHz band [18]. Despite the high fronthaul capacities these demonstrations typically rely on fixed directional antennas, such as horns, Vivaldi and Cassegrain, which are tailored for fixed backhauling of the traffic and without supporting any beam-steering, suggesting that the efficiency of the system will be enhanced if A-RoF links will be combined with the beam-steering capabilities of directional antenna beams of large phased array antennas, that include a beamforming feeder network before the multiple radiating elements. In this direction, a recent 28 GHz FiWi prototype experimentally showed downlink transmission of 5  125 MHz signals [19], and was later employed in a 2  2 MIMO setup [20]. Moreover, a 4-channel beamforming RRH was tested for steering angles of 60° degrees, as envisioned for three-sector RRH modules with 120° degrees coverage, although it was tested without including any wireless transmission [8]. All the above reveal that A-RoF/IFoF transmissions can indeed support the required spectral-efficiency and cost-requirements for the co-existence of multiple users on multiple IFoF sub-carriers [15–18], however the use of phased array antennas rather than simple fixed horn antennas [19, 20] can induce additional signal impairments and the signal quality and impact on the FiWi link also need to be investigated. In this work, we exploit the developments both in A-RoF techniques and mmWave antenna technologies, envisioning unified solution for the FiWi networks, transmitting radio signals either on the transport or even up to the access part of the network. The

A Fiber Wireless A-RoF/IFoF Uplink Transmission of up to 0.6 Gb/s

1069

conceptual schematic of a 5G fronthaul link for 5G mmWave networks is shown in Fig. 1. As illustrated, the end-to-end converged Fiber Wireless networks comprise a BBU that generates the radio signal, loads it on an optical carrier before transporting it to the RRH, that consequently steers the antenna beam and re-transmits the radio signal towards the terminal of the end user, i.e. either the rooftop antenna of a building in cased of FWA services or directly to the end user in case of eMBB services. More explicitly, we experimentally demonstrate the challenging uplink FiWi transmission using a 32-element 60 GHz phased area Rx antenna through a 10 km fiber and 1 m wireless distance. The Fi-Wi link is initially characterized in terms of supported bandwidth, exhibiting a 5 dB bandwidth and as large as 1.5 GHz frequency spectrum, allowing for the co-existence of multiple users within the relatively large available spectrum. The performance of the wireless, the basic FiWi and the full 10 km FiWi links are compared using 100Mbd QPSK, 8-PSK and 16-QAM signals, achieving user capacities of 200 Mb/s, 300 Mb/s and 400 Mb/s respectively, with EVM values well within the required 3GPP thresholds and negligible EVM penalties of less than 0.6% and 1.3% for the FiWi and 10 km FiWi, respectively. Finally, we proceed to higher user transmission rates of 300Mbd QPSK, with and without beam-steering at 15° degrees, resulting in uplink user rates of 600 Mb/s, satisfying the 5G KPIrequirements for broadband uplink access foreseen for FWA and hotspot scenarios, indicating a viable path for 5G A-RoF mmWave networks.

Fig. 2. Experimental setup used for the evaluation of the uplink with a portable 60 GHz antenna transmitting to the 32-element beam-steered antenna, that is driving an optical MZM modulator, followed by 10 km optical transmission.

2 Experimental Setup The used experimental setup is shown in Fig. 2. It includes one portable V-band transmitter (Tx), featuring a 36dBi antenna with 500 MHz baseband input, and I/Q modulator, an integrated Local Oscillator (LO) and an up-conversion stage to generate the 60 GHz wireless signals. The portable transmitter is used hereby to emulate the traffic of the end-point user terminal towards the 32-element phased array receiving antenna (Rx) by Siklu followed by the 10 km IFoF link. The data traffic is synthetized by a Keysight Arbitrary Waveform Generators (AWG) M8190A and fed to the Tx in a

1070

E. Ruggeri et al.

 At first, while performing the static differential data input configuration (I=I and Q=Q). and dynamic characterizations, the Tx is fed either by one single tone when evaluating the frequency response of the link, or fed by three different modulation formats of QPSK, 8PSK and QAM16 signals when performing data transmission over the wireless channel exclusively, over the FiWi link or over the same FiWi link extended by the 10 km SMF spool.

Fig. 3. (a) Photo of the antenna system including a Tile PCB and a Tile Feed PCB board and (b) inset photo of the 32 radiating elements operating at 60 GHz.

2.1

The 60 GHz Phased Array Antenna

The 60 GHz signals transmitted by Tx are received after 1 m V-band link by the phased array Rx antenna, composed of 2 elements: Tile PCB and Tile Feed Board PCB, both supplied with 5.3 V and consuming 1.2A and 300 mA respectively. The Tile PCB is integrated on a low-temperature ceramic and hosts the 32 radiating elements, each of which comprises a dipole with a 6dBi gain and almost isotropic radiation across the 120° sector and it is placed at the front panel of the antenna system. The 32 elements are then followed by an RF IC with 32 channels that each one features a series of a Low Noise Amplifier (LNA), a phase shifting element (u) and an integrated down-conversion stage, succeeded by a 32:1 combiner. The receiver’s clock signal is provided to the Tile Feed Board PCB by an external 10 GHz LO at an RF input power of −15 dBm, in order to generate a single 5 GHz IF output that carries the initially transmitted data. A photo of the boards of the phased antenna Rx system is shown in Fig. 3(a), while the inset Fig. 3(b) shows the 32-element Tile placed at the front panel of the antenna.

A Fiber Wireless A-RoF/IFoF Uplink Transmission of up to 0.6 Gb/s

1071

When all elements are on, they can be configured to constructively interfere over the air, concentrating the radiation in a tightly focused beam across a certain direction, that can be steered across a 120° direction, while the signal distortion across 90° range was negligible, as will be shown in the next section. The previously mentioned Tile PCB consumes 1.2 A when in isotropic mode but, when configured in beamsteering mode, with all 32 elements on, the drained current increases to 1.6 A. 2.2

The Optical Components

The wireless link was then extended with a A-RoF fiber-optical transmission of up to 10 km fiber spool. The signal received by the Rx antenna was down-converted from 60 GHz to 5 GHz at its output and then used to drive an optical transmission stage. Specifically, the −2 dBm output (0.5 Vp-p) of the Rx antenna was amplified to 5 V by a driver amplifier for driving a zero chirp LiNbO3 Mach-Zehnder Modulator (MZM) biased at the quadrature point, modulating the CW output of a 1550 nm optical carrier of a DFB Laser Source (LS). The optical IFoF signal was propagated either though a short fiber-patchcord or through a 10 km SMF spool with 0.2 dB/km losses, emulating the optical MFH distances. At the end of the fiber link, the signal is o-e converted by 10G InGaAs Avalanche Photo-Receiver (PD) and its output is captured by Signal Analyser (SA) for monitoring purposes.

3 Experimental Results

Fig. 4. Fi-Wi link IF output power vs IF output frequency.

1072

3.1

E. Ruggeri et al.

Static Characterization: Single Tone Frequency Response of the End-to-End FiWi Link

The FiWi link was first statically characterized in terms of supported bandwidth and available frequency spectrum, transmitting a single tone through the Tx, and measuring the received power after the PD. The tone frequency was swept from 3.5 GHz to 6 GHz, which was translated in sweeping of the wirelessly transmitted mmWave carrier from 58.6 to 61.3 GHz. The received power at the SA is plotted, as a function of frequency, in Fig. 4, featuring a gain-peak at 4.8 GHz, i.e. the 59.9 GHz. Based on this plot, the end-to-end FiWi link exhibits a 3 dB bandwidth of 0.95 GHz, a 5 dB bandwidth of 1.5 GHz and a 10 dB bandwidth of 2.2 GHz. 3.2

Dynamic Characterization: Transmission of QPSK, 8PSK and QAM16 Signal Over Pure Wireless, FiWi or FiWi with 10 km Spool

In order to evaluate the performance of the end-to-end channel at various sections of the link, we also performed a single user data transmission of a 100MBd QPSK, 8-PSK and 16-QAM signal, resulting in 200 Mb/s, 300 Mb/s and 400 Mb/s user rates, transmitted either exclusively through the 1 m wireless channel, through the end-to-end FiWi link or through the 10 km spool extension, while obtaining the constellation diagrams and measuring the EVM metric. The experimental results for the wireless link are illustrated in Fig. 5. More specifically, Fig. 5 (a)–(c) depict the constellation diagrams of 100Mbd QPSK, 8PSK and QAM16 signals, with respective EVMs of 13.51%, 14.55% and 11.88%, as recorded at the SA, without applying any additional offline DSP. These results have been obtained for IQ data inputs at Tx of 300 mV for the QPSK and 8PSK and 100 mV for QAM16, and all meet the EVM requirements set by 3GPP for 5G-NR systems. Following these operational conditions, we evaluated the FiWi link, and the results are shown in Fig. 5 (d)–(f). Then, after extending the link using 10 km of S-SMF fiber in order to emulate a 5G fronthaul, the constellations diagrams are illustrated in Fig. 5 (g)–(i), with the same modulation formats and baud rate of the wireless and FiWi transmissions, revealing maximum EVM penalty < hWb i

if k þ

> > : 0

if k þ

Tb

h i Wb Tb \n h i Wb Tb

ð10Þ

n

The NB-IoT devices which fall on the transmission window of the ðn  1Þth slot, will attempt its next RACH at nth slot. The EbTMb ½k is the number of UEs at ith slot, which completed the TMb number of transmissions and Cb ½k  is the number of collided devices at the kth slot in the coverage class b. We assume a static channel condition over the time period Tmax , and hence we set TMb ¼ TMG , i.e. the UE will not move to higher coverage class from lower coverage class. We assign a preamble repetition value based on the initial radio condition. Hence the device density at nth slot is given by kD;b ½n ¼

Mb ½n A

ð11Þ

kD;bs ½n ¼

kD;b ½n SCb

ð12Þ

Now the PRb ½n; Sb ½n and Cb ½n can be calculated iteratively. In our simulation, we assume that the channel conditions remain the same till Nb slots. That is, we fix the preamble repetition value based on the CE level at the beginning of the RA procedure, and UEs will not move from lower CE to higher CE level. Hence, considering the transmission in all CE levels up to Nb slots, the average success probability is given by

1126

S. Narayanan et al.

P 2 P Nb Sb ½i Ps ¼ P2b¼0 PNi¼1 b i¼1 Mb ½i b¼0

ð13Þ

We define access latency as the average time required for an NB-IoT UE to complete the RA procedure. The expressions for access latency is given by P 2 P Nb i¼1 Sb ½i Tb  ði  1Þ ð14Þ D ¼ b¼0 P P b Mb ½i Ps  2b¼0 Ni¼1

4 Numerical Results In this section, the dependency of the back-off window, repetition, and (RAO)/UE on the NPRACH success probability and access latency is shown. We consider the UEs in the single coverage class, as these parameters mainly affect the access success probability within a single coverage class. We performed simulation at CE-1. As we can observe from Fig. 3 (a) and (b) the performance of NPRACH can be enhanced by optimizing the repetition value, back-off window (BW) size, and subcarrier allocation. We consider the traffic is congested when the average number of RA resources/UE within the given time bound is >1, where we define: Average number of RA resource=UEb ¼ Nb  SCb  kB =kD;b

ð15Þ

For example, in our simulation with a predefined time-bound of 10 s and Tb ¼ k ¼ 720, and SCb ¼ 48. 640 ms; Average number of RA resource=UE ¼ 1, when kD;b B As we can observe from Fig. 3 (a), a larger BW gives better performance in highly congested traffic, while a smaller BW provides better performance at less congested traffic. This is because a larger back-off window offloads the network traffic by spreading the UE over time. When the device density is low, the preamble collisions can be resolved through retransmissions, and a lower back-off window enables a greater number of retransmission opportunities within the specified time-bound. In our model, repetition value for an NB-IoT UE is set at the beginning of the RA procedure, without considering the retransmission scheme, i.e., we set repetition value for a NB-IoT UE such that RA success probability of the NB-IoT UE is more than 99%, in a single cell NB-IoT system, and only one NB-IoT contending for the access to the network. Thus, the preamble repetition value is determined based only on the path loss parameter. If the repetition value is less than this, the probability of signal outage will increase (correspondingly, preamble detection probability decreases), and if the repetition is much higher than this value, detection probability at the PHY may increase, but improvement in the access success probability at the MAC is not guaranteed. Also, a higher repetition value increases the access latency, because more the repetition, the eNodeB has to wait for all the repeated subframe to receive to start decoding the preamble. Moreover, the repetition number determines channel allocation between NB-Physical Uplink Shared Channel (NPUSCH) and NPRACH. The

Performance Analysis of NB-IoT Random Access Channel

1127

parameter time vacancy ratio (given in Eq. (16)) gives the time available for NPUSCH within a Transmission Time Interval (TTI). Hence a redundant repetition will cause inefficient use of channel resources as well as high access latency, without any gain in the access success probability. Time vacancy ratio ¼ ðTTI  ð5:6  Repetition number ÞÞ=TTI

ð16Þ

Similarly, providing a lesser number of RAO per UE results in better access success probability in a dense MTC traffic. Providing lesser RAO effectively remove the UE from contending for the RA after performing the given number of RAO/UE, thus in the coming slots device density will be reduced. This will increase the access latency, as more devices are completing the RA by performing the retransmission scheme.

(a)

(b) Fig. 3. (a) Random-access success probability and (b) Average access latency in a multi-cell, single coverage-class NB-IoT system within a predefined time-bound of 10 s.

1128

S. Narayanan et al.

5 Conclusion The random-access procedure in NB-IoT is a software up-gradation of exiting LTE random access procedure. We model the NB-IoT Physical Random-Access Channel (NPRACH) as a multi-band, multi-channel slotted aloha system and analyzed the effect of various RAN parameters on the random-access success probability and access latency. Our performance study on NPRACH shows that the selection preamble repetition, back-off window, and random-access opportunity per UE affects the random-access success probability. The repetition value should be calculated based only on the pathloss parameter, regardless of device density. A larger back-off window gives better random access success probability in a highly congested MTC traffic, and a smaller back-off window provides better NPRACH performance in less congested traffic. Similarly, providing a lesser number of random-access opportunities per UE results in better access success probability in a dense MTC traffic. Acknowledgements. This work has been co-funded by the European Union under the Horizon 2020 research and innovation programme Marie Skłodowska Curie, grant agreement No. 722788, and also by the European Union and Greek national funds through the Operational Program Competitiveness, Entrepreneurship and Innovation, under the call RESEARCH – CREATE – INNOVATE (project CityZEN, code:T1EDK -02121).

References 1. Ratasuk, R., Mangalvedhe, N., Ghosh, A.: Overview of LTE enhancements for cellular IoT. In: IEEE International Symposium on Personal, Indoor and Mobile Radio Communications, PIMRC (2015) 2. Li, S., Da Xu, L., Zhao, S.: 5G internet of things: a survey. J. Ind. Inf. Integr. 10, 1–9 (2018) 3. Narayanan, S., Tsolkas, D., Passas, N., Merakos, L.: NB-IoT: a candidate technology for massive IoT in the 5G era. In: IEEE International Workshop on Computer Aided Modeling and Design of Communication Links and Networks, CAMAD (2018) 4. Zhou, K., Nikaein, N., Knopp, R.: Dynamic resource allocation for machine-type communications in LTE/LTE-A with contention-based access. In: IEEE Wireless Communications and Networking Conference, WCNC (2013) 5. De Andrade, T.P.C., Astudillo, C.A., Sekijima, L.R., Da Fonseca, N.L.S.: The random access procedure in long term evolution networks for the internet of things. IEEE Commun. Mag. 55(3), 124–131 (2017) 6. 2. Maximum Coupling Loss (MCL)| Coverage Analysis LTE M Interactive White Paper. http://www.coverageanalysisoflte-m.com/2-maximum-coupling-loss-mcl/. Accessed 10 Aug 2019 7. Wang, Y.P.E., et al.: A primer on 3GPP narrowband internet of things. IEEE Commun. Mag. 55(3), 117–123 (2017) 8. Gozalvez, J.: New 3GPP standard for IoT. IEEE Veh. Technol. Mag. 11(1), 14–20 (2016) 9. Jiang, N., Deng, Y., Condoluci, M., Guo, W., Nallanathan, A., Dohler, M.: RACH preamble repetition in NB-IoT network. IEEE Commun. Lett. 22(6), 1244–1247 (2018)

Author Index

A Abou El-Seoud, Samir, 888 Achouche, Mohand, 1089 Akhrouf, Samir, 879 Akkara, Sherine, 347, 357 Akrouf, Samir, 223 Alafouzou, Angeliki, 158 Aleksić, Veljko, 1051 Alexiou, Aikaterini, 158 Alfonso, Mocha-Bonilla Julio, 242 Alkasasbeh, Anas Ali, 1015 Alvarez Rodriguez, Francisco Javier, 102 Andrashko, Yu., 950 Antzoulatos, Gerasimos, 962 Anumula, V. Surya Seshagiri, 347, 357 Apostolidis, Ippokratis, 507, 629 Apostolou, Konstantinos, 91, 687 Arias T, Susana, 785 Arias, Xavier, 785 Armas-Arias, Sonia, 169 Atra, Kebede, 1089 Attia, Abdelouahab, 223 B Bălan, Alexandra, 1024 Bălan, Titus, 36, 1024 Bamidis, Panagiotis D., 437 Barbare, Inese, 765 Barkoukis, Vassilis, 147, 1058 Basse, Adrien, 327, 717, 738 Behnamghader, Mahsa, 814 Belén, Morales Jaramillo María, 242 Bergman, Gustav, 983 Berti, Margherita, 215 Bhattacharya, Abhishek, 480

Billis, Antonis, 437 Biloshchytska, S., 950 Biloshchytskyi, Andrii, 379, 950 Bratitsis, Tharrenos, 178, 189, 457, 726 Bronin, Sergiy, 379, 950 C Caicedo, Steven, 1101 Caillaud, Christophe, 1089 Calo, Cosimo, 1089 Caro-Alvaro, Sergio, 1015 Carreño León, Mónica Adriana, 102 Carrillo Ríos, Sandra, 492, 897 Cartuche, Claudia, 785 Centea, Dan, 91, 687 Cerulo, Giancarlo, 1089 Chaka, Chaka, 289 Chalabi, Nour Elhouda, 223 Chaldogeridis, Agisilaos, 147, 507, 629 Charidimou, Dimos, 705 Chatzimichail, Angelos, 468 Chehlarova, Toni, 675 Christodoulakis, Stavros, 1002 Christoulakis, Marios, 1002 Ciurea, Magdalena, 995 Cobo, Francisco Naranjo, 800 Colom, Anne-Gaelle, 232 Cumbe-Coraizaca, Dorys, 169 Curpen, Radu, 36 Czerkawski, Betul, 215 D De Smet, Cindy, 57 Decobert, Jean, 1089 Deme, Chérif Bachir, 327, 717

© Springer Nature Switzerland AG 2021 M. E. Auer and T. Tsiatsos (Eds.): IMCL 2019, AISC 1192, pp. 1129–1132, 2021. https://doi.org/10.1007/978-3-030-49932-7

1130 Demetriadis, Stavros, 693 Derouin, Estelle, 1089 Diatta, Baboucar, 327, 717, 738 Diop, Cheikh Ahmadou Lamine Yakhine, 911 Dochtsis, Romanos, 664 Douka, Stella, 401 Douka, Styliani, 514 Douligeris, Christos, 641, 664 Dubrowski, Adam, 266 Dumitrescu, Andrei, 1033 Dunwell, Ian, 594 Dychka, Ivan, 750 E El-Seoud, M. Samir Abou, 867 Eneh, Agozie H., 652 Erasme, Didier, 1089 F Fachantidis, Nikolaos, 618, 1039 Fortin, Catherine, 1089 Francisco, Lozada J., 785 Freire, Diego, 800 G Gallear, Wayne, 547 García Herrera, Darwin, 492 García-Cabot, Antonio, 1015 García-López, Eva, 1015 Gaudi, Geoffrey, 3 Ghinea, Gheorghita, 1015 Giannaraki, Marina, 582 Giannoulopoulos, Nikolaos, 641 Gkamas, Apostolos, 413 Gomez A, Hector F., 800 Gomez, Carmen, 1089 Guachimbosa, Victor Hugo, 844 Gueye, Amadou Dahirou, 297 Gueye, Assane, 297 Gueye, Bounama, 297 H Hadj, Barkat, 879 Hadzilacos, Thanasis, 539 Haedar, Hadi, 806, 825 Hall, Geoffrey, 3 Hanna, Richard, 445 Heydari, Fatemeh, 825 Holmer, Torsten, 571 Hosseinnia, Fateme, 836 I Ibrahim, Rasha, 425 Ilie, Daniel, 36

Author Index Iliou, Theodoros, 606 Inglezakis, Ioannis, 705 Ioannis, Kompatsiaris, 962 Ioannou, Andri, 367 Ioannou, Ioannis, 117 Ioannou, Michalis, 178 Isaacs, Shafika, 938 Ivanova, Malinka, 127 Izadpanah, Pegah, 814 J Jannack, Anja, 571 Johannsen, Ulf, 1076 Jordaan, Dawid Benjamin, 13 K Kalfas, George, 1067 Kalmpourtzis, George, 57 Kane, Patrick, 995 Kapralos, Bill, 3, 266 Karalis, Themistoklis, 606 Kasse, Omar, 297 Kazanidis, Ioannis, 307 Khaleghi, Ali, 806, 814, 825, 836 Khalil, Mohammad, 983 Kloss-Brandstätter, Anita, 255 Kompatsiaris, Ioannis, 468 Konstantinidis, Evdokimos, 437 Konstantinou, Dimitrios, 1076 Korosidou, Eleni, 726 Kotseva, Monka, 675 Kotsifakos, Dimitrios, 641, 664 Koukis, Alexandros, 1002 Kouloglou, Ioannis-Omiros, 962 Kourkoutas, Elias, 582 Kourtesis, Dimitrios, 962 Krasadakis, Pantelis, 276 Kuchansky, Alexander, 379, 950 Kurtz, Gila, 255 Kyriafinis, Georgios, 1051 Kyselov, Volodymyr, 379 L Lameras, Petros, 547, 559, 594 Lazuras, Lambros, 1058 Leiba, Yigal, 1067 Levinskyi, Maksym, 389 Lishou, Claude, 297 Loukovitis, Andreas, 514 M Mady, Amr S., 888 Mahmoudi, Kamran, 836

Author Index Malek, Rachid, 879 Mallampalli, Mallikarjuna Sastry, 347, 357 Mallecot, Franck, 1089 Malo, Elena, 800 Mania, Katerina, 582 Maragkoudakis, Ioannis, 1002 Margounakis, Dimitrios, 45, 606 Martin, Florence, 1089 Massler, Ute, 539 Mavropoulou, Angeliki, 629 Mavroudi, Anna, 117 McKee, Gerard, 867 Mecili, Oualid, 879 Meditskos, Georgios, 468 Mehrtash, Moein, 91 Mekhazni, Karim, 1089 Merakos, Lazaros, 1118 Mihaiescu, Dan, 25 Mikułowski, Dariusz, 773 Miladinovic, Igor, 930, 1109 Miliou, Amalia, 1067 Mocha-Bonilla, Julio, 492 Mocha-Bonilla, Julio Alfonso, 844, 897 Monekosso, Dorothy, 425 Morales, Alvaro, 1076 Morosan, Alina, 25 Moscato, Stefano, 1101 Mostert, Ingrid, 69, 81 Moumoutzis, Nektarios, 582, 1002 Mouratoglou, Sophia-Anastasia, 437 Mourtzios, Christos, 962 Murray, Niall, 480 N Narayanan, Subin, 1118 Narimani, Abbas, 806 Naumann, Fabrice, 571 Ndiaye, Ndeye Massata, 911 Nela, Barba Téllez María, 242 Neofotistou, Eleni, 158 Neofytou, Chrystalla, 539 Nikiforos, Anastasios, 705 Nikoleta, Charoumenou, 457 Nkhobo, Tlatso, 289 Noennig, Jörg Rainer, 571 Nouioua, Farid, 879 O Oertel, Lars, 559 Oldoni, Matteo, 1101 Oparaku, Ogbonna U., 652 Ouya, Samuel, 327, 717, 738

1131 P Páez-Quinde, Cristina, 169 Palaigeorgiou, George, 178, 202, 457, 971 Pandria, Niki, 437 Paneva-Marinova, Desislava, 1002 Papadimitriou, Nikolaos, 962 Paraskeva, Foteini, 158 Paris, Nektarios-Kyriakos, 45 Parvinchi, Diana, 3 Pasalidou, Christina, 1039 Passas, Nikos, 1118 Pavani, Ana M. B., 317 Pester, Andreas, 255 Petrone, Kaija, 445 Philippe, Stephanie, 559 Pilski, Marek, 773 Plaatjies, Lydia-Anne, 69, 81, 339 Pleros, Nikos, 1067 Pliasa, Sofia, 618 Politis, Dionysios, 45, 1051 Politopoulos, Nikolaos, 147, 401, 507, 629, 1058 Pommereau, Frederic, 1089 Provost, Jean-Guy, 1089 Psathas, Georgios, 693 Purdy, Wendy, 232 Q Quero, Enilda Zea, 266 R Raddo, Thiago R., 1076 Rahmani, Farzaneh, 814 Rammos, Dimitris, 189 Ravariu, Cristian, 25, 995 Roberts, Nicky, 69, 81, 136 Robu, Dan, 36, 1024 Romero, Margarida, 57 Rommel, Simon, 1076 Rozinaj, Gregor, 1015 Ruggeri, Eugenio, 1067 S Sakkopoulos, Evangelos, 276 Salenko, Anton, 750 Samarakoon, Uthpala, 109, 919 Samoila, Cornel, 995 Sánchez Guerrero, Javier, 492, 844, 897 Sandoval Bringas, Jesús Andrés, 102 Sandu, Florin, 1024 Santiago, Carolina Guachimbosa, 844 Santiago, Pullas Tapia Paúl, 242

1132

Author Index

Satratzemi, Maya, 307, 527 Savu, Tom, 1033 Schefer-Wenzl, Sigrid, 930, 1109 Semnani, Farzad, 806 Shankaranarayanan, G., 445 Shapo, Vladlen, 389 Siala, Muaad Hammuda, 867 Sidiropoulou, Vasiliki, 202 Soltaninezhad, Alireza, 825 Stamelos, Ioannis, 507 Stamkopoulos, Gregory-Telemachos, 1051 Stelzle, Benjamin, 571 Stewart, Craig, 547 Stremtan, Mircea, 995 Stylianidis, Panagiotis, 147, 401, 629, 1058 Sulema, Olga, 750 Sulema, Yevgeniya, 480, 750

U Udanor, Collins Nnalue, 652 Uribe-Quevedo, Alvaro, 3, 266 Ursutiu, Doru, 25, 995 Usoof, Hakim, 109, 919

T Tafur Monroy, Idelfonso, 1076 Takhttavani, Maedeh, 825 Tanase, Mihai, 25 Tegos, Stergios, 693 Temporão, Guilherme P., 317 Terzopoulos, George, 307, 527 Torres, Andrei B. B., 266 Towndrow, Holly, 425 Tsakyridis, Apostolos, 1067 Tsanousa, Athina, 468 Tsiatsos, Thrasyvoulos, 25, 147, 401, 507, 514, 629, 693, 1058 Tsinakos, Avgoustos, 307 Tsolkas, Dimitris, 1118 Tsompanoudi, Despina, 527 Tsoni, Rozita, 276 Tzimas, Rafail, 45

W Ware, Cédric, 1089 Wilde, Andreas, 571

V Vagionas, Christos, 1067 Varda, Christiana, 367 Veglis, Andreas, 57 Velastegui-Hernández, Santiago, 169 Verykios, Vassilios S., 276 Viberg, Olga, 983 Vrioni, Andri, 117 Vrochidis, Stefanos, 468, 962 Vroikou, Grigoria, 457

X Xefteris, Stefanos, 971 Y Yoshinov, Radoslav, 675 Ypsilanti, Antonia, 1058 Z Zapata-Mocha, Esmeralda, 897 Ziagkas, Efthymios, 401, 514 Zilidou, Vasiliki, 401, 514 Zinyuk, Olga, 379 Zoumpourtikoudi, Helen, 971