Special Topics in Artificial Intelligence and Augmented Reality: The Case of Spatial Intelligence Enhancement 3031520041, 9783031520044

Table of contents :
Foreword
Preface
Contents
Chapter 1: Introduction and Overview of AI-Enhanced Augmented Reality in Education
1.1 Overview
1.2 Motivation
1.3 Research Questions
1.4 Approach and Structure
References
Chapter 2: Review of the Literature on AI-Enhanced Augmented Reality in Education
2.1 Overview
2.2 Spatial Ability: Review of Theories
2.2.1 Spatial Ability in Engineering
2.3 Augmented Reality in Education
2.3.1 AR in Engineering Education
2.4 Learning Theories
2.4.1 The Bloom’s Taxonomy
2.4.2 The SOLO Taxonomy
2.4.3 Comparison of the Learning Theories
2.5 Literature Review
2.5.1 Planning the Review (Review Protocol)
2.5.2 Conducting the Review
2.5.3 Screening of the Evaluation Papers
2.5.4 Advantages of AR in Spatial Ability Training (RQ1)
2.5.4.1 Learner Outcomes
2.5.4.2 Pedagogical Affordances
2.5.4.3 Technical Perspectives
2.5.5 Limitations of AR in Spatial Ability Training (RQ2)
2.5.6 Exploration of the Incorporation of Adaptivity and Personalization in AR Applications (RQ3)
2.5.7 Aspects of Spatial Abilities Having Been Evaluated Using AR (RQ4)
2.5.8 Evaluation Methods Considered for AR Applications in Educational Scenarios (RQ5)
2.6 Summary
References
Chapter 3: AI-Driven and SOLO-Based Domain Knowledge Modeling in PARSAT AR Software
3.1 Overview
3.2 Domain Model
3.2.1 Objectives
3.3 Domain Knowledge Alongside SOLO Taxonomy
3.4 Examples of Learning Activities of Each SOLO Level
3.5 Summary
References
Chapter 4: Fuzzy Logic for Modeling the Knowledge of Users in PARSAT AR Software
4.1 Overview
4.2 Fuzzy Logic Algorithm
4.3 Initialization Process
4.4 Fuzzy Sets
4.5 Fuzzy Rule Base
4.6 Mamdani’s Inference System
4.7 Defuzzification
4.8 Adaptation of the Learning Activities Based on Fuzzy Weights
4.8.1 Decision Making
4.9 Summary
References
Chapter 5: Artificial Intelligence-Enhanced PARSAT AR Software: Architecture and Implementation
5.1 Overview
5.2 System Architecture
5.2.1 Hardware Layer
5.2.1.1 Tracking
5.2.1.2 Processing
5.2.1.3 Interacting
5.2.2 Software Layer
5.2.2.1 User Interface
5.2.2.2 3D Rendering Engine
5.2.3 Data Layer
5.2.3.1 Marker Database
5.2.3.2 3D Models Database
5.2.3.3 Interaction Model
5.3 Implementation of the System
5.3.1 User Interface of PARSAT
5.3.2 Fuzzy Logic Controller Implementation with C# Scripting
5.3.2.1 System Initialization
5.3.2.2 Linguistic Variables and Membership Functions
5.3.2.3 Fuzzification Process Implementation
5.3.2.4 Rules of the System
5.3.2.5 Evaluation of the Rules
5.3.2.6 Defuzzification
5.4 Summary
References
Chapter 6: Multi-model Evaluation of the Artificial Intelligence-Enhanced PARSAT AR Software
6.1 Overview
6.2 Evaluation Framework
6.2.1 Research Sample
6.2.2 Training Preparation
6.3 t-Test Analysis of Students’ Feedback
6.4 Comparative Analysis of Pre-test/Post-test Model in Achieving the Learning Outcomes
6.4.1 Discussion of the Results
6.5 Extended Technology Acceptance Model for Detecting Influencing Factors
6.5.1 Existing Acceptance Models
6.5.2 Proposed Extended Model
6.5.3 Research Model and Hypotheses
6.5.4 Research Instruments
6.5.5 Data Analysis
6.5.6 Model Validation
6.5.6.1 Measurement Model
6.5.6.2 Structural Model
6.6 Summary
References
Chapter 7: Conclusions of AI-Driven AR in Education
7.1 Overview
7.2 Conclusions and Discussion
7.3 Contribution to Intelligent Tutoring Systems
7.4 Contribution to Domain Knowledge Model
7.5 Contribution to Student Modeling
7.6 Contribution to Electronic Assessment
7.7 Future Work
References

Citation preview

Cognitive Technologies

Christos Papakostas Christos Troussas Cleo Sgouropoulou

Special Topics in Artificial Intelligence and Augmented Reality The Case of Spatial Intelligence Enhancement

Cognitive Technologies Editor-in-Chief Daniel Sonntag, German Research Center for AI, DFKI, Saarbrücken, Saarland, Germany

Titles in this series now included in the Thomson Reuters Book Citation Index and Scopus! The Cognitive Technologies (CT) series is committed to the timely publishing of high-quality manuscripts that promote the development of cognitive technologies and systems on the basis of artificial intelligence, image processing and understanding, natural language processing, machine learning and human-computer interaction. It brings together the latest developments in all areas of this multidisciplinary topic, ranging from theories and algorithms to various important applications. The intended readership includes research students and researchers in computer science, computer engineering, cognitive science, electrical engineering, data science and related fields seeking a convenient way to track the latest findings on the foundations, methodologies and key applications of cognitive technologies. The series provides a publishing and communication platform for all cognitive technologies topics, including but not limited to these most recent examples: • • • • • • • •

Interactive machine learning, interactive deep learning, machine teaching Explainability (XAI), transparency, robustness of AI and trustworthy AI Knowledge representation, automated reasoning, multiagent systems Common sense modelling, context-based interpretation, hybrid cognitive technologies Human-centered design, socio-technical systems, human-robot interaction, cognitive robotics Learning with small datasets, never-ending learning, metacognition and introspection Intelligent decision support systems, prediction systems and warning systems Special transfer topics such as CT for computational sustainability, CT in business applications and CT in mobile robotic systems

The series includes monographs, introductory and advanced textbooks, state-of-the-­ art collections, and handbooks. In addition, it supports publishing in Open Access mode.

Christos Papakostas • Christos Troussas •  Cleo Sgouropoulou

Special Topics in Artificial Intelligence and Augmented Reality The Case of Spatial Intelligence Enhancement

Christos Papakostas Department of Informatics and Computer Engineering University of West Attica Εgaleo, Greece

Christos Troussas Department of Informatics and Computer Engineering University of West Attica Egaleo, Greece

Cleo Sgouropoulou Department of Informatics and Computer Engineering University of West Attica Egaleo, Greece

ISSN 1611-2482     ISSN 2197-6635 (electronic) Cognitive Technologies ISBN 978-3-031-52004-4    ISBN 978-3-031-52005-1 (eBook) https://doi.org/10.1007/978-3-031-52005-1 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland Paper in this product is recyclable.

Foreword

In the pages of this book, titled Special Topics in Artificial Intelligence and Augmented Reality with the subtitle The Case of Spatial Intelligence Enhancement, lies a comprehensive exploration of AI and AR at the intersection of spatial intelligence. This book offers an in-depth examination of both theoretical and practical aspects, guiding the way toward enhancing spatial intelligence in the era of technological convergence. Readers will find valuable insights, diverse case studies, and a roadmap for navigating the evolving landscape of technologies that enhance human cognition. This work spans various critical subjects, including the foundations and applications of AI and AR, their symbiotic relationship, and their combined impact on the development of spatial intelligence. The authors have undertaken a commendable effort to bring together expertise from diverse domains, providing a comprehensive view of this captivating field. By addressing both the theoretical underpinnings and real-world applications, this book serves as a bridge, connecting scholarly discourse to practical implementation, offering a clear and comprehensive understanding of the subject matter. One of the notable strengths of this work is its ability to illuminate the vast potential of AI and AR in enhancing human spatial intelligence. The authors have curated a collection that not only sheds light on the transformative impact of these technologies but also encourages interdisciplinary discourse. Scholars, educators, and professionals alike will benefit from the diverse perspectives presented, fostering a deeper understanding of the profound influence of AI and AR on spatial intelligence enhancement. In conclusion, this book represents a valuable resource that underscores the critical juncture of AI, AR, and spatial intelligence. The authors’ collective expertise and their commendable effort in bringing together various facets of this field make this work an essential asset for anyone seeking a comprehensive and insightful exploration of the subject. It is my hope that this book will inspire, inform, and spark further research in this dynamic and ever-evolving domain. Professor of Computer Science, University of the Philippines Quezon City, Philippines

Jaime Caro,

v

Preface

In an ever-evolving world driven by technological advancements, the intersection of spatial intelligence, augmented reality, and fuzzy logic user modeling has emerged as a captivating frontier of research and innovation. These three interconnected realms are not only at the forefront of technological development but also have the potential to reshape and enhance a multitude of domains, including education, artificial intelligence, and industry. However, while these concepts have made waves in academic circles and technology hubs, their integration into education remains relatively uncharted territory. This book sets out to bridge this gap and provide an insightful perspective on the transformative power of spatial intelligence, augmented reality, and fuzzy logic user modeling in the realm of education. In our pursuit of this endeavor, we propose an innovative approach that combines adaptive hypermedia with mobile training applications. This groundbreaking method seeks to revolutionize the delivery of educational content by tailoring it to the specific needs and preferences of each learner. The use of augmented reality in this approach is a cornerstone of our endeavor, offering a dynamic and interactive learning environment. By superimposing digital content onto the physical world, learners can engage with educational materials in a manner that is not only immersive but also profoundly tangible. This approach not only encourages active participation but also stimulates creativity and deepens understanding through visualizations, simulations, and real-time feedback. What truly sets this approach apart is its incorporation of spatial intelligence training within the augmented reality experience. This facet enhances learners’ ability to perceive and comprehend spatial relationships, nurturing skills such as spatial visualization, mental rotation, and spatial reasoning, which hold significant value across various academic disciplines and real-world applications. Further enhancing the learning experience is the integration of fuzzy logic user modeling, which ensures that educational content adapts to the unique preferences, learning pace, and strengths of each individual. This personalized approach customizes the presentation and difficulty level of content based on the learner’s responses, thus optimizing the learning journey, and making it not only more efficient but also more effective. vii

viii

Preface

To comprehensively evaluate the effectiveness and impact of our proposed approach, this book presents a multi-model evaluation based on Lynch and Ghergulescu’s framework. This evaluation includes various methodologies such as t-test analysis, control and experimental groups, and an extended technology acceptance model (TAM) for validation. The amalgamation of these diverse evaluation methods empowers readers to draw robust conclusions about the significance of integrating adaptive hypermedia into mobile training applications with augmented reality. Through this comprehensive evaluation, we seek to underscore the potential of this approach to revolutionize education and contribute to advancements in technology-­enhanced learning and cognitive development. The findings of our study offer valuable insights for educators, policymakers, and researchers who aspire to leverage emerging technologies to create personalized and impactful learning experiences. It is our belief that this approach has the power to transform the way knowledge is disseminated and acquired, ushering in a new era of progress in the fields of education, technology, and artificial intelligence, and opening up exciting avenues for further research and exploration. As you set off on this journey through the pages of this book, we invite you to explore, learn, and imagine the future of education through the lens of spatial intelligence, augmented reality, and fuzzy logic user modeling. Together, we will embark on a transformative voyage that holds the promise of shaping a brighter and more personalized educational landscape for generations to come. Egaleo, Greece Egaleo, Greece  Egaleo, Greece 

Christos Papakostas Christos Troussas Cleo Sgouropoulou

Contents

1

Introduction and Overview of AI-Enhanced Augmented Reality in Education����������������������������������������������������������������������������������������������    1 1.1 Overview������������������������������������������������������������������������������������������    1 1.2 Motivation ����������������������������������������������������������������������������������������    2 1.3 Research Questions��������������������������������������������������������������������������    6 1.4 Approach and Structure��������������������������������������������������������������������    7 References��������������������������������������������������������������������������������������������������    8

2

Review of the Literature on AI-Enhanced Augmented Reality in Education����������������������������������������������������������������������������������������������   13 2.1 Overview������������������������������������������������������������������������������������������   14 2.2 Spatial Ability: Review of Theories��������������������������������������������������   14 2.2.1 Spatial Ability in Engineering����������������������������������������������   15 2.3 Augmented Reality in Education������������������������������������������������������   20 2.3.1 AR in Engineering Education ����������������������������������������������   21 2.4 Learning Theories ����������������������������������������������������������������������������   22 2.4.1 The Bloom’s Taxonomy��������������������������������������������������������   22 2.4.2 The SOLO Taxonomy����������������������������������������������������������   22 2.4.3 Comparison of the Learning Theories����������������������������������   23 2.5 Literature Review������������������������������������������������������������������������������   25 2.5.1 Planning the Review (Review Protocol) ������������������������������   26 2.5.2 Conducting the Review��������������������������������������������������������   27 2.5.3 Screening of the Evaluation Papers��������������������������������������   28 2.5.4 Advantages of AR in Spatial Ability Training (RQ1)����������   31 2.5.4.1 Learner Outcomes ������������������������������������������������   31 2.5.4.2 Pedagogical Affordances ��������������������������������������   33 2.5.4.3 Technical Perspectives������������������������������������������   33 2.5.5 Limitations of AR in Spatial Ability Training (RQ2) ����������   34 2.5.6 Exploration of the Incorporation of Adaptivity and Personalization in AR Applications (RQ3)��������������������   35

ix

x

Contents

2.5.7 Aspects of Spatial Abilities Having Been Evaluated Using AR (RQ4)������������������������������������������������������������������������������   37 2.5.8 Evaluation Methods Considered for AR Applications in Educational Scenarios (RQ5)�������������������������������������������   39 2.6 Summary ������������������������������������������������������������������������������������������   40 References��������������������������������������������������������������������������������������������������   41 3

AI-Driven and SOLO-Based Domain Knowledge Modeling in PARSAT AR Software ������������������������������������������������������������������������   51 3.1 Overview������������������������������������������������������������������������������������������   51 3.2 Domain Model����������������������������������������������������������������������������������   52 3.2.1 Objectives������������������������������������������������������������������������������   52 3.3 Domain Knowledge Alongside SOLO Taxonomy����������������������������   54 3.4 Examples of Learning Activities of Each SOLO Level��������������������   56 3.5 Summary ������������������������������������������������������������������������������������������   60 References��������������������������������������������������������������������������������������������������   61

4

Fuzzy Logic for Modeling the Knowledge of Users in PARSAT AR Software���������������������������������������������������������������������������������������������   65 4.1 Overview������������������������������������������������������������������������������������������   66 4.2 Fuzzy Logic Algorithm ��������������������������������������������������������������������   66 4.3 Initialization Process������������������������������������������������������������������������   68 4.4 Fuzzy Sets ����������������������������������������������������������������������������������������   69 4.5 Fuzzy Rule Base ������������������������������������������������������������������������������   72 4.6 Mamdani’s Inference System������������������������������������������������������������   79 4.7 Defuzzification����������������������������������������������������������������������������������   80 4.8 Adaptation of the Learning Activities Based on Fuzzy Weights������   84 4.8.1 Decision Making������������������������������������������������������������������   86 4.9 Summary ������������������������������������������������������������������������������������������   88 References��������������������������������������������������������������������������������������������������   89

5

Artificial Intelligence-Enhanced PARSAT AR Software: Architecture and Implementation����������������������������������������������������������   93 5.1 Overview������������������������������������������������������������������������������������������   94 5.2 System Architecture��������������������������������������������������������������������������   94 5.2.1 Hardware Layer��������������������������������������������������������������������   95 5.2.1.1 Tracking����������������������������������������������������������������   95 5.2.1.2 Processing��������������������������������������������������������������   96 5.2.1.3 Interacting��������������������������������������������������������������   97 5.2.2 Software Layer����������������������������������������������������������������������   97 5.2.2.1 User Interface��������������������������������������������������������   97 5.2.2.2 3D Rendering Engine��������������������������������������������  100 5.2.3 Data Layer����������������������������������������������������������������������������  106 5.2.3.1 Marker Database����������������������������������������������������  106 5.2.3.2 3D Models Database����������������������������������������������  110 5.2.3.3 Interaction Model��������������������������������������������������  114

Contents

xi

5.3 Implementation of the System����������������������������������������������������������  119 5.3.1 User Interface of PARSAT����������������������������������������������������  119 5.3.2 Fuzzy Logic Controller Implementation with C# Scripting��������������������������������������������������������������������������  124 5.3.2.1 System Initialization����������������������������������������������  124 5.3.2.2 Linguistic Variables and Membership Functions ��������������������������������������������������������������  125 5.3.2.3 Fuzzification Process Implementation������������������  126 5.3.2.4 Rules of the System ����������������������������������������������  126 5.3.2.5 Evaluation of the Rules������������������������������������������  127 5.3.2.6 Defuzzification������������������������������������������������������  128 5.4 Summary ������������������������������������������������������������������������������������������  128 References��������������������������������������������������������������������������������������������������  129 6

Multi-model Evaluation of the Artificial Intelligence-Enhanced PARSAT AR Software ����������������������������������������������������������������������������  131 6.1 Overview������������������������������������������������������������������������������������������  131 6.2 Evaluation Framework����������������������������������������������������������������������  132 6.2.1 Research Sample������������������������������������������������������������������  132 6.2.2 Training Preparation ������������������������������������������������������������  133 6.3 t-Test Analysis of Students’ Feedback����������������������������������������������  134 6.4 Comparative Analysis of Pre-test/Post-test Model in Achieving the Learning Outcomes ��������������������������������������������������������������������  136 6.4.1 Discussion of the Results������������������������������������������������������  138 6.5 Extended Technology Acceptance Model for Detecting Influencing Factors����������������������������������������������������������������������������  139 6.5.1 Existing Acceptance Models������������������������������������������������  139 6.5.2 Proposed Extended Model����������������������������������������������������  141 6.5.3 Research Model and Hypotheses������������������������������������������  141 6.5.4 Research Instruments������������������������������������������������������������  143 6.5.5 Data Analysis������������������������������������������������������������������������  144 6.5.6 Model Validation������������������������������������������������������������������  145 6.5.6.1 Measurement Model����������������������������������������������  145 6.5.6.2 Structural Model����������������������������������������������������  147 6.6 Summary ������������������������������������������������������������������������������������������  150 References��������������������������������������������������������������������������������������������������  150

7

 onclusions of AI-Driven AR in Education������������������������������������������  157 C 7.1 Overview������������������������������������������������������������������������������������������  157 7.2 Conclusions and Discussion ������������������������������������������������������������  158 7.3 Contribution to Intelligent Tutoring Systems�����������������������������������  163 7.4 Contribution to Domain Knowledge Model ������������������������������������  166 7.5 Contribution to Student Modeling����������������������������������������������������  169 7.6 Contribution to Electronic Assessment ��������������������������������������������  172 7.7 Future Work��������������������������������������������������������������������������������������  173 References��������������������������������������������������������������������������������������������������  174

Chapter 1

Introduction and Overview of AI-Enhanced Augmented Reality in Education

Abstract  This chapter of this book serves as an introductory chapter, offering readers a comprehensive overview of the research. It begins with an “Overview” section that outlines the main sections to provide readers with a roadmap of what to expect in the subsequent sections. The “Motivation” section explores the reasons behind conducting this research, emphasizing the significance of spatial ability in human intelligence and its connection to success in scientific and educational fields. It also discusses the potential benefits of augmented reality in enhancing spatial ability and the importance of adaptivity in training systems, which serves as a motivation for the study. In the “Research Questions” section, specific research questions are introduced, designed to address gaps in existing literature and examine the impact of a proposed blended mobile system on fostering spatial ability. These questions provide a clear focus for the study and guide the subsequent chapters. It highlights the iterative nature of the research and presents the overall structure of the book, helping readers understand how the subsequent chapters build upon each other.

1.1 Overview The first chapter introduces the purpose of the research presented in the current book. More specifically, Sect. 1.2 argues the research motivation and outlines the problem, Sect. 1.3 defines the research questions, and the final section defines the approach and structure of the book.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 C. Papakostas et al., Special Topics in Artificial Intelligence and Augmented Reality, Cognitive Technologies, https://doi.org/10.1007/978-3-031-52005-1_1

1

2

1  Introduction and Overview of AI-Enhanced Augmented Reality in Education

1.2 Motivation Einstein, Newton, Faraday, Maxwell, and Tesla are scientists who left their mark on the history of physics and technology, and they all claimed that the development of their ground-breaking theories involved the utilization of spatial visualization [1–5]. For instance, Albert Einstein gained notoriety for his utilization of creative thinking and his capacity to mentally comprehend challenging mathematical concepts. In order to create his laws of motion and gravitation, Isaac Newton also possessed a keen visual imagination and the ability to visually picture the motion of celestial bodies. The inception of the principle of the electromagnetic field and Michael Faraday’s research into electromagnetic induction are his two most notable accomplishments. Utilizing Faraday’s findings as a foundation, James Clerk Maxwell created a series of formulas that clarify the operation of electric and magnetic fields. These equations are known as Maxwell’s equations. The evolution of modern innovations like radio and television was facilitated by these equations. His research on alternating current (AC) electrical systems and his innovations linked to wireless communication and electricity transmission are what made Nikola Tesla, a contemporary of Faraday and Maxwell, so widely recognized. Tesla was also renowned for his prodigious visualization skills, which he allegedly used to plan and create a number of his inventions. Spatial visualization is the ability to mentally rotate, manipulate, and twist twoand three-dimensional stimulus objects [6]. For scientists and engineers, spatial visualization is a crucial ability since it may be utilized to develop new ideas, design new technologies, and comprehend complex data. It is an ability that can be acquired and refined with practice, and scientific and engineering education frequently emphasizes it. Psychologists and cognitive researchers have scrutinized and delineated the concept of “spatial ability” through numerous perspectives across several decades. Spatial ability, one of the most extensively studied human aptitudes, has undergone various interpretations over time [6–16]. While definitions of spatial ability may vary, most researchers concur that it entails the skill to manipulate visual and spatial data mentally. This encompasses the ability to mentally rotate or manipulate objects, comprehend spatial connections between objects, and navigate physical environments. Extensive research has established a connection between spatial ability and academic performance [17, 18]. In the past decade, studies have underscored the significant importance of spatial ability in achieving academic excellence, notably within the realms of STEM disciplines (science, technology, engineering, and mathematics) [19–24]. Spatial cognition has also shown relevance across a spectrum of vocations, encompassing STEM careers, as well as everyday tasks such as driving and navigation. While having better spatial skills as a child can predict a person’s future success in STEM fields [25], a meta-analysis of the research on spatial ability by [1] revealed that spatial skills can develop over time with the right kind of training [26].

1.2 Motivation

3

According to research, spatial ability may be developed via practice and instruction and is not a set characteristic. Although there is some research that indicates that spatial ability may have genetic roots, contextual elements including exposure to spatially demanding activities and experiences can also have a significant impact on how spatial ability develops. There are several ways to train one’s own spatial skills, including playing specific video games, engaging in mental rotation and visualization exercises, and getting specialized spatial instruction in school settings. For children’s spatial ability to develop, early exposure to challenging spatial tasks and situations can be very helpful. The importance of spatial ability, in engineering education, has been highlighted by several studies [26–30]. The visualization of abstract concepts, such as the geometry of three-dimensional objects, has been of great interest in engineering education [27]. Technical drawing design is considered to have a role in the training of spatial skills [30, 31]. The development of multimedia software, for improving 3-D spatial visualization skills, is based on the manipulation of physical models, providing kinaesthetic learners with a convenient way to absorb information [28]. Engineering education often involves the visualization and manipulation of complex three-dimensional objects and structures, and the ability to think spatially is critical for solving many engineering problems [32–34]. Technical drawing design is one way in which spatial skills can be developed in engineering education. Technical drawing involves the use of specialized software to create detailed drawings and schematics of engineering designs and requires a high degree of spatial visualization ability. By practicing technical drawing and other spatially challenging activities, engineering students can improve their spatial skills and better prepare themselves for success in their future careers. The advancement of multimedia software and other digital technologies in recent years has provided new possibilities for enhancing spatial ability in engineering education. These resources offer students the ability to engage with three-­ dimensional models and representations on the display of their devices, making learning more engaging and immersive. For kinesthetic learners, who learn best through hands-on experience and physical manipulation of items, this can be extremely helpful. Overall, the improvement of spatial ability through focused training and instruction can have a positive impact on students’ future success in their chosen field. Engineering education places a high value on this goal. The challenge of student retention usually has an impact on engineering studies too, particularly concerning first-year university students [35]. Sorby [36] argues that students are more likely to lose motivation and abandon their engineering studies if they struggle with the course material right on. However, if they attend a course with significant spatial abilities and maintain putting up the effort to improve them as the course proceeds, there is a significantly lower chance that they would drop out. Students who struggle with the course material or lack confidence in their abilities are more likely to drop out of engineering studies, particularly in the first year of university. However, enhancing spatial skills might be crucial for increasing

4

1  Introduction and Overview of AI-Enhanced Augmented Reality in Education

student retention in their coursework. Universities and educational programs can assist students in addressing the difficulties they may encounter in their studies and help them remain engaged and motivated by offering opportunities for students to enhance their spatial ability as well as support and guidance to help them succeed. Moreover, training in spatial abilities can be particularly effective in improving student retention in engineering studies. Teachers can assist students in developing the abilities and self-assurance they need to excel in their studies by giving them specialized instruction and expertise in spatial visualizing activities. Overall, addressing the issue of student retention in the classroom necessitates a multifaceted strategy [37] that includes offering students the chance to develop their spatial ability, providing them with support and advice to help them succeed, and developing a welcoming and inclusive learning environment that fosters engagement and motivation. All levels of education have examined augmented reality (AR), which offers considerable benefits like student motivation and learning effectiveness [38–42]. AR is on the verge of transforming the human–computer relationship. AR contributes to interactivity and facilitates co-creation [43]; thus, AR has the potential to create training environments and scenarios that are cost-effective, safe and personalized [22, 44–49]. This represents a good fit with the spatial ability training carried out for engineers. The immersive nature of the training using mixed reality offers a unique realistic quality, which is not generally present in traditional education in the classroom yet retains considerable cost advantages over large-scale real-life laboratories and is gaining increasing acceptance [50]. Personalized training offers great pedagogical affordances, as it provides an enhanced learning experience, improves student engagement, and promotes knowledge acquisition [51]. The adaptive systems integrate built-in components in order to offer knowledge domain adaptivity and deliver different learning activities, tailored to the student’s profile. By tailoring learning activities to a student’s individual needs and preferences, adaptive systems can improve student engagement and motivation. This approach can also help to promote knowledge acquisition by providing students with targeted feedback and support. Furthermore, adaptive technologies can offer instantaneously evaluations of a student’s progress, enabling teachers to spot potential trouble spots and offer extra assistance as necessary. As a whole, employing adaptive technologies in engineering education has an upside of enhancing student learning outcomes and delivering a more efficient and tailored learning environment. The learning material’s pedagogical potential increases with how adaptable it is to the cognitive needs and capabilities of the students. For instance, in the case of a student who studies a specific domain concept, having a high knowledge level, and has been given many learning activities that are not appropriate for that level, then the learning process may not go as expected and the student would feel frustrated [52]. Providing inappropriate learning activities can lead to disengagement, which may negatively affect the student’s motivation and learning outcomes. Maintaining student involvement and facilitating the learning process can be achieved by tailoring the learning activities to the students’ knowledge and cognitive capacities. This

1.2 Motivation

5

is where personalized learning comes into play since it enables specialized training that is catered to each student’s unique requirements and talents. The creation of AR applications that take use of the portable qualities and quick access to information that are gained with mobile devices has become popular over the past few years due to the trend of mixing mobile technologies with AR [53]. The convergence of mobile technology and augmented reality has ushered in fresh prospects for education and training by granting users access to augmented reality encounters and materials via their mobile devices. This has the potential to make learning more adaptable, personalized, and engaging [54–60]. However, the combination of AR and its application in educational settings remains an open area research. The elaboration of educational content based on augmented reality approaches or methods for the design and construction of highly interactive materials so that it can offer tailored learning in any location and at any time are not subject to any predefined rules. The potential of augmented reality (AR) in education has been the subject of some studies and trials, but there is still much to learn and investigate in terms of setting standards and best practices for creating efficient AR-based learning experiences. Additionally, there is a need for deeper and thorough evaluation techniques to determine how AR affects learning results. It is anticipated that new approaches and frameworks will emerge to direct the development and application of augmented reality in education as the area continues to grow. In light of the aforementioned, the major objective of this research is to utilize the advantages of augmented reality and the technology of adaptive systems by fusing them in a novel way to provide optimized and customized spatial ability training. By combining AR technology with adaptive systems, it may be possible to create a highly interactive and engaging learning environment that can provide personalized training in spatial ability. This could have important implications for education in fields such as engineering, where spatial ability is a key skill. Given that the undergraduate students of the Departments of Electrical and Electronic Engineering, Biomedical Engineering, Industrial Design and Production Engineering, Informatics and Computer Engineering, Surveying and Geoinformatics Engineering, Mechanical Engineering, Naval Architecture, and Civil Engineering of the School of Engineering of the University of West Attica, are offered at their curriculum, courses such as the technical drawing, which is highly correlated with an advanced level of spatial ability, this research developed a mobile application for spatial ability training. The mobile application, namely PARSAT (personalized spatial ability training application), incorporates learning theories to support the pedagogical features of the system, and fuzzy expert system for personalization and adaptivity. The incorporation of learning theories and a fuzzy expert system for personalization and adaptivity can help optimize the training experience for each individual student. In particular, PARSAT incorporates the following: • the use of the Structure of the Observed Learning Outcomes (SOLO) learning theory, for the instructional design of learning content, providing a framework

6

1  Introduction and Overview of AI-Enhanced Augmented Reality in Education

for organizing and assessing student learning outcomes based on increasing levels of complexity; • the use of fuzzy logic in an AR system in engineering education, a mathematical tool for dealing with uncertain and imprecise information, to personalize the learning experience based on the student’s performance; • the adaptive delivery of the learning activities taught to students in the AR system, which adjusts the difficulty and type of activities based on the student’s progress and needs. Students can provide highly engaging content that is individualized to their characteristics and requirements by integrating these applications into an adaptable and accessible learning process. As a result, students are able to comprehend the contents and correlate them to the actual world. Each student can benefit from a customized learning experience that takes into account their cognitive demands, skills, and preferences by integrating PARSAT with an adaptable and accessible learning process. By allowing students to apply the learning content to actual situations, this personalized learning experience can increase student engagement, knowledge acquisition, and the practical applicability of their learning. By incorporating AR technology, PARSAT can provide a more immersive and engaging learning experience, which can enhance spatial ability training in engineering education.

1.3 Research Questions The current research has the purpose to design and develop a system for training students’ spatial skills, in an innovative mobile environment, using augmented reality technology, and adaptive tutoring techniques. The research objective is to develop and evaluate a novel mobile system for spatial ability training using augmented reality technology. Hence, the research questions are formulated as follows: 1. What is the current state of research on spatial ability training through augmented reality technology? (RQ1); 2. How can the use of mobile devices enhance the effectiveness of the system for training spatial skills? (RQ2); 3. What are the key components of an effective system for training spatial skills using augmented reality technology and adaptive tutoring techniques? (RQ3); 4. What are the most effective instructional strategies and techniques for training spatial skills in a mobile augmented reality environment? (RQ4); 5. How effective is the developed AR-based spatial ability training system in improving participants’ spatial abilities? (RQ5); 6. How does the developed AR-based spatial ability training system compared to traditional spatial ability training methods in terms of effectiveness and efficiency? (RQ6);

1.4  Approach and Structure

7

7. What is the impact of the system on students’ academic achievement and motivation to train spatial skills? (RQ7); 8. What are the participants’ perceptions of the usability and effectiveness of the developed AR-based spatial ability training system? (RQ8).

1.4 Approach and Structure In essence, the aim of this research is to enhance the realm of spatial ability training by investigating the capabilities of augmented reality (AR) technology in creating a mobile training system that is both effective and efficient while also capturing the user’s engagement. The research will encompass the following activities: • • • •

a comprehensive literature review a design phase an implementation phase an evaluation phase

to answer the research questions and achieve the research objective. The structure of this book is well-organized and follows a logical progression. Following is a brief synopsis of each chapter: • Chapter 2 “Review of the Literature on AI-enhanced Augmented Reality in Education”: This chapter provides an in-depth analysis of the related literature on the topic of the book. It attempts to highlight the research’s current deficiencies and offer an extensive overview of the research setting. The chapter will also discuss the state of the art at the point in time and offer a critical assessment of earlier research. • Chapter 3 “AI-driven and SOLO-based Domain Knowledge Modeling in PARSAT AR Software”: This chapter describes the domain knowledge model used in the book and the integration of the SOLO taxonomy. In-depth analysis of the taxonomy and domain knowledge used, as well as their applicability to the research question, will also be provided in this chapter. • Chapter 4 “Fuzzy Logic for modeling the Knowledge of Users in PARSAT AR Software”: This chapter describes the student model used in the book and the integration of fuzzy logic. The student model and the fuzzy logic employed will also be thoroughly examined in this chapter, along with their applicability to the subject of the study. • Chapter 5 “Artificial Intelligence-enhanced PARSAT AR Software: Architecture and Implementation”: This chapter provides a detailed description of the architecture and implementation of the proposed system. It will also discuss the software tools and technologies used, the design decisions made, and the implementation challenges faced. • Chapter 6 “Multi-Model Evaluation of the Artificial Intelligence-Enhanced PARSAT AR Software”: This chapter evaluates the proposed system using a

8

1  Introduction and Overview of AI-Enhanced Augmented Reality in Education

multi-model approach. Additionally, it will go through the design decisions made, the implementation difficulties encountered, and the software tools and technologies employed. The chapter will also compare the proposed system to existing approaches. • Chapter 7 “Conclusions of AI-driven AR in Education”: This chapter presents and discusses the conclusions of the book. It will highlight the research’s contributions, provide a summary of the findings, and make suggestions for ­additional study. The chapter will also discuss the research’s shortcomings and the ramifications of its findings.

References 1. D.  Uttal et  al., “The Malleability of Spatial Skills: A Meta-Analysis of Training Studies,” Psychol Bull, vol. 139, Jun. 2012, https://doi.org/10.1037/a0028446. 2. D. Uttal, D. Miller, and N. Newcombe, “Exploring and Enhancing Spatial Thinking Links to Achievement in Science, Technology, Engineering, and Mathematics?,” Curr Dir Psychol Sci, vol. 22, pp. 367–373, Oct. 2013, https://doi.org/10.1177/0963721413484756. 3. A. I. Miller, “Imagery in scientific thought: Creating 20th-century physics,” 1984. 4. R. N. Shepard, “The science of imagery and the imagery of science,” in Annual Meeting of the American Psychological Society, San Francisco, 1996. 5. N. J. Nersessian, “Should physicists preach what they practice?,” Sci Educ (Dordr), vol. 4, no. 3, pp. 203–226, 1995. 6. M. G. McGee, Human Spatial Abilities: Sources of Sex Differences. in Praeger special studies. Praeger, 1979. [Online]. Available: https://books.google.gr/books?id=2HgiAAAACAAJ 7. E. L. Thorndike, “On the Organization of Intellect.,” Psychol Rev, vol. 28, no. 2, pp. 141–151, 1921, https://doi.org/10.1037/h0070821. 8. C. Spearman, “‘General intelligence,’ objectively determined and measured.,” Am J Psychol, vol. 15, pp. 201–293, 1904, https://doi.org/10.2307/1412107. 9. A. A. H. E. Koussy, An Investigation Into the Factors in Tests Involving the Visual Perception of Space. in British Journal of Psychology. Monograph Supplements. University Press, 1935. [Online]. Available: https://books.google.gr/books?id=SZh0tAEACAAJ 10. L. L. Thurstone and T. G. Thurstone, Primary mental abilities, vol. 119. University of Chicago Press Chicago, 1938. 11. L. Thurstone, “Some Primary Abilities in Visual Thinking,” Proc Am Philos Soc, vol. 94, no. 6, pp. 517–521, 1950. 12. H.  Gardner, Frames of mind  : the theory of multiple intelligences. New  York: Basic Books, 1983. 13. M. G. McGee, “Human spatial abilities: Psychometric studies and environmental, genetic, hormonal, and neurological influences.,” Psychological Bulletin, vol. 86. American Psychological Association, US, pp. 889–918, 1979. https://doi.org/10.1037/0033-­2909.86.5.889. 14. M. C. Linn and A. C. Petersen, “Emergence and Characterization of Sex Differences in Spatial Ability: A Meta-Analysis,” Child Dev, vol. 56, no. 6, pp. 1479–1498, Dec. 1985, https://doi. org/10.2307/1130467. 15. J. B. Carroll, Human cognitive abilities: A survey of factor-analytic studies, no. 1. Cambridge University Press, 1993. 16. J. W. Pellegrino and E. B. Hunt, “Cognitive models for understanding and assessing spatial abilities,” Intelligence: Reconceptualization and measurement, pp. 203–225, 1991. 17. I.  Macfarlane. Smith, Spatial ability  : its educational and social significance. San Diego, Calif.: R.R. Knapp, 1964.

References

9

18. C. L. Gohm, L. G. Humphreys, and G. Yao, “Underachievement among spatially gifted students.,” Am Educ Res J, vol. 35, pp. 515–531, 1998, https://doi.org/10.2307/1163447. 19. C. Papakostas, C. Troussas, A. Krouska, and C. Sgouropoulou, “Exploring Users’ Behavioral Intention to Adopt Mobile Augmented Reality in Education through an Extended Technology Acceptance Model,” Int J Hum Comput Interact, vol. 39, no. 6, pp. 1294–1302, 2023, https:// doi.org/10.1080/10447318.2022.2062551. 20. C. Papakostas, C. Troussas, A. Krouska, and C. Sgouropoulou, “User acceptance of augmented reality welding simulator in engineering training,” Educ Inf Technol (Dordr), vol. 27, no. 1, pp. 791–817, Jan. 2022, https://doi.org/10.1007/s10639-­020-­10418-­7. 21. A. Marougkas, C. Troussas, A. Krouska, and C. Sgouropoulou, “Virtual Reality in Education: A Review of Learning Theories, Approaches and Methodologies for the Last Decade,” Electronics (Basel), vol. 12, no. 13, 2023, https://doi.org/10.3390/electronics12132832. 22. A. Krouska, C. Troussas, K. Kabassi, and C. Sgouropoulou, “An Empirical Investigation of User Acceptance of Personalized Mobile Software for Sustainability Education,” Int J Hum Comput Interact, pp. 1–8, Aug. 2023, https://doi.org/10.1080/10447318.2023.2241614. 23. P.  Strousopoulos, C.  Troussas, C.  Papakostas, A.  Krouska, and C.  Sgouropoulou, “Revolutionizing Agricultural Education with Virtual Reality and Gamification: A Novel Approach for Enhancing Knowledge Transfer and Skill Acquisition,” in Novel & Intelligent Digital Systems: Proceedings of the 3rd International Conference (NiDS 2023), K. Kabassi, P. Mylonas, and J. Caro, Eds., Cham: Springer Nature Switzerland, 2023, pp. 67–80. 24. M. Iakovidis, C. Papakostas, C. Troussas, and C. Sgouropoulou, “Empowering Responsible Digital Citizenship Through an Augmented Reality Educational Game,” in Novel & Intelligent Digital Systems: Proceedings of the 3rd International Conference (NiDS 2023), K. Kabassi, P. Mylonas, and J. Caro, Eds., Cham: Springer Nature Switzerland, 2023, pp. 31–39. 25. J. Wai, D. Lubinski, and C. Benbow, “Spatial Ability for STEM Domains: Aligning Over 50 Years of Cumulative Psychological Knowledge Solidifies Its Importance,” J Educ Psychol, Nov. 2009, https://doi.org/10.1037/a0016127. 26. S.  Sorby, “Developing 3D spatial skills for engineering students,” Australasian Journal of Engineering Education, vol. 13, pp.  1–11, Jan. 2007, https://doi.org/10.1080/22054952.200 7.11463998. 27. A.  R. Arslan and S.  Dazkir, “Technical Drafting and Mental Visualization in Interior Architecture Education,” International Journal for the Scholarship of Teaching and Learning, vol. 11, no. 2, 2017, https://doi.org/10.20429/ijsotl.2017.110215. 28. H.  B. P.  Gerson, S.  A. Sorby, A.  Wysocki, and B.  J. Baartmans, “The development and assessment of multimedia software for improving 3-D spatial visualization skills,” Computer Applications in Engineering Education, vol. 9, no. 2, pp. 105–113, 2001. 29. J.  L. Mohler, “Computer Graphics Education: Where and How Do We Develop Spatial Ability?,” in Eurographics (Education Papers), 2006, pp. 79–86. 30. A. Šafhalter, S. Glodež, A. Šorgo, and M. Ploj Virtič, “Development of spatial thinking abilities in engineering 3D modeling course aimed at lower secondary students,” Int J Technol Des Educ, 2020, https://doi.org/10.1007/s10798-­020-­09597-­8. 31. R. H. McKim, Experiences in visual thinking. Monterey, Calif.: Brooks/Cole Pub. Co., 1980. 32. Z. Kanetaki et al., “Acquiring, Analyzing and Interpreting Knowledge Data for Sustainable Engineering Education: An Experimental Study Using YouTube,” Electronics (Basel), vol. 11, no. 14, 2022, https://doi.org/10.3390/electronics11142210. 33. A.  Krouska, C.  Troussas, and C.  Sgouropoulou, “Fuzzy Logic for Refining the Evaluation of Learners’ Performance in Online Engineering Education,” European Journal of Engineering Research and Science, vol. 4, pp.  50–56, Jun. 2019, https://doi.org/10.24018/ ejers.2019.4.6.1369. 34. C. Papakostas, C. Troussas, A. Krouska, and C. Sgouropoulou, “Exploration of Augmented Reality in Spatial Abilities Training: A Systematic Literature Review for the Last Decade,” Informatics in Education, vol. 20, no. 1, pp.  107–130, Mar. 2021, https://doi.org/10.15388/ infedu.2021.06.

10

1  Introduction and Overview of AI-Enhanced Augmented Reality in Education

35. S. Sheppard and R. Jennison, “Freshman engineering design experiences and organizational framework,” International Journal of Engineering Education, vol. 13, pp. 190–197, 1997. 36. S. A. Sorby, “Assessment of a “new and improved” course for the development of 3-D spatial skills,” The Engineering Design Graphics Journal, vol. 69, no. 3, 2005. 37. C.  Papakostas, C.  Troussas, P.  Douros, M.  Poli, and C.  Sgouropoulou, “CoMoPAR: A Comprehensive Conceptual Model for Designing Personalized Augmented Reality Systems in Education,” in Novel & Intelligent Digital Systems: Proceedings of the 3rd International Conference (NiDS 2023), K. Kabassi, P. Mylonas, and J. Caro, Eds., Cham: Springer Nature Switzerland, 2023, pp. 67–79. 38. J. L. B. Acosta, S. M. B. Navarro, R. F. Gesa, and K. Kinshuk, “Framework for designing motivational augmented reality applications in vocational education and training,” Australasian Journal of Educational Technology, vol. 35, no. 3 SE-Articles, Jun. 2019, https://doi. org/10.14742/ajet.4182. 39. S. Y. Chen and S. Y. Liu, “Using augmented reality to experiment with elements in a chemistry course,” Comput Human Behav, vol. 111, p. 106316, Apr. 2020, https://doi.org/10.1016/j. chb.2020.106418. 40. J.  Garzón, J.  Pavón, and S.  Baldiris, “Systematic review and meta-analysis of augmented reality in educational settings,” Virtual Real, vol. 23, no. 4, pp.  447–459, 2019, https://doi. org/10.1007/s10055-­019-­00379-­9. 41. M. Thees, S. Kapp, M. P. Strzys, F. Beil, P. Lukowicz, and J. Kuhn, “Effects of augmented reality on learning and cognitive load in university physics laboratory courses,” Comput Human Behav, vol. 108, p. 106316, 2020, https://doi.org/10.1016/j.chb.2020.106316. 42. C. Papakostas, C. Troussas, A. Krouska, and C. Sgouropoulou, “Measuring User Experience, Usability and Interactivity of a Personalized Mobile Augmented Reality Training System,” Sensors, vol. 21, no. 11, p. 3888, Jun. 2021, https://doi.org/10.3390/s21113888. 43. S. Alimamy, K. R. Deans, and J. Gnoth, “The Role of Augmented Reality in the Interactivity of Co-Creation: A Critical Review,” Int. J. Technol. Hum. Interact., vol. 14, no. 3, pp. 88–104, 2018, https://doi.org/10.4018/IJTHI.2018070106. 44. C.  Papakostas, C.  Troussas, A.  Krouska, and C.  Sgouropoulou, “PARSAT: Fuzzy logic for adaptive spatial ability training in an augmented reality system,” Computer Science and Information Systems. 45. C.  Papakostas, C.  Troussas, A.  Krouska, and C.  Sgouropoulou, “Personalization of the Learning Path within an Augmented Reality Spatial Ability Training Application Based on Fuzzy Weights,” Sensors, vol. 22, no. 18, 2022, https://doi.org/10.3390/s22187059. 46. C.  Troussas, C.  Papakostas, A.  Krouska, P.  Mylonas, and C.  Sgouropoulou, “Personalized Feedback Enhanced by Natural Language Processing in Intelligent Tutoring Systems,” in Augmented Intelligence and Intelligent Tutoring Systems, C.  Frasson, P.  Mylonas, and C.  Troussas, Eds., Cham: Springer Nature Switzerland, 2023, pp.  667–677. https://doi. org/10.1007/978-­3-­031-­32883-­1_58. 47. C.  Papakostas, C.  Troussas, A.  Krouska, and C.  Sgouropoulou, “On the development of a personalized augmented reality spatial ability training mobile application,” in Frontiers in Artificial Intelligence and Applications, IOS Press, 2021, pp. V–VI. https://doi.org/10.3233/ FAIA210078. 48. A.  Marougkas, C.  Troussas, A.  Krouska, and C.  Sgouropoulou, “How personalized and effective is immersive virtual reality in education? A systematic literature review for the last decade,” Multimed Tools Appl, 2023, https://doi.org/10.1007/s11042-­023-­15986-­7. 49. A. Marougkas, C. Troussas, A. Krouska, and C. Sgouropoulou, “A Framework for Personalized Fully Immersive Virtual Reality Learning Environments with Gamified Design in Education,” 2021. https://doi.org/10.3233/FAIA210080. 50. E. B. Hsu, Y. Li, J. D. Bayram, D. Levinson, S. Yang, and C. Monahan, “State of virtual reality based disaster preparedness and response training,” PLoS Curr, vol. 5, 2013. 51. A.  Soofi and M.  Uddin, “A Systematic Review of Domains, Techniques, Delivery Modes and Validation Methods for Intelligent Tutoring Systems,” International Journal of

References

11

Advanced Computer Science and Applications, vol. 10, Jan. 2019, https://doi.org/10.14569/ IJACSA.2019.0100312. 52. M. Holmes, A. Latham, K. A. Crockett, and J. D. O’Shea, “Near Real-Time Comprehension Classification with Artificial Neural Networks: Decoding e-Learner Non-Verbal Behavior,” IEEE Transactions on Learning Technologies, vol. 11, pp. 5–12, 2018. 53. G. Papagiannakis, G. Singh, and N. Thalmann, “A Survey of Mobile and Wireless Technologies for Augmented Reality Systems (Preprint),” Comput Animat Virtual Worlds, vol. 19, pp. 3–22, Feb. 2008, https://doi.org/10.1002/cav.221. 54. P. Strousopoulos, C. Papakostas, C. Troussas, A. Krouska, P. Mylonas, and C. Sgouropoulou, “SculptMate: Personalizing Cultural Heritage Experience Using Fuzzy Weights,” in Adjunct Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization, in UMAP ’23 Adjunct. New York, NY, USA: Association for Computing Machinery, 2023, pp. 397–407. https://doi.org/10.1145/3563359.3596667. 55. C. Papakostas, C. Troussas, A. Krouska, and C. Sgouropoulou, “Modeling the Knowledge of Users in an Augmented Reality-Based Learning Environment Using Fuzzy Logic,” in Lecture Notes in Networks and Systems, A. Krouska, C. Troussas, and J. Caro, Eds., Cham: Springer International Publishing, 2023, pp. 113–123. https://doi.org/10.1007/978-­3-­031-­17601-­2_12. 56. C. Troussas, A. Krouska, and C. Sgouropoulou, “Enriching Mobile Learning Software with Interactive Activities and Motivational Feedback for Advancing Users’ High-Level Cognitive Skills,” Computers, vol. 11, no. 2, 2022, https://doi.org/10.3390/computers11020018. 57. F.  Giannakas, C.  Troussas, A.  Krouska, C.  Sgouropoulou, and I.  Voyiatzis, “XGBoost and Deep Neural Network Comparison: The Case of Teams’ Performance,” in Intelligent Tutoring Systems, A. I. Cristea and C. Troussas, Eds., Cham: Springer International Publishing, 2021, pp. 343–349. 58. C. Troussas, A. Krouska, and C. Sgouropoulou, “Towards a Reference Model to Ensure the Quality of Massive Open Online Courses and E-Learning,” in Brain Function Assessment in Learning, C.  Frasson, P.  Bamidis, and P.  Vlamos, Eds., Cham: Springer International Publishing, 2020, pp. 169–175. 59. C. Troussas, A. Krouska, and C. Sgouropoulou, “Dynamic Detection of Learning Modalities Using Fuzzy Logic in Students’ Interaction Activities,” in Intelligent Tutoring Systems, V. Kumar and C. Troussas, Eds., Cham: Springer International Publishing, 2020, pp. 205–213. 60. A.  Krouska, C.  Troussas, and C.  Sgouropoulou, “A novel group recommender system for domain-independent decision support customizing a grouping genetic algorithm,” User Model User-adapt Interact, 2023, https://doi.org/10.1007/s11257-­023-­09360-­3.

Chapter 2

Review of the Literature on AI-Enhanced Augmented Reality in Education

Abstract  This chapter provides a comprehensive review of the literature regarding AI-enhanced Augmented Reality (AR). It serves as the foundational knowledge base for the study, offering insights into relevant theories, concepts, and prior research studies. The chapter begins with an “Overview” section, outlining the purpose and significance of the literature review in establishing a robust theoretical framework. It emphasizes the necessity of exploring spatial ability, AR technology, and learning theories to comprehend their interconnections and implications for the development of a mobile training system. The subsequent section, “Spatial Ability: Review of Theories,” delves into the concept of spatial ability, particularly within engineering disciplines. Various theories and models that elucidate spatial ability, its components, and its relevance in the context of success in engineering are discussed in detail, providing a theoretical underpinning. The “Augmented Reality in Education” section explores the use of AR in educational settings, with a focus on engineering education. It discusses the advantages and potential of AR technology in enhancing spatial ability and facilitating learning, considering both pedagogical and technical aspects. The “Learning Theories” section introduces different learning theories, notably Bloom’s taxonomy and the Structure of Observed Learning Outcomes (SOLO) taxonomy. It outlines these taxonomies’ principles, stages, and hierarchical levels, emphasizing their relevance to instructional design and assessing learning outcomes. The “Literature Review” section elucidates the methodology employed in conducting the literature review, including the evaluation paper screening process. Findings from the review are presented, addressing various research questions, such as the benefits and drawbacks of AR in spatial ability training, adaptive features in AR applications, evaluation methods, and the specific aspects of spatial abilities assessed using AR.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 C. Papakostas et al., Special Topics in Artificial Intelligence and Augmented Reality, Cognitive Technologies, https://doi.org/10.1007/978-3-031-52005-1_2

13

14

2  Review of the Literature on AI-Enhanced Augmented Reality in Education

2.1 Overview This chapter presents an overview of the relevant literature and is organized into five sections. More specifically, the first section of this chapter introduces the term “spatial ability”, its multiple definitions, and its components. The second section of the chapter emphasizes in the importance of spatial ability in various fields, more specifically in engineering training. The third section evaluates the use of augmented reality technology to improve spatial skills. The fourth section presents a systematic literature review (SLR) on the exploration of relevant training, based on augmented reality technology, that has been used to improve spatial ability. The last section summarizes the review of the literature.

2.2 Spatial Ability: Review of Theories Spatial ability, as a factor of human intelligence, was initially recognized and studied by [1]. Thorndike levelled criticism against the formulation of two-factor theory of intelligence [2], whose theory was based on the existence of a general intelligence factor. Thorndike’s proposed model consisted of three mutually independent abilities, namely abstract (maintained from Spearman’s theory), mechanical and social intelligence, while mechanical ability was defined as the ability to visualize the objects’ relations and understanding of the physical world. Thorndike’s theory served as the early-stage research for later studies on spatial ability and highlighted the importance of designing measuring tools for it. In [3] the author also acknowledged spatial ability as a separate independent factor, opposed to Spearman’s [2] theory. Koussy researched spatial intelligence and contributed to the creation of tools for assessing it. Koussy identified the term factor “K”, as the ability to acquire and use visual spatial images. The manipulation of spatial relations was introduced as an additional separate component of spatial ability. In [4] the author also came to a different conclusion about the nature of intelligence than [2]. Thurstone suggested that the intelligence consists of seven interrelated primary mental abilities, rather than a single general one. These primary mental abilities include a) associative memory; b) numerical ability; c) perceptual speed; d) reasoning; e) spatial visualization; f) verbal comprehension; and g) word fluency, while spatial visualization was defined as the factor involved in visualizing and manipulating objects. Thurstone’s theory of multiple factors [4] was the basis for creating each factor’s measuring tests. Thurstone [5] defined three core components of spatial ability, namely mental rotation, spatial visualization, and spatial perception. Mental rotation refers to the capacity to identify an object as it undergoes various orientations or angles of movement; spatial visualization entails the capability to discern the components of an object when it is shifted or relocated from its initial position; and spatial perception

2.2  Spatial Ability: Review of Theories

15

involves the ability to utilize one’s own bodily orientation to engage with the surrounding environment, thereby influencing spatial awareness [6]. In [7] the author put forth the theory of multiple intelligences, which challenges the idea of intelligence as a singular, overarching capability. Instead, he suggested the existence of eight distinct intelligences, each rooted in specific skills and aptitudes. Gardner’s argument stated that there is a wide range of cognitive abilities, not necessarily correlated between them, namely musical, visual-spatial, linguistic-­ verbal, logical-mathematical, bodily-kinesthetic, interpersonal, intrapersonal, and naturalistic. Visual-spatial intelligence is defined as the ability to accurately interpret the visual environment, to alter and modify one’s first perception, and to recreate some aspects of one’s visual experience, even in the lack of appropriate physical input. In [8] the author extensively studied the factor structure of spatial ability, defining two main factors, namely a) spatial visualization; and b) spatial orientation. Spatial visualization is the ability to mentally rotate, manipulate, and twist two- and three-dimensional stimulus objects [8], while spatial orientation is the comprehension of the agreement of elements within a visual stimulus pattern and the aptitude to remain unconfused by the changing orientation in which a spatial configuration may be presented [9]. Linn and Petersen[10] researched the gender differences in the aspects of spatial ability and labeled the spatial ability factors as: a) spatial perception; b) mental rotation; and c) spatial visualization, separating mental rotation from spatial visualization. In [11] the author also proposed three factors for spatial ability, giving them the slightly different names: a) spatial visualization; b) spatial orientation; and c) speeded rotation. Many recent theories proposed additional factors or cognitive processes, in their effort to better understand spatial ability [12, 13]. Each research has added significantly to the definition of spatial ability, as a form of intelligence, where a person demonstrates the capacity to mentally generate, transform, and rotate a visual image and thus, understand and recall spatial relationships between real and imagined objects. Table  2.1 summarizes the definitions of spatial ability and its structure factors.

2.2.1 Spatial Ability in Engineering In [14] the author reported a list of 84 different occupations that require top-level spatial ability, and that engineering, and graphics-related occupations were well-­ represented on this list. 26 out of 84 occupations were related to engineering, while 14 of them were related to graphics (Table 2.2). There is a significant correlation between spatial ability and many scientific fields, such as geometry, physics and technical drawing [15, 16]. Spatial ability stands as a pivotal cognitive prowess significantly applicable across numerous scientific and technical domains such as engineering, graphics,

16

2  Review of the Literature on AI-Enhanced Augmented Reality in Education

Table 2.1  Spatial ability definitions Spatial Author(s) factor(s) [1] Mechanical ability [3] “K” factor [4] Spatial visualization [5] Mental rotation Spatial visualization Spatial perception [8] Spatial visualization Spatial orientation

[10]

[13]

Spatial perception Mental rotation Spatial visualization Spatial visualization Spatial relations Closure speed

Closure flexibility Perceptual speed

Definition The ability of visualizing the objects’ relations and understanding the physical world. The ability of acquiring and using visual spatial images Visualizing and manipulating objects The ability to recognize an object being moved in different directions or angles The ability to recognize the parts of an object when it is moved or displaced from its original position The ability to use the human’s own body orientation to interact with the environment The ability to mentally rotate, manipulate, and twist two- and three-dimensional stimulus objects The comprehension of the agreement of elements within a visual stimulus pattern and the aptitude to remain unconfused by the changing orientation in which a spatial configuration may be presented The ability to determine spatial relationships with respect to the orientation of the subject’s own body Rotation of a two- or three-dimensional figure rapidly and accurately The ability to involve complicated, multi-step manipulations of spatially presented information The ability in manipulating visual patterns, as indicated by level of difficulty and complexity in visual stimulus material that can be handled successfully, without regard to the speed of task solution Speed in manipulating relatively simple visual patterns by whatever means (mental rotation, transformation, or otherwise) Speed in apprehending and identifying a visual pattern, without knowing in advance what the pattern is, when the pattern is disguised or obscured in some way Speed in finding, apprehending, and identifying a visual pattern, knowing in advance what is to be apprehended, when the pattern is disguised or obscured in some way Speed in finding a known visual pattern, or in accurately comparing one or more patterns, in a visual field such that the patterns are not disguised or obscured

physics, and geometry [17, 18]. Those possessing robust spatial abilities are inclined toward triumph in these areas, and fostering spatial competence becomes a paramount objective for educational initiatives focused on equipping students for STEM careers. Beyond its significance in achieving success within the realm of engineering, spatial ability also plays a vital role in advancing innovative technologies. Engineers possessing robust spatial skills are more adept at conceptualizing and crafting novel

17

2.2  Spatial Ability: Review of Theories Table 2.2  Occupations requiring top-level spatial ability [14] 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36. 37. 38. 39. 40. 41. 42.

Occupation Air-plane Designer Architect, Marine Botanist Cartoonist Cartoonist, Motion Picture Chemist, Metallurgical Chemist, Physical Detailer Die Checker Die Designer Draughtsman, Aeronautical '' Apprentice Marine '' Architectural '' Construction '' Hull '' Marine '' Mechanical '' Mine '' Patent '' Refrigeration '' Ship Detail '' Ship Engineering '' Structural '' Tool Design Engineer, Agricultural '' Air-conditioning '' Automotive '' Ceramic '' Chemical Research '' Combustion '' Electrical Reearch '' Gas Distribution '' Hydraulic '' Methods '' Mining '' Petroleum '' Plant '' Steam-power Plant '' Production '' Radio '' Railroad '' Refrigeration

43. 44. 45. 46. 47. 48. 49. 50. 51. 52. 53. 54. 55. 56. 57. 58. 59. 60. 61. 62. 63. 64. 65. 66. 67. 68. 69. 70. 71. 72. 73. 74. 75. 76. 77. 78. 79. 80. 81. 82. 83. 84.

Occupation Engineer, Safety '' Salvage '' Sanitary '' Sheet-metal '' Systems '' Time-study '' Traffic '' Utilization '' Welding Geophysicist Industrial Designer Int. Combustion Engine Designer Machinery and Tool Designer Manager, Tarde Mark and © Mathematician Memorial Designer Modeller Neurologist Obstetrician Oculist Oral Surgeon Orthodontist Orthopedic Surgeon Osteopath Painter Pattern Checker Pattern Lay-out Man Pediatrician Physicist Production Planner Psychiatrist Psycologist, Industrial Public Health Officer Sculptor Stage-scenery Designer Surgeon Tool Designer Urologist Veterinarian Veterinarty, Bacteriologist '' Pathologist '' Surgeon

18

2  Review of the Literature on AI-Enhanced Augmented Reality in Education

technologies that align with societal requirements. To illustrate, in the domain of robotics, spatial ability is indispensable for crafting robotic systems capable of traversing intricate surroundings and executing precise object manipulation. Spatial ability performs as an indicator for success in engineering students. In [19] the author designed a 52-h course aiming to develop skills in representing spatially visualized objects through projections. The authors of [20] also launched a fast-remedial course based on three-dimensional modeling for improving spatial abilities of engineering students. Designing three-dimensional objects is highly considered to be a crucial factor in the development of spatial skills [21, 22]. In engineering university education, the enhancement of the visualization skills of the students is essential for the development of the design skills in many fields of engineering [23, 24]. Spatial skills are the most crucial indicator of success in objects’ manipulation and interaction [25]. Students, who have the opportunity to enhance their spatial skills, exhibit higher levels of self-efficacy, achieve better results in the fields of math and science and are more likely to maintain in engineering [26]. In [27] the author identified the challenge of improving the spatial ability of engineering students. In [28] the author reported that students’ spatial ability level is directly correlated to the success in the fields of engineering, mathematics, and architecture. Engineering drawing plays an important role in the efforts in improving students’ spatial ability. The School of Engineering includes first-year university programs for various departments, such as Electrical and Electronic Engineering, Biomedical Engineering, Industrial Design and Production Engineering, Informatics and Computer Engineering, Surveying and Geoinformatics Engineering, Mechanical Engineering, Naval Architecture, and Civil Engineering. In these programs, students are primarily exposed to courses like engineering drawing, technical drawing, and computer-aided design (CAD). According to a study by [29], courses related to technical drawing instruct students in the utilization of manual tools like pencils, paper, and drafting boards to create two-dimensional designs. Students must sketch the appropriate views of an object, such as a construction building, using orthographic and isometric projection techniques, as well as fundamental engineering graphics rules. First-year engineering students have difficulties in drawing orthographic views (such as the front view, the top view, and the right-side view), and perspectives of a 3D view (Fig.  2.1), since they find it difficult to understand 3D shapes from 2D views [30, 31]. Engineering students often struggle with identifying some of the views of a structure and recognizing the hidden lines [32]. This is because spatial ability, which is necessary for accurately visualizing and manipulating objects in the mind, is not a skill that comes naturally to everyone. In particular, understanding engineering drawings and technical diagrams requires a high level of spatial ability. Students need the capacity to mentally construct three-dimensional objects from two-dimensional representations, comprehend diverse views and projections of structures (Fig. 2.1), and identify concealed lines and other critical elements. These proficiencies are fundamental for success in engineering but can pose difficulties in attainment.

2.2  Spatial Ability: Review of Theories

19

Fig. 2.1  Spatial orthographic projections (a, b, c, d) of a 3D construction

To tackle these challenges, educators and instructional designers have devised an array of pedagogical techniques and tools aimed at enhancing the spatial ability of engineering students. These methods encompass the utilization of visual aids like three-dimensional models, computer-based simulations, and virtual reality environments. Moreover, educators may incorporate spatial visualization training programs and exercises to facilitate students in enhancing their spatial ability. Moreover, it’s crucial to acknowledge that students possess diverse learning styles and varying strengths and weaknesses [33, 34]. Spatial ability may be a challenge for some students, while others may demonstrate exceptional proficiency in this domain. Hence, educators should offer a range of instructional approaches and materials to support students in honing their spatial skills, guaranteeing that every student has the chance to thrive in the field of engineering. The creation of accurate engineering drawings requires a specific set of technical guidelines and spatial skills. The technical guidelines are typically covered in engineering departments’ drawing courses. Consequently, having strong spatial skills is essential for producing high-quality engineering drawings [35]. Numerous studies have emphasized the significance of spatial ability and have suggested various pedagogical approaches to enhance the development of this ability in engineering students [23, 24, 36]. In [37] the authors emphasize the importance of strong spatial skills, as a basic competency for future engineers, including the ability to visualize the rotation of objects and their different perspectives. A large number of studies has indicated that integrating the appropriate training materials, can result in the enhancement of the training of spatial skills [35, 38–40]. Freshmen engineering students, attending engineering graphics courses, showed substantive improvement in their spatial visualisation skills [41–44].

20

2  Review of the Literature on AI-Enhanced Augmented Reality in Education

2.3 Augmented Reality in Education All technologically enhanced realities fall under the broad concept of Extended Reality (XR), which combines the experiences of augmented reality (AR), virtual reality (VR), and mixed reality (MR) [45]. To improve this experience, AR overlays virtual content on top of the already existing real-world environment [46]. Contrarily, VR immerses viewers in an entirely new environment that is often developed and rendered by computers [47–50]. Finally, MR is a user environment that combines digital content and physical reality in a manner that makes it possible for users to interact with both real-world and virtual objects [51]. Augmented Reality (AR) serves as an optimal conduit for Internet of Things (IoT) applications by overlaying digital data concerning intelligent objects and services onto a user’s real-world perspective [52, 53]. An AR system possesses the following characteristics: a) it blends actual and virtual elements within a genuine setting; b) it operates interactively and instantaneously; and c) it aligns real and virtual components with one another [46]. A substantial body of published research exists, detailing the benefits, constraints, and obstacles associated with the use of Augmented Reality (AR) in educational settings [54–59]. Various interactive and innovative applications generated by AR technology [57, 60, 61] have given great potential in different learning subjects and specifically in STEM (Science, Technology, Engineering, and Mathematics) education [62]. Many studies have explored the positive effects of AR technology in education [17, 63, 64], as compared to the traditional methods of teaching and learning. Previous authors [61] tested a firefighting training system using AR, which was more cost-effective and safer, compared to large-scale real-life training. Simulating firefighting scenarios helped the trainees evaluate their knowledge and deal with risky circumstances. Another study [65] presented a mobile AR travel guide, supporting personalized recommendations. The authors explored the relationship between system properties, user emotions, and adoption behavior. More specifically, the developed AR application built a user profile, based on users’ preferences, and according to the updated profile, users are offered extra media features. AR is considered one of the most disruptive technologies in the field of marketing [66]. Consumers derive tangible benefits from AR technology and expect it as part of their purchasing process. AR has been incorporated into store catalog applications, which allow consumers to visualize what different products would look like in different environments. With AR technology, marketers are able to carry out successful digital campaigns. AR aids marketing as it can let the customers try before they buy, augment touring and assistance, and finally, augment branding materials. In [67] the authors explored the potential of VR, AR, and MR in both dental education and clinical dentistry. AR and VR technology can be beneficial not only for dental students but also for patients, as AR and VR can reduce dental anxiety and treat dental phobias. Furthermore, a systematic review [68] investigated the

2.3  Augmented Reality in Education

21

usability of AR in the area of health sciences on the aspect of the psychopedagogy of students in higher education. The usage of AR improved many aspects of the learning process, including motivation, satisfaction, and autonomous learning [57, 69, 70]. The literature review reveals numerous studies focusing on the integration of AR technology in fields such as education [58, 60, 61, 71, 72], tourism [65, 73–76], industry [77–79], marketing [66, 80, 81], and medical [67, 68].

2.3.1 AR in Engineering Education Engineering drawing was used to be taught using chalk and board, as well as model blocks back in the day. Nowadays, the development of technology has introduced the use of computer-aided software in the teaching of engineering drawing in higher institutions [82–86]. Recent studies have shown that AR is considered to be one of the best alternative teaching approaches to cover these issues [23, 56, 87]. A study by [88] examined the use of AR in teaching students and future teachers of Descriptive Geometry, Engineering and Computer Graphics (DGECG). Their results showed that there is an absence of scientifically substantiated and proven programs and training material for training students of DGECG using AR, needing further scientific research in this field. However, the study examined solely nine articles without focusing on a detailed overview of the research in the area of enhancing spatial skills within AR environments; the authors mainly identified the indispensable need for comprehensive future research in the area of descriptive geometry. Diao and Shih [89] conducted another systematic review of literature concerning AR in architectural and civil engineering education. Analysis was performed based on fundamental information, application domains, AR development tools, system types, teaching devices, teaching methods, learning strategies and research methods. The study focused on domain-specific studies of architects and civil engineers, and ignored cross-domain contributions. Additionally, the study lacks in the evaluation of the contribution of AR to spatial skills training. While many prior reviews of AR applications have focused on the field of education [90], there is an absence of systematic literature review on the use of AR in training spatial skills. According to [91], very little or no systematic research work has been conducted studying the effects of VR/AR systems in training spatial skills. Moreover, no commercial VR/AR applications have been developed for this training purpose either. Therefore, a review of research studies in spatial ability training using AR technology can suggest areas in which future research can be oriented.

22

2  Review of the Literature on AI-Enhanced Augmented Reality in Education

2.4 Learning Theories 2.4.1 The Bloom’s Taxonomy Bloom’s taxonomy of educational objectives has served as the foundation for the development of the majority of cognition and achievement assessments [92]. Since its publication in 1956, the taxonomy has thousands of citations. Numerous tests, created by teachers, have been based on the Bloom taxonomy, which has been widely utilized in teacher education to recommend learning and teaching tactics. The ranked classifications range from “knowledge”, at the most fundamental level, to “evaluation”, at the level requiring the most cognitive state, arranging different processes hierarchically. Each level is dependent on the student’s competence in the level or levels before it. Bloom’s taxonomy was developed to help teachers define learning objectives and serves as a framework for creating assessment criteria relevant to the varieties of cognitive domains, and mental and cognitive abilities being evaluated. Certain limitations of the first version of Bloom’s Taxonomy are acknowledged in a revised edition [93], which also adds a new dimension to knowledge types, and rearranges the hierarchy of the cognitive processes. The revised Bloom taxonomy actually defines the expected cognitive process and the type of knowledge underlying a learning objective. Both taxonomies are represented as hierarchical frameworks, in which, the upper more complicated level, encases all lower levels. Despite the Bloom cognitive taxonomy’s popularity, there is no evidence to support its utility in planning curriculum, instruction, or assessment. The primary issue, with using Bloom’s Taxonomy to control the development of questions, is the misconception that its categories constitute a range of classifications that are arranged hierarchically [94]. Some of the issues arising from the Bloom’s taxonomies are the fact that the Bloom taxonomy assumes that there must be a connection between the questions raised, and the expected responses. The assumption used while applying Bloom’s taxonomy is that the question will elicit a specific type of Bloom response. However, there is no definite connection, because a student may answer the supposedly simpler question with a very profound explanation, or in a similar vein, a student might give a perfunctory response.

2.4.2 The SOLO Taxonomy The Structure of Observed Learning Outcomes (SOLO) cognitive processing taxonomy was developed by [95] and describes the levels of students’ understanding in ascending order of complexity. In particular, the model consists of five levels of understanding, namely pre-­ structural (L1), unistructural (L2), multi-structural (L3), relational (L4), and

2.4  Learning Theories

23

extended abstract (L5). This model can be used to deliver the most appropriate learning activity to students with the intention of improving their learning outcomes [96]. Therefore, the PARSAT uses the SOLO model to offer each student the learning activities that suit them best. Table  2.3 illustrates the learning goals and the corresponding activities per SOLO level. The SOLO taxonomy is divided into two main categories, each of which has two progressively more complicated stages. Unistructural and multi-structural are part of the surface level, whereas relational and extended abstract are included within the depth level. Responses, that are unistructural, need the knowledge or application of a single fact, or piece of information, that was directly derived from the situation. Multiple pieces of information or facts must be known or employed, in order to complete multi-structural responses or queries, as well as two or more distinct steps without any concept integration. Fundamentally, this is an unstructured, unsorted list. As opposed to surface-level inquiries, deep processes represent a shift in the nature of thinking that is cognitively more difficult. Relational answers or inquiries must integrate at least two distinct pieces of knowledge that, when combined, provide the solution to the problem. In other words, relational inquiries demand that students put an organizing structure on the content they are given. Extended abstract, the highest level of the SOLO taxonomy, requires for the answer to go beyond the information, knowledge, or concepts provided and deduce a more universal rule or proof that actually applies in all cases. The learner is driven to go beyond the information provided and to draw on relevant, prior knowledge concepts or information in order to form a response, a prediction, or a hypothesis that applies the information provided to a wider range of scenarios.

2.4.3 Comparison of the Learning Theories Although Bloom’s taxonomy is widely adopted for educational outcomes, the SOLO taxonomy provides useful tools for assessing the levels attained in students’ effort. The SOLO taxonomy allows teachers to evaluate students’ tasks, based on their quality, rather than what questions they answered correctly and incorrectly. The taxonomy allows teachers to evaluate students’ tasks based on their quality rather than what they did correctly and incorrectly. There are five stages with each level increasing in complexity and skill. Students completely miss the point at the pre-structural level, which is the lowest. Students can only describe one feature of a topic at the following level (unistructural), and they can only identify multiple unrelated aspects at the third level (multi-structural). Students can combine several ideas into a single structure at the relational level, which is the fourth level. Finally, students are able to generalize and formulate hypotheses at the greatest level (extended abstract).

24

2  Review of the Literature on AI-Enhanced Augmented Reality in Education

Table 2.3  Learning goals and activities per SOLO level [97] SOLO level Pre-­ structural (L0)

Uni-­ structural (L1)

Multi-­ structural (L2)

Learning goal Learning activities Students get information Define concepts on the subject List items Match information Name facts

Description of the activities Introduction to Technical Drawing: A history and current importance of drawing are presented. Students are asked TO illustrate the significance of drawing by presenting applications and reports of both good and negative uses of the skill Setting up a model space in Identify content to be Students define, CAD software by defining recognize, name, sketch, memorized, show limits, grid, snap, layers, and reproduce, recite, follow examples object snap Provide disciplinary simple instructions, Video tutorials on standard context calculate, reproduce, views, views alignment, Mnemonics in groups arrange, find Repetition of procedures completion of activity sheet, and setting up the model Games space Repetitive testing and Border creation with a matching Peer testing (one student completed title block to be used for all future drawings, asks, one answers) and drawing templates with all the settings necessary saved within it Glossaries of key terms Orthographic drawing Students describe, list, creation. with definitions, classify, structure, classifications, examples Lines, layers enumerate, conduct, Isometric object drawing to build disciplinary complete, illustrate, Video tutorials on linetype, vocabulary solve lineweight and isometric Simple laboratory drawing creation of objects exercises in the activity Define terms, compare to glossary Games modelled on Trivial Pursuit, Family Feud (continued)

2.5  Literature Review

25

Table 2.3 (continued) SOLO level Relational (L3)

Extended abstract (L4)

Learning goal Students relate, analyze, compare, integrate, plan, construct, implement, summarize

Students generalize, hypothesize, theorize, predict, judge, evaluate, assess, predict, reason, criticize

Learning activities Case studies, simulations, and complex lab exercises Concept maps Research projects and experiential learning cycles Application of theoretical models Reflective journals Student seminars and debates Syndicate groups (each group is part of whole) Problem-Based Learning and Inquiry Learning Self-directed projects involving research, design, application, argumentation, evaluation Case studies involving extensive analysis, debate, reflection, argumentation, evaluation, forecasting Development of a theory or model Experiential learning cycles Problem Based Learning and Inquiry learning

Description of the activities Scaling the border and title block to fit the orthographic drawing Dimensioning an orthographic drawing Video tutorials on basic dimensioning rules and parts of dimensions Filling in a title block, including Name, Date, Title, Drawing No., and the correct scale Snapping and Text commands

Printing the drawing on 8.5″ × 11″ paper (letter size) in landscape orientation Video tutorial on cutting plane, half and full sections Printer/plotter settings Export/plot an object that has been drawn in CAD so it can be exported or printed to a variety of other applications CAD software to create objects that are more precise and sometimes easier to draw in CAD than in other software

2.5 Literature Review This literature review follows the guideline for systematic reviews, appropriate for software engineering researchers, proposed by [98]. There is a variety of review designs and existing guidelines intended to aid medical researchers. The purpose of a review of healthcare literature is to summarize the knowledge around a specific topic and support health professionals make decisions about a care issue. Kitchenham’s guideline is based on a review of three existing medical guidelines and adapts them to the need of software engineering researchers. In particular, software engineering research has relatively little empirical research compared with the large quantities of research available on medical issues, and therefore, research methods used by software engineers are not as rigorous as those used by medical

26

2  Review of the Literature on AI-Enhanced Augmented Reality in Education

researchers. The guideline has been adapted to reflect the specific problems of software engineering research [99]. The guideline covers three phases of a systematic review: planning the review, conducting the review and reporting the review. It is at a relatively high level [98]. At the first phase of the planning of the review, a review protocol is developed. The protocol serves as a roadmap for the review and specifies the objectives, methods, and outcomes of primary interest of the systematic review. The purpose of having a protocol is to promote transparency of the methods. A protocol defines the search terms, the inclusion and exclusion criteria and the data that will be analyzed. At the second phase, the review is conducted. Once the protocol has been developed, the review starts. This procedure involves the finding of studies relating to the research questions, and then, the study of their actual relevance. Subsequently, inclusion and exclusion criteria are employed to refine the outcomes. Data collection forms are devised to gather all the requisite information from each study, and data synthesis is utilized to compile and condense the findings of these studies. The third phase of the review is very important as it communicates the results of the systematic literature review. Figure 2.2 illustrates the three primary stages of the literature review. This section focuses on the examination of the initial two phases, while the subsequent section will cover the third phase, which involves reporting the review, including result analysis, discussion of findings, identification of trends, and drawing conclusions.

2.5.1 Planning the Review (Review Protocol) For this review, it was conducted a thorough search of scientific articles, mainly in the Scopus and Google Scholar databases. The results were filtered through keywords in the paper title, abstract and keywords list. The search results discovered 154 papers, based on certain keywords “augmented reality” and (“spatial ability” or “spatial skills”) and (“training” or “teaching” or “education”). Afterwards, the inclusion and exclusion criteria were defined. Considering the research questions, general criteria defining the time frame for the studies and the type of relevant studies were devised. This research aims to review the most recent literature regarding using AR in spatial abilities training. The inclusion criteria were the time span of the last 12 years from 2010 to 2022, and the document language was determined as “English”. Journal and/or conference literature review papers were included, while Master’s or Ph.D. theses were excluded from the systematic review. Papers that were not directly related to AR and spatial abilities’ training were also determined as exclusion criteria. In the final step of the first phase, a group of analysis categories was defined with their corresponding sub-categories, according to each research question. These categories constitute the evaluation criteria used in the data extraction forms for each selected study. They were also helpful in grouping studies according to their shared

2.5  Literature Review

27

Fig. 2.2  Systematic literature review phases

characteristics [100]. The list of evaluation criteria for the data extraction, classified by the research questions, is showed in Table 2.4.

2.5.2 Conducting the Review All the 154 papers from the first phase were thoroughly examined to determine their relevance to the study. As they did not align with the study’s objectives, 114 papers were deemed unsuitable and consequently excluded based on the exclusion criteria. This led to the inclusion of a total of 40 papers, which comprised the final list for

28

2  Review of the Literature on AI-Enhanced Augmented Reality in Education

Table 2.4  Evaluation criteria

Research question RQ 1 RQ 2 RQ 3

RQ 4 RQ 5

Evaluation criteria AR advantages AR limitations Learning tool AR approach AR type Software Type of adaptation process Spatial abilities Aspects Test used Research sample Research method Data collection method

analysis. Among these, 29 were journal articles, 8 were conference papers, and 3 were either books or book chapters. Every one of the studies underwent assessment and analysis in accordance with the research inquiries. An article review form was developed, as a data collection tool, to examine the articles to be reviewed. The data collection tool, developed by [101], was revised according to the research questions in the present research, and was implemented as a table matrix in Microsoft Excel. It is composed of six sections, one section for the screening of the evaluation papers, including article’s general information, and five more sections, one for each research question. Each one of the 40 studies was thoroughly reviewed in order to determine the advantages and the limitations of the first and second research questions (RQ1 & RQ2). Advantages were arranged into three categories: learner outcomes, pedagogical contributions and technical perspectives. In the case that an article included more than one advantage and/or limitation, it was separately recorded.

2.5.3 Screening of the Evaluation Papers The first section of the data extraction form addresses the publication year, the target group, the level of education and the country. The year is the date of the publication of the journal or the conference and has a value from 2010 to 2022. The level of education was divided into seven sub-categories of participants: preschool students, primary education students, secondary education students, technical education students, higher education students, teachers (of any level of education) and not specified (the learner type was not clearly specified). The 40 publications on AR in spatial abilities training were analyzed to determine the position of the subject in academia. Figure 2.3 shows the number of studies according to their publication year. During the initial 7 years within the timeframe

29

2.5  Literature Review 8

8 7 7

6

Number of studies

6

5

4 3

3

3 2

2

2

2

2

2 1

1

1

1 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022

Years Fig. 2.3  The number of studies published per year

(2010–2016), there were relatively few studies conducted, typically numbering one or two per year. However, in 2017, there was a notable increase in interest, with the number of publications rising from one to seven compared to the preceding year. From 2017 onwards, there has been a consistent trend of at least six papers being published each year. During the years 2017–2019, the average of publications per year was seven. The past 2½ years (2020–mid 2022), there is a steady research interest in the field of spatial ability training through AR.  This suggests that the research interest in AR and its implementation in spatial abilities training has been increasing since 2017 and a similar level of interest will continue in 2022 and after. This finding is significant since it presents the value of this study to guide future studies in the field. The countries that explored the application of AR in educational research are recorded and the major contributing countries in the studies are Malaysia, the United States of America (USA), Spain, Turkey and Taiwan (Fig. 2.4). Three studies [102– 104] of the leading contributing country, Malaysia, are conducted due to scholarships. More specifically, Universiti Teknologi Malaysia, Universiti Malaysia Pahang, Universiti Tun Hussein Onn and the Ministry of Higher Education (MOHE) Malaysia provided the funds to support the researchers. Two studies [24, 105] of the second contributing country, the USA, were also supported by funds such as the

30

2  Review of the Literature on AI-Enhanced Augmented Reality in Education

1 1 11 1

0

2

4

8

6

8

7

11 11

5

4

10 12 14 16 18 20 22 24 26

7

28 30 32 34 36 38 40

Number of studies Australia

Hungary

India

Indonesia

Latvia

Malaysia

Mexico

Netherlands

Peru

Portugal

Spain

Taiwan

Turkey

USA

Fig. 2.4  The number of studies published per country

National Science Foundation (NSF). Spanish researchers [106] acknowledge the support from the Spanish Ministry of Economy and Competitiveness. Finally, two of the three studies [107, 108] from Taiwan, were funded to deploy their studies from the University department and the Ministry of Science and Technology, respectively. Conclusively, the leading contributing countries USA, Taiwan, Spain and Malaysia fund their researchers through grants. Regarding the “Level of education”, this category refers to the level of education of the participants in the experiments that the study of AR in spatial abilities training was carried out. The majority of the studies (63%) involved University students (Fig. 2.5). Preschool, Technical, Primary and Secondary education have very low percentages in the literature, 2% the first two, 5%, and 10% for Primary and Secondary education respectively, showing a general tendency in the field of spatial skills’ training to students of higher education (Fig. 2.5).

2.5  Literature Review

31 Preschool 2%

not specified 18%

Primary education 5%

Secondary education 10% Technical education 2%

Higher education 63%

Fig. 2.5  Level of education applying AR on spatial skills’ training

2.5.4 Advantages of AR in Spatial Ability Training (RQ1) The first evaluation criterion, analyzed in this systematic literature review, deals with the reported advantages of AR in spatial abilities training. After extracting the data from the published research, the identified advantages were arranged into three categories, namely learner outcomes, pedagogical affordances, and technical perspectives. The results of the reported advantages are presented, in a descriptive manner, as frequencies in Table 2.5. 2.5.4.1 Learner Outcomes The advantages of AR in spatial abilities training that are related to either spatial visualization or spatial perception or mental rotation are grouped under the sub-­ section of the learning outcomes of enhancing spatial ability. Most of the studies (29 out of 40) reported that AR technology in education leads to this learning outcome. Numerous studies (12 out of 40) have indicated that AR increases students’ understanding. For instance, AREDApps [103] was developed as an alternative to help increase students’ understanding, enhance visualization skills and attract students’

32

2  Review of the Literature on AI-Enhanced Augmented Reality in Education

Table 2.5  The advantages of AR in spatial abilities training Categories Learning outcomes

Pedagogical affordances

Technical perspectives

Sub-categories Enhances spatial ability Increases students’ understanding Enhances students’ motivation Improves students’ academic performance Visualizes abstract concepts Positive attitude towards the course Reduces cognitive load Students observe models from different perspectives Enhances satisfaction Assists students in solving given problems Students memorize better the learning material Attracts students’ interest Students manipulate virtual objects in real environment Enhances enjoyability Autonomous training Increases engagement in teaching and learning process Interaction with the immersive environment Promotes self-directed learning Personalization of learning Allows students to be an active learner Develops collaborative work in students Learning by doing Student centered learning Easy to use Cost effective

Frequency 29 12 6 6 6 5 4 3 2 1 1

Sample research [6, 109–111] [112] [113] [114] [104] [115, 116] [107, 117] [118] [119, 120] [121] [91, 122]

13 8

[113] [123]

6 4 4

[124] [106] [125]

4 4 2 1 1 1 1 11 8

[31] [91] [105] [123] [113] [91] [6] [126] [116]

interests in engineering drawing. Moreover, AR provides the ability to help students develop a deeper understanding [104]. The review findings also indicate that AR can enhance students’ motivation, improve students’ academic performance, enhance positive attitude, and visualize abstract concepts. Students feel more motivated in class when these tools are implemented in pedagogical activities in the classroom [113]. Students’ motivation to the instructional activities can be increased due to the enriched features of the learning environment [91]. Furthermore, the use of AR materials, integrated in the educational environment, improves the students’ academic achievement [114]. According to [116], the faculty members that participated in the validation study perceived a very positive and receptive attitude by the students. With the use of 3D imagery,

2.5  Literature Review

33

students can visualize the abstract concept [127], which cannot be easily seen in a real-life setting [104]. Some researchers reported specific AR-related learning outcomes such as the reduction of cognitive load, the enhancement of satisfaction, student observation from different perspectives, the provision of assistance to students for solving given problems and better memorization of learning material. As an example, [107] conducted a paired-sample t-test to assess mental effort and mental load scores independently. This analysis revealed a noteworthy distinction in terms of mental effort and mental load between the experimental and control groups. In another study by [120], students expressed a high level of satisfaction when Augmented Reality (AR) technology was employed. 2.5.4.2 Pedagogical Affordances According to the pedagogical affordances of AR, the most prominent contributions are the attraction of students’ interest, the manipulation of virtual objects in real environment, and the enhancement of enjoyability. Most of the students show a great interest, expressing awe and revelation in their faces while exploring with the new tools until they found new purposes [113, 128]. Lee et al. [108] concluded that the participants in the experimental group showed great interest in using their AR training system. AR provides a platform for students to manipulate a virtual object freely from various perspectives, as they can use their bare hands to make manipulation [104]. In addition to that, all of the students stated that the AR application made the classes more enjoyable [114]. AR technology is valuable, engaging, and useful in the engineering design graphics domain, particularly when visualizing challenging models. Excitement and engagement could be easily observed in the participants during the exercise, although this could be attributed to the fact that many had never used or experienced AR technology before [129]. On the other hand, the indications of self-directed and/or personalization of learning and autonomous training were reported in very few studies within the 40 reviewed research articles, and they all suggest that further study is warranted, regarding potential benefits for the development of the ability and the confidence in spatial skills. 2.5.4.3 Technical Perspectives The last part of Table 2.5 shows two advantages that could not be arranged to either the learner’s outcomes or the pedagogical affordances. In [126], the authors deployed an AR teaching and learning kit, named AREDKit, which is practical to be used in classrooms since it has a low production cost. Furthermore, it has easy to use functions which can be operated using smartphones. Chandrasekera and Yoon [130] intended to incorporate AR into the architectural curriculum, so they also chose to

34

2  Review of the Literature on AI-Enhanced Augmented Reality in Education

construct a cost-effective and easy-to-use system. The AR system, which was presented by [129], was low cost and easy to implement. In addition to the printed materials, just a computer with the proper software installed and a web camera were required.

2.5.5 Limitations of AR in Spatial Ability Training (RQ2) Though AR provides many advantages, researchers have reported some limitations imposed by this technology (Table 2.6). The most frequently reported limitation is that the content is still poor. Some improvement actions may be incorporated into the application, such as expanding the content by increasing the amount and types of exercises and including self-evaluation tests [106]. Some other studies had the limitation of the small sample size [117–119]. While the results of the studies are significant, a broader sampling size in order to include more students of varying background could better validate the research questions. Another limitation was the absence of a control group. Therefore, the improvement of the posttest score of students may be made not only due to the AR learning

Table 2.6  The limitations in AR in spatial abilities’ training Limitations Necessity for more content Small sample size Control group missing Help sections missing Country location specific results The participants had limited time for the training Data collection during very long periods of time Focuses only on beginners Gender related differences in performance could not be examined due to unequal gender ratio Limited financial resources limit a further development and wider application of this technology in the education process Needs a re-design in terms of pedagogical instructions Needs many environmental settings and pre-settings of the system No experimental testing Relatively new technology Requires long-term training and practice to master The app was under continued development during the course of this study The application needs supportive printed material The end product must achieve certain qualities

Frequency 7 3 2 2 2 2 1 1 1

Sample research [106] [121] [104] [121] [130] [107] [113] [107] [131]

1

[131]

1 1 1 1 1 1

[121] [107] [31] [118] [107] [124]

1 1

[132] [126]

2.5  Literature Review

35

experiment but also by other variables. It is recommended to have a control group in the future investigation [104]. Additional training modules or help sections could be added to the application to familiarize the learner with the necessary know-how of the application usage [121]. A couple of studies reported the limitation that the results are specific to the location of the study [130, 132], while two others stated that the participants had limited time for the training because many had routine school homework to do and meetings to participate in [108, 121, 122]. The rest of the limitations involve application-related and technical problems. Future technological developments are expected to fix most of the current limitations.

2.5.6 Exploration of the Incorporation of Adaptivity and Personalization in AR Applications (RQ3) Table 2.7 shows 13 AR teaching applications, which have been developed since 2010, all aiming to improve students’ spatial skills and provide better perception of three-dimensional geometry of an object. The results show that the developers from 2010 until 2012 were developing mainly desktop AR applications. For instance, an augmented book called “AR-Dehaes”, designed to provide 3D virtual models helps students to perform visualization tasks promoting the development of their spatial ability during a short remedial course [116]. The “AR models” are the virtual models which can superimpose 3D graphics of typical geometries on real-time video and dynamically vary view perspective in real-time to be seen as real objects. The AR model was developed using the ARToolKitPlus library, including all the geometrical features generally taught in engineering graphics courses or technical drawing courses [125]. Both of these applications require programming knowledge in C++ language, using Brainstorm eStudio and ARToolKitPlus respectively in their development process, which can be a significant drawback. “AR enhanced exercise book” is another desktop approach presented in 2012 which aims to improve the spatial ability of freshman engineering students [134]. Since 2014, the developers started to use mobile AR, mainly due to the proliferation of mobile technology (such as mobile phones, tablets, wireless network etc.). The freedom degree of the user increases when using mobile AR applications, compared to desktop AR applications. Figueiredo et  al. [31] presented the creation of a low-cost prototype, namely “EducHolo”, which enabled the visualization and interaction with holograms. Their aim was to provide a better perception of the model 3D shape improving the ability to make the 2D orthographic views and perspectives that the first year of mechanical engineer studied. They used Augment software which is free without requiring programming. The “DiedricAR” application allowed students to learn autonomously by using their own mobile devices that work as AR displays over training material

36

2  Review of the Literature on AI-Enhanced Augmented Reality in Education

Table 2.7  AR applications training spatial ability

Article Learning tool [112] AR-Dehaes augmented book [133] AR models [132]

AR approach Desktop Desktop

AR type Marker-­ based Marker-­ based Marker-­ based Marker-­ based Marker-­ based Not specified

Desktop

[31]

AR enhanced exercise book EducHolo

[106]

DiedricAR

Mobile

[122]

AREDKit

Not specified

[117]

GeoSolvAR

Mobile

[104] [119]

ARScience Magic Desktop/ Book Laptop MAR Mobile

[103]

AREDApps

Mobile

Marker-­ based

[113]

Marker-­ based

[131]

Augmented Mobile Reality Chemistry (ARC) Geogebra AR Mobile

[134]

Spatial-AR

Mobile

Mobile/ smart glasses

Marker-­ based Marker-­ based Marker-­ based

Marker-­ based Marker-­ based

Software Brainstorm eStudio ARToolKitPlus not specified AutoCAD Augment Unity 3D Vuforia SDK AutoDesk 3DS Max Unity 3D Android Studio Unity 3D Vuforia SDK Not specified AutoDesk 3DS Max Unity 3D Android Studio Unity 3D Vuforia SDK Android Studio Unity 3D Vuforia SDK Android Studio Not specified AutoDesk 3DS Max Unity 3D Android Studio

Adaptivity? (Yes/No/ Content Some) 3D No models 3D No models 3D No models 3D No models 3D No models 3D No models

3D models 3D models 3D models

No

3D models

No

3D models

No

–

No

3D models

No

No No

[106]. Other applications developed during the years 2018–2019 are: i. “AREDKit” developing to allow manipulation of 3D virtual models with the purpose to help students to perform visualization tasks during the process of teaching and learning [126], ii. “GeoSolvAR” focusing on middle school students to improve their visualization skills [121], iii. “ARScience Magic Book” Learning System (AR-SMB) developing to facilitate students in learning science concept, hence, enhancing their spatial visualization ability [104], iv. “MAR” and “AREDApps” being the most recently known approaches in the teaching and learning of orthographic projection [102, 103].

2.5  Literature Review

37

Finally, during the years 2020–2022, three more applications were presented, namely “ARC”, “Geogebra AR”, and “Spatial-AR”, which train the spatial skills of secondary students in the fields of Chemistry and Mathematics [110, 117, 135]. Except for the “ARScience Magic Book” and the “Geogebra AR”, that the authors did not clearly explain the software they used, the rest applications were all developed using Unity 3D software to create the AR platform. This is an object-­ oriented framework with graphic interface that can build applications for many different platforms including iOS and Android. Unlike other frameworks, Unity3D allows an easy handling of virtual models. Additionally, some of the applications used Autodesk 3D Studio Max as 3D modeling software to create and render 3D models and Android Studio as the official integrated development environment (IDE) for Google’s Android operating system. Vuforia is also used as a good Software Development Kit (SDK) that provides functionalities for the development of AR applications on mobile, using images or objects for targets. For the development and deployment of the application to the mobile devices, Unity3D is used. Unity3D is a game engine that can be integrated with Vuforia allowing the development of AR applications. Among all these teaching applications, it can be noted that the trend in the usage of AR approach is 3D model content. The justification for choosing three-­ dimensional model as the augmented objects is made based on the major trends and the effectiveness of learning using three-dimensional models towards improving spatial visualization skills. Table 2.7 also illustrates that there is still no approach, desktop or mobile, that has included adaptive or personalized processes. A personalized AR-system could possibly provide agent-customized training for students’ learning performance enhancement[136–139]. An agent-based infrastructure could support the customization of the spatial abilities’ assignments, as triggered by the performance of the trainee. Another example of adaptive AR system could be the user modeling [140– 143], so that the contents and flow of the learning could be customized. Based on the reviewed literature, AR implementation has never been considered yet for this topic. There is a need for further research on the personalization aspect in a learning environment, which should meet the needs and interests of individual learners.

2.5.7 Aspects of Spatial Abilities Having Been Evaluated Using AR (RQ4) The fourth research question investigates the aspects of spatial abilities that have been evaluated using AR.  The assessment of spatial ability is critical as specific standardized tests offer a valuable piece of knowledge about specific spatial subcomponents. It is important for researchers or employers to know what aspect they want to evaluate, so that they select a test with strong reliability and validity evidence.

38

2  Review of the Literature on AI-Enhanced Augmented Reality in Education

Currently, studies on spatial abilities follow two lines of research concerning the definition of factors. The first one is the proposal of three factors: i) spatial perception; ii) spatial orientation or mental rotation, considered to be unique; and iii) spatial visualization. The second line is the proposal of two factors: i) spatial relation, or mental rotation; and ii) spatial [6]. Numerous assessments exist for evaluating spatial ability, reflecting the absence of a single, unified definition of this skill. Instead, spatial ability is typically described in relation to multiple sub-elements. Numerous research studies have explored the spatial skills of students and developed standardized tests to gauge their proficiency in various facets of spatial ability. As seen in Table  2.8, the Purdue Spatial Visualization Tests: Visualization of Rotations (PSVT:R) has been commonly used to predict students’ success in spatial skills’ training. PSVT was developed by [144] and consists of 36 questions. More specifically, the test consists of three 12-item subtests entitled Developments, Rotations, and Views, respectively. The PSVT:R is an extended version of the subtest, Rotations, to measure the 3-D mental rotation ability of individuals aged 13 or above, in 20 min. The PSVT:R has 30 items consisting of 13 symmetrical and 17 nonsymmetrical 3-D objects that are drawn in a 2-D isometric format. In each item, the respondents’ task is to mentally rotate an object in the same direction as indicated visually in the instructions, and then to select an answer from among five possible options [145]. In the second place, the most frequent tests are the DAT:SR and MRT tests. Differential Aptitude Tests: Space Relations (DAT:SR) measures learner’s ability to move from 2D to 3D world. It consists of 50 questions about paper folding. Mental Rotation Test (MRT) of [146] was developed to measure students’ improvement in spatial skills. MRT is one of the most frequently used tests that measure the spatial relations. Mental rotation is the perceptual process of visualizing an item at different angles in a three-dimensional world. The students compared objects depicted on a paper or monitor and find the identical ones. The original image and the identical ones are displayed at different rotations. There are 20 items in MRT, in each item, the left side consists of a target figure and the right side consists of four (or three in some revised versions) sample stimuli. The participant needs to choose the correct figure that represents the rotation of the target figure.

Table 2.8  Standardized tests to measure spatial skills Test Frequency Percentage Purdue Spatial Visualization Tests: Visualization of Rotations (PSVT:R) 12 33.33 Differential Aptitude Tests: Space Relations (DAT:SR) 9 25.00 Mental Rotation Test (MRT) 7 19.44 Mental Cutting Test (MCT) 3 8.33 Middle Grades Mathematics Project (MGMP) 1 2.78 Minnesota Paper Form Board Test (MPFBT) 1 2.78 Picture Rotation Test (PRT) 1 2.78 Purdue Spatial Visualization Tests: for Development (PSVT:D) 1 2.78 Spatial Perception Scale (SPS) 1 2.78 Spatial Orientation Test (SOT) 0 0.00

2.5  Literature Review

39

The Mental Cutting Test (MCT) measures the ability to visualize object cutting and was first developed for a university entrance examination in the USA, College Entrance Examination Board (CEEB) in 1939. The test consists of 25 items. For each problem on the exam, students are shown a criterion figure which is to be cut with an assumed plane. They must choose the correct resulting cross-section from among five alternatives. Middle Grades Mathematics Project (MGMP) [114] and Minnesota Paper Form Board Test (MPFBT) [147] have been used each one of them in just one research study. This low usability is also observed in Picture Rotation Test (PRT) and Spatial Perception Scale (SPS) respectively. The first one measures the rotation skills of pre-and-early primary school children (ages from 4 to 6) [148], while the second one measures the visualization and orientation skills of 6-year-old children [118], but this infrequency of use is mainly due to the preschool domain, which is not among the target groups of AR spatial skills training. The last one named Spatial Orientation Test (SOT) was not found in any of the selected studies. This is due to the fact that the majority of authors and researchers recognize two factors: mental rotation; and visualization. They do not really include the third category: spatial orientation as well as the specific instrument for measuring it.

2.5.8 Evaluation Methods Considered for AR Applications in Educational Scenarios (RQ5) The fifth research question examines the methodology in terms of AR applications. It is very important for the researchers to identify the correct strategy for their study so that the results provide valid information. It was found that the most preferred research sample size in educational AR in spatial ability training studies, is between 11 and 50 (45.0%) (Table 2.9). This sample size is followed by the sample size of 51–100 (25.0%), and the sum of both of them raises to 80.0%. The sample size of above 100 (10.0%) is coming third place, 5.0% of the studies were conducted with 1–10 participants, whereas 15.0% of the studies did not provide any sample size information. It was found that the most common research method in educational AR in spatial ability training studies is mixed (37.5%) and the least common is qualitative (10.0%) (Table 2.10). Other common research method is quantitative (32.5%), whereas 20.0% of the studies did not provide information regarding the applied research method.

Table 2.9  Distribution of research sample size

Sample size Frequency % Between 1–10 2 5.0 Between 11–50 18 45.0 Between 51–100 10 25.0 Above 100 4 10.0 Not specified 6 15.0

40

2  Review of the Literature on AI-Enhanced Augmented Reality in Education

Table 2.10 Research methods applied

Table 2.11 Data collection method

Research method Frequency % Quantitative 13 32.5 Qualitative 4 10.0 Mixed 15 37.5 Not specified 8 20.0

Data collection tool Case study Interview Post-test Pre-test Questionnaire Other Not specified

Frequency 2 4 22 21 17 4 7

% 2.6 5.2 28.6 27.3 22.1 5.2 9.1

The results show that the most common data collection tool is post-test (28.6%), and the least common employed data collection tool is the case study (2.6%) (Table 2.11). At the same category of the least common collection tools, there are the observation, the survey, the long-term test, and the instructor’s evaluation, which are all included in the category named “Other”. Other data collection tools are pre-­tests (27.3%), questionnaires (22.1%), and interviews (5.2%). It’s worth noting that among the 21 studies reviewed, a pre-test method was consistently employed alongside a post-test method. This approach allowed for the comparison of learning achievements. Drawing from the literature discussed in this chapter, a handful of Augmented Reality (AR) applications [116, 125, 134] were designed with the objective of assisting students in improving their spatial skills using a desktop-based approach. Conversely, certain other applications [31, 104, 106, 121, 149] opted for a mobile-­centered approach to achieve the same goal. All of the already available augmented reality applications were created without reference to any particular framework but rather based on the writers’ previous programming and/or technological knowledge. Furthermore, intelligent tutoring systems (ITS) offering more independence to students during the training sessions have been implemented in different educational fields and should be combined with AR, in order to provide next generation advanced learning systems. Intelligent augmented reality tutoring systems could provide a personalized interface which is currently absent.

2.6 Summary In this chapter, a comprehensive review regarding AR in spatial ability training has been conducted and the technologies, application areas, and future research directions have been identified. General reviews of AR applications focusing on education have been made; however, a systematic literature review is absent when it

References

41

comes to the use of AR in spatial ability training. The identified research gaps are: i) the absence of commercial AR applications developed for the training of spatial abilities; ii) the absence of adaptivity of the systems in the existing literature; and iii) the need for developing a novel framework to focus on the design elements of an AR application. Trends of AR are a) the development of mobile (smartphones and tablets) AR applications; b) the use of Unity3D as the development platform; c) the use of Vuforia as the AR software development kit; and d) 3ds Max as 3D modeling software. Spatial ability is very important in engineering education and AR technology can suggest areas in which to invest new research. The review offers new insight to researchers as it points toward unexplored regions of engineering education and urges educators to incorporate AR into their teaching methods. Based on the review of 40 studies, there is an increase in the research studies during the last few years. The primary education level of the target group is higher education, and more specifically first-year engineering students as the development of spatial skills is important for their future studies and career. AR offers unique advantages to the learners, such as the improvement of their spatial ability, the better understanding of the topic and gradually the improved academic performance and motivation. When it comes to pedagogical contributions, AR is shown to attract students’ interest, enhances enjoyability, and increases their engagement in the teaching and learning process. The development of AR applications is transitioning from desktop-based to mobile ones, especially with the global ease of use of mobile phones and tablets. The majority of AR applications, reviewed in this chapter, are marker-based and their content is 3D models, as 3D modeling is shown to be very effective when it comes to improving spatial visualization skills. However, none of them considered the inclusion of combined adaptivity or personalization processes, pointing out the research interest of the current book.

References 1. E. L. Thorndike, “On the Organization of Intellect.,” Psychol Rev, vol. 28, no. 2, pp. 141–151, 1921, https://doi.org/10.1037/h0070821. 2. C. Spearman, “‘General intelligence,’ objectively determined and measured.,” Am J Psychol, vol. 15, pp. 201–293, 1904, https://doi.org/10.2307/1412107. 3. A. A. H. E. Koussy, An Investigation Into the Factors in Tests Involving the Visual Perception of Space. in British Journal of Psychology. Monograph Supplements. University Press, 1935. [Online]. Available: https://books.google.gr/books?id=SZh0tAEACAAJ 4. L. L. Thurstone, “Primary mental abilities. Psychometric Monograph 1.” Chicago, IL: Univer. of Chicago Press, 1938. 5. L. Thurstone, “Some Primary Abilities in Visual Thinking,” Proc Am Philos Soc, vol. 94, no. 6, pp. 517–521, 1950. 6. C. Roca-González, J. Martin-Gutierrez, M. García-Dominguez, and M. del C. M. Carrodeguas, “Virtual technologies to develop visual-spatial ability in engineering students,” Eurasia Journal of Mathematics, Science and Technology Education, vol. 13, no. 2, pp.  441–468, 2017, https://doi.org/10.12973/eurasia.2017.00625a.

42

2  Review of the Literature on AI-Enhanced Augmented Reality in Education

7. H.  Gardner, Frames of mind  : the theory of multiple intelligences. New  York: Basic Books, 1983. 8. M. G. McGee, Human Spatial Abilities: Sources of Sex Differences. in Praeger special studies. Praeger, 1979. [Online]. Available: https://books.google.gr/books?id=2HgiAAAACAAJ 9. M. G. McGee, “Human spatial abilities: Psychometric studies and environmental, genetic, hormonal, and neurological influences.,” Psychological Bulletin, vol. 86. American Psychological Association, US, pp. 889–918, 1979. https://doi.org/10.1037/0033-­2909.86.5.889. 10. M.  C. Linn and A.  C. Petersen, “Emergence and Characterization of Sex Differences in Spatial Ability: A Meta-Analysis,” Child Dev, vol. 56, no. 6, pp.  1479–1498, Dec. 1985, https://doi.org/10.2307/1130467. 11. D.  F. Lohman, “Spatial abilities as traits, processes, and knowledge.,” in Advances in the psychology of human intelligence, Vol. 4., Hillsdale, NJ, US: Lawrence Erlbaum Associates, Inc, 1988, pp. 181–248. 12. J. W. Pellegrino and E. B. Hunt, “Cognitive models for understanding and assessing spatial abilities,” Intelligence: Reconceptualization and measurement, pp. 203–225, 1991. 13. J. B. Carroll, Human cognitive abilities: A survey of factor-analytic studies, no. 1. Cambridge University Press, 1993. 14. I.  Macfarlane. Smith, Spatial ability  : its educational and social significance. San Diego, Calif.: R.R. Knapp, 1964. 15. H. Ault and S. John, “Assessing and Enhancing Visualization Skills of Engineering Students in Africa: A Comparative Study,” Engineering Design Graphics Journal, vol. 74, pp. 12–20, Jun. 2010. 16. S. Hsi, M. Linn, and J. Bell, “The Role of Spatial Reasoning in Engineering and the Design of Spatial Instruction,” Journal of Engineering Education, vol. 86, Apr. 1997, https://doi. org/10.1002/j.2168-­9830.1997.tb00278.x. 17. C. Papakostas, C. Troussas, A. Krouska, and C. Sgouropoulou, “PARSAT: Fuzzy logic for adaptive spatial ability training in an augmented reality system.,” Computer Science and Information Systems. 18. C.  Papakostas, C.  Troussas, A.  Krouska, and C.  Sgouropoulou, “Personalization of the Learning Path within an Augmented Reality Spatial Ability Training Application Based on Fuzzy Weights,” Sensors, vol. 22, no. 18, 2022, https://doi.org/10.3390/s22187059. 19. B. W. Field, “A course in spatial visualization,” Journal for Geometry and Graphics, vol. 3, no. 2, pp. 201–209, 1999. 20. N. Martín-Dorta, J. L. Saorín, and M. Contero, “Development of a fast remedial course to improve the spatial abilities of engineering students,” Journal of Engineering Education, vol. 97, no. 4, pp. 505–513, 2008. 21. A. Šafhalter, S. Glodež, A. Šorgo, and M. Ploj Virtič, “Development of spatial thinking abilities in engineering 3D modeling course aimed at lower secondary students,” Int J Technol Des Educ, 2020, https://doi.org/10.1007/s10798-­020-­09597-­8. 22. R. H. McKim, Experiences in visual thinking. Monterey, Calif.: Brooks/Cole Pub. Co., 1980. 23. D.  F. Ali, M.  Omar, M.  Mokhtar, N.  Suhairom, A.  H. Abdullah, and N.  D. A.  Halim, “A review on augmented reality application in engineering drawing classrooms,” Man India, vol. 97, no. 19, pp.  195–204, 2017, [Online]. Available: https://www.researchgate.net/ publication/320878073 24. T. Serdar, E. S. S. Aziz, S. K. Esche, and C. Chassapis, “Integration of augmented reality into the CAD process,” ASEE Annual Conference and Exposition, Conference Proceedings, 2013. 25. K.  L. Norman, “Spatial Visualization—A Gateway to Computer-Based Technology:,” Journal of Special Education Technology, vol. 12, no. 3, pp.  195–206, 1994, https://doi. org/10.1177/016264349401200303. 26. S. Sorby, “Developing 3D spatial skills for engineering students,” Australasian Journal of Engineering Education, vol. 13, pp. 1–11, Jan. 2007, https://doi.org/10.1080/22054952.200 7.11463998.

References

43

27. C. Ferguson, “A Comparison of Instructional Methods for Improving the Spatial-Visualization Ability of Freshman Technology Seminar Students,” NA, vol. NA, no. NA. p. NA-NA, 2008. doi: NA. 28. E. E. Ghiselli, “THE VALIDITY OF APTITUDE TESTS IN PERSONNEL SELECTION,” Pers Psychol, vol. 26, no. 4, pp. 461–477, 1973, https://doi.org/10.1111/j.1744-­6570.1973. tb01150.x. 29. E. Wiebe, T. Branoff, and N. Hartman, Constraint Based, Three Dimensional Solid Modeling In An Introductory Engineering Graphics Course: Re Examining The Curriculum. 2001. https://doi.org/10.18260/1-­2%2D%2D9035. 30. A.  R. Arslan and S.  Dazkir, “Technical Drafting and Mental Visualization in Interior Architecture Education,” International Journal for the Scholarship of Teaching and Learning, vol. 11, no. 2, 2017, https://doi.org/10.20429/ijsotl.2017.110215. 31. M.  Figueiredo, P.  J. S.  Cardoso, J.  M. F.  Rodrigues, and R.  Alves, “Learning Technical Drawing with Augmented Reality and Holograms,” Recent Advances in Educational Technologies and Methodologies, pp. 1–20, 2014. 32. M. Garmendia Mujika, J. Guisasola, and E. Sierra, “First-year engineering students’ difficulties in visualization and drawing tasks,” European Journal of Engineering Education, vol. 32, pp. 315–323, Jun. 2007, https://doi.org/10.1080/03043790701276874. 33. C. Papakostas, C. Troussas, A. Krouska, and C. Sgouropoulou, “On the development of a personalized augmented reality spatial ability training mobile application,” in Frontiers in Artificial Intelligence and Applications, IOS Press, 2021, pp. V–VI. https://doi.org/10.3233/ FAIA210078. 34. C. Papakostas, C. Troussas, A. Krouska, and C. Sgouropoulou, “Modeling the Knowledge of Users in an Augmented Reality-Based Learning Environment Using Fuzzy Logic,” in Lecture Notes in Networks and Systems, A. Krouska, C. Troussas, and J. Caro, Eds., Cham: Springer International Publishing, 2023, pp. 113–123. https://doi.org/10.1007/978-­3-­031-­17601-­2_12. 35. T.  Kösa and F.  Karakuş, “The effects of computer-aided design software on engineering students’ spatial visualisation skills,” European Journal of Engineering Education, vol. 43, pp. 296–308, 2018. 36. S.  Strong, R.  Smith, and G.  Communications, “Spatial Visualization: Fundamentals and Trends in Engineering Graphics,” Journal of Industrial Technology, vol. 18, Jan. 2002. 37. M. Contero, F. Naya, P. Company, J. L. Saorin, and J. Conesa, “Improving visualization skills in engineering education,” IEEE Comput Graph Appl, vol. 25, no. 5, pp. 24–31, 2005, https:// doi.org/10.1109/MCG.2005.107. 38. S.  A. ; L.  Burnett David M., “Effects of academic instruction on spatial visualization,” Intelligence, vol. 4, no. 3, pp. 233–242, 1980, https://doi.org/10.1016/0160-­2896(80)90021-­5. 39. T. R. Lord, “Enhancing the visuo-spatial aptitude of students,” J Res Sci Teach, vol. 22, no. 5, pp. 395–405, 1985, https://doi.org/10.1002/tea.3660220503. 40. J. Torner, F. Alpiste, and M. Brigos, “Spatial Ability in Computer-Aided Design Courses,” Comput Aided Des Appl, vol. 12, no. 1, pp. 36–44, Jan. 2015, https://doi.org/10.1080/1686436 0.2014.949572. 41. C.  Leopold, R.  Górska, and S.  Sorby, “International experiences in developing visualization abilities of engineering students,” Journal for Geometry and Graphics Volume, vol. 5, pp. 81–91, Jan. 2001. 42. S.  A. Sorby, “Improving the Spatial Skills of Engineering Students: Impact on Graphics Performance and Retention.,” Engineering Design Graphics Journal, vol. 65, no. 3, pp. 31–36, 2001. 43. M. Alias, T. R. Black, and D. Gray, “The relationship between spatial visualization ability and problem solving in structural design,” World Transaction on Engineering and Technology Education, vol. 2, pp. 273–276, Jan. 2003. 44. Z.  Jeli, B.  Popokonstantinovic, and M.  Stojicevic, “Usage of 3D Computer Modelling in Learning Engineering Graphics,” D.  Cvetkovic, Ed., Rijeka: IntechOpen, 2016, p. Ch. 4. https://doi.org/10.5772/65217.

44

2  Review of the Literature on AI-Enhanced Augmented Reality in Education

45. A. Çöltekin et al., “Extended Reality in Spatial Sciences: A Review of Research Challenges and Future Directions,” ISPRS International Journal of Geo-Information, vol. 9, no. 7. 2020. https://doi.org/10.3390/ijgi9070439. 46. R. Azuma, Y. Baillot, R. Behringer, S. Feiner, S. Julier, and B. MacIntyre, “Recent advances in augmented reality,” IEEE Comput Graph Appl, vol. 21, no. 6, pp. 34–47, 2001, https://doi. org/10.1109/38.963459. 47. E. Zaretsky and V. Bar, “Intelligent virtual reality and its impact on spatial skills and academic achievements,” in The 10th International Conference on Information Systems Analysis and Synthesis: ISAS 2004 and International Conference on Cybernetics and Information Technologies, Systems and Applications: CITSA, 2004, pp. 107–113. 48. P.  Strousopoulos, C.  Troussas, C.  Papakostas, A.  Krouska, and C.  Sgouropoulou, “Revolutionizing Agricultural Education with Virtual Reality and Gamification: A Novel Approach for Enhancing Knowledge Transfer and Skill Acquisition,” in Novel & Intelligent Digital Systems: Proceedings of the 3rd International Conference (NiDS 2023), K. Kabassi, P. Mylonas, and J. Caro, Eds., Cham: Springer Nature Switzerland, 2023, pp. 67–80. 49. A. Marougkas, C. Troussas, A. Krouska, and C. Sgouropoulou, “Virtual Reality in Education: A Review of Learning Theories, Approaches and Methodologies for the Last Decade,” Electronics (Basel), vol. 12, no. 13, 2023, https://doi.org/10.3390/electronics12132832. 50. A.  Marougkas, C.  Troussas, A.  Krouska, and C.  Sgouropoulou, “A Framework for Personalized Fully Immersive Virtual Reality Learning Environments with Gamified Design in Education,” 2021. https://doi.org/10.3233/FAIA210080. 51. S.  Rokhsaritalemi, A.  Sadeghi-Niaraki, and S.-M.  Choi, “A Review on Mixed Reality: Current Trends, Challenges and Prospects,” Applied Sciences, vol. 10, no. 2. 2020. https:// doi.org/10.3390/app10020636. 52. G.  White, C.  Cabrera, A.  Palade, and S.  Clarke, “Augmented reality in IoT,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11434 LNCS, pp.  149–160, 2019, https://doi. org/10.1007/978-­3-­030-­17642-­6_13. 53. C.  Papakostas, C.  Troussas, P.  Douros, M.  Poli, and C.  Sgouropoulou, “CoMoPAR: A Comprehensive Conceptual Model for Designing Personalized Augmented Reality Systems in Education,” in Novel & Intelligent Digital Systems: Proceedings of the 3rd International Conference (NiDS 2023), K. Kabassi, P. Mylonas, and J. Caro, Eds., Cham: Springer Nature Switzerland, 2023, pp. 67–79. 54. M. Akçayır and G. Akçayır, “Advantages and challenges associated with augmented reality for education: A systematic review of the literature,” Educ Res Rev, vol. 20, pp. 1–11, Feb. 2017, https://doi.org/10.1016/j.edurev.2016.11.002. 55. M.  Karakus, A.  Ersozlu, and A.  Clark, “Augmented Reality Research in Education: A Bibliometric Study,” Eurasia Journal of Mathematics, Science and Technology Education, vol. 15, no. 10, 2019, https://doi.org/10.29333/ejmste/103904. 56. R. M. Yilmaz, “Augmented Reality Trends in Education between 2016 and 2017 Years,” State of the Art Virtual Reality and Augmented Reality Knowhow, 2018, https://doi.org/10.5772/ intechopen.74943. 57. C. Papakostas, C. Troussas, A. Krouska, and C. Sgouropoulou, “Exploring Users’ Behavioral Intention to adopt Mobile Augmented Reality in education through an extended Technology Acceptance Model,” Int J Hum Comput Interact, vol. 39, no. 6, pp. 1294–1302, 2023, https:// doi.org/10.1080/10447318.2022.2062551. 58. C. Papakostas, C. Troussas, A. Krouska, and C. Sgouropoulou, “Exploration of Augmented Reality in Spatial Abilities Training: A Systematic Literature Review for the Last Decade,” Informatics in Education, vol. 20, no. 1, pp.  107–130, 2021, https://doi.org/10.15388/ infedu.2021.06. 59. A. Marougkas, C. Troussas, A. Krouska, and C. Sgouropoulou, “Virtual Reality in Education: Reviewing Different Technological Approaches and Their Implementations,” in Novel & Intelligent Digital Systems: Proceedings of the 2nd International Conference (NiDS 2022),

References

45

A. Krouska, C. Troussas, and J. Caro, Eds., Cham: Springer International Publishing, 2023, pp. 77–83. 60. M. Iakovidis, C. Papakostas, C. Troussas, and C. Sgouropoulou, “Empowering Responsible Digital Citizenship Through an Augmented Reality Educational Game,” in Novel & Intelligent Digital Systems: Proceedings of the 3rd International Conference (NiDS 2023), K. Kabassi, P. Mylonas, and J. Caro, Eds., Cham: Springer Nature Switzerland, 2023, pp. 31–39. 61. C. Papakostas, C. Troussas, A. Krouska, and C. Sgouropoulou, “Measuring User Experience, Usability and Interactivity of a Personalized Mobile Augmented Reality Training System,” Sensors, vol. 21, no. 11, p. 3888, Jun. 2021, https://doi.org/10.3390/s21113888. 62. N.  Pellas and I.  Kazanidis, “Developing and assessing augmented reality applications for mathematics with trainee instructional media designers: An exploratory study on user experience,” Journal of Universal Computer Science, vol. 25, no. 5, pp. 489–514, 2019. 63. A.  Kapetanaki, A.  Krouska, C.  Troussas, and C.  Sgouropoulou, “A Novel Framework Incorporating Augmented Reality and Pedagogy for Improving Reading Comprehension in Special Education,” in Novelties in Intelligent Digital Systems, IOS Press, 2021, pp. 105–110. 64. A.  Kapetanaki, A.  Krouska, C.  Troussas, and C.  Sgouropoulou, “Exploiting Augmented Reality Technology in Special Education: A Systematic Review,” Computers, vol. 11, no. 10, 2022, https://doi.org/10.3390/computers11100143. 65. P. Kourouthanassis, C. Boletsis, C. Bardaki, and D. Chasanidou, “Tourists responses to mobile augmented reality travel guides: The role of emotions on adoption behavior,” Pervasive Mob Comput, vol. 18, pp. 71–87, 2015, https://doi.org/10.1016/j.pmcj.2014.08.009. 66. T. Bekus, “THE APPLICATION OF AUGMENTED REALITY IN MARKETING,” Zeszyty Naukowe Wyższej Szkoły Humanitas Zarządzanie, vol. 19, pp. 279–296, Oct. 2018, https:// doi.org/10.5604/01.3001.0013.0068. 67. R. Monterubbianesi et al., “Augmented, Virtual and Mixed Reality in Dentistry: A Narrative Review on the Existing Platforms and Future Challenges,” Applied Sciences, vol. 12, no. 2, 2022, https://doi.org/10.3390/app12020877. 68. C. Rodríguez-Abad, J.-C. Fernández-de-la-Iglesia, A.-E. Martínez-Santos, and R. Rodríguez-­ González, “A Systematic Review of Augmented Reality in Health Sciences: A Guide to Decision-Making in Higher Education,” Int J Environ Res Public Health, vol. 18, p. 4262, Apr. 2021, https://doi.org/10.3390/ijerph18084262. 69. J. Buchner, K. Buntins, and M. Kerres, “The impact of augmented reality on cognitive load and performance: A systematic review,” J Comput Assist Learn, vol. 38, no. 1, pp. 285–303, Feb. 2022, https://doi.org/10.1111/jcal.12617. 70. G. Makransky and G. B. Petersen, “The Cognitive Affective Model of Immersive Learning (CAMIL): a Theoretical Research-Based Model of Learning in Immersive Virtual Reality,” Educ Psychol Rev, vol. 33, no. 3, pp.  937–958, 2021, https://doi.org/10.1007/ s10648-­020-­09586-­2. 71. J. Yu, A. R. Denham, and E. Searight, “A systematic review of augmented reality game-based Learning in STEM education,” Educational technology research and development, 2022, https://doi.org/10.1007/s11423-­022-­10122-­y. 72. N.  Khan et  al., “An Adaptive Game-Based Learning Strategy for Children Road Safety Education and Practice in Virtual Space,” Sensors, vol. 21, no. 11, 2021, https://doi. org/10.3390/s21113661. 73. D.-I. Han, M. C. tom Dieck, and T. Jung, “User experience model for augmented reality applications in urban heritage tourism,” Journal of Heritage Tourism, vol. 13, no. 1, pp. 46–61, Jan. 2018, https://doi.org/10.1080/1743873X.2016.1251931. 74. H. Kim, T. Matuszka, J.-I. Kim, J. Kim, and W. Woo, “Ontology-based mobile augmented reality in cultural heritage sites: information modeling and user study,” Multimed Tools Appl, vol. 76, no. 24, pp. 26001–26029, 2017, https://doi.org/10.1007/s11042-­017-­4868-­6. 75. P. Strousopoulos, C. Papakostas, C. Troussas, A. Krouska, P. Mylonas, and C. Sgouropoulou, “SculptMate: Personalizing Cultural Heritage Experience Using Fuzzy Weights,” in Adjunct Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization,

46

2  Review of the Literature on AI-Enhanced Augmented Reality in Education

in UMAP ’23 Adjunct. New York, NY, USA: Association for Computing Machinery, 2023, pp. 397–407. https://doi.org/10.1145/3563359.3596667. 76. C. Troussas, C. Papakostas, A. Krouska, P. Mylonas, and C. Sgouropoulou, “Personalized Feedback Enhanced by Natural Language Processing in Intelligent Tutoring Systems,” in Augmented Intelligence and Intelligent Tutoring Systems, C.  Frasson, P.  Mylonas, and C.  Troussas, Eds., Cham: Springer Nature Switzerland, 2023, pp.  667–677. https://doi. org/10.1007/978-­3-­031-­32883-­1_58. 77. S.  S. Alam, S.  Susmit, C.-Y.  Lin, M.  Masukujjaman, and Y.-H.  Ho, “Factors Affecting Augmented Reality Adoption in the Retail Industry,” Journal of Open Innovation: Technology, Market, and Complexity, vol. 7, no. 2, 2021, https://doi.org/10.3390/joitmc7020142. 78. F. de Pace, F.  Manuri, and A.  Sanna, “Augmented Reality in Industry 4.0,” American Journal of Computer Science and Information Technology, vol. 06, Jan. 2018, https://doi. org/10.21767/2349-­3917.100017. 79. C.  Papakostas, C.  Troussas, A.  Krouska, and C.  Sgouropoulou, “User acceptance of augmented reality welding simulator in engineering training,” Educ Inf Technol (Dordr), vol. 27, no. 1, pp. 791–817, Jan. 2022, https://doi.org/10.1007/s10639-­020-­10418-­7. 80. K. Obara, “AUGMENTED REALITY AND MARKETING,” Mar. 2021. 81. K. Mitrovic, N. Novakovic, J. Spajić, and I. Cosic, “Augmented Reality in Marketing – State of Art,” 2021, pp. 566–575. https://doi.org/10.2507/32nd.daaam.proceedings.081. 82. Z. Jian, “Teaching of Engineering Drawing in the 21st century,” in 2011 Second International Conference on Mechanic Automation and Control Engineering, 2011, pp. 1713–1715. https:// doi.org/10.1109/MACE.2011.5987287. 83. Z. Kanetaki, C. Stergiou, C. Troussas, and C. Sgouropoulou, “Developing Novel Learning Spaces Through Social Media Channels for Sustainable CAD Engineering Education,” in Novel & Intelligent Digital Systems: Proceedings of the 2nd International Conference (NiDS 2022), A. Krouska, C. Troussas, and J. Caro, Eds., Cham: Springer International Publishing, 2023, pp. 359–371. 84. A. Sideris, C. Troussas, A. Krouska, and C. Sgouropoulou, “A 2D Platform Game Offering Personalized Guidance to Players to Promote Environmental Awareness,” in Novel & Intelligent Digital Systems: Proceedings of the 2nd International Conference (NiDS 2022), A. Krouska, C. Troussas, and J. Caro, Eds., Cham: Springer International Publishing, 2023, pp. 134–146. 85. M. Go et al., “Evaluating digital mathematical games in improving the basic mathematical skills of university students,” Int J Math Educ Sci Technol, pp. 1–23, Jul. 2022, https://doi. org/10.1080/0020739X.2022.2089604. 86. Z. Kanetaki et al., “Acquiring, Analyzing and Interpreting Knowledge Data for Sustainable Engineering Education: An Experimental Study Using YouTube,” Electronics (Basel), vol. 11, no. 14, 2022, https://doi.org/10.3390/electronics11142210. 87. O. Huerta et al., “A design-based approach to enhancing technical drawing skills in design and engineering education using VR and AR tools,” VISIGRAPP 2019  – Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, vol. 3, no. March, pp.  306–313, 2019, https://doi. org/10.5220/0007566003060313. 88. M. Voronina, Z. Tretyakova, E. Krivonozhkina, S. Buslaev, and G. Sidorenko, “Augmented Reality in Teaching Descriptive Geometry, Engineering and Computer Graphics – Systematic Review and Results of the Russian Teachers’ Experience,” Eurasia Journal of Mathematics, Science and Technology Education, vol. 15, no. 12, 2019, https://doi.org/10.29333/ ejmste/113503. 89. P. H. Diao and N. J. Shih, “Trends and research issues of augmented reality studies in architectural and civil engineering education-A review of academic journal publications,” Applied Sciences (Switzerland), vol. 9, no. 9, 2019, https://doi.org/10.3390/app9091840.

References

47

90. J.  Garzón, J.  Pavón, and S.  Baldiris, “Systematic review and meta-analysis of augmented reality in educational settings,” Virtual Real, vol. 23, no. 4, pp. 447–459, 2019, https://doi. org/10.1007/s10055-­019-­00379-­9. 91. C.  Tuker, “Training Spatial Skills with Virtual Reality and Augmented Reality,” in Encyclopedia of Computer Graphics and Games, N. Lee, Ed., Cham: Springer International Publishing, 2018, pp. 1–9. https://doi.org/10.1007/978-­3-­319-­08234-­9_173-­1. 92. B. S. Bloom, M. D. Engelhart, E. J. Furst, W. H. Hill, and D. R. Krathwohl, “Handbook I: cognitive domain,” New York: David McKay, 1956. 93. A. LW et al., A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. 2001. 94. J. H. McMillan, Classroom assessment: Principles and practice for effective standards-based instruction. Pearson, 2014. 95. J. B. Biggs and K. F. Collis, Evaluating the Quality of Learning: The SOLO Taxonomy (structure of the Observed Learning Outcome). in Educational psychology. Academic Press, 1982. 96. R. el Mouhayar, Levels of generalization and the SOLO taxonomy. 2018. 97. J.  Biggs, “Teaching for Quality Learning at University. Society for Research into Higher Education,” The Higher Education Academy.(2008). Groupwork, Retrieved August, vol. 6, p. 2008, Jan. 2003. 98. B. Kitchenham, “Procedures for Performing Systematic Literature Reviews,” Joint Technical Report, Keele University TR/SE-0401 and NICTA TR-0400011T.1, p. 33, 2004. 99. K.  BA and S.  Charters, “Guidelines for performing Systematic Literature Reviews in Software Engineering,” vol. 2, Jan. 2007. 100. J.  Bacca-Acosta, S.  Baldiris, R.  Fabregat, S.  Graf, and D.  Kinshuk, “Augmented Reality Trends in Education: A Systematic Review of Research and Applications,” Educational Technology and Society, vol. 17, pp. 133–149, Oct. 2014. 101. Y. Goktas et al., “Educational Technology Research Trends in Turkey: A Content Analysis of the 2000-2009 Decade,” Jan. 2012. 102. M. Omar, D. F. Ali, M. Mokhtar, N. M. Zaid, H. Jambari, and N. H. Ibrahim, “Effects of Mobile Augmented Reality (MAR) towards Students’ Visualization Skills when Learning Orthographic Projection,” International Journal of Emerging Technologies in Learning (iJET), vol. 14, no. 20, p. 106, 2019, https://doi.org/10.3991/ijet.v14i20.11463. 103. M.  Omar, D.  F. Ali, A.  N. Nasir, and M.  S. Sunar, “AREDApps: Integrating mobile augmented reality in orthographic projection teaching and learning,” International Journal of Recent Technology and Engineering, vol. 8, no. 1C2, pp. 821–825, 2019. 104. D. N. A. L. E. Phon, M. H. A. Rahman, N. I. Utama, M. B. Ali, N. D. A. Halim, and S. Kasim, “The effect of augmented reality on spatial visualization ability of elementary school student,” Int J Adv Sci Eng Inf Technol, vol. 9, no. 2, pp.  624–629, 2019, https://doi.org/10.18517/ ijaseit.8.5.4971. 105. S. Tumkor, “Personalization of engineering education with the mixed reality mobile applications,” Computer Applications in Engineering Education, vol. 26, no. 5, pp. 1734–1741, 2018, https://doi.org/10.1002/cae.21942. 106. E. G. de Ravé, F. J. Jiménez-Hornero, A. B. Ariza-Villaverde, and J. Taguas-Ruiz, “DiedricAR: a mobile augmented reality system designed for the ubiquitous descriptive geometry learning,” Multimed Tools Appl, vol. 75, no. 16, pp.  9641–9663, 2016, https://doi.org/10.1007/ s11042-­016-­3384-­4. 107. I. J. Lee, “Using augmented reality to train students to visualize three-dimensional drawings of mortise–tenon joints in furniture carpentry,” Interactive Learning Environments, vol. 28, no. 7, pp. 930–944, 2020, https://doi.org/10.1080/10494820.2019.1572629. 108. I. J. Lee, T. C. Hsu, T. L. Chen, and M. C. Zheng, “The application of ar technology to spatial skills learning in carpentry training,” International Journal of Information and Education Technology, vol. 9, no. 1, pp. 56–60, 2019, https://doi.org/10.18178/ijiet.2019.9.1.1173.

48

2  Review of the Literature on AI-Enhanced Augmented Reality in Education

109. B. Özçakır and E. Çakıroğlu, “Fostering spatial abilities of middle school students through augmented reality: Spatial strategies,” Educ Inf Technol (Dordr), vol. 27, no. 3, pp. 2977–3010, 2022, https://doi.org/10.1007/s10639-­021-­10729-­3. 110. B.  Ozcakir and E.  Cakiroglu, “An augmented reality learning toolkit for fostering spatial ability in mathematics lesson: Design and development,” European Journal of Science and Mathematics Education, vol. 9, no. 4, pp.  145–167, 2021, https://doi.org/10.30935/ SCIMATH/11204. 111. J.  Weidinger, S.  Schlauderer, and S.  Overhage, “Information Technology to the Rescue? Explaining the Acceptance of Emergency Response Information Systems by Firefighters,” IEEE Trans Eng Manag, vol. PP, pp. 1–15, Jan. 2021, https://doi.org/10.1109/ TEM.2020.3044720. 112. D.  F. Ali et  al., “The Use of Augmented Reality Learning Environment in Enhancing Students’ Mental Rotation Skills,” Adv Sci Lett, vol. 24, no. 5, pp. 3705–3708, 2018, https:// doi.org/10.1166/asl.2018.11470. 113. L.  Medina Herrera, J.  Castro Pérez, and S.  Juárez Ordóñez, “Developing spatial mathematical skills through 3D tools: augmented reality, virtual environments and 3D printing,” International Journal on Interactive Design and Manufacturing (IJIDeM), vol. 13, no. 4, pp. 1385–1399, Dec. 2019, https://doi.org/10.1007/s12008-­019-­00595-­2. 114. E. T. Gün and B. Atasoy, “The effects of augmented reality on elementary school students’ spatial ability and academic achievement,” Egitim ve Bilim, vol. 42, no. 191, pp. 31–51, 2017, https://doi.org/10.15390/EB.2017.7140. 115. D.  Sumardani, E.  R. Sipayung, and P.-S.  Chiu, “Enabling spatial thinking through an augmented reality for teaching crystal structure,” Innovations in Education and Teaching International, 2022, https://doi.org/10.1080/14703297.2022.2076716. 116. J.  Martín-Gutiérrez, J.  Luís Saorín, M.  Contero, M.  Alcañiz, D.  C. Pérez-López, and M. Ortega, “Design and validation of an augmented book for spatial abilities development in engineering students,” Computers and Graphics (Pergamon), vol. 34, no. 1, pp. 77–91, 2010, https://doi.org/10.1016/j.cag.2009.11.003. 117. S. Habig, “Who can benefit from augmented reality in chemistry? Sex differences in solving stereochemistry problems using augmented reality,” British Journal of Educational Technology, vol. 51, no. 3, pp. 629–644, May 2020, https://doi.org/10.1111/bjet.12891. 118. Z. Gecu-Parmaksiz and Ö. Delialioğlu, “The effect of augmented reality activities on improving preschool children’s spatial skills,” Interactive Learning Environments, vol. 0, no. 0, pp. 1–14, 2018, https://doi.org/10.1080/10494820.2018.1546747. 119. H. C. ; M.-G. Gómez-Tone Jorge; Anci, Lili Valencia; Luis, Carlos Efrén Mora, “International Comparative Pilot Study of Spatial Skill Development in Engineering Students through Autonomous Augmented Reality-Based Training,” Symmetry (Basel), vol. 12, no. 9, pp. 1401-NA, 2020, https://doi.org/10.3390/sym12091401. 120. J. M. Gutiérrez, M. G. Domínguez, and C. R. González, “Using 3D virtual technologies to train spatial skills in engineering,” International Journal of Engineering Education, vol. 31, no. 1, pp. 323–334, 2015. 121. N.  Kaur, R.  Pathan, U.  Khwaja, and S.  Murthy, “GeoSolvAR: Augmented reality based solution for visualizing 3D Solids,” Proceedings – IEEE 18th International Conference on Advanced Learning Technologies, ICALT 2018, pp. 372–376, 2018, https://doi.org/10.1109/ ICALT.2018.00093. 122. C. Tuker and M. S. Fine, Training Spatial Skills with, no. January. 2018. 123. M. A. Omar Dayana Farzeeha; Mokhtar, Mahani; Zaid, Norasykin Mohd; Jambari, Hanifah; Ibrahim, Nor Hasniza, “Effects of Mobile Augmented Reality (MAR) towards Students’ Visualization Skills when Learning Orthographic Projection,” International Journal of Emerging Technologies in Learning (iJET), vol. 14, no. 20, pp. 106–119, 2019, https://doi. org/10.3991/ijet.v14i20.11463. 124. J.  Bell et  al., “A Study of Augmented Reality for the Development of Spatial Reasoning Ability,” Jun. 2017. https://doi.org/10.18260/1-­2%2D%2D27831.

References

49

125. Y.  C. C.  Chen Hung-Lin; Hung, Wei-Han; Kang, Shih-Chung, “Use of Tangible and Augmented Reality Models in Engineering Graphics Courses,” Journal of Professional Issues in Engineering Education and Practice, vol. 137, no. 4, pp. 267–276, 2011, https:// doi.org/10.1061/(asce)ei.1943-­5541.0000078. 126. M.  Omar, D.  Farzeeha, and M.  Mokhtar, “USING AREDKIT TO IMPROVE SPATIAL VISUALIZATION SKILLS FOR ORTHOGRAPHIC PROJECTION,” 2018. 127. D. F. Ali, M. Omar, N. H. Ibrahim, J. Surif, M. Ali, and S. Ismail, “Overcoming the problems faced by student’s in learning engineering drawing with the implementation of augmented reality learning environment,” Man India, vol. 97, no. 17, pp. 147–159, 2017. 128. A.  Krouska, C.  Troussas, and C.  Sgouropoulou, “Mobile game-based learning as a solution in COVID-19 era: Modeling the pedagogical affordance and student interactions,” Educ Inf Technol (Dordr), vol. 27, no. 1, pp.  229–241, 2022, https://doi.org/10.1007/ s10639-­021-­10672-­3. 129. J.  Dorribo-Camba and M.  Contero, “Incorporating augmented reality content in engineering design graphics materials,” Proceedings  – Frontiers in Education Conference, FIE, pp. 35–40, 2013, https://doi.org/10.1109/FIE.2013.6684784. 130. T. Chandrasekera and S. Y. Yoon, “Adopting augmented reality in design communication: Focusing on improving spatial abilities,” International Journal of Architectonic, Spatial, and Environmental Design, vol. 9, no. 1, pp.  1–14, 2015, https://doi.org/10.18848/2325-­1662/ CGP/v09i01/38384. 131. Z.  Veide, V.  Strozheva, and M.  Dobelis, “Application of Augmented Reality for teaching Descriptive Geometry and Engineering Graphics Course to First-Year Students,” Joint International Conference on Engineering Education & International Conference on Information Technology, pp. 158–164, 2014. 132. A.  Buchori, P.  Setyosari, I.  Wayan Dasna, and S.  Ulfa, “Mobile augmented reality media design with waterfall model for learning geometry in college,” International Journal of Applied Engineering Research, vol. 12, no. 13, pp. 3773–3780, 2017. 133. J. Kim and J. Irizarry, “Assessing the effectiveness of augmented reality on the spatial skills of postsecondary construction management students in the U.S.,” in ISARC 2017 – Proceedings of the 34th International Symposium on Automation and Robotics in Construction, 2017, pp. 173–180. https://doi.org/10.22260/isarc2017/0023. 134. M. Contero, J. M. Gomis, F. Naya, F. Albert, and J. Martin-Gutierrez, “Development of an augmented reality based remedial course to improve the spatial ability of engineering students,” Proceedings – Frontiers in Education Conference, FIE, 2012, https://doi.org/10.1109/ FIE.2012.6462312. 135. F. del Cerro Velázquez and G.  Morales Méndez, “Application in Augmented Reality for Learning Mathematical Functions: A Study for the Development of Spatial Intelligence in Secondary Education Students,” Mathematics, vol. 9, no. 4, p. 369, Feb. 2021, https://doi. org/10.3390/math9040369. 136. C. Troussas, A. Krouska, and C. Sgouropoulou, “Enriching Mobile Learning Software with Interactive Activities and Motivational Feedback for Advancing Users’ High-Level Cognitive Skills,” Computers, vol. 11, no. 2, 2022, https://doi.org/10.3390/computers11020018. 137. F. Giannakas, C. Troussas, A. Krouska, C. Sgouropoulou, and I. Voyiatzis, “XGBoost and Deep Neural Network Comparison: The Case of Teams’ Performance,” in Intelligent Tutoring Systems, A. I. Cristea and C. Troussas, Eds., Cham: Springer International Publishing, 2021, pp. 343–349. 138. A. Krouska, C. Troussas, and C. Sgouropoulou, “Fuzzy Logic for Refining the Evaluation of Learners’ Performance in Online Engineering Education,” European Journal of Engineering Research and Science, vol. 4, pp. 50–56, Jun. 2019, https://doi.org/10.24018/ ejers.2019.4.6.1369. 139. C. Troussas, A. Krouska, and C. Sgouropoulou, “Towards a Reference Model to Ensure the Quality of Massive Open Online Courses and E-Learning,” in Brain Function Assessment

50

2  Review of the Literature on AI-Enhanced Augmented Reality in Education

in Learning, C.  Frasson, P.  Bamidis, and P.  Vlamos, Eds., Cham: Springer International Publishing, 2020, pp. 169–175. 140. A.  Marougkas, C.  Troussas, A.  Krouska, and C.  Sgouropoulou, “How personalized and effective is immersive virtual reality in education? A systematic literature review for the last decade,” Multimed Tools Appl, 2023, https://doi.org/10.1007/s11042-­023-­15986-­7. 141. C. Troussas, A. Krouska, and C. Sgouropoulou, “Dynamic Detection of Learning Modalities Using Fuzzy Logic in Students’ Interaction Activities,” in Intelligent Tutoring Systems, V. Kumar and C. Troussas, Eds., Cham: Springer International Publishing, 2020, pp. 205–213. 142. A.  Krouska, C.  Troussas, and C.  Sgouropoulou, “A novel group recommender system for domain-independent decision support customizing a grouping genetic algorithm,” User Model User-adapt Interact, 2023, https://doi.org/10.1007/s11257-­023-­09360-­3. 143. A. Krouska, C. Troussas, K. Kabassi, and C. Sgouropoulou, “An Empirical Investigation of User Acceptance of Personalized Mobile Software for Sustainability Education,” Int J Hum Comput Interact, pp. 1–8, Aug. 2023, https://doi.org/10.1080/10447318.2023.2241614. 144. Roland. Guay, Purdue spatial visualization test. [West Layfette, Ind.]: Purdue University, 1976. 145. Y.  Maeda, S.  Y. Yoon, G.  Kim-Kang, and P.  K. Imbrie, “Psychometric properties of the revised PSVT:R for measuring First Year Engineering students’ spatial ability,” International Journal of Engineering Education, vol. 29, no. 3, pp. 763–776, 2013. 146. S. G. Vandenberg and A. R. Kuse, “Mental rotations, a group test of three-dimensional spatial visualization,” Percept Mot Skills, vol. 47, no. 2, pp. 599–604, 1978, https://doi.org/10.2466/ pms.1978.47.2.599. 147. D. F. Ali, M. Omar, H. Mohamed, N. M. Zaid, M. Mokhtar, and A. H. Abdullah, “Application of Augmented Reality Learning Environment in Enhancing Students’ Mental Cutting Skills and Mental Folding Skills,” Adv Sci Lett, vol. 24, no. 5, pp. 3701–3704, 2018, https://doi. org/10.1166/asl.2018.11469. 148. C. Quaiser-Pohl, “The Mental Cutting Test ‘Schnitte’ and the Picture Rotation Test-Two New Measures to Assess Spatial Ability,” Int J Test, vol. 3, no. 3, pp. 219–231, Sep. 2003, https:// doi.org/10.1207/S15327574IJT0303_2. 149. M.  Omar, D.  Farzeeha, and M.  Mokhtar, “USING AREDKIT TO IMPROVE SPATIAL VISUALIZATION SKILLS FOR USING AREDKIT TO IMPROVE SPATIAL VISUALIZATION,” 2018.

Chapter 3

AI-Driven and SOLO-Based Domain Knowledge Modeling in PARSAT AR Software

Abstract  This chapter of this book centers on the enrichment of the domain knowledge model through the incorporation of the Structure of Observed Learning Outcomes (SOLO) taxonomy. It investigates the correlation between the domain knowledge model and the SOLO taxonomy, offering practical instances of learning tasks aligned with each SOLO level. The “Overview” section introduces the chapter’s purpose, emphasizing the significance of aligning learning activities with SOLO-defined cognitive levels. The “Domain Model” section outlines the model’s objectives and relevance in spatial ability training, highlighting specific knowledge areas targeted in the mobile training system. In the “Domain Knowledge alongside SOLO Taxonomy” section, the integration of the SOLO taxonomy into the domain model is explored. This section underscores the importance of gradually developing students’ spatial ability through scaffolded learning experiences. The “Examples of Learning Activities of Each SOLO Level” section furnishes detailed examples of learning activities spanning from prestructural to extended abstract SOLO levels. These examples illustrate the practical application of the SOLO taxonomy within the domain knowledge model. The “Summary” section concludes by summarizing key points, highlighting the integration of the SOLO taxonomy as a scaffolding mechanism to enhance spatial ability training. This chapter serves as the foundation for subsequent chapters, which delve into the implementation and evaluation of the mobile training system.

3.1 Overview In this chapter, the training system’s domain knowledge is presented, considering the Structure of Observed Learning Outcomes (SOLO) taxonomy. The content of the domain model [1, 2] is a critical component of the application’s structure, whereas the combination of the learning theory with adaptive learning activities [3–6] enhances the students’ motivation and improves their learning outcome [7–17].

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 C. Papakostas et al., Special Topics in Artificial Intelligence and Augmented Reality, Cognitive Technologies, https://doi.org/10.1007/978-3-031-52005-1_3

51

52

3  AI-Driven and SOLO-Based Domain Knowledge Modeling in PARSAT AR Software

3.2 Domain Model The content of the domain knowledge consists of three levels (Table 3.1).

3.2.1 Objectives The Technical Drawing (TD) objectives, in detail, are as follows: 1. Recognize the exploratory potential of technical drawing while acknowledging the universality of objective language in information transmission and comprehension. 2. Strengthen the skills necessary for them to represent the graphical solutions precisely and objectively. 3. Have a basic understanding of technical drawing so that students can utilize it to read and interpret simple designs and artistic creations as well as to develop well-thought-out solutions to mathematical challenges in both the plane and space. 4. Recognize normalization as the optimum realist for condensing communication and giving it a more universal tone. 5. Include technical drawing tasks in a study area where aesthetic considerations are relevant, such as art, architecture, or industrial design. 6. Recognize and depict shapes in accordance with ISO standards. 7. Recognize how different approaches enhance the traditional idea of technical drawing. 8. Include the information provided by technical drawing in technological, artistic, or scientific research process. 9. Encourage method and rationality in sketching, as a way to convey scientific and technological concepts. 10. Acquire abilities that enable the expression of graphical solutions with accuracy, clarity, and objectivity. 11. Skillfully employ the specialized tools of technical drawing, and pay attention to the drawing’s proper execution, as well as the enhancements that various graphical styles can provide to the depiction. 12. Master the art of sketching to improve the speed and accuracy while expressing graphically. 13. Connect the space to the plane, recognizing the requirement to complete exercises from the activity book.

53

3.2  Domain Model Table 3.1  Domain knowledge Level Reading—perquisition, background knowledge, ISO 128-1:2020 general principles of graphical representation of objects on technical drawings Basic

Topics

Objective

Introduction to Technical Drawing Drawing instruments and accessories Isometric views Introduction to orthographic projections The six principal views The glass box method Standard views Alignment of the views

Identify the most popular drawing instruments and equipment, along with their purposes.

Visualization—understanding shapes and mentally rotate them in two dimensions whilst comparing to a model Intermediate Line types Line weights Creating an orthographic projection Orthographic projection drawing details Basics of dimensioning rules Dimension types Dimension parts Dimensioning of the views Drawing the visualizations Advanced Scaling Types of scaling Cutting plane Cutting plane line Section lining Full sections Half sections Technical symbols

Accurately and unambiguously capture all the geometric features of a 3D model

Convey all the required information that will allow a manufacturer to produce the designed model

54

3  AI-Driven and SOLO-Based Domain Knowledge Modeling in PARSAT AR Software

3.3 Domain Knowledge Alongside SOLO Taxonomy Several studies have highlighted the broader applicability of the learning theories in the design of effective training systems, showcasing their utility in organizing domain knowledge, structuring learning activities, and boosting student engagement, thus offering a blueprint for creating more personalized and impactful learning experiences across various domains [18–23]. SOLO taxonomy was created, within a constructivist context, as a tool for teaching students how to use basic rubrics to think more thoroughly about their own understanding. In addition to evaluation, SOLO is used in the developed system to design the curriculum according to the expected level of learning outcomes, which helps in establishing constructive alignment. The choice of SOLO taxonomy versus the Bloom’s taxonomy was made based on the following reasons: • Bloom’s theory has uncleared hierarchical connection between levels, whereas SOLO is based on stages of progressive cognitive complexity • Bloom’s theory is concerned with knowledge, whereas SOLO is a theory about teaching and learning • While Bloom’s model was not designed to, or could not be used, to compare outcomes to each task, SOLO provides task and outcome to be at separate levels • Aa task’s cognitive complexity and its difficulty can be separated using SOLO • Each level of SOLO contains clear verb usage, while Bloom’s verb use across levels is perplexing. The clarity of verb level is a strong advantage when educators are planning and writing the learning objectives • Levels of declarative knowledge and functioning knowledge, including metacognitive reflection, can be examined using SOLO • Students, even of younger age, can use SOLO to examine their own learning outcomes and the learning outcomes of their teammates, since it is rigorously simple • The SOLO approach teaches students that learning comes from work and strategies, and it is not an innate talent, displaying their individual learning progress PARSAT [2, 24] has used the SOLO taxonomy in the development of the assessment items for the learning objectives in technical drawing. These items had to fit to curriculum objectives and levels and measure both surface and deep cognitive states. Throughout the PARSAT development, experienced faculty members in the field of technical drawing have been involved in designing and reviewing the assessment tasks according to the SOLO taxonomy. Table 3.2 illustrates the learning goals and the corresponding activities per SOLO level.

3.3  Domain Knowledge Alongside SOLO Taxonomy

55

Table 3.2  Learning goals and activities per SOLO level [25, 26] SOLO level Pre-­ structural (L0)

Uni-­ structural (L1)

Multi-­ structural (L2)

Learning goal Learning activities Students get information Define concepts on the subject List items Match information Name facts

Description of the activities Introduction to Technical Drawing: A history and current importance of drawing are presented Students are asked to illustrate the significance of drawing by presenting applications and reports of both good and negative uses of the skill Setting up a model space in Identify content to be Students define, CAD software by defining recognize, name, sketch, memorized, show limits, grid, snap, layers, and reproduce, recite, follow examples object snap Provide disciplinary simple instructions, Video tutorials on standard context calculate, reproduce, views, views’ alignment, Mnemonics in groups arrange, find Repetition of procedures completion of activity sheet, and setting up the model Games space Repetitive testing and Border creation with a matching Peer testing (one student completed title block to be used for all future drawings, asks, one answers) and drawing templates with all the settings necessary saved within it Glossaries of key terms Orthographic drawing Students describe, list, creation with definitions, classify, structure, classifications, examples Lines, layers enumerate, conduct, Isometric object drawing to build disciplinary complete, illustrate, Video tutorials on linetype, vocabulary solve lineweight and isometric Simple laboratory drawing creation of objects exercises in the activity Define terms, compare to glossary Games modelled on Trivial Pursuit, Family Feud (continued)

56

3  AI-Driven and SOLO-Based Domain Knowledge Modeling in PARSAT AR Software

Table 3.2 (continued) SOLO level Relational (L3)

Extended abstract (L4)

Learning goal Students relate, analyze, compare, integrate, plan, construct, implement, summarize

Students generalize, hypothesize, theorize, predict, judge, evaluate, assess, predict, reason, criticize

Learning activities Case studies, simulations, and complex lab exercises Concept maps Research projects and experiential learning cycles Application of theoretical models Reflective journals Student seminars and debates Syndicate groups (each group is part of whole) Problem-Based Learning and Inquiry Learning Self-directed projects involving research, design, application, argumentation, evaluation Case studies involving extensive analysis, debate, reflection, argumentation, evaluation, forecasting Development of a theory or model Experiential learning cycles Problem Based Learning and Inquiry learning

Description of the activities Scaling the border and title block to fit the orthographic drawing Dimensioning an orthographic drawing Video tutorials on basic dimensioning rules and parts of dimensions Filling in a title block, including Name, Date, Title, Drawing No., and the correct scale Snapping and Text commands

Printing the drawing on 8.5″ × 11″ paper (letter size) in landscape orientation Video tutorial on cutting plane, half and full sections Printer/plotter settings Export/plot an object that has been drawn in CAD so it can be exported or printed to a variety of other applications CAD software to create objects that are more precise and sometimes easier to draw in CAD than in other software

3.4 Examples of Learning Activities of Each SOLO Level A simple uni-structural assignment is presented in Fig. 3.1, while the student has to observe the object in 3D, place herself/himself on the spot that the black arrow points to, and identify the object’s front view in 2D, ignoring the other views, and following simple procedure of the general principles of graphical representation of objects on technical drawings. In the uni-structural assignment of Fig. 3.1, the student is asked to identify the front view of an object in 2D. To this direction, the student needs to imagine herself/ himself standing in front of the object and looking straight at it. The front view corresponds to the perspective that a student would have when observing the object from a specific angle.

3.4  Examples of Learning Activities of Each SOLO Level

57

Fig. 3.1  Sample activity of uni-structural SOLO level

In technical drawing, this perspective is usually depicted as a 2D projection of the object onto a flat surface, like a sheet of paper. This projection should accurately represent the object’s shape and features, as well as any relevant measurements or dimensions. In order to produce a front view projection of an object, the student must apply fundamental principles of graphical representation, which encompass techniques like orthographic projection and dimensioning. Orthographic projection entails generating various perspectives of the object from different angles, which can subsequently be employed to construct a 2D portrayal of the object from any viewpoint. Dimensioning, on the other hand, involves incorporating precise measurements and dimensions into the drawing to guarantee the accuracy of the final depiction of the object. Overall, creating a front view projection of an object in 2D requires careful observation, an understanding of technical drawing principles, and attention to detail. A sample question of multi-structural SOLO level is presented in Fig. 3.2, while the student has to observe the 3D model, place herself/himself on the spot that the black arrow points to, and identify the object’s both front view and top view in 2D, among 12 available multiple choices. Now the student needs to visualize the

58

3  AI-Driven and SOLO-Based Domain Knowledge Modeling in PARSAT AR Software

Fig. 3.2  Multi-structural activity of SOLO taxonomy

geometry of the object and combine multiple views which are related, yet handled independently; however, its full perspective is still not completed. In the multi-structural task illustrated in Fig. 3.2, students are tasked with recognizing the frontal and overhead perspectives of a three-dimensional object when presented in a two-dimensional format. To successfully complete this assignment, students must mentally envision the object’s geometric structure and grasp how it would appear when projected onto a flat surface. The front view corresponds to the perspective one would have while directly facing the object, and the top view reflects the viewpoint from above when looking down upon the object. In technical drawing, these aspects are usually depicted using orthographic projection, a method that involves projecting various angles/views of the object onto a flat surface. To identify the front view and top view of the object in question, the student needs to combine multiple views that are related but handled independently. To approach this assignment, the student should start by carefully examining the available views of the object and identifying the ones that correspond to the front and top views. An effective strategy involves searching for visual hints, such as the alignment of edges and characteristics, the positioning of critical elements, and any

3.4  Examples of Learning Activities of Each SOLO Level

59

recognizable patterns or symmetries that can assist in determining the accurate perspectives. After identifying the correct perspectives, the student will then have to integrate them in a manner that faithfully portrays the object’s geometry. This could entail modifying the scale or orientation of the perspectives or aligning significant features to ensure their proper alignment. Overall, identifying the front view and top view of a 3D object in 2D requires careful observation, an understanding of technical drawing principles, and the ability to combine multiple views into a coherent representation of the object’s geometry. A deeper thinking level requires framing a question about a relationship within the given material, rather than having a surface approach. Figure 3.3 presents an activity from the relational level of SOLO taxonomy. The student must visualize the

Fig. 3.3  Relational level question of SOLO taxonomy

60

3  AI-Driven and SOLO-Based Domain Knowledge Modeling in PARSAT AR Software

3D model and find its front view with its corresponding top view and left-side view, so that relating them will result into the integration of the object’s structure. In the instructional scenario described in Fig. 3.3, regarding the relational SOLO level, students are tasked with evaluating the distinct perspectives of the object and assessing their interconnections. They must pinpoint significant attributes and components in each viewpoint and discern how these elements amalgamate to construct a comprehensive portrayal of the object’s structure and form. Additionally, students are encouraged to apply critical thinking skills to assess the inherent constraints of each perspective and find strategies to make them mutually reinforcing. As an illustration, the front view may adeptly communicate the object’s dimensions in terms of height and width, while the top view might be better suited to illustrate its depth and overall form. In essence, this assignment demands that students employ higher-order thinking skills and contemplate the interplay between diverse pieces of information to construct a more unified and comprehensive grasp of the object. The final level of SOLO taxonomy, namely extended abstraction, within the domain of technical drawing is achieved by paying detailed attention to more widely valid principles. In an extended abstract assignment, the student has to compare the given front views, with their corresponding top views and left-side views respectively, and afterwards, relate them integrating into the object’s structure. The students would need to demonstrate a deep understanding of the underlying principles of technical drawing, including orthographic projection and geometric construction. They would be required to elucidate how these principles are employed to generate precise and logically consistent depictions of three-dimensional objects in a two-­ dimensional format. Furthermore, students would have to put these principles into practice when generating and fusing front, top, and left-side perspectives of a three-dimensional object. This task entails scrutinizing the connections between these various viewpoints and recognizing shared characteristics and components that can be harnessed to produce a more comprehensive and unified portrayal of the object’s structure and form. Overall, this assignment requires students to engage in advanced levels of abstraction and demonstrate a deep understanding of the underlying principles and processes involved in technical drawing.

3.5 Summary This chapter discussed the use of the SOLO taxonomy in developing the training system. The SOLO taxonomy is a framework used to categorize learning outcomes and can be used to design effective teaching strategies. In this context, the chapter outlined how the domain knowledge of the subject of the technical drawing course is categorized according to the SOLO taxonomy. The

References

61

chapter also discussed how the learning activities within the training system are structured and how they align with the SOLO levels to promote effective learning. Furthermore, the chapter presented how the use of adaptive learning activities based on the SOLO levels can enhance student motivation and improve learning outcomes. By providing activities that match the student’s current level of understanding, the training system can better support their learning journey and provide a more personalized experience. Overall, the chapter focused on how the SOLO taxonomy and adaptive learning activities can be used to design an effective training system for technical drawing. In addition, the incorporated learning theory was explained, providing detailed learning goals per SOLO level, as well as a description of each learning activity. Finally, examples of assignments, and activities per SOLO level, were presented.

References 1. C.  Papakostas, C.  Troussas, A.  Krouska, and C.  Sgouropoulou, “On the development of a personalized augmented reality spatial ability training mobile application,” in Frontiers in Artificial Intelligence and Applications, IOS Press, 2021, pp. V–VI. https://doi.org/10.3233/ FAIA210078. 2. C. Papakostas, C. Troussas, A. Krouska, and C. Sgouropoulou, “Exploration of Augmented Reality in Spatial Abilities Training: A Systematic Literature Review for the Last Decade,” Informatics in Education, vol. 20, no. 1, pp.  107–130, Mar. 2021, https://doi.org/10.15388/ infedu.2021.06. 3. P. Strousopoulos, C. Papakostas, C. Troussas, A. Krouska, P. Mylonas, and C. Sgouropoulou, “SculptMate: Personalizing Cultural Heritage Experience Using Fuzzy Weights,” in Adjunct Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization, in UMAP ’23 Adjunct. New York, NY, USA: Association for Computing Machinery, 2023, pp. 397–407. https://doi.org/10.1145/3563359.3596667. 4. C.  Troussas, C.  Papakostas, A.  Krouska, P.  Mylonas, and C.  Sgouropoulou, “Personalized Feedback Enhanced by Natural Language Processing in Intelligent Tutoring Systems,” in Augmented Intelligence and Intelligent Tutoring Systems, C.  Frasson, P.  Mylonas, and C.  Troussas, Eds., Cham: Springer Nature Switzerland, 2023, pp.  667–677. https://doi. org/10.1007/978-­3-­031-­32883-­1_58. 5. C. Papakostas, C. Troussas, A. Krouska, and C. Sgouropoulou, “Exploring Users’ Behavioral Intention to Adopt Mobile Augmented Reality in Education through an Extended Technology Acceptance Model,” Int J Hum Comput Interact, vol. 39, no. 6, pp. 1294–1302, 2023, https:// doi.org/10.1080/10447318.2022.2062551. 6. C. Papakostas, C. Troussas, A. Krouska, and C. Sgouropoulou, “Modeling the Knowledge of Users in an Augmented Reality-Based Learning Environment Using Fuzzy Logic,” in Lecture Notes in Networks and Systems, A. Krouska, C. Troussas, and J. Caro, Eds., Cham: Springer International Publishing, 2023, pp. 113–123. https://doi.org/10.1007/978-­3-­031-­17601-­2_12. 7. Z. Kanetaki et al., “Acquiring, Analyzing and Interpreting Knowledge Data for Sustainable Engineering Education: An Experimental Study Using YouTube,” Electronics (Basel), vol. 11, no. 14, 2022, https://doi.org/10.3390/electronics11142210. 8. C. Troussas, A. Krouska, and C. Sgouropoulou, “Enriching Mobile Learning Software with Interactive Activities and Motivational Feedback for Advancing Users’ High-Level Cognitive Skills,” Computers, vol. 11, no. 2, 2022, https://doi.org/10.3390/computers11020018.

62

3  AI-Driven and SOLO-Based Domain Knowledge Modeling in PARSAT AR Software

9. A.  Krouska, C.  Troussas, and C.  Sgouropoulou, “Fuzzy Logic for Refining the Evaluation of Learners’ Performance in Online Engineering Education,” European Journal of Engineering Research and Science, vol. 4, pp.  50–56, Jun. 2019, https://doi.org/10.24018/ ejers.2019.4.6.1369. 10. A.  Marougkas, C.  Troussas, A.  Krouska, and C.  Sgouropoulou, “How personalized and effective is immersive virtual reality in education? A systematic literature review for the last decade,” Multimed Tools Appl, 2023, https://doi.org/10.1007/s11042-­023-­15986-­7. 11. A. Marougkas, C. Troussas, A. Krouska, and C. Sgouropoulou, “A Framework for Personalized Fully Immersive Virtual Reality Learning Environments with Gamified Design in Education,” 2021. https://doi.org/10.3233/FAIA210080. 12. C. Troussas, A. Krouska, and C. Sgouropoulou, “Dynamic Detection of Learning Modalities Using Fuzzy Logic in Students’ Interaction Activities,” in Intelligent Tutoring Systems, V. Kumar and C. Troussas, Eds., Cham: Springer International Publishing, 2020, pp. 205–213. 13. A.  Krouska, C.  Troussas, and C.  Sgouropoulou, “A novel group recommender system for domain-independent decision support customizing a grouping genetic algorithm,” User Model User-adapt Interact, 2023, https://doi.org/10.1007/s11257-­023-­09360-­3. 14. C.  Papakostas, C.  Troussas, P.  Douros, M.  Poli, and C.  Sgouropoulou, “CoMoPAR: A Comprehensive Conceptual Model for Designing Personalized Augmented Reality Systems in Education,” in Novel & Intelligent Digital Systems: Proceedings of the 3rd International Conference (NiDS 2023), K. Kabassi, P. Mylonas, and J. Caro, Eds., Cham: Springer Nature Switzerland, 2023, pp. 67–79. 15. C.  Papakostas, C.  Troussas, A.  Krouska, and C.  Sgouropoulou, “Personalization of the Learning Path within an Augmented Reality Spatial Ability Training Application Based on Fuzzy Weights,” Sensors, vol. 22, no. 18, 2022, https://doi.org/10.3390/s22187059. 16. C. Troussas, A. Krouska, and C. Sgouropoulou, “Towards a Reference Model to Ensure the Quality of Massive Open Online Courses and E-Learning,” in Brain Function Assessment in Learning, C.  Frasson, P.  Bamidis, and P.  Vlamos, Eds., Cham: Springer International Publishing, 2020, pp. 169–175. 17. A. Krouska, C. Troussas, K. Kabassi, and C. Sgouropoulou, “An Empirical Investigation of User Acceptance of Personalized Mobile Software for Sustainability Education,” Int J Hum Comput Interact, pp. 1–8, Aug. 2023, https://doi.org/10.1080/10447318.2023.2241614. 18. A. Marougkas, C. Troussas, A. Krouska, and C. Sgouropoulou, “Virtual Reality in Education: A Review of Learning Theories, Approaches and Methodologies for the Last Decade,” Electronics (Basel), vol. 12, no. 13, 2023, https://doi.org/10.3390/electronics12132832. 19. F.  Giannakas, C.  Troussas, A.  Krouska, C.  Sgouropoulou, and I.  Voyiatzis, “XGBoost and Deep Neural Network Comparison: The Case of Teams’ Performance,” in Intelligent Tutoring Systems, A. I. Cristea and C. Troussas, Eds., Cham: Springer International Publishing, 2021, pp. 343–349. 20. M. Iakovidis, C. Papakostas, C. Troussas, and C. Sgouropoulou, “Empowering Responsible Digital Citizenship Through an Augmented Reality Educational Game,” in Novel & Intelligent Digital Systems: Proceedings of the 3rd International Conference (NiDS 2023), K. Kabassi, P. Mylonas, and J. Caro, Eds., Cham: Springer Nature Switzerland, 2023, pp. 31–39. 21. P.  Strousopoulos, C.  Troussas, C.  Papakostas, A.  Krouska, and C.  Sgouropoulou, “Revolutionizing Agricultural Education with Virtual Reality and Gamification: A Novel Approach for Enhancing Knowledge Transfer and Skill Acquisition,” in Novel & Intelligent Digital Systems: Proceedings of the 3rd International Conference (NiDS 2023), K. Kabassi, P. Mylonas, and J. Caro, Eds., Cham: Springer Nature Switzerland, 2023, pp. 67–80. 22. C. Papakostas, C. Troussas, A. Krouska, and C. Sgouropoulou, “User acceptance of augmented reality welding simulator in engineering training,” Educ Inf Technol (Dordr), vol. 27, no. 1, pp. 791–817, Jan. 2022, https://doi.org/10.1007/s10639-­020-­10418-­7. 23. C. Papakostas, C. Troussas, A. Krouska, and C. Sgouropoulou, “Measuring User Experience, Usability and Interactivity of a Personalized Mobile Augmented Reality Training System,” Sensors, vol. 21, no. 11, p. 3888, Jun. 2021, https://doi.org/10.3390/s21113888.

References

63

24. C.  Papakostas, C.  Troussas, A.  Krouska, and C.  Sgouropoulou, “PARSAT: Fuzzy logic for adaptive spatial ability training in an augmented reality system,” Computer Science and Information Systems, vol. 20, no. 4, 2023, https://doi.org/10.2298/CSIS230130043P. 25. J.  Biggs, “Teaching for Quality Learning at University. Society for Research into Higher Education,” The Higher Education Academy.(2008). Groupwork, Retrieved August, vol. 6, p. 2008, Jan. 2003. 26. J. Biggs and C. Tang, Teaching for Quality Learning at University. in UK Higher Education OUP Humanities & Social Sciences Higher Education OUP. McGraw-Hill Education, 2011. [Online]. Available: https://books.google.gr/books?id=VC1FBgAAQBAJ

Chapter 4

Fuzzy Logic for Modeling the Knowledge of Users in PARSAT AR Software

Abstract  This chapter of the book centers on the application of fuzzy logic for modeling students’ knowledge, with a primary focus on enhancing spatial ability training through personalized and adaptive learning experiences. The chapter begins with an overview, emphasizing the utility of fuzzy logic in capturing and adapting to students’ knowledge levels. It underscores the significance of tailoring learning activities to individual students’ needs. The core components of the fuzzy logic algorithm are elaborated upon in detail. This includes an explanation of how fuzzy logic handles imprecise and uncertain knowledge through linguistic variables and fuzzy sets. The initialization process is discussed, outlining how the model is set up to capture students’ knowledge levels at the outset of training, underscoring the importance of accurate initialization for effective adaptation of learning activities. The concept of fuzzy sets and their role in representing linguistic variables is explored, shedding light on how they measure the degree of membership or fuzzy truth values associated with various knowledge levels. The construction of the fuzzy rule base is explained, detailing how rules are defined to link linguistic variables and their corresponding fuzzy sets to appropriate learning activities, emphasizing the rule-based decision-making nature of fuzzy logic. Mamdani’s inference system, a crucial component of the fuzzy logic model, is examined in terms of how it combines fuzzy rules to determine adaptive learning activities based on students’ knowledge levels. The process of defuzzification is described, highlighting its role in converting fuzzy outputs into actionable decisions. The chapter concludes by illustrating how fuzzy weights obtained through the fuzzy logic model are employed for real-time adaptation of learning activities, influencing the selection and customization of learning materials.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 C. Papakostas et al., Special Topics in Artificial Intelligence and Augmented Reality, Cognitive Technologies, https://doi.org/10.1007/978-3-031-52005-1_4

65

66

4  Fuzzy Logic for Modeling the Knowledge of Users in PARSAT AR Software

4.1 Overview In this chapter, the principles of the design of the student model, within the spatial ability training platform, are discussed. The model allows steering of the sequence of the educational material and the deliverable learning activities, through the incorporation of fuzzy logic, quantitative inputs, and fuzzy weights [1, 2]. The design of the student model in the spatial ability training platform involves the use of fuzzy logic, quantitative inputs, and fuzzy weights to control the sequence of educational material and learning activities [3–6]. Fuzzy logic is a type of logic that allows for the handling of uncertain or imprecise information by using linguistic variables and fuzzy sets to represent qualitative concepts. In the context of the spatial ability training platform, fuzzy logic could be used to represent the student’s level of understanding or mastery of certain concepts, and to make decisions about which educational material or learning activities to present next. Quantitative inputs refer to numeric data that can be employed as input variables for a fuzzy system. In the context of a student model, these inputs could encompass measurements like a student’s past performance in educational tasks, the time dedicated to each task, or the student’s self-assessed confidence in their grasp of the subject matter. Fuzzy weights are used to assign importance or priority to different inputs or rules in the fuzzy system. Within the student model framework, fuzzy weights could be applied to give priority to specific learning activities, considering their perceived significance or alignment with the student’s learning objectives. In the exploration of fuzzy logic’s application for personalized spatial ability training in this chapter, a significant body of research is referenced, drawing on influential works, as well as insightful contributions from the educational [7–21]. In general, incorporating fuzzy logic, numerical inputs, and fuzzy weights into the construction of the student model can enhance the customization of the learning journey for individual students, enabling them to advance through the educational content autonomously and with increased control over their progress.

4.2 Fuzzy Logic Algorithm Users of PARSAT are students with various levels of prior knowledge in the field of technical drawing, and, as a result, they have different learning needs. The significance of the parameter of prior knowledge is in line with [22], as according to them, the engineering background of the students should be taken into consideration while planning the curriculum and lessons. An AR system, namely PARSAT, was developed to identify each learner’s knowledge level and instructional needs, in order to provide them the most appropriate learning path and content. The student model, which may be found in the majority of the latest adaptive educational software [23], is responsible for defining the student’s knowledge level.

4.2  Fuzzy Logic Algorithm

67

A student model serves as a vital element within adaptive educational software, preserving data pertaining to a student’s knowledge, competencies, preferences, and educational objectives. This information is leveraged to individualize the learning process for each student, encompassing the selection of suitable learning tasks, offering tailored feedback, and adjusting the content’s difficulty level to align with the student’s skill level. In the case of PARSAT, the student model likely takes into account the student’s prior knowledge of technical drawing, as well as other factors such as their performance on previous learning activities, and their self-reported level of confidence. By using this information to personalize the learning experience, PARSAT can help each student to progress through the content at their own pace and with a greater degree of engagement and motivation. The purpose of the student model is to represent the students’ current level of knowledge [2, 4, 24], and the system needs to provide the necessary level of customization on every student’s learning requirement. Other approaches regarding adaptivity, are those of neural networks, machine learning, fuzzy logic networks, etc., which can be utilized to build the student model [19, 25–28]. The backbone of PARSAT’s student model is fuzzy logic, which defines the students’ current level of knowledge. Fuzzy logic is a type of logic that allows for the handling of uncertain or imprecise information by using linguistic variables and fuzzy sets to represent qualitative concepts. In the realm of student modeling, fuzzy logic offers a valuable tool for depicting a student’s grasp or proficiency in specific topics. Fuzzy logic stands out as an ideal choice for portraying intricate, ambiguous, or vague systems, especially when it comes to human cognition and the learning process. By using fuzzy logic to define the student’s current level of knowledge, PARSAT can more accurately assess their learning needs and personalize the learning experience accordingly. PARSAT’s fuzzy system consists of three main parts: a) the part of the linguistic variables, b) the part of the membership functions, and c) the rules, which are described as follows: Linguistic variables: Linguistic variables are used in fuzzy logic to represent qualitative concepts. In the context of PARSAT’s student model, linguistic variables might be used to represent the student’s level of understanding or mastery of certain concepts. Membership functions: Membership functions are used to map the linguistic variables onto fuzzy sets. A membership function specifies the degree of membership of a particular input value in a given fuzzy set. In the context of PARSAT’s student model, membership functions might be used to map the student’s performance on a particular learning activity onto a fuzzy set representing their level of understanding of the material. Rules: Rules are used to define the relationships between the input variables (in this case, the linguistic variables and their associated membership functions) and the output variable (the student’s level of knowledge). In the context of PARSAT’s student model, rules might be used to specify how the student’s performance on different learning activities should be combined to determine their overall level of understanding.

68

4  Fuzzy Logic for Modeling the Knowledge of Users in PARSAT AR Software

The general process of designing a fuzzy system for PARSAT’s student model would involve selecting appropriate linguistic variables, designing appropriate membership functions to map the input values onto fuzzy sets, and defining appropriate rules to combine the input variables and produce the output variable. The design process involved iterative refinement and testing to ensure that the fuzzy system accurately captures the relevant aspects of the student’s learning needs and produces appropriate recommendations for learning activities. This section describes the general process of designing the fuzzy system.

4.3 Initialization Process The process of developing the fuzzy system starts by defining the linguistic variables, which represent, in words, the system’s input and output variables. Each linguistic variable is described by a specific number of linguistic values, while in most cases 3–7 terms are enough. In most cases, 3–7 linguistic values are sufficient for representing a linguistic variable. However, the number and specificity of the linguistic values will depend on the particular application and the level of detail required. The proposed fuzzy model has four inputs, namely prior knowledge (PRK), video-based learning duration (VLD), augmented-reality interaction duration (ARID), and the number of errors (NoE). The first input is derived from the student profile, while the remaining three inputs are derived from the interaction model. Furthermore, the output value and its linguistic name is the current level of knowledge (CLK). Table 4.1 presents the input linguistic variables and their ranges. Table 4.1  Linguistic input variables and their ranges Linguistic value Notation Linguistic variable: Prior Knowledge (PRK)     Poor PRK_PR     Medium PRK_MDM     Good PRK_GD Linguistic variable: Video-based Learning Duration (VLD)     Short VLD_SRT     Normal VLD_NRML     Long VLD_LNG Linguistic variable: Augmented-Reality Interaction Duration (ARID)     Short ARID_SRT     Normal ARID_NRML     Long ARID_LNG Linguistic variable: Number of Errors (NoE)     Small NoE_SMLL     Medium NoE_MDM     Large NoE_LRG

Range (normalized) [0, 0.35] [0.30, 0.75] [0.70, 1.00] [0, 0.35] [0.30, 0.70] [0.60, 1.00] [0, 0.60] [0.40, 0.80] [0.70, 1.00] [0, 0.40] [0.35, 0.65] [0.60, 1.00]

4.4  Fuzzy Sets

69

4.4 Fuzzy Sets In the second part of the process of developing the fuzzy system for PARSAT’s student model, fuzzy sets are determined by mapping all input values into fuzzy ones using membership functions. In this case, trapezoidal membership functions are used, which are assigned to each linguistic variable. The membership functions are formed using straight lines, having the advantage of simplicity. A trapezoidal membership function is assigned to each linguistic variable. A curve characterized by four distinct points (a, b, c, d), where they correspond to a minimum boundary (a), a maximum boundary (b), a lower support boundary (c), and an upper support boundary (d), is referred to as a trapezoidal membership function (Fig. 4.1). It has the advantage of simplicity and specificity, making it a good choice for mapping input values onto fuzzy sets. Using trapezoidal membership functions, the input variables of PRK, VLD, ARID, and NoE can be mapped onto their corresponding linguistic variables with fuzzy sets. These fuzzy sets would have degrees of membership between 0 and 1, representing the uncertainty and imprecision of the input values. Once the input values have been mapped onto fuzzy sets, appropriate rules can be defined to combine the input variables and produce the output variable, CLK. The rules would take into account the degrees of membership for each fuzzy set and use them to determine the appropriate level of knowledge for the student. Based on the attained knowledge level, personalized suggestions for learning activities that align with the student’s unique requirements and learning preferences can be offered. The values of the curve span from 0 to 1. Real values between b and c are represented by degree of membership 1. Values between a and b have a higher degree of membership as they move closer to element b, whereas values between c and d have a lower degree of membership as they move closer to element d. The membership degree is zero in all other cases. Figures 4.2, 4.3, 4.4 and 4.5 present the equation of the membership function for every input fuzzy variable. These equations can be used to map input values onto fuzzy sets, which can then be used in the rule-based system to determine the appropriate level of knowledge for the student.

Fig. 4.1 Trapezoidal membership function

70

4  Fuzzy Logic for Modeling the Knowledge of Users in PARSAT AR Software

Fig. 4.2  Membership functions describing student’s prior knowledge

Fig. 4.3  Membership functions describing video-based learning duration

The student’s video-based learning duration for each topic is measured in seconds and then normalized to obtain the linguistic variable values of this input. The three linquistic values for this input are “short” (VLD_SRT), “normal” (VLD_ NRML) and “long” (VLD_LNG), which are represented by trapezoidal membership functions (Fig.  4.3). The specific values for the trapezoidal membership functions depend on the specific implementation of the fuzzy system. These fuzzy values can then be used in the rule-based system to determine the appropriate level of knowledge for the student based on their video-based learning duration. Similar to the video-based learning duration (VLD) input, the augmented reality interaction duration (ARID) of each 3D model is recorded in seconds and

4.4  Fuzzy Sets

71

Fig. 4.4  Membership functions describing augmented-reality interaction duration

Fig. 4.5  Membership functions describing number of errors

normalized to obtain the values of the linguistic variable for this input. There are three linguistic categories for ARID, namely “short” (referred to as ARID_SRT), “normal” (referred to as ARID_NRML), and “long” (referred to as ARID_LNG). These categories are represented using trapezoidal membership functions. (Fig. 4.4). Lastly, the fourth input variable, which pertains to the number of errors, is determined based on the average performance of students in the level’s test, where their scores are rated on a 100-point scale. This variable is categorized into three linguistic values: “small” (represented as NoE_SMLL), “medium” (denoted as NoE_ MDM), and “large” (referred to as NoE_LRG) (Fig. 4.5).

72

4  Fuzzy Logic for Modeling the Knowledge of Users in PARSAT AR Software

The outcome produced by the fuzzy system, along with its associated linguistic label, represents the student’s existing level of knowledge (referred to as CLK). This level can fall into one of the following categories: “Novice” (N), “Beginner” (B), “Competent” (C), “Very Good” (VG), “Proficient” (P), or “Expert” (E). Each of these levels is defined by specific attributes as follows: • Novice (N): the student has minimal or textbook knowledge of the educational material, without connecting it to the practice. The student has a low level of knowledge and understanding of the topic. • Beginner (B): the student has working knowledge of key aspects of the educational material. The student has some basic understanding of the topic but lacks experience and depth of knowledge. • Competent (C): the student has good working and background knowledge of the educational material. The student possesses a solid grasp of the subject matter and is able to execute tasks with a reasonable degree of competence. • Very Good (VG): the student has very good knowledge of the educational material. The student has a strong understanding of the topic and can perform tasks with high proficiency • Proficient (P): The student demonstrates a profound comprehension of the educational content. They have attained an advanced level of expertise in the subject and can execute intricate tasks with a remarkable level of skill. • Expert (E): the student has authoritative knowledge of the educational material and a deep tacit understanding across the domain.

4.5 Fuzzy Rule Base In the fuzzy IF-THEN rules, each input variable is assigned a fuzzy set, and the output variable is assigned a fuzzy set as well. The IF-THEN rules define how the input variables will affect the output variable. Every rule comprises two components: an antecedent (IF) and a consequent (THEN). The antecedent defines the circumstances for which the rule is relevant, and the consequent outlines the action to be executed when those circumstances are satisfied. Professionals with expertise in the field are responsible for crafting these fuzzy IF-THEN rules. They leverage their knowledge and experience to determine the most suitable rules for the system. These rules undergo testing and refinement over time to ensure their accuracy and effectiveness. The quantity of rules (r) within the fuzzy system follows an exponential relationship with the number of inputs (m) and the number of linguistic values (w) that each input can assume. All four inputs (PRK, VLD, ARID, and NoE) have a constant number of three alternative values, so the maximum number of rules is given by Eq. (4.1) from [29]:

r = wm

(4.1)

4.5  Fuzzy Rule Base

73

A set of 81 fuzzy rules were formulated, and they have been incorporated into the proposed system (Table 4.2), which were specifically designed to create the logic outcome. The remaining part of this subsection presents a representative sample of the aforementioned rules, showing how inputs affect the output. These rules consider different combinations of input values and their fuzzy sets and output the corresponding linguistic term of the current level of knowledge. The complete set of rules in Table 4.2 provide a comprehensive representation of the logic behind the system’s decision-making process. Example Rule 1: IF PRK is PRK_PR AND VLD is VLD_LNG AND ARID is ARID_LNG AND NoE is NoE_LRG THEN CLK is N This rule (rule 1) is an example of how the PARSAT fuzzy inference system uses the input variables (PRK, VLD, ARID, NoE) to classify the student’s current level of knowledge (CLK) as “Novice” based on certain conditions. In this case, the rule suggests that if the student has poor prior knowledge, spends a long time on video tutorials (maybe by replaying them all the time, or constantly pausing and rewinding them) and AR interactions (maybe finding it difficult to conceptualize their geometry), and scores a large number of errors, then she/he is likely to be classified as a novice student. Indeed, Fig. 4.6 is an example of assigning the input variables PRK, VLD, ARID and NoE, with the crisp values 15, 88, 82 and 91, respectively. The fuzzification process turns crisp inputs into their corresponding fuzzy inputs, the Mamdani’s inference engine triggers the appropriate rule in line 27 of Table 4.2 among the 81 rules located in the rule base, resulting in the fuzzy output, namely novice. Example Rule 2: IF PRK is PRK_GD AND VLD is VLD_NRML AND ARID is ARID_NRML AND NoE is NoE_SMLL THEN CLK is E The above rule (rule 2) suggests that if a student has a strong prior knowledge background, performs well in the assessments with few errors, and spends a long time interacting with the AR models, then she/he can be classified as an expert. This rule also emphasizes the importance of a strong foundation in technical drawing for achieving expert level proficiency in the PARSAT system. As seen in Fig. 4.7, the crisp input values are 88, 49, 53, and 14 of the variables PRK, VLD, ARID, and NoE, respectively, and the output fuzzy output is an expert student. Example Rule 3: IF PRK is PRK_MDM AND VLD is VLD_NRML AND ARID is ARID_NRML AND NoE is NoE_SMLL THEN CLK is P This rule (rule 3) indicates that a student with medium prior knowledge, spending a normal amount of time in both video-based learning and augmented reality interaction, and making a medium number of errors in the assessment, can become proficient if appropriate personalized learning activities are provided. This rule (Fig. 4.8) highlights the importance of adaptive and personalized learning in improving students’ knowledge and skills. The next two rules (rule 4 and rule 5) consider the various inputs that all contribute to the knowledge level rating of the student being a beginner.

74

4  Fuzzy Logic for Modeling the Knowledge of Users in PARSAT AR Software

Table 4.2  Fuzzy rules PRK

VLD

ARID

NoE

CLK

PRK

VLD

ARID

NoE

CLK

1 Poor

Short

Short

Small

VG

41 Medium

Normal

Normal

Medium

VG

2 Poor

Short

Normal

Small

VG

42 Medium

Normal

Long

Medium

VG

3 Poor

Short

Long

Small

C

43 Medium

Long

Short

Medium

C

4 Poor

Normal

Short

Small

C

44 Medium

Long

Normal

Medium

VG

5 Poor

Normal

Normal

Small

VG

45 Medium

Long

Long

Medium

C

6 Poor

Normal

Long

Small

VG

46 Medium

Short

Short

Large

B

7 Poor

Long

Short

Small

VG

47 Medium

Short

Normal

Large

C

8 Poor

Long

Normal

Small

C

48 Medium

Short

Long

Large

B

9 Poor

Long

Long

Small

C

49 Medium

Normal

Short

Large

B

10 Poor

Short

Short

Medium

VG

50 Medium

Normal

Normal

Large

C

11 Poor

Short

Normal

Medium

VG

51 Medium

Normal

Long

Large

B

12 Poor

Short

Long

Medium

C

52 Medium

Long

Short

Large

C

13 Poor

Normal

Short

Medium

VG

53 Medium

Long

Normal

Large

C

14 Poor

Normal

Normal

Medium

VG

54 Medium

Long

Long

Large

B

15 Poor

Normal

Long

Medium

C

55 Good

Short

Short

Small

E

16 Poor

Long

Short

Medium

C

56 Good

Short

Normal

Small

E

17 Poor

Long

Normal

Medium

VG

57 Good

Short

Long

Small

E

18 Poor

Long

Long

Medium

C

58 Good

Normal

Short

Small

E

19 Poor

Short

Short

Large

N

59 Good

Normal

Normal

Small

E

20 Poor

Short

Normal

Large

N

60 Good

Normal

Long

Small

E

21 Poor

Short

Long

Large

N

61 Good

Long

Short

Small

E

22 Poor

Normal

Short

Large

N

62 Good

Long

Normal

Small

E

23 Poor

Normal

Normal

Large

C

63 Good

Long

Long

Small

E

24 Poor

Normal

Long

Large

B

64 Good

Short

Short

Medium

P

25 Poor

Long

Short

Large

N

65 Good

Short

Normal

Medium

P

26 Poor

Long

Normal

Large

B

66 Good

Short

Long

Medium

VG

27 Poor

Long

Long

Large

N

67 Good

Normal

Short

Medium

P

28 Medium

Short

Short

Small

P

68 Good

Normal

Normal

Medium

P

29 Medium

Short

Normal

Small

P

69 Good

Normal

Long

Medium

VG

30 Medium

Short

Long

Small

P

70 Good

Long

Short

Medium

VG

31 Medium

Normal

Short

Small

P

71 Good

Long

Normal

Medium

P

32 Medium

Normal

Normal

Small

P

72 Good

Long

Long

Medium

VG

33 Medium

Normal

Long

Small

P

73 Good

Short

Short

Large

C

34 Medium

Long

Short

Small

P

74 Good

Short

Normal

Large

C

35 Medium

Long

Normal

Small

P

75 Good

Short

Long

Large

C

36 Medium

Long

Long

Small

P

76 Good

Normal

Short

Large

C

37 Medium

Short

Short

Medium

VG

77 Good

Normal

Normal

Large

C

38 Medium

Short

Normal

Medium

VG

78 Good

Normal

Long

Large

B

39 Medium

Short

Long

Medium

C

79 Good

Long

Short

Large

C

40 Medium

Normal

Short

Medium

VG

80 Good

Long

Normal

Large

B

81 Good

Long

Long

Large

B

4.5  Fuzzy Rule Base

75

Fig. 4.6  Fuzzy algorithm example 1

Fig. 4.7  Fuzzy algorithm example 2

Example Rule 4: IF PRK is PRK_PR AND VLD is VLD_NRML AND ARID is ARID_LNG AND NoE is NoE_LRG THEN CLK is B If prior knowledge is poor and video-based learning duration is normal and augmented-­reality interaction duration is long and number of errors is large, then the current level of knowledge is beginner. This rule (rule 4) implies that a student with poor prior knowledge, who has spent anormal time watching the video tutorials, long time in manipulating the 3D models, and scored a large average number of errors in the assessment, is classified as a beginner.

76

4  Fuzzy Logic for Modeling the Knowledge of Users in PARSAT AR Software

Fig. 4.8  Fuzzy algorithm example 3

Fig. 4.9  Fuzzy algorithm example 4

Given the crisp input values are 5, 58, 88, and 81 of the variables PRK, VLD, ARID, and NoE, respectively, the fuzzy output is a beginner student (Fig. 4.9). Example Rule 5: IF PRK is PRK_PR AND VLD is VLD_LNG AND ARID is ARID_NRML AND NoE is NoE_LRG THEN CLK is B Given the crisp input values are 9, 91, 57, and 84 of the variables PRK, VLD, ARID, and NoE, respectively, the output fuzzy output is a beginner student (Fig. 4.10). The next rule (rule 6) indicates that spending normal time in watching the educational tutorials and interacting with the virtual models, results in a competent knowledge rating. Indeed, in Fig. 4.11 the input values are 12, 46, 52, and 82 of the

4.5  Fuzzy Rule Base

77

Fig. 4.10  Fuzzy algorithm example 5

Fig. 4.11  Fuzzy algorithm example 6

variables PRK, VLD, ARID, and NoE, respectively. The values trigger the rule located in the line 23 of Table 4.2, resulting in the output of competent student. Example Rule 6: IF PRK is PRK_PR AND VLD is VLD_NRML AND ARID is ARID_NRML AND NoE is NoE_LRG THEN CLK is C Example rules 7 (Fig. 4.12) and 8 (Fig. 4.13) highlight a very good student and a proficient student, respectively. In the case of a very good student, the inference engine adjudged the input values 52, 19, 58 and 55 for the variables PRK, VLD, ARID, and NoE, respectively.

78

4  Fuzzy Logic for Modeling the Knowledge of Users in PARSAT AR Software

Fig. 4.12  Fuzzy algorithm example 7

Fig. 4.13  Fuzzy algorithm example 8

In the case of a proficient student, the inference engine’s output is due to the input values of 82, 59, 17 and 56 for the variables PRK, VLD, ARID, and NoE, respectively. Example Rule 7: IF PRK is PRK_MDM AND VLD is VLD_SRT AND ARID is ARID_NRML AND NoE is NoE_MDM THEN CLK is VG Example Rule 8: IF PRK is PRK_GD AND VLD is VLD_NRML AND ARID is ARID_SRT AND NoE is NoE_MDM THEN CLK is VG

4.6  Mamdani’s Inference System

79

4.6 Mamdani’s Inference System After fuzzification, the rules are applied to the inputs, and the system’s output is obtained using the fuzzy inference mechanism. The fuzzy inference system (FIS) aggregates the rules and calculates the degree of membership of the output variable for each of the linguistic terms. The output variable’s crisp value is obtained through the defuzzification process. In the proposed system, the centroid method was used for defuzzification (Fig. 4.14). 81 fuzzy rules were designed to classify the inputs into the appropriate output linguistic terms, or student knowledge levels, using the fuzzy inference engine. The output linguistic terms were defined as “Novice”, “Beginner”, “Competent”, “Very Good”, “Proficient”, and “Expert”. The fuzzy inference engine uses the membership functions and fuzzy rules to compute the appropriate degree of membership for each output linguistic term, and then applies the defuzzification method to generate the crisp output value, or the final student knowledge level. There are two main types of fuzzy inference systems: Mamdani FIS and Sugeno FIS. Mamdani fuzzy inference was first introduced as a method to create a control system, by synthesizing a set of linguistic control rules obtained from experienced human operators [30]. In a Mamdani system, the output of each rule is a fuzzy set. Since Mamdani systems have more intuitive and easier to understand rule bases, they are well-suited to expert system applications, where the rules are created from human expert knowledge. The output of each rule is a fuzzy set, derived from the output membership function and the implication method of the FIS. These output fuzzy sets are combined into a single fuzzy set using the aggregation method of the FIS. Then, to compute a final crisp output value, the combined output fuzzy set is defuzzified.

Fig. 4.14  Architecture of the fuzzy inference system

80

4  Fuzzy Logic for Modeling the Knowledge of Users in PARSAT AR Software

Table 4.3  Comparison between Mamdani and Sugeno fuzzy inference system Mamdani FIS There is an output membership function Surface produces a discontinuous output Consequent crisp values are obtained through defuzzification Applied in Multiple Input and Single Output (MISO) and Multiple Input and Multiple Output (MIMO) systems Effectively suited to human input

Sugeno FIS There is no output membership function Surface produces a continuous output There is no defuzzification Applied only in Multiple Input and Single Output (MISO) Effectively suited to mathematical analysis

In a Sugeno fuzzy inference system, also referred to as Takagi-Sugeno-Kang (TSK) fuzzy inference, the output membership functions are represented by constants or linear functions of the input variables, rather than fuzzy sets as in a Mamdani system. The output of a Sugeno system is a crisp value, rather than a fuzzy set. This makes the defuzzification process simpler, as it involves computing a weighted sum or average of a few data points rather than computing the centroid of a fuzzy set [31]. Sugeno systems are often used in applications where the output needs to be a precise numerical value, rather than a fuzzy set. Table 4.3 presents a comparison between Mamdani and Sugeno fuzzy inference system. In this book, the Mamdani FIS is employed, as it is typically used to capture expert knowledge. It enables us to communicate more naturally, while describing the expertise. The fuzzy inputs must be combined into a single fuzzy output, by using the Mamdani inference engine’s fuzzy implication. The fuzzy input variables for each of the rules are then connected using the AND operator. This operator’s function is to extract the minimum membership function value from the fuzzy input variables. Using the value obtained from the input component, the fuzzy output variable is truncated. By taking the maximum value of the membership degree, the entire shortened output is therefore aggregated into a single graph and employed in the final stage of the fuzzy logic system.

4.7 Defuzzification The defuzzifier procedure maps the fuzzy output to a crisp value according to the membership function of the output variable. In order to get the crisp value, a diverse method is required. This defuzzification is not part of the “mathematical fuzzy logic” and various methods are possible [32–34], such as:

4.7 Defuzzification

81

• Center of Sums Method (COS): This method calculates the center of gravity or the center of the area of the fuzzy output set. The defuzzification process is accomplished by computing the weighted average of the values of the output variable for all the fuzzy sets that are active. The COS method involves the following steps: 1. Divide the output variable universe of discourse into a number of discrete points. 2. Evaluate the degree of membership of the fuzzy output set at each of these points. 3. Multiply each point’s degree of membership by the point’s position (i.e., the crisp value of the output variable at that point). 4. Sum the results obtained in step 3. 5. Divide the result obtained in step 4 by the sum of the degrees of membership of the fuzzy output set. The final result obtained in step 5 is the crisp output value of the fuzzy logic system. The COS method is easy to understand and implement, but it may not always provide the best results for all types of fuzzy systems. • Center of gravity (COG)/Centroid of Area (COA) Method: This is another commonly used defuzzification technique. It calculates the center of mass of the fuzzy output’s area, which represents the final crisp value. Mathematically, the COG/COA method can be expressed as:



Crisp output =

∑ ( x ∗ µ ( x )) ∑ µ ( x)

(4.2)

where x is the input value of the fuzzy output variable and μ(x) is the membership function value of the fuzzy output variable at input value x. The numerator calculates the weighted sum of the input values, where the weight is the membership function value of each input value. The denominator calculates the sum of the membership function values. The result is a weighted average of the input values, representing the crisp output value. The COG/COA method is often preferred over the COS method because it takes into account the shape of the fuzzy output’s membership function, rather than just the center point. It also produces a more accurate and stable result for non-­symmetric membership functions. • Center of Area/Bisector of Area Method (BOA): In this method, the centroid or center of gravity of the area under the membership function curve is calculated. The BOA method can be used for any type of membership function, including non-symmetric ones. To calculate the BOA, the output membership function is first divided into small intervals along the x-axis. Then, the area of each interval is calculated by

82

4  Fuzzy Logic for Modeling the Knowledge of Users in PARSAT AR Software

multiplying the membership value of that interval with its width. The total area under the curve is then divided into two equal halves, and the point where the two halves intersect is determined as the BOA. The BOA method is more computationally expensive than the COS method, but it provides better results for non-symmetric membership functions. • Weighted Average Method: This method, also known as the weighted mean method, is a defuzzification method that calculates the crisp output value as the weighted average of the centroid values of the fuzzy sets in the output variable. The formula for calculating the crisp output value using the weighted average method is:



Y=

( Sum of ( wi ∗ yi ) ) ( Sum of wi )

(4.3)

where Y is the crisp output value, wi is the weight of the i-th fuzzy set, and yi is the centroid value of the i-th fuzzy set. The weights can be determined in several ways, such as using the degree of membership of each fuzzy set in the output variable or using the confidence level of each rule that fired during the inference process. • Maxima Methods: This method, also known as the max-of-max method, selects the output value that has the highest degree of membership among all the output membership functions. This method is simple and easy to implement, but it may not be the most accurate method in some cases where there is overlap between the membership functions. • First of Maxima Method (FOM): This method selects the crisp output value as the first value in the output membership function that achieves the maximum membership degree. In other words, the output value corresponding to the first peak of the membership function is chosen. This method is simple to compute but may not provide an accurate representation of the output value, especially if there are multiple peaks in the membership function. • Last of Maximum Method (LOM): This method selects the output value corresponding to the last maximum value in the aggregated output fuzzy set. This method can be useful in situations where it is important to prioritize the higher output values, as the last maximum value corresponds to the highest membership degree. It can be expressed mathematically as:

y = max ( x )

(4.4)

where y is the crisp output value and x is the aggregated fuzzy output set. The maximum function selects the highest value in the set, corresponding to the last maximum membership degree. • Mean of Maximum Method (MOM): This method is a type of defuzzification method used in fuzzy logic. It is a weighted average of the output values corresponding to the highest membership degrees of the output variable.

83

4.7 Defuzzification

In the MOM method, the output variable is first divided into a set of discrete points, and the membership values for each point are computed using the fuzzy inference system. Then, the points with the highest membership degrees are identified, and their corresponding output values are averaged, with the degree of membership serving as the weight for each point. Mathematically, the MOM method can be expressed as:



Σ ( yi ∗ mi )

Crisp Output =

Σ ( mi )

(4.5)

where yi is the output value at the ith point, mi is the membership degree of the ith point, and the summation is taken over all points with non-zero membership degrees. The defuzzification process involves converting a fuzzy set of the aggregated output into a single numerical value. The Centroid of Gravity (COG) method is the most prevalent and physically appealing of all the defuzzification methods [31], which is a popular and physically intuitive method of defuzzification, where the center of gravity of the aggregated output fuzzy set is calculated to obtain a single numerical value. The COG method involves calculating the weighted average of the membership function values of the fuzzy set. This weighted average is calculated using the centroid formula, which is the ratio of the first moment of the fuzzy set to its area. The COG method evaluates the center of area under the curve of the aggregated output fuzzy set. This involves calculating the first moment of the fuzzy set, which is the weighted sum of the input values, and dividing it by the area of the fuzzy set. The resulting value is the center of gravity or the centroid of the fuzzy set, which is used as the defuzzified output. Overall, the COG method provides a simple and intuitive approach to defuzzification and is widely used in various applications of fuzzy logic. The basic principle in the COG method is a centroid approach, which finds the point where a vertical line slices the aggregate set into two equal masses. Consider μ(x) to be the output membership function of the output variable x, then the position of the center of gravity of a fuzzy set A, in the interval ab, is given by (4.6), as follows: b

∫ x µ ( x ) dx COG = ∫ µ ( x ) dx a



A

b

a

(4.6)

A

There are multiple sub-areas within the overall area of the membership function distribution, that are utilized to represent the combined control action. To determine the defuzzified value for a discrete fuzzy set, the area and centroid of each sub-area are determined, and the sum of all these sub-areas is then used.

84

4  Fuzzy Logic for Modeling the Knowledge of Users in PARSAT AR Software

4.8 Adaptation of the Learning Activities Based on Fuzzy Weights Technical drawing assumes a high-level training in spatial ability, while it is achieved by using adaptive learning activities considering the student’s level of knowledge. This is accomplished by converting students’ current knowledge level to fuzzy weights to deliver appropriate both, in quantity and level of difficulty, learning activities. To calculate the six fuzzy weights that represent the student’s knowledge level, membership functions are used. These membership functions are used to determine the extent to which the student’s knowledge level belongs to each of the six fuzzy sets. The resulting values for each fuzzy set are then combined to calculate the sextet that best defines the student’s current level of knowledge. The membership functions which are used to calculate the sextet that best defines the student’s current level of knowledge are presented in Fig. 4.15. By using these membership functions, the approach can determine the student’s current level of knowledge in the domain of technical drawing and provide appropriate learning activities that match the student’s current level of ability. This approach uses fuzzy logic techniques to adapt learning activities to the student’s current level of knowledge and ability, which can potentially improve learning outcomes in the domain of technical drawing. The membership functions discussed in the aforementioned context are limited between 0 and 1. These membership functions are used in both the fuzzification and defuzzification steps of the fuzzy logic system to map non-fuzzy input values to fuzzy linguistic terms and vice versa. In the fuzzification step, non-fuzzy input values are mapped to fuzzy linguistic terms using membership functions. These membership functions determine the degree to which the input values belong to each fuzzy set and assign a membership value between 0 and 1 for each fuzzy set. The resulting membership values represent the degree of membership of the input values in each fuzzy set and are used to form fuzzy sets. In the defuzzification step, the output of the fuzzy logic system is converted from fuzzy linguistic terms back to a non-fuzzy numerical value. This is done by calculating a single numerical value that represents the degree of truth of the aggregated output fuzzy set. This process involves using a defuzzification method, namely the COG method, which uses the membership values of the fuzzy set to calculate a single numerical value that represents the output of the system. The membership functions play a crucial role in the fuzzification and defuzzification steps of the fuzzy logic system. The developed fuzzy logic system uses membership functions which are formed using straight lines, which provide the advantage of simplicity, and specifically, the trapezoidal membership function is being used (Fig. 4.16). The trapezoidal membership function is a common type of fuzzy membership function utilized in fuzzy logic systems. It is recognized by its trapezoid shape, where the membership value

4.8  Adaptation of the Learning Activities Based on Fuzzy Weights

85

Fig. 4.15  Membership functions describing student’s knowledge level

rises from 0 to 1 as the input value progresses from the lower base of the trapezoid to its upper base. The two points where the membership value is 1 define the upper and lower bases, while the sides of the trapezoid are determined by two points where the membership value is 0. By combining trapezoidal membership functions constructed with straight lines, the system can harness the simplicity of linear functions while still capturing the

86

4  Fuzzy Logic for Modeling the Knowledge of Users in PARSAT AR Software

Fig. 4.16  Fuzzy weights of knowledge level in linguistic terms.

intricacy and precision associated with trapezoidal membership functions. This can potentially improve the performance and ease of implementation of the fuzzy logic system. Student’s knowledge level is described by the sextet (N, B, C, VG, P, and E), and as such, the student may be fully assigned to one, or partially assigned to more fuzzy sets, meaning that student’s knowledge level can be described as ‘Competent’ or partially ‘Very Good’ and partially ‘Proficient’, respectively. As an example, a student’s sextet of (0, 1, 0, 0, 0, 0), classifies the student as 100% ‘Beginner’. An additional instance of a student’s sextet, represented as (0, 0, 0, 0.70, 0.30, 0), categorizes the student as 70% ‘Very Good’ and 30% ‘Proficient.’ However, it’s important to note that regardless of the specific values within the sextet, the equation μN(x) + μB(x) + μC(x) + μVG(x) + μP(x) + μE(x) = 1 remains constant.

4.8.1 Decision Making A teaching strategy is being developed that adapts to the student’s knowledge level using fuzzy logic [35–39]. In this section, the analysis of the rules, in combination with the fuzzy weights, is presented to achieve this. The teaching strategy involves dynamically defining the number of learning activities that the student must learn in each chapter based on their current level of knowledge. This is done by analyzing the rules and combining them with the fuzzy

4.8  Adaptation of the Learning Activities Based on Fuzzy Weights

87

weights that represent the student’s knowledge level. The rules define the relationship between the student’s knowledge level and the number of learning activities that they need to complete [40, 41]. By tailoring the teaching approach to match the student’s knowledge level, the system can offer suitable learning tasks that align with the student’s existing skill level. This approach helps prevent situations where the student is either swamped with overly challenging material or disengaged due to overly simplistic content [42, 43]. The formulation of rules is of paramount importance in shaping the quantity and complexity of the learning activities offered to students. In this case, the rules were developed by eight professors who utilized fuzzy rules and related thresholds at the membership functions. The rules are related to technical drawing knowledge levels that students acquire during the course of an entire semester. The fact that all faculty members have over 15 years of experience teaching technical drawing in academic settings indicates that they have a wealth of knowledge and expertise in this area. Their contributions to matching each learning activity with the corresponding SOLO level was significant, as SOLO is a framework for assessing and describing the complexity of student learning outcomes. The use of experienced faculty members to develop rules and match learning activities with SOLO levels suggests a thoughtful and intentional approach to designing the course and assessing student progress. By utilizing the expertise of these instructors, the course can be tailored to the needs of the students and provide a comprehensive and effective learning experience. An example highlights how the rules take into account the different knowledge levels of beginner and expert students. Beginners may need to study many topics of low difficulty (uni-structural and multi-structural level), while experts can focus on fewer topics but at higher levels of complexity (relational and extended abstract). By utilizing the SOLO taxonomy and involving experienced faculty members in the development of the rules and matching of learning activities with SOLO levels, the course has been designed to effectively address the needs of students with varying levels of knowledge and experience in technical drawing. Adopting a deliberate and methodical course design is crucial to establishing well-defined learning goals and aligning course materials and tasks effectively to support students in attaining those objectives. Additionally, it aids in ensuring that students are adequately stimulated and engaged throughout the course, ultimately resulting in improved learning outcomes and student achievement. Table 4.4 contains the complete set of rules, which have been structured regarding the delivered learning activities (LAs). According to Table 4.4, a student who was assigned a crisp output value of 76%, has been classified as partially “very good” and partially “proficient”, and will be delivered learning activities (LAs), as follows: • • • • •

no learning activity of SOLO-L0; no learning activity of SOLO-L1; one learning activities of SOLO-L2; four learning activities of SOLO-L3; and two learning activities of SOLO-L4.

88

4  Fuzzy Logic for Modeling the Knowledge of Users in PARSAT AR Software

Table 4.4  Decision rules for adaptive instruction Current Level of Knowledge μΝ = 1 μΝ < 1 and μΒ < 1 μΒ = 1 μΒ < 1 and μC < 1 μC = 1 μC < 1 and μVG < 1 μVG = 1 μVG < 1 and μP < 1 μP = 1 μP < 1 and μE < 1 μE = 1

L0

L1 9 8 6 5 4 3 1 0 0 0 0

L2 5 5 6 5 4 3 2 1 0 0 0

L3 0 0 0 1 2 3 4 4 3 2 0

L4 0 0 0 0 0 0 1 2 3 3 4

Sum of LAs 14 13 12 11 10 9 8 7 6 5 4

Fig. 4.17  Adaptation of learning activities based on fuzzy weights

Figure 4.17 presents an example of using the decision rules of Table 4.4. The linguistic output fuzzy term is proficient, when PRK, VLD, ARID, and NoE are 83, 57, 19 and 53, respectively. The crisp output of CLK is 83.07, triggering the decision rule of the ninth line of Table 4.4, and as such, the system adapts its delivering content to the student by three learning activities of SOLO-L3 three learning activities of SOLO-L4.

4.9 Summary In this chapter, fuzzy logic was introduced. More specific, the problem was specified, and the linguistic variables were defined. Afterwards, the fuzzy sets were determined, while 81 fuzzy rules were constructed. Fuzzy sets, fuzzy rules and procedures were encoded to perform fuzzy inference, and finally, determine the system’s adaptation as far as the learning activities are considered.

References

89

Overall, the use of fuzzy logic and the specific classifications and learning activities assigned to students based on their output values and SOLO levels suggests a detailed and customized approach to learning and assessment that can be beneficial for students. By tailoring the course to individual students’ needs and knowledge levels, the current proposed system can provide an effective learning experience that helps students achieve their learning objectives.

References 1. C.  Papakostas, C.  Troussas, A.  Krouska, and C.  Sgouropoulou, “PARSAT: Fuzzy logic for adaptive spatial ability training in an augmented reality system.,” Computer Science and Information Systems, vol. 20, no. 4, 2023, https://doi.org/10.2298/CSIS230130043P. 2. C.  Papakostas, C.  Troussas, A.  Krouska, and C.  Sgouropoulou, “Personalization of the Learning Path within an Augmented Reality Spatial Ability Training Application Based on Fuzzy Weights,” Sensors, vol. 22, no. 18, 2022, https://doi.org/10.3390/s22187059. 3. C.  Troussas, C.  Papakostas, A.  Krouska, P.  Mylonas, and C.  Sgouropoulou, “Personalized Feedback Enhanced by Natural Language Processing in Intelligent Tutoring Systems,” in Augmented Intelligence and Intelligent Tutoring Systems, C.  Frasson, P.  Mylonas, and C.  Troussas, Eds., Cham: Springer Nature Switzerland, 2023, pp.  667–677. https://doi. org/10.1007/978-­3-­031-­32883-­1_58. 4. C. Papakostas, C. Troussas, A. Krouska, and C. Sgouropoulou, “Modeling the Knowledge of Users in an Augmented Reality-Based Learning Environment Using Fuzzy Logic,” in Lecture Notes in Networks and Systems, A. Krouska, C. Troussas, and J. Caro, Eds., Cham: Springer International Publishing, 2023, pp. 113–123. https://doi.org/10.1007/978-­3-­031-­17601-­2_12. 5. P. Strousopoulos, C. Papakostas, C. Troussas, A. Krouska, P. Mylonas, and C. Sgouropoulou, “SculptMate: Personalizing Cultural Heritage Experience Using Fuzzy Weights,” in Adjunct Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization, in UMAP ’23 Adjunct. New York, NY, USA: Association for Computing Machinery, 2023, pp. 397–407. https://doi.org/10.1145/3563359.3596667. 6. C. Papakostas, C. Troussas, A. Krouska, and C. Sgouropoulou, “User acceptance of augmented reality welding simulator in engineering training,” Educ Inf Technol (Dordr), vol. 27, no. 1, pp. 791–817, Jan. 2022, https://doi.org/10.1007/s10639-­020-­10418-­7. 7. M. Iakovidis, C. Papakostas, C. Troussas, and C. Sgouropoulou, “Empowering Responsible Digital Citizenship Through an Augmented Reality Educational Game,” in Novel & Intelligent Digital Systems: Proceedings of the 3rd International Conference (NiDS 2023), K. Kabassi, P. Mylonas, and J. Caro, Eds., Cham: Springer Nature Switzerland, 2023, pp. 31–39. 8. P.  Strousopoulos, C.  Troussas, C.  Papakostas, A.  Krouska, and C.  Sgouropoulou, “Revolutionizing Agricultural Education with Virtual Reality and Gamification: A Novel Approach for Enhancing Knowledge Transfer and Skill Acquisition,” in Novel & Intelligent Digital Systems: Proceedings of the 3rd International Conference (NiDS 2023), K. Kabassi, P. Mylonas, and J. Caro, Eds., Cham: Springer Nature Switzerland, 2023, pp. 67–80. 9. C.  Papakostas, C.  Troussas, P.  Douros, M.  Poli, and C.  Sgouropoulou, “CoMoPAR: A Comprehensive Conceptual Model for Designing Personalized Augmented Reality Systems in Education,” in Novel & Intelligent Digital Systems: Proceedings of the 3rd International Conference (NiDS 2023), K. Kabassi, P. Mylonas, and J. Caro, Eds., Cham: Springer Nature Switzerland, 2023, pp. 67–79. 10. C. Papakostas, C. Troussas, A. Krouska, and C. Sgouropoulou, “Exploration of Augmented Reality in Spatial Abilities Training: A Systematic Literature Review for the Last Decade,” Informatics in Education, vol. 20, no. 1, pp.  107–130, Mar. 2021, https://doi.org/10.15388/ infedu.2021.06.

90

4  Fuzzy Logic for Modeling the Knowledge of Users in PARSAT AR Software

11. Z. Kanetaki et al., “Acquiring, Analyzing and Interpreting Knowledge Data for Sustainable Engineering Education: An Experimental Study Using YouTube,” Electronics (Basel), vol. 11, no. 14, 2022, https://doi.org/10.3390/electronics11142210. 12. A. Marougkas, C. Troussas, A. Krouska, and C. Sgouropoulou, “Virtual Reality in Education: A Review of Learning Theories, Approaches and Methodologies for the Last Decade,” Electronics (Basel), vol. 12, no. 13, 2023, https://doi.org/10.3390/electronics12132832. 13. C. Troussas, A. Krouska, and C. Sgouropoulou, “Enriching Mobile Learning Software with Interactive Activities and Motivational Feedback for Advancing Users’ High-Level Cognitive Skills,” Computers, vol. 11, no. 2, 2022, https://doi.org/10.3390/computers11020018. 14. F.  Giannakas, C.  Troussas, A.  Krouska, C.  Sgouropoulou, and I.  Voyiatzis, “XGBoost and Deep Neural Network Comparison: The Case of Teams’ Performance,” in Intelligent Tutoring Systems, A. I. Cristea and C. Troussas, Eds., Cham: Springer International Publishing, 2021, pp. 343–349. 15. A.  Krouska, C.  Troussas, and C.  Sgouropoulou, “Fuzzy Logic for Refining the Evaluation of Learners’ Performance in Online Engineering Education,” European Journal of Engineering Research and Science, vol. 4, pp.  50–56, Jun. 2019, https://doi.org/10.24018/ ejers.2019.4.6.1369. 16. C. Troussas, A. Krouska, and C. Sgouropoulou, “Towards a Reference Model to Ensure the Quality of Massive Open Online Courses and E-Learning,” in Brain Function Assessment in Learning, C.  Frasson, P.  Bamidis, and P.  Vlamos, Eds., Cham: Springer International Publishing, 2020, pp. 169–175. 17. A.  Marougkas, C.  Troussas, A.  Krouska, and C.  Sgouropoulou, “How personalized and effective is immersive virtual reality in education? A systematic literature review for the last decade,” Multimed Tools Appl, 2023, https://doi.org/10.1007/s11042-­023-­15986-­7. 18. A. Marougkas, C. Troussas, A. Krouska, and C. Sgouropoulou, “A Framework for Personalized Fully Immersive Virtual Reality Learning Environments with Gamified Design in Education,” 2021. https://doi.org/10.3233/FAIA210080. 19. C. Troussas, A. Krouska, and C. Sgouropoulou, “Dynamic Detection of Learning Modalities Using Fuzzy Logic in Students’ Interaction Activities,” in Intelligent Tutoring Systems, V. Kumar and C. Troussas, Eds., Cham: Springer International Publishing, 2020, pp. 205–213. 20. A.  Krouska, C.  Troussas, and C.  Sgouropoulou, “A novel group recommender system for domain-independent decision support customizing a grouping genetic algorithm,” User Model User-adapt Interact, 2023, https://doi.org/10.1007/s11257-­023-­09360-­3. 21. A. Krouska, C. Troussas, K. Kabassi, and C. Sgouropoulou, “An Empirical Investigation of User Acceptance of Personalized Mobile Software for Sustainability Education,” Int J Hum Comput Interact, pp. 1–8, Aug. 2023, https://doi.org/10.1080/10447318.2023.2241614. 22. T. Hailikari, N. Katajavuori, and S. Lindblom-Ylänne, “The Relevance of Prior Knowledge in Learning and Instructional Design,” Am J Pharm Educ, vol. 72, p. 113, Nov. 2008, https://doi. org/10.5688/aj7205113. 23. N. Medina-Medina and L. García-Cabrera, “A taxonomy for user models in adaptive systems: special considerations for learning environments,” Knowl Eng Rev, vol. 31, pp. 124–141, Mar. 2016, https://doi.org/10.1017/S0269888916000035. 24. C.  Papakostas, C.  Troussas, A.  Krouska, and C.  Sgouropoulou, “On the development of a personalized augmented reality spatial ability training mobile application,” in Frontiers in Artificial Intelligence and Applications, IOS Press, 2021, pp. V–VI. https://doi.org/10.3233/ FAIA210078. 25. E.  Mousavinasab, N.  Zarifsanaiey, S.  R. Niakan Kalhori, M.  Rakhshan, L.  Keikha, and M. Ghazi Saeedi, “Intelligent tutoring systems: a systematic review of characteristics, applications, and evaluation methods,” Interactive Learning Environments, vol. 29, no. 1, pp. 142–163, Jan. 2021, https://doi.org/10.1080/10494820.2018.1558257. 26. A. Krouska, C. Troussas, and C. Sgouropoulou, “Fuzzy logic for refining the evaluation of learners’ performance in online engineering education,” European Journal of Engineering and Technology Research, vol. 4, no. 6, pp. 50–56, 2019.

References

91

27. C.  Troussas, A.  Krouska, C.  Sgouropoulou, and I.  Voyiatzis, “Ensemble Learning Using Fuzzy Weights to Improve Learning Style Identification for Adapted Instructional Routines,” Entropy, vol. 22, no. 7, 2020, https://doi.org/10.3390/e22070735. 28. C.  Troussas, F.  Giannakas, C.  Sgouropoulou, and I.  Voyiatzis, “Collaborative activities recommendation based on students’ collaborative learning styles using ANN and WSM,” Interactive Learning Environments, pp.  1–14, May 2020, https://doi.org/10.1080/1049482 0.2020.1761835. 29. Y. H. Kim, S. C. Ahn, and W. H. Kwon, “Computational complexity of general fuzzy logic control and its simplification for a loop controller,” Fuzzy Sets Syst, vol. 111, no. 2, pp. 215–224, Apr. 2000, https://doi.org/10.1016/S0165-­0114(97)00409-­0. 30. E.  H. Mamdani and S.  Assilian, “An experiment in linguistic synthesis with a fuzzy logic controller,” Int J Man Mach Stud, vol. 7, no. 1, pp.  1–13, 1975, https://doi.org/10.1016/ S0020-­7373(75)80002-­2. 31. T. Takagi and M. Sugeno, “Fuzzy identification of systems and its applications to modeling and control,” IEEE Trans Syst Man Cybern, vol. SMC-15, no. 1, pp. 116–132, 1985, https:// doi.org/10.1109/TSMC.1985.6313399. 32. H. Diab, “Defuzzification methods and new techniques for fuzzy controllers,” Iranian Journal of Electrical and Computer Engineering, vol. 3, Jul. 2004. 33. A.  Chandramohan, M.  V. C.  Rao, and M.  Senthil Arumugam, “Two New and Useful Defuzzification Methods Based on Root Mean Square Value,” Soft comput, vol. 10, no. 11, pp. 1047–1059, 2006, https://doi.org/10.1007/s00500-­005-­0042-­6. 34. N.  Mogharreban and L.  Dilalla, Comparison of Defuzzification Techniques for Analysis of Non-interval Data. 2006. https://doi.org/10.1109/NAFIPS.2006.365418. 35. N. Elghouch, M. Kouissi, and E.-N. el Mokhtar, “Multi-Agent System of an Adaptive Learning Hypermedia Based on Incremental Hybrid Case-Based Reasoning,” 2020, pp.  143–156. https://doi.org/10.1007/978-­3-­030-­37629-­1_12. 36. T.  K. F.  Chiu and I.  Mok, “Learner expertise and mathematics different order thinking skills in multimedia learning,” Comput Educ, vol. 107, pp. 147–164, Apr. 2017, https://doi. org/10.1016/j.compedu.2017.01.008. 37. A.  Khamparia and B.  Pandey, “SVM and PCA Based Learning Feature Classification Approaches for E-Learning System,” International Journal of Web-Based Learning and Teaching Technologies, vol. 13, pp.  32–45, Apr. 2018, https://doi.org/10.4018/ IJWLTT.2018040103. 38. C.  Troussas, A.  Krouska, F.  Giannakas, C.  Sgouropoulou, and I.  Voyiatzis, “Redesigning Teaching Strategies through an Information Filtering System,” in 24th Pan-Hellenic Conference on Informatics, in PCI 2020. New York, NY, USA: Association for Computing Machinery, 2021, pp. 111–114. https://doi.org/10.1145/3437120.3437287. 39. A. Krouska, C. Troussas, A. Voulodimos, and C. Sgouropoulou, “A 2-tier fuzzy control system for grade adjustment based on students’ social interactions,” Expert Syst Appl, vol. 203, p. 117503, 2022, https://doi.org/10.1016/j.eswa.2022.117503. 40. C.  Troussas, A.  Krouska, and C.  Sgouropoulou, “Improving Learner-Computer Interaction through Intelligent Learning Material Delivery Using Instructional Design Modeling,” Entropy, vol. 23, no. 6, 2021, https://doi.org/10.3390/e23060668. 41. C. Troussas, A. Krouska, and C. Sgouropoulou, “A Novel Teaching Strategy Through Adaptive Learning Activities for Computer Programming,” IEEE Transactions on Education, vol. 64, no. 2, pp. 103–109, 2021, https://doi.org/10.1109/TE.2020.3012744. 42. C. Papakostas, C. Troussas, A. Krouska, and C. Sgouropoulou, “Exploring Users’ Behavioral Intention to Adopt Mobile Augmented Reality in Education through an Extended Technology Acceptance Model,” Int J Hum Comput Interact, vol. 39, no. 6, pp. 1294–1302, 2023, https:// doi.org/10.1080/10447318.2022.2062551. 43. C. Papakostas, C. Troussas, A. Krouska, and C. Sgouropoulou, “Measuring User Experience, Usability and Interactivity of a Personalized Mobile Augmented Reality Training System,” Sensors, vol. 21, no. 11, p. 3888, Jun. 2021, https://doi.org/10.3390/s21113888.

Chapter 5

Artificial Intelligence-Enhanced PARSAT AR Software: Architecture and Implementation

Abstract  This chapter offers a comprehensive exploration of the architecture and practical implementation of a mobile training system, enriched with augmented reality (AR) features and adaptive capabilities based on fuzzy logic. It commences with an overview that encapsulates the core concepts and objectives of the system, followed by a detailed exposition of its structural underpinnings. The “Overview” section provides a high-level synopsis of the mobile training system’s architecture, with a specific emphasis on its integration of AR features and its pivotal role in enhancing spatial ability training. The subsequent section, “System Architecture,” conducts an intricate examination of the system’s architecture, delineating the distinct layers involved, namely the hardware layer, software layer, and data layer. This section elucidates the interplay of components within each layer, fostering a comprehensive understanding of the system’s holistic structure. The “Hardware Layer” section delves into the physical components of the system, elucidating their roles in tracking user movements, computational processes, and facilitating user interactions in real-time. In “Software Layer,” the focus shifts to the software components, encompassing the user interface’s interactive capabilities and the 3D rendering engine’s pivotal role in creating and presenting virtual elements within the real-­ world context. The “Data Layer” section addresses data storage and management, encompassing marker databases for AR tracking, 3D models repositories, and interaction models defining system rules and behaviors. The chapter further illuminates the practical implementation of the system, specifically detailing the user interface’s design and its interaction with AR learning activities. Additionally, it elucidates the incorporation of a fuzzy logic controller through C# scripting, facilitating adaptive learning based on fuzzy weight parameters.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 C. Papakostas et al., Special Topics in Artificial Intelligence and Augmented Reality, Cognitive Technologies, https://doi.org/10.1007/978-3-031-52005-1_5

93

94

5  Artificial Intelligence-Enhanced PARSAT AR Software: Architecture…

5.1 Overview This chapter provides a comprehensive overview of a mobile training system featuring augmented reality (AR) capabilities and adaptive functionality driven by fuzzy logic, including an exploration of its architecture, hardware and software layers, and practical implementation [1–14]. The current chapter provides a thorough view of the system’s architecture, analyzing all the layers involved in the training application, while the second section of the chapter presents the system’s implementation.

5.2 System Architecture The research described in the current book involves the implementation of the novel approach of a spatial ability training system, incorporating fuzzy logic for the automatic recognition of the students’ knowledge level, and augmented reality digital technology for the spatial ability training. In this study, we present PARSAT [15, 16], a pioneering mobile platform designed for the enhancement of spatial skills through the utilization of augmented reality (AR) technology. The core essence of our AR system is predicated upon a harmonious convergence of intricate hardware and software constituents, which are intricately complemented by an expansive data repository. This repository is meticulously curated to provide exhaustive descriptions that seamlessly bridge the realms of physical reality and virtual content, thereby offering an immersive and integrated AR experience. Figure 5.1 illustrates the layout of PARSAT’s architecture, which is organized into three distinct layers: the upper section houses the hardware components, the middle section is dedicated to the software components, and the lower half is where the data layer is situated. The utilization of fuzzy logic for automatic assessment of students’ knowledge levels is particularly intriguing. This approach offers a more adaptable and flexible training system capable of catering to the unique needs and abilities of individual students. Augmented reality technology also holds substantial potential in the realm of spatial ability training, enabling students to engage with virtual objects in a more immersive and lifelike manner. In summary, the book presents an innovative and promising approach to spatial ability training. Its findings and implications hold substantial promise for education and training across various domains, especially those that rely on robust spatial reasoning skills.

5.2  System Architecture

95

Fig. 5.1  The architecture of PARSAT

5.2.1 Hardware Layer 5.2.1.1 Tracking Within the context of this book, a diverse array of sensors, comprising the accelerometer, gyroscope, magnetometer, and GPS, assumes a central role in the establishment of the system’s spatial position and orientation. This orchestration of sensors is instrumental in ensuring the seamless alignment of virtual content with the

96

5  Artificial Intelligence-Enhanced PARSAT AR Software: Architecture…

physical environment. Importantly, the prevailing landscape of contemporary mobile devices, inclusive of smartphones and tablets, uniformly incorporates these sensors as integral components. These sensors are consistently featured in contemporary mobile devices, thereby endowing them with the intrinsic capability to facilitate augmented reality applications that seamlessly superimpose virtual data onto the tangible reality. Notably, the accelerometer functions to quantitatively assess changes in acceleration and tilt, while the gyroscope is pivotal in quantifying rotations and orientation shifts. The magnetometer, in turn, is dedicated to the precise measurement of magnetic fields, whereas the GPS system provides indispensable geolocation data. This collective integration of sensors collaborates in real-time to furnish precise and immediate information pertaining to the dynamic position and orientation of the device. It is imperative to underscore that such precision assumes a paramount role in the realization of an uninterrupted and fully immersive augmented reality experience for end-users. 5.2.1.2 Processing Within the framework of the AR system, it is imperative to acknowledge that critical computational processes, namely the fuzzy inference system, 3D rendering, and the overarching user interface, impose substantial demands on hardware resources. Consequently, contemporary mobile devices are fortified with potent processing units, firmly establishing them as capable platforms for the execution of these computational tasks. These mobile devices, typically exemplified by smartphones and tablets, house robust processors that encompass both central processing units (CPUs) and graphics processing units (GPUs). The CPUs, constituting the brains of these devices, are adept at executing user instructions and adeptly overseeing the allocation of system resources. In parallel, GPUs assume the mantle of processing extensive volumes of graphical data with swiftness and precision. This tandem operation between CPU and GPU is pivotal in ensuring the seamless functioning of our augmented reality system and the delivery of an uninterrupted and engaging user experience. In essence, this hardware synergy underscores the competence of contemporary mobile devices as formidable computational powerhouses, well-suited for the intricate demands of 3D rendering, fuzzy logic inference, and other resource-intensive computational facets intrinsic to augmented reality technology. This symbiotic relationship between hardware and software represents a cornerstone of the system’s capacity to realize the AR experience.

5.2  System Architecture

97

5.2.1.3 Interacting This element includes various sensors like touch surfaces, gesture recognition, and biometric sensors, all of which interpret the user’s engagement with the system. The tactile sensor is integrated into PARSAT so that the identified information is due to the contact of the student’s fingers on the mobile screen. All mobile devices support touch commands, which are identified by key components, such as a tactile sensor. Tactile sensors are widely employed in mobile devices to perceive touch inputs, enhancing the interactivity of the user experience. Whenever a user interacts with a mobile device’s screen, the tactile sensor identifies the touch and transmits a signal to the device’s processor. Subsequently, the processor interprets this signal, triggering particular actions like launching an application or navigating through a webpage. Gesture recognition and biometrics are also becoming more common in mobile devices, allowing for more advanced and secure methods of user interaction.

5.2.2 Software Layer 5.2.2.1 User Interface A user interface’s success is determined by how discretely people may use it, without interruptions from other interface components. In the context of AR applications, this is also accurate. Due to AR’s immersive and captivating nature, PARSAT’s user interface aims to focus on how students engage with the system. The primary objective of PARSAT’s user interface is to furnish students with an immersive and uninterrupted learning experience, concurrently mitigating potential distractions that might impede the learning process. This endeavor is underpinned by a deliberate intent to reduce any extraneous interruptions, thereby optimizing the efficacy of the learning environment. The design philosophy governing the user interface is rooted in principles of intuitiveness and accessibility. It is meticulously crafted to facilitate user navigation, offering lucid and concise guidance to steer students through the training modules seamlessly. AR technology, integral to PARSAT’s interface, contributes to the realization of a more natural and instinctive user interaction paradigm. Students engage with virtual objects in a manner that mirrors their interaction with real-world counterparts, employing familiar gestures and movements. The incorporation of tactile sensors and gesture recognition further elevates the user experience, enabling students to engage with the system in an intuitive and organic manner. In summation, PARSAT’s user interface is unwavering in its commitment to prioritizing user-friendliness, intuitiveness, and immersive qualities. By meticulously addressing these facets, it ensures that students are positioned to embark on an engaging and highly effective learning journey, characterized by minimal

98

5  Artificial Intelligence-Enhanced PARSAT AR Software: Architecture…

disruptions and maximal educational benefit. In summary, PARSAT’s user interface prioritizes user-friendliness, intuitiveness, and immersion, ensuring that students benefit from an engaging and effective learning journey with minimal disruptions. PARSAT’s user interface is designed by focusing on the essential User Interface (UI)—User Experience (UX) AR pillars [17–19] as listed below: • Environment: in the realm of AR application design, meticulous attention to the environmental context within which users engage with the application is imperative[20, 21]. This encompasses a comprehensive evaluation of various environmental facets, encompassing lighting conditions, spatial configuration, and the safety parameters governing user interactions. Within the context of PARSAT, an AR-based educational platform, students were afforded the opportunity to interact with the application within the controlled confines of their university laboratories, environments thoughtfully designed to optimize user ergonomics and safety. The contextual backdrop within which AR applications are deployed bears profound implications, influencing both the application’s performance and the quality of the user experience. Notably, the interplay between environmental factors and application functionality is a pivotal determinant of efficacy. Factors such as lighting levels and spatial dimensions must be meticulously calibrated to harmonize with the intended application usage, thereby ensuring a congruent and conducive environment. In the case of PARSAT, the tailored design was intrinsically aligned with its designated utilization within university laboratories. These settings are notably characterized by a deliberate emphasis on user comfort and safety. Consequently, lighting conditions and spatial layouts within these environments are attuned to the precise requisites of the application, fostering an optimal user experience. Furthermore, stringent safety protocols are typically instituted to safeguard students’ well-being during application usage. The conscientious consideration of the environmental milieu in which the application operates underscores the strategic design approach adopted by PARSAT.  This prescient approach ensures that the application seamlessly integrates into its designated environment, facilitating both safe and effective utilization as envisaged. • Interaction design: this parameter is also crucial, as the interaction design determines how the user interacts with the context of PARSAT. The primary gestures essential for operating the application and enhancing the AR experience include: –– Tapping, executed with a gentle touch of the student’s finger, primarily employed for pressing buttons and making selections. –– Double tapping, utilized to zoom in on 3D models. –– Pinching, involving two fingers brought close together or spread apart to adjust the size of 3D models. –– Rotating, the fundamental gesture for comprehending the spatial aspects of 3D models from various angles, thereby revealing hidden views. These interactions are of paramount importance and necessitate thoughtful design to ensure they are intuitive, user-friendly, and efficacious in achieving the desired outcomes. The gestures should feel natural and comfortable to the user, and the

5.2  System Architecture

99

system should offer clear visual feedback to elucidate how user actions impact the AR environment. Furthermore, interaction design should maintain consistency throughout different sections of the application to prevent confusion and facilitate rapid user comprehension and mastery of the system. • Colors: The discipline of color theory, widely acknowledged in diverse design domains encompassing print, mobile applications, and web interfaces, bears equal relevance in the domain of AR design. In the context of PARSAT, an educational AR platform, the choice of colors has been scrupulously tailored to align with its pedagogical objectives. This strategic color selection ensures that textual content remains clearly discernible, with fonts chosen judiciously to optimize readability. It is noteworthy that the selection of font type may vary depending on contextual factors, with San Serif fonts often favored for enhanced legibility. Critical to the effectiveness of color selection in AR applications is the establishment of optimal contrast schemes. This typically entails the utilization of light text on a dark background, a configuration empirically recognized as conducive to comfortable reading experiences. It is imperative to underscore that the curation of color palettes within AR applications necessitates a comprehensive assessment of various parameters. Environmental factors, including ambient lighting conditions specific to the application’s usage setting, merit meticulous consideration. Additionally, user-centric factors, including visual abilities and individual preferences, play a significant role in color selection. For instance, high-contrast color schemes may be more suitable for users with visual impairments, ensuring enhanced visibility. Conversely, subdued and muted color palettes may be preferred in scenarios where a tranquil and less distracting user experience is deemed essential. In summation, the selection of colors in AR applications, exemplified by PARSAT, transcends aesthetics to embrace functional and contextual considerations. Each color choice is not only a visual element but a strategic component in the optimization of user experience, aligning closely with the specific educational objectives and the nuances of the AR application’s operational context. Such considerations underscore the interdisciplinary nature of AR design, where visual and functional parameters intersect to cultivate engaging and effective user interactions. • Feedback: it is a critical parameter which is considered, defining how students will be informed of their activities and the results or outcome of those actions. Whether it is the feedback on the assessment score, or feedback encouraging the student to continue the effort on training, it is a parameter which adaptive systems usually integrate. Feedback is an important parameter in any interactive system, and especially in adaptive systems like PARSAT. Feedback serves as a motivational tool for students, assisting them in monitoring their advancement and pinpointing areas requiring enhancement. Feedback can be conveyed in diverse formats, including visual, auditory, or haptic modes. For example, in PARSAT, the system provides visual feedback by highlighting the correct or incorrect answers, and by displaying a progress bar indicating the completion of a task. Auditory feedback could be provided through sounds or speech, such as

100

5  Artificial Intelligence-Enhanced PARSAT AR Software: Architecture…

congratulatory remarks when the student successfully completes a task. Haptic feedback could be provided through vibrations or touch, such as a vibration when a button is pressed. 5.2.2.2 3D Rendering Engine The 3D rendering engine is a combination of the software integrated into the PARSAT application. More specifically, this engine maintains an internal 3D representation of the virtual scene augmenting the real world. This internal representation is updated in real-time according to several factors such as the user’s profile, student’s interactions, the 3D objects’ behavior, the updated knowledge domain, and the fuzzy inference adaptation. Both the hardware components, including the CPU and GPU, as well as the data components, are dedicated to empowering the 3D rendering engine for crafting the user interface screens. The 3D rendering engine holds the responsibility of crafting the 3D graphics presented on the screen, utilizing a blend of algorithms and mathematical models. It takes into consideration the user’s viewpoint and movements to craft an authentic and immersive encounter. Additionally, the engine employs lighting and shading effects to infuse depth and realism into virtual objects. The engine is frequently fine-tuned to maximize the utilization of accessible hardware resources, notably the GPU, to ensure a fluid and responsive graphical performance. Data components, such as 3D models and textures, are stored in the device’s memory and are summoned by the engine as required. In the context of PARSAT, the 3D rendering engine stands as a pivotal component in delivering an engaging and interactive learning journey. It empowers students to delve into and interact with 3D models and visualizations, thereby enriching their comprehension of intricate concepts. PARSAT was implemented in a HP Pavilion TP01-2014nv PC (Ryzen 7-5700G/8GB DDR4/512GB SSD/GeForce GTX 1650/W10, to which a large format monitor was connected. The Ryzen 7-5700G processor and 8GB of DDR4 memory provided good performance for running Unity and other software. The 512GB SSD also helped with faster boot times and loading of large files. The GeForce GTX 1650 graphics card, classified as a mid-range GPU, exhibited commendable performance in executing PARSAT with a high level of graphical fidelity, particularly when coupled with a spacious display monitor. In the development of PARSAT on the PC platform, a comprehensive software toolkit played a pivotal role, comprising several integral components. Noteworthy among these components were the utilization of the Unity 3D cross-platform game engine, specifically version 2020.3.43f1 LTS, alongside the Vuforia Software Development Kit (SDK), Autodesk 3ds Max 2020, meticulously crafted custom scripts in the C# programming language, the esteemed Visual Studio 2019 integrated development environment (IDE), and the Android Studio Integrated Development Environment (IDE).

5.2  System Architecture

101

Unity 3D distinguishes itself by virtue of its extensive array of tools and functionalities, catering to the needs of developers and designers engaged in the creation, design, and animation of 3D elements. These tools empower creators by providing features such as a robust audio system, networking capabilities, and a sophisticated physics engine. The audio system seamlessly integrates soundscapes and audio effects, enriching the immersive quality of applications developed using this platform. Networking capabilities inherent to Unity 3D foster the seamless implementation of multiplayer experiences, substantially enhancing the platform’s versatility. This specific feature plays a pivotal role in the development of online multiplayer games and collaborative virtual environments, thereby expanding the scope of Unity 3D beyond traditional gaming. A pivotal advantage of Unity lies in its cross-platform compatibility, enabling developers to produce applications that seamlessly function across multiple platforms. This compatibility, complemented by a unified codebase, streamlines the development process, curtails time-to-market, and eliminates the necessity for extensive platform-specific modifications. It empowers developers to efficiently target a diverse array of devices. Unity 3D’s unwavering commitment to continual evolution is clearly evident through its regular iterative updates, introducing fresh features and enhancements. This unwavering dedication ensures that Unity 3D consistently maintains its position at the forefront of 3D content creation, adapting adeptly to evolving industry trends and technological advancements. Unity 3D finds itself in direct competition with Unreal Engine, another formidable contender in the game engine and application development sphere. While both engines offer robust capabilities for crafting interactive 3D content, each possesses its own distinctive strengths and limitations. Unity is highly regarded for its user-friendly interface, which fosters accessibility for developers across different skill levels. Its intuitive design and visual scripting tools simplify the development process, enabling swift prototyping and experimentation. This accessibility has significantly contributed to the widespread adoption of Unity, particularly among indie developers. In contrast, Unreal Engine has garnered renowned for its advanced graphics capabilities and extensive utilization of the C++ scripting language. It excels in rendering photorealistic visuals and lends robust support to high-end graphics and rendering pipelines, making it the preferred choice for AAA game development and the creation of high-fidelity simulations. It is imperative to acknowledge that Unity 3D and Unreal Engine represent merely a fraction of the available game engines, as numerous alternatives exist for developers seeking to craft interactive 3D content. Prominent among these alternatives are Godot, CryEngine, and GameMaker Studio, among others. Ultimately, the selection of an engine hinges upon the specific requirements and preferences of the development team. In summary, Unity 3D and Unreal Engine stand as potent game engines and application development platforms, facilitating the creation of interactive 3D

102

5  Artificial Intelligence-Enhanced PARSAT AR Software: Architecture…

content. The ensuing discussion provides a brief comparative overview of these two engines, emphasizing their respective strengths and distinctive attributes: • Ease of Use: Unity 3D is widely acknowledged for its user-friendliness and lower learning curve, making it an attractive choice for beginners. In contrast, Unreal Engine may appear more intricate and challenging to newcomers due to its advanced feature set and interface. • Graphics: Unreal Engine is celebrated for its cutting-edge graphical prowess, excelling in rendering photorealistic environments with intricate lighting and shadow effects. In contrast, Unity 3D, though making substantial progress in enhancing its graphics, has not achieved the same advanced level of realism as Unreal Engine. • Scripting Languages: Unity 3D predominantly employs the C# scripting language, known for its accessibility and productivity benefits, particularly for smaller development teams. Unreal Engine, on the flip side, utilizes C++, a language known for its power and adaptability, although it comes with a more challenging learning curve. • Licensing: Unity 3D provides a range of licensing options, including a free version with limited features and paid versions offering enhanced features and support. In contrast, Unreal Engine offers no initial cost for usage but mandates developers to pay a royalty fee based on project revenue. Platforms: Unity 3D stands as a cross-platform engine, accommodating an extensive array of platforms, encompassing desktop, mobile, console, and virtual/augmented reality. Unreal Engine is also versatile in platform support but is particularly suited for high-end PC and console development. In conclusion, the choice between Unity 3D and Unreal Engine is contingent upon the unique requirements and preferences of the development team. Each engine presents its own set of advantages and drawbacks, with the capacity to deliver exceptional interactive 3D content. PARSAT was integrated with Unity 3D. Unity 3D stands as a potent and versatile game engine ideal for crafting interactive 3D applications, encompassing simulations and educational content such as PARSAT. Leveraging Unity 3D equips developers with an array of resources and functionalities to construct captivating 3D worlds. These resources include a visual editor, a robust scripting language (C#), and compatibility with numerous platforms and devices. In addition to Unity 3D, the development of PARSAT encompassed the utilization of the Vuforia Software Development Kit (SDK). Within the augmented reality (AR) development domain, Vuforia faces competition from several notable counterparts, including: –– ARToolKit: 1 An open-source AR development kit renowned for its longevity since its inception in the late 1990s. ARToolKit offers a comprehensive suite of features, such as marker-based tracking, image recognition, and 3D model ren http://www.hitl.washington.edu/artoolkit.html

1

5.2  System Architecture

––

–– ––

––

103

dering. It maintains an active community of developers, ensuring ongoing maintenance and support. Wikitude: 2 A versatile AR development kit that supports marker-based tracking, image recognition, and location-based AR. Wikitude provides tools for AR content creation and management, accompanied by SDKs compatible with multiple platforms, including iOS, Android, and Unity. EasyAR: 3 Another AR development kit offering marker-based tracking, image recognition, and 3D model rendering. EasyAR includes a visual editor and extends support to various platforms, such as iOS, Android, and Unity. Kudan: 4 An AR development kit specializing in marker-based tracking, image recognition, and 3D model rendering. Kudan also provides tools for AR content creation and management, along with SDKs for platforms like iOS, Android, and Unity. Maxst: 5 Maxst is an AR development kit offering marker-based tracking, image recognition, and 3D model rendering. It features a visual editor and caters to multiple platforms, including iOS, Android, and Unity.

It’s crucial to recognize that these are just a few options among the many AR development kits available today. Selecting a specific development kit depends on various factors, such as project needs, the intended platform, and the developer’s skills and preferences. PARSAT was seamlessly incorporated with Vuforia due to its widespread popularity and strong support, offering an array of features and extensive platform compatibility, rendering it a highly suitable option for numerous AR projects. Unity supports C# and Python as scripting languages for game development. A brief analysis of each is as follows: • C#: 6 C# is a prevalent object-oriented programming language created by Microsoft. It enjoys popularity in game development due to its user-friendly syntax, which bears similarities to Java and C++, making it an accessible choice. Moreover, C# finds robust support within Unity, boasting a sizable community of developers and an abundance of tools and resources. • Python: 7 Python is a general-purpose programming language that is known for its simplicity and ease of use Python is a versatile programming language with widespread use in data science, machine learning, and web development. In the context of PARSAT, the decision to adopt C# as the primary programming language was based on its extensive adoption and prominence within the Unity ecosystem. This choice was underpinned by C#’s commendable performance  https://www.wikitude.com/  https://www.easyar.com/ 4  https://www.kudan.io/ 5  https://developer.maxst.com/ 6  https://learn.microsoft.com/en-us/dotnet/csharp/ 7  https://www.python.org/ 2 3

104

5  Artificial Intelligence-Enhanced PARSAT AR Software: Architecture…

attributes and the robust support it enjoys within the Unity game engine. Python can serve as a valuable tool for certain tasks, especially for developers already proficient in it. Furthermore, in the developmental process of PARSAT, Visual Studio was employed as the designated integrated development environment (IDE), a selection frequently endorsed for the creation of diverse software applications, encompassing game development. It is noteworthy that Visual Studio assumes the role of the recommended IDE for C# scripting within the Unity framework. It is pertinent to acknowledge the existence of alternative IDEs endowed with commensurate functionality and features, thereby warranting contemplation: • Eclipse: 8 Eclipse is a free and open-source IDE that is widely used for Java development, but also supports other programming languages such as C++, Python, and PHP. • IntelliJ IDEA: 9 IntelliJ IDEA is a popular IDE for Java development that offers a range of features including code completion, debugging, and testing. • NetBeans: 10 NetBeans is a free and open-source IDE that supports several programming languages including Java, C++, and PHP. Ultimately, the choice of the integrated development environment (IDE) was contingent upon the specific requirements of the project and the preferences and expertise of the developer. C# is a programming language that is used when developing PARSAT in Unity for scripting gameplay logic, fuzzy logic, Mamdani’s inference algorithm, decision making with fuzzy weights, user interfaces, and other components of games. Visual Studio offers an array of features and tools designed to facilitate the creation, debugging, and management of C# code in Unity, encompassing: • IntelliSense: Visual Studio’s IntelliSense feature provides code completion and suggestions as you type, making it easier to write code and avoid errors. • Debugging tools: Visual Studio provides a range of debugging tools, including breakpoints, call stacks, and exception handling, that make it easier to identify and fix errors in your code. • Integrated development: Visual Studio integrates seamlessly with Unity, allowing you to edit and debug your scripts directly in the Unity editor. • Community support: Visual Studio has a large and active community of developers, with a range of resources and tools available for Unity developers. In summary, the combination of Visual Studio and C# within Unity offers a potent and adaptable development environment for crafting games and interactive applications. This pairing aids in streamlining the development journey, enhancing code quality, and optimizing performance.

 https://www.eclipse.org/  https://www.jetbrains.com/idea/ 10  https://netbeans.apache.org/ 8 9

5.2  System Architecture

105

These tools collectively played a pivotal role in enabling the developer to craft and enhance various facets of the PARSAT application, including its augmented reality features, 3D models, and user interfaces [22–27]. Altogether, the PARSAT development harnessed a comprehensive array of tools and technologies to deliver a top-tier educational experience that is engaging and of high quality. Figure 5.2 presents the combination of the software tools used for developing PARSAT. Through the synergistic use of these software tools, the PARSAT developer successfully crafted an interactive 3D application seamlessly integrating AR technology, optimized for Android devices. The amalgamation of Unity 3D, Vuforia SDK, Autodesk 3ds Max, C#, and Visual Studio furnished an all-encompassing toolkit for the complete lifecycle of designing, developing, testing, and deploying PARSAT. Evaluating PARSAT on mobile devices is crucial for assessing its practicality and effectiveness. However, it’s essential to acknowledge that results may not apply universally to all mobile devices due to disparities in hardware and software specifications. One common challenge in implementing AR solutions is the disparity in technology across AR devices. Different mobile devices exhibit varying hardware and

Fig. 5.2  Combination of the software tools used for developing PARSAT

106

5  Artificial Intelligence-Enhanced PARSAT AR Software: Architecture…

software capabilities, resulting in inconsistent performance and user experiences. This poses a significant challenge for developers striving to create applications that seamlessly function across a spectrum of devices. In the case of PARSAT, the comprehensive AR design faced compatibility issues with certain students’ smartphones due to hardware limitations. This underscores the importance of considering device compatibility during the design and development of AR applications. Developers may need to adapt their applications to different hardware configurations or narrow down the scope to ensure broader compatibility. To address this challenge, the PARSAT author resolved the issue by installing the application on personal tablet devices (specifically, three Samsung Galaxy Tab S5e 10.5″) that met the necessary hardware and software prerequisites for effective operation.

5.2.3 Data Layer 5.2.3.1 Marker Database PARSAT integrates marker-based AR, requiring a trigger image or a QR code to activate the AR experience. The student detects and scans the marker using the mobile device’s camera, the image is identified as a marker, and then, the device renders the virtual content on top of the marker. This feature allows the student to move around the marker and observe the perspectives of the 3D content. The incorporation of marker-based AR in PARSAT enhances the learning experience for students, offering a more interactive and immersive approach. This hands­on engagement fosters improved comprehension and retention of the subject matter. PARSAT’s ability to enable students to navigate around markers and view 3D content from various perspectives allows for a more intuitive exploration of intricate concepts. Overall, the use of marker-based AR in PARSAT can enhance the learning experience and help to make education more engaging and effective. Cloud-based or device-localized are the two categories of marker-based AR. In the first category, since the AR assets must be downloaded from the server, a cloud-­ based AR experience may require a few additional minutes to load. In the second category, since the AR assets have already been pre-downloaded to the student’s mobile device via the application, a localized AR experience may be accessed instantly. For greater storage capacity, the choice of the cloud-based AR is preferred, but localized AR is less expensive and not dependent on network availability. PARSAT integrates device-localized marker-based AR, which means that the AR assets are pre-downloaded onto the user’s device via the application, enabling instant access to the AR experience. This approach is less dependent on network availability, making it a suitable choice for educational environments where internet connectivity may not always be reliable. Additionally, since the AR assets are stored

5.2  System Architecture

107

locally, the device-localized approach can be more cost-effective than cloud-based AR, as there is no need for additional storage or bandwidth on the server side. The marker-based AR experience is created using a software development kit (SDK), namely Vuforia, one of the best-known AR tools sets available. Vuforia provides advanced computer vision functionality that enables the AR experiences to realistically interact with the 3D geometrical objects displayed at each level. Vuforia offers a significant advantage by accommodating a wide array of devices, including Android and iOS smartphones, tablets, as well as AR headsets like Microsoft Hololens and Magic Leap. This versatility broadens the scope for a more adaptable and immersive AR experience, ensuring accessibility to a larger audience. Furthermore, Vuforia equips developers with the necessary tools and resources for the swift and efficient creation of robust AR experiences, allowing them to concentrate on delivering engaging and effective educational content. The prerequisite stage for the development of the marker database is the registration as a Vuforia engine developer. 11 Once the account is setup and the login is successful, the developer selects either the basic or the premium plan from the license manager. Both basic and premium plans, allow the publication of unlimited applications under a single plan, which makes them a suitable choice for developing educational AR applications like PARSAT.  However, the image targets feature, included in the free basic plan (Fig. 5.3), was crucial for the selection of the basic plan and license key (Fig. 5.4) of PARSAT. Image targets are a critical component of marker-based AR, as they enable the AR system to identify and track the marker, allowing the virtual content to be rendered accurately. The basic plan of Vuforia provides this essential feature, making it a suitable choice for PARSAT.  The license key is used to enable the AR

Fig. 5.3  Basic plan obtained through the license manager

11

 https://developer.vuforia.com/

108

5  Artificial Intelligence-Enhanced PARSAT AR Software: Architecture…

Fig. 5.4  PARSAT’s license key

functionality within the PARSAT application, allowing students to scan and view the AR content using their mobile devices. The image targets in PARSAT are created using the Vuforia Target Manager. The supported formats for the input images are JPEG or PNG images in either RGB or grayscale format. Size limitations are in place when generating image targets. Input images should not exceed 2.25MB in file size and must possess a minimum width of 320 pixels. These specifications guarantee that the image targets are appropriately sized for processing on various devices. The features of the input images are extracted and stored in the marker database, which is then packaged together with the PARSAT application. When the user scans the marker using their mobile device’s camera, the system matches the features of the input image with those stored in the marker database to identify the marker and display the associated AR content. This process allows the AR system to render the virtual content accurately on top of the marker, creating an immersive and engaging educational experience for students. Glossiness and light source reflections have an impact on the recognition and tracking of image targets in AR applications like PARSAT. It is recommended that the image target is viewed in bright, evenly lit diffused lighting to maintain an optimal AR experience. Any planar image that provides enough detail for the Vuforia Engine to recognize it can be used as an image target and stored in the marker database (Fig.  5.5). This flexibility permits developers to employ a diverse selection of images as targets, encompassing photographs, logos, and illustrations.

5.2  System Architecture

109

Fig. 5.5  Marker database creation

Creating image targets involves two primary steps. The initial step revolves around the design of the target image itself. This entails the selection of a suitable image and its preparation for use as an image target. Such preparations may involve fine-tuning brightness and contrast, as well as the removal of undesired backgrounds or noise. The second step is the actual creation of the image target using the Vuforia Target Manager (Fig. 5.6). This procedure entails the uploading of the target image while specifying its dimensions and additional parameters, including whether it constitutes a single image or a multi-image target. Subsequent to the creation of the image target, it is incorporated into the marker database and becomes available for utilization within the PARSAT application. The Vuforia Target Manager is an online platform that is used to create and manage image targets for use in the AR application. The Target Manager provides a user-friendly interface that allows image uploading and processing to create image targets. Once the image targets have been created, they can be downloaded as a package and integrated into the PARSAT application. This allows the PARSAT application to recognize and track the image targets, and display AR content on top of them. By using the Vuforia Target Manager, developers can easily create and manage image targets, and ensure that the AR experience in the PARSAT application is accurate and engaging. When an image is uploaded to the Target Manager, it undergoes processing to create data and visual representations of its attributes. This process involves

110

5  Artificial Intelligence-Enhanced PARSAT AR Software: Architecture…

Fig. 5.6  Marker database content

extracting features from the image and evaluating its visual traits, such as contrast and brightness. The Vuforia Target Manager provides an anticipated detection and tracking performance score for each image target, which can help developers evaluate the suitability of the image for use as a marker in the PARSAT application. The performance score takes into account factors such as the image quality, lighting conditions, and contrast, and can help developers choose the best image targets for their AR experience. Once the image targets have been created and evaluated Fig.  5.7, they can be downloaded as a package ready for integration into the PARSAT application. This package can be easily imported into Unity, the development platform used to create the PARSAT application, allowing developers to easily incorporate the image targets into their AR experiences. By integrating the image targets into the PARSAT application, developers can ensure that the AR content is accurately rendered on top of the image targets for an immersive and engaging learning experience. 5.2.3.2 3D Models Database The 3D model library within the PARSAT application plays a pivotal role in enabling student interaction with virtual models and fostering spatial visualization skills. In the endeavor to produce these indispensable 3D models, developers are presented

5.2  System Architecture

111

Fig. 5.7  Image targets creation

with a spectrum of choices within the realm of 3D modeling software. These alternatives can be systematically categorized into two principal types: those that are available free of charge and those that necessitate a paid license for usage. Within the domain of freely accessible options, prominent software tools such as Blender, SketchUp, and Tinkercad emerge as noteworthy selections. These applications offer a comprehensive suite of functionalities encompassing 3D modeling, texturing, and

112

5  Artificial Intelligence-Enhanced PARSAT AR Software: Architecture…

animation. These choices are particularly attractive for beginners and those with budget constraints. Some popular software programs, freely available and geared specifically toward users staring with 3D modeling are: a) SketchUp Free, 12 an online browser-based free 3D modeling software which is the minimal version of its popular desktop software, b) Blender, 13 a completely free and open source industry leader in 3D character modeling, sculpting, and animation, c) Fusion 360, 14 which is a 3D ­modeling software with powerful features that go beyond simple 3D modeling, d) Shapr 3D, 15 a freemium software with a free tier available with all 3D modeling tools, and e) Vectary, 16 a browser-based application that has a very intuitive user interface, for creating great presentations of 3D objects. Conversely, licensed 3D modeling software introduces alternatives that encompass industry-standard applications, including but not limited to Autodesk 3ds Max, Maya, and Cinema 4D, among others. These tools offer a more sophisticated set of features and capabilities, rendering them particularly suitable for professional utilization across domains like architecture, engineering, and animation. In addition to free 3D modeling software, there are also paid versions available, typically offered through monthly or yearly subscription plans. Noteworthy software in this category includes a) 3ds Max, 17 which is used by engineers and designers to create 3D designs, models, animations, and renders. It delivers photorealistic designs and offers a variety of creative tools to users looking to produce a true work of art, b) Rhinoceros, 18 uninhibited design is the focus of Rhinoceros, providing all the tools and features a designer would expect in the best 3D modeling software, but with an easy-to-learn toolset that allows users to get cracking right away. It is compatible with just about everything and has a well-designed rendering engine that can process even complex animations without unexpected slowdown, c) Maya, 19 which is a 3D modeling software tool that can also be used for animation, simulation, and rendering. With one creative toolset, Maya helps artists share their stories with realistic characters, stunning visuals, and blockbuster-worthy effects, and d) Cinema 4D, 20 which is a professional 3D modeling, animation, simulation, and rendering software solution. Users enjoy a fast, flexible, powerful, and stable list of tools that make 3D workflows more efficient. Irrespective of the software employed, the resultant 3D models assume a pivotal role within the framework of the PARSAT application. Their meticulous design and

 https://www.sketchup.com/plans-and-pricing/sketchup-free  https://www.blender.org/ 14  https://www.autodesk.com/products/fusion-360/personal 15  https://www.shapr3d.com/ 16  https://www.vectary.com/ 17  https://www.autodesk.com/products/3ds-max/overview?term=1-YEAR&tab=subscription 18  https://www.rhino3d.com/ 19  https://www.autodesk.com/products/maya/overview?term=1-YEAR&tab=subscription 20  https://www.maxon.net/en/cinema-4d 12 13

5.2  System Architecture

113

optimization are imperative, as they must facilitate swift and precise rendering across a diverse array of devices. The selection of the 3D modeling software considered several factors including: • Object design: Different software is better suited for different types of objects. Some software is optimized for designing characters, while others are better for architectural or mechanical designs. • Interface: The user interface of the software is crucial to consider as it can impact the ease of use and productivity of the user. • Community support: Having a supportive community behind the software can make a significant difference in the learning process. • Cost: The cost of the software is also an important factor to consider, especially if a paid license is required. Considering these factors, PARSAT’s 3D model’s database was prepared using Autodesk 3ds Max due to the following reasons: • The author’s background is based on engineering education, and as such, the author is trained, both at an undergraduate and professional level, at Autodesk’s products of AutoCAD, 3ds Max, and Tinkercad. • Additionally, being a Ph.D. student at the Laboratory of Educational Technology and e-Learning Systems, the author has free access to Autodesk’s products and services, renewable as long as the author remains eligible, making it a cost-­ effective option. • Furthermore, 3ds Max is widely used in many industries for generating graphics, including mechanical and organic objects. Its toolset is highly capable, making it a powerful tool for modeling. The software is also commonly used in the engineering, manufacturing, educational, and medical industries for visualization needs, making it a versatile choice. Figure 5.8 exhibits a depiction of a 3D model that has been generated through the utilization of Autodesk 3ds Max. It is worth noting that this specific model is integral constituent of a more extensive database comprising 3D models, all of which have been crafted employing the same Autodesk 3ds Max software. Autodesk 3ds Max enjoys renowned for its proficiency in the creation of high-caliber 3D models, distinguished by a rich assortment of features and tools at its disposal. All 3D models were seamlessly transferred from Autodesk 3ds Max to Unity, employing the Autodesk .fbx format as the chosen conduit. The selection of this format was predicated on its manifold advantages during the process of importing meshes from 3ds Max into Unity. Primarily, the .fbx format facilitates the import of nodes imbued with attributes such as position, rotation, and scale from 3ds Max to Unity. This affords the preservation of the intricate structure and hierarchy inherent in the 3D models originally conceived in 3ds Max. Such preservation significantly economizes time and effort that would otherwise be expended in reconstituting these models within the Unity environment.

114

5  Artificial Intelligence-Enhanced PARSAT AR Software: Architecture…

Fig. 5.8  3D model of PARSAT’s 3D models database

Secondly, the .fbx format lends itself to the importation of meshes replete with vertex colors from 3ds Max to Unity. The granular control over vertex colors permits the creation of meticulously detailed and lifelike textures. Thirdly, the format accommodates the import of materials complete with diffuse textures and colors from 3ds Max to Unity. This seamless transition ensures that the materials and textures meticulously applied to the 3D models within 3ds Max are faithfully transposed into Unity, simplifying the endeavor of crafting realistic and intricate virtual environments. Lastly, animations painstakingly fashioned in 3ds Max find a harmonious integration into Unity through the .fbx format. This facilitates the incorporation of complex and dynamic interactions between objects and characters within the game or application. In summation, the capability to import nodes, vertex-colored meshes, materials, and animations from 3ds Max into Unity via the .fbx format serves as a profound streamlining mechanism for the construction of 3D environments and characters in Unity. 5.2.3.3 Interaction Model Within the domain of augmented reality (AR) technology, the interaction model expounded in this chapter epitomizes a quintessential archetype. AR fundamentally revolves around the overlaying of digital content onto the tangible real-world environment. In the case at hand, the AR application endeavors to enrich the educational journey of students by granting them the capacity to interact with 3D models of objects, thus affording a more verisimilar and immersive experience. The crux of the interaction model hinges on the deployment of markers, predefined images meticulously cataloged within a database. When students align their

5.2  System Architecture

115

mobile device’s camera with one of these markers, the application adeptly discerns the marker’s identity and overlays the corresponding 3D model atop it. This engenders an illusion where students perceive the 3D model coexisting within the tangible world, akin to a corporeal entity. Moreover, students are empowered to engage actively with the 3D model by manipulating its size, position, and orientation. This facet of interaction bequeaths a deeper comprehension of the spatial intricacies inherent in the object. Such interactive pedagogy assumes paramount significance in educational contexts, as it empowers students to probe and manipulate 3D objects in a manner heretofore inconceivable with conventional learning materials. In summary, the interaction model epitomizes an influential and innovative pedagogical approach that engages students actively in the learning process, thereby augmenting their grasp of 3D objects and their contextual understanding. The previously referenced term encapsulates the intricate process of amalgamating Vuforia with Unity to institute augmented reality (AR) functionalities within an application. Vuforia stands as a venerated AR platform, replete with tools for the detection and tracking of image targets. These image targets serve as conduits for the superimposition of 3D content onto tangible real-world objects. The inaugural step in the Vuforia-Unity amalgamation entails the procurement of the Vuforia package, which is subsequently imported into the Unity editor. Subsequent to this importation, the Unity main camera is supplanted with the specialized Vuforia engine AR camera, tailored to the precise demands of detecting and tracking image targets. The subsequent pivotal step revolves around the integration of the Vuforia license key into the Unity project. This key, graciously furnished by Vuforia, stands as the linchpin for enabling AR functionalities within the application. Following this, the marker database, an inventory of predefined images primed for service as image targets within the AR application, is seamlessly incorporated into the Unity project. The denouement of this intricate process materializes when the application adroitly overlays 3D content upon an image target, facilitated by the Vuforia engine. Once the camera identifies a designated image target, the corresponding 3D content is artfully superimposed, birthing an immersive augmented reality experience for the end-user. In synthesis, the orchestration of Vuforia with Unity unfolds through a meticulously orchestrated sequence of steps, encompassing the importation of the Vuforia package, the substitution of the main camera with the Vuforia AR camera, the infusion of the Vuforia license key, and the integration of the marker database. The culmination of these steps bequeaths the application with the capability to seamlessly drape 3D content over tangible real-world objects, ultimately engendering a more immersive and interactive user experience. More specifically, after downloading the Vuforia engine package from the official website, the next step is to import it into the Unity editor (Fig. 5.9). The verification of the successful import of the Vuforia engine is made through the “Hierarchy” panel in the Unity Editor.

116

5  Artificial Intelligence-Enhanced PARSAT AR Software: Architecture…

Fig. 5.9  Addition of Vuforia engine into the Unity project

To this direction, the “Import Package” option was selected from the “Assets” menu in the Unity editor and then the Vuforia package, that was downloaded, was chosen. Once the package is imported, it can be verified that the Vuforia engine has been successfully imported by checking the “Hierarchy” panel in the Unity editor. The Vuforia engine will have created a new hierarchy in the panel, which includes the “ARCamera” object, among others. The “ARCamera” object is the main camera used for augmented reality applications in Unity with Vuforia. It is responsible for detecting and tracking image targets and rendering 3D content on top of them. Once the Vuforia engine is successfully imported and the “ARCamera” object is present in the Unity hierarchy, the next step involves setting up the image targets and 3D content to be overlaid on top of them. In the setup process for using Vuforia in Unity, the main camera in Unity is deleted, and the AR camera from the Vuforia package is added instead. This is because the AR camera is responsible for detecting and tracking image targets and rendering 3D content on top of them. After incorporating the AR camera into the Unity scene, the subsequent action involves including the license key, which is furnished via the Vuforia developer’s portal, into Unity’s assets section (Fig.  5.10). This license key serves the pivotal purpose of authenticating the application with Vuforia’s servers, thereby unlocking the full spectrum of Vuforia’s features and capabilities. Finally, the marker database is imported into the Unity project. The marker database contains the image targets that Vuforia will use to detect and track in order to overlay 3D content on top of them. Once the marker database is imported, the user can associate the image targets with the corresponding 3D models in the Unity scene and configure the interactions and behaviors of the 3D content.

5.2  System Architecture

117

Fig. 5.10  Vuforia license key configuration

In Unity with Vuforia, the rendering of 3D content on top of an image target is configured in the image target behavior, which is a script that can be attached to an image target object in the Unity scene (Fig. 5.11). The image target behavior script includes properties for setting the type of image target, the marker database to use, and the marker to associate with the image target. These properties are used to specify which image target Vuforia should detect and track, and which 3D content should be rendered on top of it. Following the configuration of the image target behavior script, users may advance by establishing an association between the script and an image target object situated

118

5  Artificial Intelligence-Enhanced PARSAT AR Software: Architecture…

Fig. 5.11  Rendering 3D content on top of an image target

within the Unity scene. Furthermore, they possess the capability to affix the 3D content designated for superimposition onto the image target, assigning it as a subordinate element of said image target. As a result of this procedural arrangement, the opportunity arises to meticulously refine the conduct of the aforementioned 3D content, enhancing its capacity to responsively engage with user interactions, which may encompass tactile input or gesture-based commands. This intricate interactive response framework is instrumental in furnishing users with both visual feedback and pertinent informational content, thereby enriching their overall experiential engagement. The successful import of the Vuforia engine is verified by checking the “Hierarchy” panel in the Unity Editor. After importing the Vuforia package, the Vuforia engine’s AR camera appears in the “Hierarchy” panel. The AR camera is responsible for detecting and tracking image targets, and rendering 3D content on top of them. In the case of PARSAT, the AR camera did not appear in the “Hierarchy” panel at first, so it was necessary to restart Unity and re-import the Vuforia package.

5.3  Implementation of the System

119

Additionally, it was important to make sure that the correct version of the Vuforia package was imported for the version of Unity being used, as compatibility issues occasionally cause errors or issues with the AR camera and other Vuforia features.

5.3 Implementation of the System 5.3.1 User Interface of PARSAT In this subsection, an overview of screenshots of PARSAT is presented. More specifically, Figs. 5.12 and 5.13 present the start screen of PARSAT, and the recording of students’ personal information, respectively. The third parameter of prior knowledge is being used as an input for the student model in PARSAT. This parameter is related to the student’s prior knowledge in the subject of technical drawing and can be used to tailor the learning experience to their individual needs. Figure 5.14 shows the application’s training screen of Level I, having a bottom side menu of grayed-out elements, which are enabled when the student gradually completes the training tasks, namely the video courses, the interaction with the 3D models, the assignments, and each level’s assessment test. It is important to point out that the feature of the grayed-out menu items, which are enabled as the student progresses through the training tasks, can help provide a sense of achievement and progress, which in turn can be motivating for students. Figure 5.15 serves to depict an inventory comprising the video courses encompassed within Level I, while Fig. 5.16 offers glimpses of the initial video courses incorporated within the educational framework. The strategic incorporation of

Fig. 5.12  Start screen of PARSAT

120

5  Artificial Intelligence-Enhanced PARSAT AR Software: Architecture…

Fig. 5.13  Student profiling

Fig. 5.14  PARSAT’s main menu

asynchronous content bears notable advantages, particularly catering to students who exhibit a preference for self-paced learning. Furthermore, this approach contributes to mitigating cognitive load by affording students the liberty to pause and revisit content as necessary, thereby fostering a more effective and adaptable learning experience. The asynchronous content of PARSAT’s videos, lasting from a minimum of 4 to a maximum of 7 min, help students understand technical drawing course concepts and reduces the amount of friction or extraneous cognitive load.

5.3  Implementation of the System

121

Fig. 5.15  Video courses at the first stage of training Level I

Fig. 5.16  Interface of the video courses

Figure 5.17 is giving feedback to the student, about the completion of the video courses, it congratulates the student on completing the series of all video courses for Level I and provides information on the time spent watching the videos. It also informs the student that the visualization of the 3D models feature is now enabled, urging the student to return to the main menu, allowing them to proceed to the AR environment.

122

5  Artificial Intelligence-Enhanced PARSAT AR Software: Architecture…

Fig. 5.17  Feedback to the student about the completion of the video courses

Fig. 5.18  The 3D visualization option is enabled

Figure 5.18 shows that the “3D Visualization” option is now unlocked and not grayed out anymore, indicating that the student can access it. The operation of augmenting a 3D model involves the following steps: • Printing out a marker that correspond to the specific 3D model of each SOLO level. The marker is designed in such a way that PARSAT can recognize it and use it as an anchor point for overlaying the 3D model.

5.3  Implementation of the System

123

• Launching the PARSAT on the mobile device and pointing the camera at the marker. The application uses the device’s camera to detect the marker and overlay the 3D model onto the corresponding marker position. • Once the 3D model is overlaid on the marker, the student can interact with and can rotate and zoom in/out using touch gestures on the device’s screen. Figure 5.19 presents an example of operation with screenshot of the student’s mobile device, while the 3D models are augmented onto the view of their corresponding markers printed in the booklet. Figure 5.20 shows a question about the assessment test that students take at the end of the level. The student’s assessment score is the fourth parameter fed the input

Fig. 5.19  Example of operation with the augmentation of a 3D model of the uni-structural SOLO level

Fig. 5.20  Assessment test at the end of level I

124

5  Artificial Intelligence-Enhanced PARSAT AR Software: Architecture…

of the student model, and in combination with the recorded duration of the rest two parameters, contribute as inputs in the fuzzy logic system.

5.3.2 Fuzzy Logic Controller Implementation with C# Scripting Fuzzy logic controller (FLC) is based on Mamdani’s inference engine, which was developed in the 1970s as a way to deal with imprecise or uncertain information. FLC uses fuzzy sets and fuzzy rules to approximate human reasoning and decision-­ making processes. In the fuzzy logic controller, crisp inputs are first translated into fuzzy values using linguistic variables, which represent the qualitative aspects of the system being controlled. The fuzzy rules in the FLC specify how the input fuzzy values should be combined to produce output fuzzy values, which are then defuzzified to produce crisp outputs. Mamdani’s inference engine uses a method called “max-min composition” to combine the input fuzzy values according to the fuzzy rules. This involves taking the maximum of the minimum values of the fuzzy sets that correspond to each input and rule. The resulting output fuzzy set is then defuzzified to produce a crisp output value. Overall, fuzzy logic controller is useful for controlling systems that are complex, nonlinear, or poorly understood, as it can capture and approximate the behavior of these systems in a more intuitive way than traditional control methods. 5.3.2.1 System Initialization The first step in implementing the fuzzy controller in C# programming language, was the initialization of the system. The system was configured with the “and” logic connection (Fig. 5.21).

public static void Initalize() { InputVariables = new List(); OutputVariables = new List(); Rules = new List(); Configuration = new Config(ImpMethod.Prod, ConnMethod.Min); Configuration.DefuzzificationType = DefuzzifcationType.ModifiedHeight; FuzzyControl = new FLC(Configuration); }

Fig. 5.21  System initialization

5.3  Implementation of the System

125

5.3.2.2 Linguistic Variables and Membership Functions The implementation in C# programming language of the linguistic variables, namely prior knowledge, video-based learning duration, augmented-reality interaction duration and number of errors is shown in Fig. 5.22. The part of the code shown in Fig. 5.22 has also the membership functions of the aforementioned input variables. Figure 5.23 presents the C# code for the output linguistic variable, namely current knowledge level, and its corresponding membership function.

// Inputs LingVariable PRK = new LingVariable("PRK", VarType.Input); PRK.setRange(0, 100); PRK.addMF(new Trapmf("Poor", 0, 0, 20, 35)); PRK.addMF(new Trapmf("Medium", 30, 40, 60, 75)); PRK.addMF(new Trapmf("Good", 70, 80, 100, 100)); LingVariable VLD = new LingVariable("VLD", VarType.Input); VLD.setRange(0, 100); VLD.addMF(new Trapmf("Short", 0, 0, 20, 35)); VLD.addMF(new Trapmf("Normal", 30, 40, 60, 70)); VLD.addMF(new Trapmf("Long", 60, 80, 100, 100)); LingVariable ARID = new LingVariable("ARID", VarType.Input); ARID.setRange(0, 100); ARID.addMF(new Trapmf("Short", 0, 0, 40, 60)); ARID.addMF(new Trapmf("Normal", 40, 50, 70, 80)); ARID.addMF(new Trapmf("Long", 70, 80, 100, 100)); LingVariable NoE = new LingVariable("NoE", VarType.Input); NoE.setRange(0, 100); NoE.addMF(new Trapmf("Small", 0, 0, 20, 40)); NoE.addMF(new Trapmf("Medium", 30, 40, 60, 65)); NoE.addMF(new Trapmf("Large", 60, 80, 100, 100)); FuzzyApp.InputVariables.Add(PRK); FuzzyApp.InputVariables.Add(VLD); FuzzyApp.InputVariables.Add(ARID); FuzzyApp.InputVariables.Add(NoE);

Fig. 5.22  Input linguistic variables and its membership functions

//Output LingVariable CLK = new LingVariable("CLK", VarType.Output); CLK.setRange(0, 100); CLK.addMF(new Trapmf("Novice", 0, 0, 10, 20)); CLK.addMF(new Trapmf("Beginner", 10, 20, 30, 40)); CLK.addMF(new Trapmf("Competent", 30, 40, 50, 60)); CLK.addMF(new Trapmf("VeryGood", 50, 60, 70, 80)); CLK.addMF(new Trapmf("Profocient", 70, 80, 90, 95)); CLK.addMF(new Trapmf("Expert", 90, 95, 100, 100)); FuzzyApp.OutputVariables.Add(CLK); Fig. 5.23  Output linguistic variable and its membership function

126

5  Artificial Intelligence-Enhanced PARSAT AR Software: Architecture…

5.3.2.3 Fuzzification Process Implementation The fuzzification process evaluates the crisp inputs against the membership functions of the linguistic variables. The membership functions are an abstract class, with the method “getOutput”. The objects, which inherit the class, need to define their getOutput(double) function. The fuzzy controller capabilities are extended by implementing all the necessary membership functions. The fuzzification method returns the list of the fuzzy numbers, each of which has two parts, namely the name of the function and the amount of contribution. Then, the fuzzy numbers are added to a fuzzy set for further process. After the fuzzification of all inputs is completed, they are added to a list of fuzzy sets. 5.3.2.4 Rules of the System Figure 5.24 presents the implementation of the rule base, which is used by the system, and specifically, Fig. 5.24 is a screenshot of a part of the definition of the 81 rules, consisting of the rule base.

//Rules List rule1in = new List(); List rule1out = new List(); List rule2in = new List(); List rule2out = new List(); List rule3in = new List(); List rule3out = new List(); List rule4in = new List(); List rule4out = new List(); List rule5in = new List(); List rule5out = new List(); List rule9in = new List(); List rule9out = new List(); List rule10in = new List(); List rule10out = new List(); List rule11in = new List(); List rule11out = new List(); List rule12in = new List(); List rule12out = new List(); List rule13in = new List(); List rule13out = new List(); List rule14in = new List(); List rule14out = new List(); List rule15in = new List(); List rule15out = new List(); List rule16in = new List(); List rule16out = new List(); List rule17in = new List(); List rule17out = new List(); // ... this part includes the rest of the code of the 81 rule s List rule81in = new List(); List rule81out = new List();

Fig. 5.24  Rule base definition

5.3  Implementation of the System

127

Figure 5.25 presents the C# implementation of the rules presented in Table 4.2 and specifically, the “if-then” part of the rules. For every rule, there is a duet of code lines, the first line which constitutes the “if” part with four rule items and the second line which constitutes the “then” part in the rule. Finally, Fig. 5.26 shows how the list of the rules is passed to the inference engine. 5.3.2.5 Evaluation of the Rules The evaluation of the rules refers to the firing strength of the output and it is determined inside the inference system (Fig. 5.27). // ... if ... then ... code of each rule rule1in.AddRange(new RuleItem[4] { new RuleItem("PRK", "Poor"), new RuleItem("VLD", "Short"), new RuleItem("ARID", "Short"), new RuleItem("NoE", "Small")}); rule1out.AddRange(new RuleItem[1] { new RuleItem("CLK", "VeryGood") }); rule2in.AddRange(new RuleItem[4] { new RuleItem("PRK", "Poor"), new RuleItem("VLD", "Short"), new RuleItem("ARID", "Normal"), new RuleItem("NoE", "Small")}); rule2out.AddRange(new RuleItem[1] { new RuleItem("CLK", "VeryGood") }); rule3in.AddRange(new RuleItem[4] { new RuleItem("PRK", "Poor"), new RuleItem("VLD", "Short"), new RuleItem("ARID", "Long"), new RuleItem("NoE", "Small")}); rule3out.AddRange(new RuleItem[1] { new RuleItem("CLK", "Competent") }); rule4in.AddRange(new RuleItem[4] { new RuleItem("PRK", "Poor"), new RuleItem("VLD", "Normal"), new RuleItem("ARID", "Short"), new RuleItem("NoE", "Small")}); rule4out.AddRange(new RuleItem[1] { new RuleItem("CLK", "Competent") }); rule5in.AddRange(new RuleItem[4] { new RuleItem("PRK", "Poor"), new RuleItem("VLD", "Normal"), new RuleItem("ARID", "Normal"), new RuleItem("NoE", "Small")}); rule5out.AddRange(new RuleItem[1] { new RuleItem("CLK", "VeryGood") }); rule6in.AddRange(new RuleItem[4] { new RuleItem("PRK", "Poor"), new RuleItem("VLD", "Normal"), new RuleItem("ARID", "Long"), new RuleItem("NoE", "Small")}); rule6out.AddRange(new RuleItem[1] { new RuleItem("CLK", "VeryGood") }); // ... this part includes the rest of the code of the 81 rule s rule81in.AddRange(new RuleItem[4] { new RuleItem("PRK", "Good"), new RuleItem("VLD", "Long"), new RuleItem("ARID", "Long"), new RuleItem("NoE", "Large")}); rule81out.AddRange(new RuleItem[1] { new RuleItem("CLK", "Beginner") });

Fig. 5.25  If-then selection coding

128

5  Artificial Intelligence-Enhanced PARSAT AR Software: Architecture…

// ... this is the list of the rules for the inference engine List rules = new List(); FuzzyApp.Rules.Add(new Rule(rule1in, rule1out, Connector.And)); FuzzyApp.Rules.Add(new Rule(rule2in, rule2out, Connector.And)); FuzzyApp.Rules.Add(new Rule(rule3in, rule3out, Connector.And)); FuzzyApp.Rules.Add(new Rule(rule4in, rule4out, Connector.And)); FuzzyApp.Rules.Add(new Rule(rule5in, rule5out, Connector.And)); FuzzyApp.Rules.Add(new Rule(rule6in, rule6out, Connector.And)); FuzzyApp.Rules.Add(new Rule(rule7in, rule7out, Connector.And)); FuzzyApp.Rules.Add(new Rule(rule8in, rule8out, Connector.And)); FuzzyApp.Rules.Add(new Rule(rule9in, rule9out, Connector.And)); FuzzyApp.Rules.Add(new Rule(rule10in, rule10out, Connector.And)); FuzzyApp.Rules.Add(new Rule(rule11in, rule11out, Connector.And)); FuzzyApp.Rules.Add(new Rule(rule12in, rule12out, Connector.And)); FuzzyApp.Rules.Add(new Rule(rule13in, rule13out, Connector.And)); FuzzyApp.Rules.Add(new Rule(rule14in, rule14out, Connector.And)); FuzzyApp.Rules.Add(new Rule(rule15in, rule15out, Connector.And)); // ... this part includes the rest of the code of the 81 rules FuzzyApp.Rules.Add(new Rule(rule81in, rule81out, Connector.And));

Fig. 5.26  And operator coding

InferEngine engine = new InferEngine(configure, rules, input_sets); List fuzzy_out = engine.evaluateRules();

Fig. 5.27  Rules evaluation at the inference engine double crisp_CLK = c.DeFuzzification(fuzzy_out, CLK);

Fig. 5.28  Crisp output after defuzzification

5.3.2.6 Defuzzification The defuzzification process is a method to calculate the fuzzy output and convert it to a crisp value. In the proposed system, COG is used for defuzzification (Fig. 5.28).

5.4 Summary Chapter 5 presented the architecture of the system, analyzing in depth all the layers involved in the training application, namely the hardware layer, the software layer, and the data layer. For each layer, a detailed representation of the module was made. The second part of Chap. 5 presented the implementation of the system, by providing screenshots of the system’s environment, the user interface of a training scenario, while the last section presents parts of the scripts in C# programming language, used for the implementation of the fuzzy system controller.

References

129

References 1. Z. Kanetaki et al., “Acquiring, Analyzing and Interpreting Knowledge Data for Sustainable Engineering Education: An Experimental Study Using YouTube,” Electronics (Basel), vol. 11, no. 14, 2022, https://doi.org/10.3390/electronics11142210. 2. A. Marougkas, C. Troussas, A. Krouska, and C. Sgouropoulou, “Virtual Reality in Education: A Review of Learning Theories, Approaches and Methodologies for the Last Decade,” Electronics (Basel), vol. 12, no. 13, 2023, https://doi.org/10.3390/electronics12132832. 3. C. Troussas, A. Krouska, and C. Sgouropoulou, “Enriching Mobile Learning Software with Interactive Activities and Motivational Feedback for Advancing Users’ High-Level Cognitive Skills,” Computers, vol. 11, no. 2, 2022, https://doi.org/10.3390/computers11020018. 4. F.  Giannakas, C.  Troussas, A.  Krouska, C.  Sgouropoulou, and I.  Voyiatzis, “XGBoost and Deep Neural Network Comparison: The Case of Teams’ Performance,” in Intelligent Tutoring Systems, A. I. Cristea and C. Troussas, Eds., Cham: Springer International Publishing, 2021, pp. 343–349. 5. A.  Krouska, C.  Troussas, and C.  Sgouropoulou, “Fuzzy Logic for Refining the Evaluation of Learners’ Performance in Online Engineering Education,” European Journal of Engineering Research and Science, vol. 4, pp.  50–56, Jun. 2019, https://doi.org/10.24018/ ejers.2019.4.6.1369. 6. C. Troussas, A. Krouska, and C. Sgouropoulou, “Towards a Reference Model to Ensure the Quality of Massive Open Online Courses and E-Learning,” in Brain Function Assessment in Learning, C.  Frasson, P.  Bamidis, and P.  Vlamos, Eds., Cham: Springer International Publishing, 2020, pp. 169–175. 7. A.  Marougkas, C.  Troussas, A.  Krouska, and C.  Sgouropoulou, “How personalized and effective is immersive virtual reality in education? A systematic literature review for the last decade,” Multimed Tools Appl, 2023, https://doi.org/10.1007/s11042-023-15986-7. 8. A. Marougkas, C. Troussas, A. Krouska, and C. Sgouropoulou, “A Framework for Personalized Fully Immersive Virtual Reality Learning Environments with Gamified Design in Education,” 2021. https://doi.org/10.3233/FAIA210080. 9. C. Troussas, A. Krouska, and C. Sgouropoulou, “Dynamic Detection of Learning Modalities Using Fuzzy Logic in Students’ Interaction Activities,” in Intelligent Tutoring Systems, V. Kumar and C. Troussas, Eds., Cham: Springer International Publishing, 2020, pp. 205–213. 10. A.  Krouska, C.  Troussas, and C.  Sgouropoulou, “A novel group recommender system for domain-independent decision support customizing a grouping genetic algorithm,” User Model User-adapt Interact, 2023, https://doi.org/10.1007/s11257-023-09360-3. 11. A. Krouska, C. Troussas, K. Kabassi, and C. Sgouropoulou, “An Empirical Investigation of User Acceptance of Personalized Mobile Software for Sustainability Education,” Int J Hum Comput Interact, pp. 1–8, Aug. 2023, https://doi.org/10.1080/10447318.2023.2241614. 12. C.  Papakostas, C.  Troussas, A.  Krouska, and C.  Sgouropoulou, “Personalization of the Learning Path within an Augmented Reality Spatial Ability Training Application Based on Fuzzy Weights,” Sensors, vol. 22, no. 18, 2022, https://doi.org/10.3390/s22187059. 13. C.  Troussas, C.  Papakostas, A.  Krouska, P.  Mylonas, and C.  Sgouropoulou, “Personalized Feedback Enhanced by Natural Language Processing in Intelligent Tutoring Systems,” in Augmented Intelligence and Intelligent Tutoring Systems, C.  Frasson, P.  Mylonas, and C.  Troussas, Eds., Cham: Springer Nature Switzerland, 2023, pp.  667–677. https://doi. org/10.1007/978-3-031-32883-1_58. 14. C. Papakostas, C. Troussas, A. Krouska, and C. Sgouropoulou, “Measuring User Experience, Usability and Interactivity of a Personalized Mobile Augmented Reality Training System,” Sensors, vol. 21, no. 11, p. 3888, Jun. 2021, https://doi.org/10.3390/s21113888. 15. C.  Papakostas, C.  Troussas, A.  Krouska, and C.  Sgouropoulou, “PARSAT: Fuzzy logic for adaptive spatial ability training in an augmented reality system,” Computer Science and Information Systems, vol. 20, no. 4, 2023, https://doi.org/10.2298/CSIS230130043P.

130

5  Artificial Intelligence-Enhanced PARSAT AR Software: Architecture…

16. C.  Papakostas, C.  Troussas, A.  Krouska, and C.  Sgouropoulou, “On the development of a personalized augmented reality spatial ability training mobile application,” in Frontiers in Artificial Intelligence and Applications, IOS Press, 2021, pp. V–VI. https://doi.org/10.3233/ FAIA210078. 17. X.  Wang, S.  K. Ong, and A.  Y. C.  Nee, “A comprehensive survey of augmented reality assembly research,” Adv Manuf, vol. 4, no. 1, pp.  1–22, 2016, https://doi.org/10.1007/ s40436-015-0131-4. 18. F.  Redzuan, A.-N.  A. Khairuddin, and N.  A. Daud, “Emotional augmented reality-based mobile learning design elements: a kansei engineering approach,” Indones. J.  Electr. Eng. Comput. Sci, vol. 14, no. 1, pp. 413–420, 2019. 19. T. Masood and J. Egger, “Adopting augmented reality in the age of industrial digitalisation,” Comput Ind, vol. 115, p. 103112, 2020. 20. C. Papakostas, C. Troussas, A. Krouska, and C. Sgouropoulou, “User acceptance of augmented reality welding simulator in engineering training,” Educ Inf Technol (Dordr), vol. 27, no. 1, pp. 791–817, Jan. 2022, https://doi.org/10.1007/s10639-020-10418-7. 21. C. Papakostas, C. Troussas, A. Krouska, and C. Sgouropoulou, “Exploring Users’ Behavioral Intention to Adopt Mobile Augmented Reality in Education through an Extended Technology Acceptance Model,” Int J Hum Comput Interact, vol. 39, no. 6, pp. 1294–1302, 2023, https:// doi.org/10.1080/10447318.2022.2062551. 22. M. Iakovidis, C. Papakostas, C. Troussas, and C. Sgouropoulou, “Empowering Responsible Digital Citizenship Through an Augmented Reality Educational Game,” in Novel & Intelligent Digital Systems: Proceedings of the 3rd International Conference (NiDS 2023), K. Kabassi, P. Mylonas, and J. Caro, Eds., Cham: Springer Nature Switzerland, 2023, pp. 31–39. 23. P.  Strousopoulos, C.  Troussas, C.  Papakostas, A.  Krouska, and C.  Sgouropoulou, “Revolutionizing Agricultural Education with Virtual Reality and Gamification: A Novel Approach for Enhancing Knowledge Transfer and Skill Acquisition,” in Novel & Intelligent Digital Systems: Proceedings of the 3rd International Conference (NiDS 2023), K. Kabassi, P. Mylonas, and J. Caro, Eds., Cham: Springer Nature Switzerland, 2023, pp. 67–80. 24. C.  Papakostas, C.  Troussas, P.  Douros, M.  Poli, and C.  Sgouropoulou, “CoMoPAR: A Comprehensive Conceptual Model for Designing Personalized Augmented Reality Systems in Education,” in Novel & Intelligent Digital Systems: Proceedings of the 3rd International Conference (NiDS 2023), K. Kabassi, P. Mylonas, and J. Caro, Eds., Cham: Springer Nature Switzerland, 2023, pp. 67–79. 25. P. Strousopoulos, C. Papakostas, C. Troussas, A. Krouska, P. Mylonas, and C. Sgouropoulou, “SculptMate: Personalizing Cultural Heritage Experience Using Fuzzy Weights,” in Adjunct Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization, in UMAP ’23 Adjunct. New York, NY, USA: Association for Computing Machinery, 2023, pp. 397–407. https://doi.org/10.1145/3563359.3596667. 26. C. Papakostas, C. Troussas, A. Krouska, and C. Sgouropoulou, “Exploration of Augmented Reality in Spatial Abilities Training: A Systematic Literature Review for the Last Decade,” Informatics in Education, vol. 20, no. 1, pp.  107–130, Mar. 2021, https://doi.org/10.15388/ infedu.2021.06. 27. C. Papakostas, C. Troussas, A. Krouska, and C. Sgouropoulou, “Modeling the Knowledge of Users in an Augmented Reality-Based Learning Environment Using Fuzzy Logic,” in Lecture Notes in Networks and Systems, A. Krouska, C. Troussas, and J. Caro, Eds., Cham: Springer International Publishing, 2023, pp. 113–123. https://doi.org/10.1007/978-3-031-17601-2_12.

Chapter 6

Multi-model Evaluation of the Artificial Intelligence-Enhanced PARSAT AR Software

Abstract  This chapter of the book delves into the meticulous evaluation of an Artificial Intelligence-enhanced Augmented Reality (AR) mobile training system designed to enhance spatial ability training. This chapter adopts a multi-model evaluation approach, employing various research methods and techniques to comprehensively assess the system’s effectiveness and impact. The chapter commences with an overview stressing the importance of evaluating the system’s impact on spatial ability training and introduces the need for a comprehensive evaluation framework. It then delves into the “Evaluation Framework,” outlining the overall structure, research sample, and participant preparation for the training phase. The “t-Test Analysis of Students’ Feedback” section focuses on analyzing participant feedback, utilizing t-test analysis to identify differences in feedback between the experimental and control groups. This sheds light on participants’ perceptions and satisfaction with the system. Next, the “Comparative Analysis of Pre-Test/Post-Test Model” assesses the system’s impact on spatial ability development through a pre-­ test and post-test model, providing valuable insights into its effectiveness. The chapter also introduces an “Extended Technology Acceptance Model” tailored to evaluate the human-system interaction, exploring factors influencing system acceptance and usability.

6.1 Overview The current chapter presents the evaluation of the system, which is a significant phase, determining the software’s usability. The evaluation is deployed in multiple levels, in terms of quantitative measurements, questionnaires’ reliability, t-tests, pre-test and post-test analysis, and an extended acceptance model [1, 2], so that both pedagogical affordance and learning outcomes have been thoroughly evaluated.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 C. Papakostas et al., Special Topics in Artificial Intelligence and Augmented Reality, Cognitive Technologies, https://doi.org/10.1007/978-3-031-52005-1_6

131

132

6  Multi-model Evaluation of the Artificial Intelligence-Enhanced PARSAT AR Software

6.2 Evaluation Framework The evaluation of the proposed system is an important stage of the current research, and it is based on well-established evaluation frameworks [3, 4], in order to produce more accurate results and improve the usability of the software. According to the overview on evaluation of training [5], the experimental measurement is one of the most popular and efficient methodologies, and as such, this technique is integrated into the current evaluation. The Lynch and Ghergulescu framework [6] is used for PARSAT’s evaluation, which is specifically oriented to evaluate adaptive and intelligent learning systems. The proposed framework comprises four distinctive evaluation criteria, namely a) learning, and training, b) system, c) user experience, and d) affective dimension. More specific, each criterion serves as a separate factor contributing to an overall framework. Learning and training assessment involves the evaluation of the effectiveness of the system by measuring the volume of the finished learning activities, and the duration of the time spent on each learning improvement. In terms of the system’s accuracy, the algorithmic approach is evaluated on how precisely the system fits standardized tests and determines the student model, which is critical for the adaptivity process. The criterion of the user experience evaluates the ease with which users operate the system and their level of satisfaction and actual intention of using the system in the future. This dimension involves the human-computer interaction (HCI) user experience, the system’s usefulness and the students’ behavioral intention towards using it. Finally, the affective-related evaluation includes the assessment of the students’ motivation and engagement in the learning process. The proposed evaluation framework is presented in Table 6.1.

6.2.1 Research Sample The evaluation of PARSAT took place during the winter academic semester 2022–2023, while students took the following courses: (a) Computer-Aided-Design, (b) Technical Drawing, and (c) 3D Monument Modeling, at an undergraduate curriculum of a public University, located in the nation’s capital. In particular, three educators, and 240 second-, third-, and fourth-year undergraduate students, participated in the evaluation process. The gender and age measurements were obtained from a randomly selected sample and do not influence the research outcomes. The demographic analysis can be found in Table 6.2. The assessment process involved the active participation of three faculty members, who contributed by presenting the system to students, demonstrating its functionality, and providing guidance to students throughout the experiment.

133

6.2  Evaluation Framework Table 6.1  Evaluation framework [6] Evaluation direction Attribute Learning and Effectiveness training

References [7–10]

System

[7, 11]

Usability/ User experience Affective

Description Learning improvements (with and without revisions); Amount of completed, or studied content (in comparison with other learning instructions) Efficiency How efficient is the use of the time Accuracy How accurate the user model is (i.e., how accurate is the system grading in comparison with their tests/exams) and how accurate the recommendation algorithms are (higher accuracy scores or lower predictive errors) Ease of use and How easy is to use the system satisfaction Engagement, motivation

[8, 12, 13]

How engaged are the learners both in class and out [6, 14] of class

Table 6.2  Population demographics Measure Sample size Gender

Age

Level of prior knowledge Computer skills Motivation

Item Male Female Non-binary 15–17 18–19 Over 20 None Technical background Knowledge of computers at a high level All students wanted to achieve a high grade at the attended course

Percentage Frequency (%) 240 100.0 151 62.9 88 36.7 1 0.4 0 0 199 82.9 41 17.1 198 82.5 42 17.5

6.2.2 Training Preparation The instructors evenly divided the population into two groups of 120 students. The experimental group, namely group A, was instructed to run the PARSAT independently while utilizing the system’s adaptability. For instance, the modeling of students’ knowledge levels enabled them to watch video tutorials of various lengths and rotate 3D objects of various complexity to see and comprehend their structures, and generally engage in various learning activities adjusted to their unique profiles. The control group, namely group B, used the same instructional material and exercises, without any customization based on the students’ individual profiles. The

134

6  Multi-model Evaluation of the Artificial Intelligence-Enhanced PARSAT AR Software

students were given instructions on how to carry out the learning activities, which included the same content and approach of the learning activity presented in group A. However, the visualization and explanation techniques were different for both groups. The experimental group with AR application could use PARSAT and see the system in action on their smartphones and/or tablets and leverage the proposed framework to derive pedagogically meaningful semantics, while group B did not use PARSAT.  The educators were involved in the educational procedure in both groups. To carry out the measurement tests, a procedure was implemented for each participant in both the experimental and control groups. The PSVT:R test was administered on the days and times corresponding to the start of individual training sessions for the experimental group participants. Each participant received a designated time and day for their training sessions, which were spread across various days. Each session, lasting 45 min in duration, was individually administered to distinct groups of participants. These sessions were distributed across various days throughout the week. Both the experimental and control groups were again contacted to conduct the tests, after each participant had finished the sessions and, consequently, the training. Pre- and posttest results from the group that engaged in a visuospatial mental task were acquired in order to compare and confirm if there were significant differences. Under the guidance of the three faculty members, the students were handed questionnaires to complete at the completion of the semester. The questions were organized based on the Lynch-Ghergulescu framework.

6.3 t-Test Analysis of Students’ Feedback At the end of the semester, after the successful completion of all three courses, the two groups were delivered a questionnaire [4, 15, 16] and were asked to respond to the following questions, using a 10-point Likert scale ranging from “Not at all” (0), to “Very much” (10): • How much did the activities match your level of knowledge? (Q1); • Was the quantity of the activities used efficient? (Q2); • Did the activities’ level of complexity enhance your learning? (Q3). To further evaluate the statistical significance of the results and address the research inquiries, a t-test was employed. The statistical significance of questions 1, 2, and 3 is detailed in Tables 6.3, 6.4 and 6.5. First, two conditions are formed, namely an experimental condition, in which students receive the AR-assisted approach, and a control condition in which they do not. The two conditions were compared to define whether the difference between them was clear enough or not. In this context, the t-test was utilized to ascertain

6.3  t-Test Analysis of Students’ Feedback Table 6.3 t-Test results of Q1

Table 6.4 t-Test results of Q2

Table 6.5 t-Test results of Q3

135

Mean Variance Observations Hypothesized mean difference df t Stat P (T ≤ t) one-tail t Critical one-tail P (T ≤ t) two-tail t Critical two-tail

Group A Group B 8.236 6.581 1.053 0.368 120 120 0 239 16.900