Biomedical Visualisation: Volume 11 (Advances in Experimental Medicine and Biology, 1356) [1st ed. 2022] 3030877787, 9783030877781

Table of contents :
Preface
Acknowledgements
About the Book
Contents
Editor and Contributors
1: Creating Interactive Three-Dimensional Applications to Visualise Novel Stent Grafts That Aid in the Treatment of Aortic Ane...
1.1 Introduction
1.2 Background
1.2.1 Aortic Aneurysm Background
1.2.1.1 Thoracic Aortic Aneurysms
1.2.1.2 Abdominal Aortic Aneurysms
1.2.2 Surgical Interventions for AAAs and TAAs
1.2.2.1 Open Surgical Repair and Endovascular Aneurysm Repair of AAAs
1.2.2.2 Open Surgical Repair and Endovascular Aneurysm Repair of TAAs
1.2.3 Potential of Medical Visualisations for Surgical Techniques
1.2.3.1 Imaging Modalities in a Healthcare Setting
1.2.3.2 Public Engagement for Medical Visualisation
1.3 Methods
1.3.1 Conceptual Development (Storyboard/Outline)
1.3.2 Digital 3D Content Production
1.3.2.1 Segmentation of the Aorta, Kidneys and Associated Vessels
1.3.2.2 Bifrost Visual Programming
1.3.2.2.1 Voxel Volume Remeshing Using Bifrost Graph Editor
1.3.2.3 Retopology and Sculpting
1.3.2.4 Modelling of the Heart
1.3.2.5 Modelling of Relay Endograft
1.3.2.6 Modelling of Fenestrated Anaconda Endograft
1.3.2.6.1 Wires and Stitching of Stent Graft
1.3.2.6.2 Stitches and Fine Details of Graft
1.3.2.6.3 Additional Stent Body Models
1.3.2.6.4 Deployment Devices
1.3.2.7 Texturing in Substance Painter
1.3.2.8 Informational Animations
1.3.2.8.1 Animations for the Fenestrated Anaconda Stent Graft
1.3.2.8.2 Animations for the Proximal Relay Stent Graft
1.3.2.8.3 Red Blood Cell Flow Animations
1.3.2.8.4 Post Processing
1.3.2.9 Application Development
1.3.2.9.1 Home Screen
1.3.2.9.2 Features Section
1.3.2.9.3 Clinical Performance and Deployment Sections
1.4 Results
1.4.1 Outcomes from Evaluating the Finished Application with Clinical Professionals
1.5 Discussion
1.5.1 Discussion of Development Process
1.5.2 Discussion of Application Feedback
1.5.3 Benefits and Drawbacks of the Application/3D Visualisation Technique
1.5.4 Limitations
1.5.5 Further Development
1.6 Conclusion
References
2: Using Confocal Microscopy to Generate an Accurate Vascular Model for Use in Patient Education Animation
2.1 Introduction
2.2 Blood Pressure
2.3 Blood Pressure Regulation
2.4 Pathophysiology of Hypertension
2.5 Peripheral Resistance Artery Structure and Vascular Remodelling in Hypertension
2.6 Treatment of Hypertension
2.7 Medication Adherence
2.8 Patient Education Can Improve Medication Adherence
2.9 Generating Digital 3D Models Using Confocal Microscopy
2.10 Building a Complete Vessel 3D Model from a Partial Confocal Microscopy Dataset
2.11 Modelling the Tunica Intima
2.12 Tunica Media
2.13 Tunica Externa
2.14 Simple Effects in Animation
2.15 Vascular Wall Remodelling Using Blend Shapes
2.16 Maya´s MASH Toolkit
2.17 Materials (Shaders)
2.18 Lighting
2.19 Rendering
2.20 Results
2.21 Discussion and Evaluation
References
3: Methods and Applications of 3D Patient-Specific Virtual Reconstructions in Surgery
3.1 Introduction
3.2 Methods of 3D Virtual Reconstructions
3.2.1 Segmentation
3.2.1.1 Manual Segmentation
3.2.1.2 Algorithmic Approaches to Segmentation
3.2.2 Rendering Methods for 3D Virtual Models
3.2.2.1 Volumetric Rendering
3.2.2.2 Surface Rendering Techniques
3.2.3 Post-Processing of Surface Polygon Mesh
3.2.3.1 Decimation
3.2.3.2 Smoothing
3.2.4 Advanced 3D Modelling Techniques
3.2.4.1 Complex 3D Modelling and Digital Sculpture
3.2.4.2 Retopology
3.2.4.3 UV Unwrapping
3.2.4.4 Texture Maps and Physically Based Rendering
3.3 Applications of 3D Models in Surgical Practice
3.3.1 3D Models in Surgical Planning
3.3.1.1 Anatomical Understanding
3.3.1.2 Patient-Specific Simulation
3.3.1.3 Resection Planning
3.3.1.4 Reconstruction
3.3.2 Intraoperative Navigation
3.3.3 3D Models in Surgical Patient Education
3.4 Conclusion
References
4: Proof of Concept for the Use of Immersive Virtual Reality in Upper Limb Rehabilitation of Multiple Sclerosis Patients
4.1 Rationale
4.2 Multiple Sclerosis and Conventional Physiotherapy
4.3 Virtual Reality-Based Rehabilitation
4.3.1 Interaction
4.3.2 Visualisation
4.3.3 HMDs in MS Rehabilitation
4.4 Treatment Adherence and Motivation
4.4.1 Feedback
4.5 Aims and Objectives
4.6 Methods
4.6.1 Workflow (Fig. 4.1)
4.6.1.1 Materials
4.6.2 Design and Development Process
4.7 Developmental Outcomes
4.7.1 Menu Scene
4.7.2 Piano Scene
4.7.3 Maze Scene
4.7.4 Evaluation
4.7.4.1 Participants
4.7.4.2 Experimental Set-Up and Procedure
4.7.4.3 Ethics
4.7.4.4 Data Analysis
4.8 Results
4.9 Discussion
4.9.1 Future Works
4.10 Conclusion
References
5: Virtual Wards: A Rapid Adaptation to Clinical Attachments in MBChB During the COVID-19 Pandemic
5.1 Introduction
5.2 Theoretical Underpinnings
5.2.1 Dual-Process Theory
5.2.2 Script Theory
5.2.3 Cognitive Load Theory
5.2.4 Situated Cognition
5.3 Technological Considerations
5.3.1 Flexibility of Content
5.3.2 Inclusion of Automatically Marked Questions
5.3.3 Control over Non-linear Lesson Flow
5.3.4 Large Amount of Information in a Single Click
5.3.5 Embedding H5G Interactive Content
5.3.6 Tips for Virtual Ward Developers
5.4 Description of the Virtual Wards
5.4.1 The Content Covered by the Virtual Wards
5.4.2 The Format of the Modules
5.4.3 The Interactive Cases
5.4.3.1 Setting the Scene
5.4.3.2 Interactive History-Taking
5.4.3.3 Observations and Examination
5.4.3.4 Investigations: Selection and Interpretation
5.4.3.5 Refining the Differential
5.4.3.6 Management
5.5 Evaluation and Future
5.5.1 Asynchronous Engagement with Virtual Wards
5.5.2 Issues Working with Multiple New Technologies
5.5.3 Clinician Time Involved to Create Content
5.5.4 Simultaneous Virtual Wards
5.5.5 Quality Control of Benevolent Contributor Content
5.5.6 A Reflection on the Faculty Experience
5.5.7 The Students´ Perspective
5.5.7.1 The Virtual Ward Format
5.5.7.2 Feedback on Content
5.5.7.3 Amount of Content
5.5.7.4 Technical Difficulties
5.5.7.5 Loss of Clinical Contact
5.5.8 Lessons Learnt
5.6 Tips for Setting Up Virtual Wards
5.7 The Future of Virtual Wards
References
6: Artificial Intelligence: Innovation to Assist in the Identification of Sono-anatomy for Ultrasound-Guided Regional Anaesthe...
6.1 Introduction
6.2 Part 1: Challenges in Ultrasound Image Interpretation and Ultrasound-Guided Regional Anaesthesia
6.2.1 What Is Ultrasound-Guided Regional Anaesthesia?
6.2.2 Why Is Regional Anaesthesia Difficult?
6.2.2.1 Selection of the Right Block
6.2.2.2 Acquiring and Interpreting an Optimised Ultrasound Image
6.2.2.2.1 Operator Dependence
6.2.2.2.2 Anatomical Variation
6.2.2.2.3 Learning Materials Depict Ideal Versions of Sono-anatomy
6.2.2.2.4 Comorbidity
6.2.2.2.5 Inattentional Blindness
6.2.2.2.6 Satisfaction of Search
6.2.2.2.7 Fatigability
6.2.2.3 Planning a Safe Needle Path and Visualising the Needle Tip
6.2.2.4 Ensuring Accurate Deposition of Local Anaesthetic Around the Target Structure
6.2.2.5 Post-Procedure Monitoring Both to Ensure Effect and to Monitor for any Complications
6.2.3 Education in Ultrasound-Guided Regional Anaesthesia
6.3 Part 2: An Introduction to Artificial Intelligence for Clinicians
6.3.1 What Is Artificial Intelligence?
6.3.2 Machine Learning Categories
6.3.3 The Computational Problem
6.3.4 Rule-Based vs Model-Based Techniques
6.3.4.1 Rule-Based Techniques
6.3.4.2 Model-Based Techniques
6.3.5 Convolutional Neural Networks
6.3.6 The U-Net Architecture
6.3.7 How Models Train
6.3.8 Model Evaluation
6.4 Part 3: The Current State of AI in Ultrasound Image Interpretation for Ultrasound-Guided Regional Anaesthesia
6.4.1 How Can Technology Be Used to Augment UGRA?
6.4.2 Summary of Different Approaches
6.4.3 Segmentation
6.4.3.1 Deep Learning Approaches
6.4.3.2 Non-deep Learning Approaches
6.4.4 Tracking Methods
6.4.4.1 How Does Tracking Fit in with Segmentation?
6.4.4.2 Approaches
6.4.5 Summary and Future Directions
6.5 Part 4: A Case Study: ScanNav Anatomy Peripheral Nerve Block
6.6 Part 5: The Future: Artificial Intelligence and Ultrasound-Guided Regional Anaesthesia
6.6.1 Supporting Practice
6.6.2 Changing How We Learn
6.6.3 The Extra Dimension
6.6.4 The Future of Clinical Practice
References
7: A Systematic Review of Randomised Control Trials Evaluating the Efficacy and Safety of Open and Endoscopic Carpal Tunnel Re...
7.1 Introduction
7.1.1 Carpal Tunnel Syndrome
7.1.2 The Surgical Interventions
7.1.3 Aims and Objectives
7.2 Methods
7.2.1 Study Identification
7.2.2 Study Screening and Selection
7.2.3 Assessment of Patient Outcomes
7.2.4 Risk of Bias Assessment
7.2.5 Data Analysis
7.3 Results
7.3.1 Study Identification, Screening and Inclusion
7.3.2 Study Characteristics
7.3.3 Patient Outcomes
7.3.4 Risk of Bias Assessment
7.4 Discussion
7.4.1 Main Findings
7.4.2 Study Quality
7.4.3 Limitations
7.4.4 Conclusions
Appendices
Appendix 1. Table of Individual Participant and Study Characteristics
Appendix 2. Table of Participant Outcome Assessment
Appendix 3. Table of Individual Study Bias Assessment
Appendix 4. Characteristics of Excluded Studies
References
Included References
Excluded References
Additional References
8: Exploring Visualisation for Embryology Education: A Twenty-First-Century Perspective
8.1 Introduction
8.2 History of Visualisation in Embryology and Challenges in the Twenty-First Century
8.2.1 In the Nineteenth Century
8.2.2 In the Twentieth Century
8.2.3 In the Twenty-First Century
8.2.3.1 Challenges in Embryology Teaching in the Twenty-First Century
8.3 Learning Theories
8.3.1 Cognitive Load Theory
8.4 Current Resources in Embryology Teaching
8.4.1 Videos and YouTube
8.4.2 Animations
8.4.3 Virtual Reality
8.4.4 Virtual Dissection Tables
8.5 Summary of Evidence-Based Studies on Using Visualisation in Embryology Teaching
8.6 Case Study: Integrating 3D Embryology Learning Resources Within a Medical School Curriculum
8.6.1 Educational Context
8.6.1.1 Pedagogical Basis
8.6.1.2 Pre-pandemic Curriculum
8.6.1.3 Post-Pandemic Curriculum
8.6.2 Pre-Covid-19 Innovations for Embryology Learning
8.6.2.1 Social Media and Creative Art-Based Approaches
8.6.2.2 Development of a Prototype Digital Embryology Resource
8.6.3 Approaches to Asynchronous Embryology Education During Covid-19
8.6.3.1 Integrated Embryology VLE Tutorial
8.6.3.2 HDBR Atlas
8.6.3.3 Three-Dimensional Atlas of Human Embryology
8.7 Conclusion
References
9: How Artificial Intelligence and Machine Learning Is Assisting Us to Extract Meaning from Data on Bone Mechanics?
9.1 An Introduction to the Book Chapter
9.2 The Applications of Artificial Intelligence and Machine Learning to Bone Mechanics Research
9.2.1 What Are Artificial Intelligence and Machine Learning?
9.2.2 Richness and Abundance of Data as Well as Powerful Computational Tools Motivate the Application of ML in Bone Mechanics
9.2.3 Main Areas of Bone Mechanics Where Machine Learning Is Worth-Employing
9.3 Machine Learning Algorithms
9.3.1 Types of Machine Learning Based on Learning Paradigm
9.3.2 Main Steps Involved in Machine Learning
9.3.3 Performance Metrics
9.3.4 Training Algorithm
9.3.5 Training, Validation, and Testing Datasets
9.4 Artificial Neural Networks
9.5 Applications of Artificial Neural Networks to Bone Mechanics
9.6 Perspectives, Conclusions, and Future Directions
References
10: Visual Communication and Creative Processes Within the Primary Care Consultation
10.1 Ethics
10.2 Combining Visual Communication and the Medical Consultation
10.3 Medical Consultation Models
10.4 Examples of Illustration and Visual Communication in my GP Consultations
10.5 Congestive Heart Failure
10.6 Ear Nose and Throat Conditions
10.7 Analogy and Metaphor
10.8 Patient Drawings and Pain
10.9 Diabetes
10.10 Gynaecology
10.11 Urology
10.12 Colour
10.13 Inclusive Visual Communication in Dementia, Autistic Spectrum Disorder and Learning Disability
10.14 Art Therapy and Use of Allegory
10.15 Summary
References
11: Digital 2D, 2.5D and 3D Methods for Adding Photo-Realistic Textures to 3D Facial Depictions of People from the Past
11.1 Introduction
11.1.1 Existing Facial Reconstruction Methods
11.1.2 What Is the Purpose of a Facial Depiction?
11.2 3D Digital Texture Methods
11.2.1 2D Digital Composite Method
11.2.1.1 Workflow
11.2.2 3D Digital Painting and Rendering Method
11.2.2.1 Workflow
11.2.3 2.5D Digital Composite Method
11.2.3.1 Workflow
11.3 Discussion
11.3.1 Comparing Methods for Adding Digital Textures to 3D Facial Reconstructions
11.3.2 Artistic Proficiency and Cognitive Biases
11.4 Conclusion
References
12: Teaching with Cadavers Outside of the Dissection Room Using Cadaveric Videos
12.1 Introduction
12.1.1 Transition During Covid-19
12.1.2 Anatomy at Brighton and Sussex Medical School
12.2 Cadaveric Videos
12.2.1 Student Opinion
12.2.2 Learning Gain
12.2.3 Engagement
12.3 Cognitive Load Theory
12.3.1 Split Attention and Modality Effects
12.3.2 Task Complexity and Self-Efficacy
12.3.3 Task Fidelity and Affect
12.4 Sharing Cadaveric Images Online with Students
12.4.1 Opportunity to Develop Digital Professionalism and Fluency
12.4.2 Storage of Cadaveric Images
12.4.3 Existing Online Learning Resources Versus Bespoke Cadaveric Video Content
12.5 Conclusion
References
13: A Novel Cadaveric Embalming Technique for Enhancing Visualisation of Human Anatomy
13.1 History of Embalming
13.2 Modern Approaches to Cadaveric Preservation
13.2.1 Phenol
13.2.2 Formaldehyde
13.2.3 Thiel
13.2.4 Alternative Fixatives
13.2.5 Fresh-Frozen Preservation
13.3 Learning and Teaching with Cadavers
13.3.1 Curricular Integration
13.3.2 Visualisation, Sensation and Emotion
13.4 The Newcastle Experience
13.4.1 Educational and Technical Context
13.4.2 Newcastle Formaldehyde-Phenol Mix Embalming
13.4.3 Newcastle-WhitWell Embalming Protocol
13.5 Summary, Conclusions, and Implications for Practice
References
14: Assessing the Impact of Interactive Educational Videos and Screencasts Within Pre-clinical Microanatomy and Medical Physio...
14.1 Introduction
14.1.1 The Use of Video in Clinical Anatomy
14.1.2 The Use of Screencasts in Clinical Anatomy
14.1.3 The Use of Interactive Video in Anatomy Education
14.1.4 How Do These Resources Improve Learning?
14.1.4.1 The Spatial-Contiguity Principle
14.1.4.2 The Temporal Contiguity Principle
14.1.4.3 The Modality Principle
14.1.5 Methods for Assessing Learning Gain
14.1.6 Rationale for a Study in Basic Medical Sciences
14.2 Aims
14.3 Methods
14.3.1 Video Production and Design
14.3.2 Participant Recruitment
14.3.3 Knowledge Testing
14.4 Results
14.4.1 Demographics
14.4.2 Knowledge Testing
14.4.2.1 Histology Resources: Learning Gain and Retention
14.4.2.2 Pain Physiology Resources: Learning Gain and Retention
14.4.3 Student Attitudes, Perceptions, and User Experience
14.4.3.1 Standalone Questions: Post-teaching Perceptions
14.4.3.2 Histology Perceptions: Paired Questions
14.4.3.3 Physiology Perceptions: Paired Questions
14.4.4 Comparison Between Cohorts
14.4.4.1 Perceived Confidence
14.5 Discussion
14.5.1 Introduction
14.5.2 Learning Gain and Knowledge Retention
14.5.3 Assessing Interactive Video
14.5.4 Curriculum Integration of Video Resources
14.5.5 Student Perceptions and Preferences
14.5.6 Screencasts and Standard Video Formats
14.5.7 The Learner Experience
14.5.8 Methodological Approaches
14.5.9 Limitations
14.5.10 Future Work
14.6 Conclusion
References
Correction to: How Artificial Intelligence and Machine Learning Is Assisting Us to Extract Meaning from Data on Bone Mechanics?
Correction to: Chapter 9 in: P. M. Rea (ed.), Biomedical Visualisation, Advances in Experimental Medicine and Biology 1356, ht...

Citation preview

Advances in Experimental Medicine and Biology 1356

Paul M. Rea   Editor

Biomedical Visualisation Volume 11

Advances in Experimental Medicine and Biology Volume 1356 Series Editors Wim E. Crusio, Institut de Neurosciences Cognitives et Intégratives d’Aquitaine, CNRS and University of Bordeaux, Pessac Cedex, France Haidong Dong, Departments of Urology and Immunology, Mayo Clinic, Rochester, MN, USA Heinfried H. Radeke, Institute of Pharmacology & Toxicology, Clinic of the Goethe University Frankfurt Main, Frankfurt am Main, Hessen, Germany Nima Rezaei , Research Center for Immunodeficiencies, Children’s Medical Center, Tehran University of Medical Sciences, Tehran, Iran Ortrud Steinlein, Institute of Human Genetics, LMU University Hospital, Munich, Germany Junjie Xiao, Cardiac Regeneration and Ageing Lab, Institute of Cardiovascular Science, School of Life Science, Shanghai University, Shanghai, China

Advances in Experimental Medicine and Biology provides a platform for scientific contributions in the main disciplines of the biomedicine and the life sciences. This series publishes thematic volumes on contemporary research in the areas of microbiology, immunology, neurosciences, biochemistry, biomedical engineering, genetics, physiology, and cancer research. Covering emerging topics and techniques in basic and clinical science, it brings together clinicians and researchers from various fields. Advances in Experimental Medicine and Biology has been publishing exceptional works in the field for over 40 years, and is indexed in SCOPUS, Medline (PubMed), Journal Citation Reports/Science Edition, Science Citation Index Expanded (SciSearch, Web of Science), EMBASE, BIOSIS, Reaxys, EMBiology, the Chemical Abstracts Service (CAS), and Pathway Studio. 2020 Impact Factor: 2.622 More information about this series at https://link.springer.com/bookseries/5584

Paul M. Rea Editor

Biomedical Visualisation Volume 11

Editor Paul M. Rea Anatomy Facility, School of Life Sciences University of Glasgow Glasgow, United Kingdom

ISSN 0065-2598 ISSN 2214-8019 (electronic) Advances in Experimental Medicine and Biology ISBN 978-3-030-87778-1 ISBN 978-3-030-87779-8 (eBook) https://doi.org/10.1007/978-3-030-87779-8 # The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2022, corrected publication 2023 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG. The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Preface

The utilisation of technologies in the biomedical and life sciences, medicine, dentistry, surgery, veterinary medicine and surgery, and the allied health professions has grown at an exponential rate over recent years. The way we view and examine data now is significantly different to what has been done over recent years. With the growth, development and improvement of imaging and data visualisation techniques, the way we are able to interact with data is much more engaging than it has ever been. These technologies have been used to enable improved visualisation in the biomedical fields, but also how we engage our future generations of practitioners when they are students within our educational environment. Never before have we had such a wide range of tools and technologies available to engage our end-stage user. Therefore, it is a perfect time to bring this together to showcase and highlight the great investigative works that is going on globally. This book will truly showcase the amazing work that our global colleagues are investigating, and researching, ultimately to improve student and patient education, understanding and engagement. By sharing best practice and innovation we can truly aid our global development in understanding how best to use technology for the benefit of society as a whole. Glasgow, UK

Paul M. Rea

v

Acknowledgements

I would like to truly thank every author who has contributed to the eleventh edition of Biomedical Visualisation. By sharing our innovative approaches we can truly benefit students, faculty, researchers, industry, and beyond, in our quest for the best uses of technologies and computers in the field of life sciences, medicine, the allied health professions, and beyond. In doing so, we can truly improve our global engagement and understanding about best practice in the use of these technologies for everyone. Thank you! I would also like to extend a personal note of thanks to the team at Springer Nature who have helped make this possible. The team I have been working with have been so incredibly kind and supportive, and without you, this would not have been possible. Thank you kindly!

vii

About the Book

Following on from the success of the first ten volumes, Biomedical Visualisation, Volume 11, will demonstrate the numerous options we have in using technology to enhance, support, and challenge education, clinical settings, and professional training. The chapters presented here highlight the wide use of tools, techniques, and methodologies we have at our disposal in the digital age. These can be used to image the human body, to educate patients, the public, faculty, and students in the plethora of how to use cutting-edge technologies in visualising the human body and its processes, to create and integrate platforms for teaching and education, to visualise biological structures and pathological processes, and to aid visualisation of the historical arenas. The chapters presented in this volume cover such a diverse range of topics, with something for everyone. We present here chapters on 3D visualising novel stent grafts to aid treatment of aortic aneurysms, confocal microscopy constructed vascular models in patient education, 3D patient-specific virtual reconstructions in surgery, virtual reality in upper limb rehabilitation in patients with multiple sclerosis, and virtual clinical wards. In addition, we present chapters on artificial intelligence in ultrasoundguided regional anaesthesia, carpal tunnel release visualisation techniques, visualising for embryology education, and artificial intelligence data on bone mechanics. Finally, we conclude with chapters on visualising patient communication in a general practice setting, digital facial depictions of people from the past, instructor made cadaveric videos, novel cadaveric techniques for enhancing visualisation of the human body, and finally, interactive educational videos and screencasts.

ix

Contents

1

Creating Interactive Three-Dimensional Applications to Visualise Novel Stent Grafts That Aid in the Treatment of Aortic Aneurysms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sara Bakalchuk, Caroline Walker, Craig Daly, Louise Hill, and Matthieu Poyade

1

2

Using Confocal Microscopy to Generate an Accurate Vascular Model for Use in Patient Education Animation . . . . . 31 Angela Douglass, Gillian Moffat, and Craig Daly

3

Methods and Applications of 3D Patient-Specific Virtual Reconstructions in Surgery . . . . . . . . . . . . . . . . . . . . . . . . . . . Jordan Fletcher

53

Proof of Concept for the Use of Immersive Virtual Reality in Upper Limb Rehabilitation of Multiple Sclerosis Patients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rachel-Anne Hollywood, Matthieu Poyade, Lorna Paul, and Amy Webster

73

4

5

Virtual Wards: A Rapid Adaptation to Clinical Attachments in MBChB During the COVID-19 Pandemic . . . . 95 Camille Huser, Kerra Templeton, Michael Stewart, Safiya Dhanani, Martin Hughes, and James G. Boyle

6

Artificial Intelligence: Innovation to Assist in the Identification of Sono-anatomy for Ultrasound-Guided Regional Anaesthesia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 James Lloyd, Robert Morse, Alasdair Taylor, David Phillips, Helen Higham, David Burckett-St. Laurent, and James Bowness

7

A Systematic Review of Randomised Control Trials Evaluating the Efficacy and Safety of Open and Endoscopic Carpal Tunnel Release . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 Eilidh MacDonald and Paul M. Rea

xi

xii

Contents

8

Exploring Visualisation for Embryology Education: A Twenty-First-Century Perspective . . . . . . . . . . . . . . . . . . . . 173 Eiman M. Abdel Meguid, Jane C. Holland, Iain D. Keenan, and Priti Mishall

9

How Artificial Intelligence and Machine Learning Is Assisting Us to Extract Meaning from Data on Bone Mechanics? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 Saeed Mouloodi, Hadi Rahmanpanah, Colin Burvill, Colin Martin, Soheil Gohari, and Helen M. S. Davies

10

Visual Communication and Creative Processes Within the Primary Care Consultation . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 Holly Quinton

11

Digital 2D, 2.5D and 3D Methods for Adding Photo-Realistic Textures to 3D Facial Depictions of People from the Past . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245 Mark Roughley and Ching Yiu Jessica Liu

12

Teaching with Cadavers Outside of the Dissection Room Using Cadaveric Videos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281 Danya Stone, Catherine M. Hennessy, and Claire F. Smith

13

A Novel Cadaveric Embalming Technique for Enhancing Visualisation of Human Anatomy . . . . . . . . . . . . . . . . . . . . . . 299 Brian Thompson, Emily Green, Kayleigh Scotcher, and Iain D. Keenan

14

Assessing the Impact of Interactive Educational Videos and Screencasts Within Pre-clinical Microanatomy and Medical Physiology Teaching . . . . . . . . . . . . . . . . . . . . . . 319 Alistair Robson, Yarrow Scantling-Birch, Stuart Morton, Deepika Anbu, and Scott Border

Correction to: How Artificial Intelligence and Machine Learning Is Assisting Us to Extract Meaning from Data on Bone Mechanics? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C1 Saeed Mouloodi, Hadi Rahmanpanah, Colin Burvill, Colin Martin, Soheil Gohari, and Helen M. S. Davies

Editor and Contributors

About the Editor Paul M. Rea is a Professor of Digital and Anatomical Education at the University of Glasgow. He is qualified with a medical degree (MBChB), an MSc (by research) in craniofacial anatomy/surgery, a PhD in neuroscience, the Diploma in Forensic Medical Science (DipFMS), and an MEd with Merit (Learning and Teaching in Higher Education). He is an elected Fellow of the Royal Society for the Encouragement of Arts, Manufactures and Commerce (FRSA), elected Fellow of the Royal Society of Biology (FRSB), Senior Fellow of the Higher Education Academy, professional member of the Institute of Medical Illustrators (MIMI), and a registered medical illustrator with the Academy for Healthcare Science. Paul has published widely and presented at many national and international meetings, including invited talks. He sits on the Executive Editorial Committee for the Journal of Visual Communication in Medicine, is Associate Editor for the European Journal of Anatomy, and reviews for 25 different journals/publishers. He is the Public Engagement and Outreach lead for anatomy coordinating collaborative projects with the Glasgow Science Centre, NHS, and Royal College of Physicians and Surgeons of Glasgow. Paul is also a STEM ambassador and has visited numerous schools to undertake outreach work. His research involves a long-standing strategic partnership with the School of Simulation and Visualisation, The Glasgow School of Art. This has led to multi-million pound investment in creating world-leading 3D digital datasets to be used in undergraduate and postgraduate teaching to enhance learning and assessment. This successful collaboration resulted in the creation of the world’s first taught MSc Medical Visualisation and Human Anatomy combining anatomy and digital technologies. The Institute of Medical Illustrators also accredits it. It has created college-wide, industry, multi-institutional, and NHS research linked projects for students. Paul is the Programme Director for this degree.

Contributors Deepika Anbu Centre for Learning Anatomical Sciences, Primary Care, Population Sciences and Medical Education, Mailpoint 845, University Hospital Southampton, Southampton, UK xiii

xiv

Sara Bakalchuk Anatomy Facility, School of Life Sciences, College of Medical, Veterinary and Life Sciences, University of Glasgow, Glasgow, UK School of Simulation and Visualisation, Glasgow School of Art, Glasgow, UK Scott Border Centre for Learning Anatomical Sciences, Primary Care, Population Sciences and Medical Education, Mailpoint 845, University Hospital Southampton, Southampton, UK James G. Boyle School of Medicine, University of Glasgow, Glasgow, UK NHS Greater Glasgow and Clyde, Glasgow, UK James Bowness Aneurin Bevan University Health Board, University of Oxford, Oxford, UK David Burkett-St Laurent Royal Cornwall Hospitals NHS Trust, Truro, UK Colin Burvill Department of Mechanical Engineering, The University of Melbourne, Melbourne, Australia Craig Daly School of Life Sciences, College of Medical, Veterinary and Life Sciences, University of Glasgow, Glasgow, UK Helen M. S. Davies Department of Veterinary Biosciences, University of Melbourne, Melbourne, VIC, Australia Safiya Dhanani School of Medicine, University of Glasgow, Glasgow, UK NHS Greater Glasgow and Clyde, Glasgow, UK Angela Douglass Anatomy Facility, School of Life Sciences, College of Medical, Veterinary and Life Sciences, University of Glasgow, Glasgow, UK School of Simulation and Visualisation, Glasgow School of Art, Glasgow, UK Jordan Fletcher St Mark’s Hospital, Harrow, UK Soheil Gohari Department of Mechanical Engineering, University of Melbourne, Melbourne, VIC, Australia Emily Green School of Medical Education, Newcastle University, Newcastle upon Tyne, UK Helen Higham Oxford University Hospitals NHS Foundation Trust, London, UK Louise Hill Terumo Aortic, Renfrewshire, UK Jane C. Holland Department of Anatomy, RCSI University of Medicine and Health Sciences, Dublin, Ireland Rachel-Anne Hollywood Department of Physiotherapy and Paramedicine, Glasgow Caledonian University, Glasgow, UK Martin Hughes School of Medicine, University of Glasgow, Glasgow, UK NHS Greater Glasgow and Clyde, Glasgow, UK

Editor and Contributors

Editor and Contributors

xv

Camille Huser School of Medicine, University of Glasgow, Glasgow, UK Iain D. Keenan School of Medical Education, Newcastle University, Newcastle upon Tyne, UK Jessica Liu Liverpool School of Art and Design, Liverpool John Moores University, Liverpool, UK James Lloyd Aneurin Bevan University Health Board, Newport, UK Eilidh Macdonald Anatomy Facility, School of Life Sciences, College of Medical, Veterinary and Life Sciences, University of Glasgow, Glasgow, UK Colin Martin Department of Veterinary Biosciences, University of Melbourne, Melbourne, VIC, Australia Eiman M. Abdel Meguid Centre for Biomedical Sciences Education, School of Medicine, Dentistry and Biomedical Sciences, Queen’s University Belfast, Belfast, UK Priti Mishall Department of Anatomy and Structural Biology and Department of Ophthalmology and Visual Sciences, Albert Einstein College of Medicine, Bronx, NY, USA Gillian Moffat School of Simulation and Visualisation, Glasgow School of Art, Glasgow, UK Stuart Morton Centre for Learning Anatomical Sciences, Primary Care, Population Sciences and Medical Education, Mailpoint 845, University Hospital Southampton, Southampton, UK Robert Morse Intelligent Ultrasound Limited, Cardiff, UK Saeed Mouloodi Department of Mechanical Engineering, University of Melbourne, Melbourne, VIC, Australia Lorna Paul Department of Physiotherapy and Paramedicine, Glasgow Caledonian University, Glasgow, UK David Phillips Aneurin Bevan University Health Board, Newport, UK Matthieu Poyade School of Simulation and Visualisation, Glasgow School of Art, Glasgow, UK Holly Quinton TBC, London, UK Hadi Rahmanpanah Department of Mechanical Engineering, University of Melbourne, Melbourne, VIC, Australia Paul M. Rea Anatomy Facility, School of Life Sciences, College of Medical, Veterinary and Life Sciences, University of Glasgow, Glasgow, UK Alistair Robson Centre for Learning Anatomical Sciences, Primary Care, Population Sciences and Medical Education, Mailpoint 845, University Hospital Southampton, Southampton, UK Mark Roughley Liverpool School of Art and Design, Liverpool John Moores University, Liverpool, UK

xvi

Yarrow Scantling-Birch Centre for Learning Anatomical Sciences, Primary Care, Population Sciences and Medical Education, Mailpoint 845, University Hospital Southampton, Southampton, UK Kayleigh Scotcher School of Medical Education, Newcastle University, Newcastle upon Tyne, UK Michael Stewart School of Medicine, University of Glasgow, Glasgow, UK NHS Greater Glasgow and Clyde, Glasgow, UK Alasdair Taylor NHS Tayside, Dundee, UK Kerra Templeton School of Medicine, University of Glasgow, Glasgow, UK NHS Greater Glasgow and Clyde, Glasgow, UK Brian Thompson School of Medical Education, Newcastle University, Newcastle upon Tyne, UK Caroline Walker Anatomy Facility, School of Life Sciences, College of Medical, Veterinary and Life Sciences, University of Glasgow, Glasgow, UK School of Simulation and Visualisation, Glasgow School of Art, Glasgow, UK Amy Webster Department of Physiotherapy and Paramedicine, Glasgow Caledonian University, Glasgow, UK

Editor and Contributors

1

Creating Interactive Three-Dimensional Applications to Visualise Novel Stent Grafts That Aid in the Treatment of Aortic Aneurysms Sara Bakalchuk, Caroline Walker, Craig Daly, Louise Hill, and Matthieu Poyade

Abstract

Three-Dimensional (3D) medical animations incorporated into applications are highly beneficial for clinical outreach and medical communication purposes that work towards educating the clinician and patient. Aortic aneurysms are a clinically important area to communicate with multiple audiences about various treatment options; both abdominal and thoracic aortic aneurysms were selected to create 3D animations and applications to educate medical professionals and patients regarding treatment options. Fenestrated endovascular aortic repair (FEVAR) and thoracic endovascular aortic repair (TEVAR) are both tried and tested minimally invasive surgical methods for treating thoracic aortic

S. Bakalchuk · C. Walker (*) Anatomy Facility, School of Life Sciences, College of Medical, Veterinary and Life Sciences, University of Glasgow, Glasgow, UK School of Simulation and Visualisation, Glasgow School of Art, Glasgow, UK e-mail: [email protected] C. Daly School of Life Sciences, College of Medical, Veterinary and Life Sciences, University of Glasgow, Glasgow, UK L. Hill Terumo Aortic, Glasgow, UK M. Poyade School of Simulation and Visualisation, Glasgow School of Art, Glasgow, UK

aneurysms respectively. The Terumo Aortic Custom Relay Proximal Scalloped stent graft and Fenestrated Anaconda stent graft were both designed specifically for these procedures; however, it can be difficult to visually communicate to clinicians and patients in a straightforward way how these devices work. Therefore, we have developed two interactive applications that use 3D visualisation techniques to demonstrate how these aortic devices function and are implemented. The objective of these applications is to engage both clinicians and patients, therefore demonstrating that the addition of anatomically accurate 3D visualisations within an interactive interface would have a positive impact on public engagement while also ensuring that clinicians will have the best possible understanding of the potential uses of both devices, enabling them to exploit their key features to effectively broaden the treatable patient population. Detailed anatomical modelling and animation was used to generate realistic and accurate rendered videos showcasing both products. These videos were integrated into an interactive application within a modern, professional graphic interface that allowed the user to explore all aspects of the stent device. The resulting applications were broken down into three modules: deployment, clinical performance and features. Following application development, these applications were

# The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 P. M. Rea (ed.), Biomedical Visualisation, Advances in Experimental Medicine and Biology 1356, https://doi.org/10.1007/978-3-030-87779-8_1

1

2

S. Bakalchuk et al.

evaluated by professionals in the field. Overall, positive feedback was received regarding the user-friendly nature of the applications and highly effective animations to showcase the products. The clinical applications and feature modules were particularly successful, while the deployment modules had a neutral response. Biomedical applications such as these show great potential for communicating the key features of medical devices and promoting discussion between clinicians and patients; further testing would need to be conducted on a larger group of participants in order to validate the learning effectiveness of the applications. Keywords

Abdominal aortic aneurysms · Thoracic aortic aneurysms · Interactive application · 3D modelling · Medical animation · Medical communication

1.1

Introduction

Communicating novel medical devices in a visually logical and innovative way is essential in the changing healthcare climate, specifically for diseases with alternative treatment options that healthcare professionals may not be aware of. The methodology described in this paper outlines the use of cutting-edge technologies to visualise treatment approaches that clinicians can use to enhance treatment of aortic aneurysms. Aortic aneurysms can be subcategorised into abdominal and thoracic aneurysms, depending on the location of the swelling and site of rupture. Aortic aneurysms are leading causes of death globally. While both abdominal aortic aneurysms (AAAs) and thoracic aortic aneurysms (TAAs) contribute to this, abdominal aneurysms specifically are the tenth leading cause of death globally (Howard et al. 2015). Although advanced treatment options can help reduce mortality rates, it is essential to increase the awareness of clinicians about novel stent grafts that can reduce the risk of rupture and help treat both AAAs and TAAs with complex aortic anatomy (Filardo et al. 2015).

Fenestrated endovascular aortic repair (FEVAR) and thoracic endovascular aortic repair (TEVAR) are tried and tested minimally invasive surgical methods for treating AAAs and TAAs with novel stent grafts. It can be difficult to visually communicate to clinicians how these stent grafts work in real time; thus, the visualisations created would subsequently benefit patient outcome specifications through a series of threedimensional (3D) animations embedded into interactive applications. These applications ensure that clinicians have the best possible understanding of the potential uses of the medical devices, enabling them to exploit key features to effectively broaden the treatable patient population of both AAAs and TAAs. Thus, the resulting applications should have a positive impact while ensuring that clinicians understand the potential uses and customisations of the two visualised stent grafts—the Fenestrated Anaconda stent graft and the Relay Proximally Scalloped stent graft—to treat juxtarenal aneurysms and thoracic aneurysms involving the arch, respectively. Advances in stent graft surgical techniques and treatment options have led to the need for the creation of interactive applications to engage healthcare workers and increase their knowledge of alternative devices for better patient treatment. This visualisation research explores the methodology of creating anatomically correct models and animations for an interactive application. Through collaboration with Terumo Aortic, two applications were generated by creating multiple 3D medical animations from anatomically accurate models for use in public and medical engagement outreach. Various 3D modelling and animation techniques, as well as application development software, were utilised in order to produce the animations that would showcase the most important characteristics of the Fenestrated and Relay devices. The visualisation approaches used to create scientific animations in 3D programs were unique and are discussed throughout the paper. The research and methodologies used to create these scientific animations and corresponding applications show a useful way of communicating advanced medical devices through cutting-edge, intuitive, technological methods.

1

Creating Interactive Three-Dimensional Applications to Visualise Novel. . .

1.2

Background

This chapter reviews the global problem of aortic aneurysms in the elderly population and explores the surgical interventions and technology needed to visualise surgical techniques, thus generating understanding and awareness of the disease.

1.2.1

Aortic Aneurysm Background

An aortic aneurysm can be defined as a swelling or bulging at any point along the aorta. An aneurysm is a blood vessel that results in a permanent dilation by at least 150% of the regular vessel diameter (Shaw et al. 2020). A neurysms are most likely to occur at the section of the aorta where the wall is weakened and has lost its elastic properties, as it does not return to its normal shape after the blood has passed through (British Heart Foundation 2020). If the dilation is left untreated, vessel wall degeneration progresses, leading to an increase in swelling and thinning of the vessel wall. The risk of rupture is directly correlated with the increase of aneurysm diameter (Avishay and Reimon 2020). Aortic aneurysms can be subcategorised into abdominal and thoracic aortic aneurysms, depending on the location of the swelling. Aortic aneurysms are classified as either small or large, 3–5 cm and > 5 cm, respectively (Powell et al. 2011). Approximately of aortic aneurysms occur in the abdomen; the remainder are thoracic aneurysms (Kuivaniemi et al. 2015). The aneurysm requires close monitoring or treatment once detected as the aortic wall can continue to weaken over time and be unable to withstand the pressing blood pressure forces, resulting in rupture (Filardo et al. 2015). The level of stress on the wall is directly proportional to an increase in aortic diameter and is considered to be a significant factor in the growth rate of aneurysms.

1.2.1.1 Thoracic Aortic Aneurysms Thoracic aortic aneurysms (TAAs) make up approximately of aneurysms (British Heart Foundation 2020) and are characterised by swelling or bulging of the aorta within the region of the chest. TAAs may involve one or more aortic segments,

3

either the aortic root, ascending aorta, arch, or descending aorta. Sixty percent of thoracic aortic aneurysms involve the aortic root and/or ascending aorta, involve the descending aorta, involve the arch, and involve the thoracoabdominal aorta (with some involving >1 segment) (Isselbacher 2005). The treatment of thoracic aneurysms differs for each of these segments. If the aneurysm were to rupture, immediate surgery is required to repair the aorta with numerous possible surgical interventions depending on the severity and progression. While open surgery used to be the industry standard for this type of procedure, it is becoming increasingly common to perform minimally invasive endovascular operations using a custom stent graft (Ben Abdallah et al. 2016).

1.2.1.2 Abdominal Aortic Aneurysms Abdominal aortic aneurysm (AAA) is a lifethreatening condition that affects the aorta and account for 75% of all aortic aneurysms (Kuivaniemi et al. 2015). AAAs are often detected incidentally due to their asymptomatic nature (Kuivaniemi et al. 2015). AAAs are responsible for approximately 5000 deaths in the UK every year and more than 175,000 deaths globally (Howard et al. 2015). One percent of deaths in men over 68 years are attributed to ruptured AAA, coming in as the tenth leading cause of death in older men (Avishay and Reimon 2020). The mortality rate of a ruptured AAA is over; thus early diagnosis, monitoring and treatment are critical before the aneurysm’s rupture (Howard et al. 2015). Undetected, untreated AAAs result in expansion, rupture, haemorrhage and often death (Howard et al. 2015).

1.2.2

1.2.2.1

Surgical Interventions for AAAs and TAAs

Open Surgical Repair and Endovascular Aneurysm Repair of AAAs Treatment of large abdominal aortic aneurysms is changing over time to include more advanced surgical techniques for higher-risk patients. Treatment is suggested when the aneurysm reaches 5 cm to 5.5 cm to prevent rupture and its

4

corresponding high mortality rate (Wang et al. 2018). Treatment options include open surgical repair (OSR), endovascular aneurysm repair (EVAR) and FEVAR (Wang et al. 2018). Open surgical repair (OSR) has been the traditional clinical method for AAA (Wang et al. 2018). While rupture has an 80% mortality rate (Dueck et al. 2004; Heikkinen et al. 2002; Ashton et al. 2002), elective surgical repair of unruptured AAA has only a 2–6% 30-day mortality rate (Brown et al. 2012; Cosford and Leng 2007). Endovascular aneurysm repair (EVAR) is a recommended surgical substitute for AAA higher-risk patients who are unsuitable for open repair (De Bruin et al. 2010). However, up to 45% of individuals with AAA are not suitable for EVAR. Advanced AAA cases such as patients with juxtarenal, thoracoabdominal and pararenal aneurysms, or short, angulated or reversed aortic neck anatomy, cannot be treated by EVAR endografts (Dijkstra et al. 2014). This percentage of AAAs are unsuitable for EVAR surgery but can safely undergo fenestrated endovascular aneurysm (FEVAR) device surgery. FEVAR treatment is a minimally invasive surgery that allows the aorta to be repaired in the patient’s groin or arms while preserving blood flow to major arteries (Park et al. 1996). EVAR treatment with a fenestrated endovascular aneurysm (FEVAR) device was introduced in 1996 to combat complex aneurysms (Park et al. 1996). Custom-made devices for FEVAR are now utilised in patients that are higher risk for open repair surgery. These stent grafts allow clinicians to accommodate renal and mesenteric vessels by inserting custom devices where disease progression has occurred, higher up in the aorta (Colgan et al. 2018). FEVAR has since shown high technical success rate, low mortality, and short, intermediate and long-term complication rates (Bungay et al. 2011). As a result, Terumo Aortic has developed a fenestrated graft stent system, the Fenestrated Anaconda™, as seen in Fig. 1.1.

1.2.2.2

Open Surgical Repair and Endovascular Aneurysm Repair of TAAs In order to decide whether to perform elective, pre-emptive aneurysmectomy, the specific risk

S. Bakalchuk et al.

Fig. 1.1 (a) Fenestrated Anaconda stent graft and (b) Fenestrated Anaconda device and aneurysm with angulation device stents and legs (Vascutek Lmtd. 2020)

versus benefit from resection needs to be estimated, which ultimately depends on the surgical method chosen (Elefteriades 2002). Openchest surgical repair using prosthetic graft interposition has been the conventional treatment for TAAs (Abraha et al. 2009). However, despite improvements in surgical procedures, perioperative complications remain significant. The alternative option of thoracic endovascular aneurysm repair (TEVAR) is considered a less invasive and potentially safer technique, with lower morbidity and mortality compared to open surgical repair, making it an appealing therapeutic option (Alsafi et al. 2014). Fortunately, this approach is becoming increasingly common, as both acute and chronic traumatic lesions of the descending aorta can be treated via an endovascular approach in specialised centres, with low morbidity and mortality rates (Kato et al. 1997). Endovascular repair is particularly attractive for treating patients whose associated injuries or comorbid conditions put them at greater risk for the open-chest repair surgery (Kasirajan et al. 2003).

1

Creating Interactive Three-Dimensional Applications to Visualise Novel. . .

The main advantages of the endovascular procedure include shorter time and lower operative risk. If the patient is not affected by other highpriority life-threatening injuries, endovascular repair should be performed first before any other surgical treatment in order to eliminate the risk of sudden aortic rupture (Ferrari et al. 2006). Another benefit of this surgical technique is the absence of cardiopulmonary bypass and the low-dose systemic heparinisation (Ferrari et al. 2006). Despite great achievements from endovascular stent grafts, several complications of endovascular stenting have remained. Although complications do not occur frequently, endoleak, stent collapse, subclavian occlusion, stroke, embolisation, bronchial obstruction, implant syndrome, dissection, migration and paralysis may develop (Karmy-Jones et al. 2009). When the treatment segment of the thoracic aorta, most commonly the aortic arch, present certain anatomical challenges, technically complex stent graft designs are sometimes needed. Therefore, endovascular treatment of aortic arch disease may require a custom-made endograft to maintain the patency of the supra-aortic trunks (SATs) in the event of a short healthy proximal aortic neck (Fernández-Alonso et al. 2018). In specific cases such as these, the custom Terumo Aortic Relay proximal scalloped stent graft has been particularly successful (Fig. 1.2). Consequently, anatomically accurate and interactive 3D visualisations of this product would be helpful for communicating its correct usage and other various benefits to both patients and clinicians.

1.2.3

1.2.3.1

Potential of Medical Visualisations for Surgical Techniques

Imaging Modalities in a Healthcare Setting In today’s clinical setting, medical imaging is an essential component of the entire healthcare continuum, from wellness and screening to early diagnosis, treatment selection and follow-up. Some of the most common imaging modalities used today are computed tomography (CT) and magnetic resonance imaging (MRI) (Liu et al. 2007). CT imaging allows for clinicians to gain

5

high levels of detail for anatomical structures and soft tissue, while MRI [T1 and T2] can assess not only morphology but also metabolic function detected by changes in blood flow. Advancements in computer graphics have allowed for fast, real-time medical imaging interpretation, giving radiologists the ability to process a huge amount of data, compare prior studies and create multiplanar and three-dimensional image reconstructions. This is done through very thin (fractions of a millimetre) slices obtained, for example, in the DICOM image format, and subsequently segmented in an editing software such as 3D Slicer, Amira, MITK, ITK and Osirix. The 3D volume generated from these imaging modalities allows for accurate and efficient visualisation of the patient’s anatomy and physiology (Bercovich and Javitt 2018). CT and MRI visualisation combined with interactive technology allowing for 3D reconstructions will continue to further the understanding of human disease and allow a personalised approach to treatment plans. The process of converting medical imaging into 3D visualisations follows a general imaging pipeline (Fig. 1.3). A summary of this pipeline is as follows: acquiring data, analysing and visualising medical images for use in diagnosis, education, or research purposes. Medical imaging is especially useful to help guide surgical procedures and enable correct spatial accuracy visualisation in connection with understanding cardiac-related complications. Advanced cardiac CT visualisation technology such as coronary CT angiography (CTA) is an innovative clinical imaging tool that allows a non-invasive and highly specific approach for cardiac diagnosis and treatment (Saremi 2017; Coelho-Filho et al. 2013; Burt et al. 2014). As aforementioned, generation of a 3D model from a CT scan can be used for many reasons to increase patient understanding, surgical planning and informing clinicians understanding of medical devices for specific patient treatment (Bercovich and Javitt 2018).

1.2.3.2

Public Engagement for Medical Visualisation When examining 3D visualisation for education, it is also important to consider how well these technologies are able to educate or otherwise

6

S. Bakalchuk et al.

Fig. 1.2 (a) Terumo Aortic Relay proximal scalloped endograft. (b) Schematic showing a scalloped stent graft with the radiopaque markers used during positioning (Alsafi et al. 2014)

engage a broader audience (Holliman et al. 2009). Public engagement with science is an important aspect of society at a number of levels (Taylor and Wessels 2019). The ability of the public to understand healthcare information is particularly important, as it has been shown that adults without sufficient health literacy are twice as likely to be hospitalised than those with an adequate understanding (Baker et al. 1998; Baker et al. 2002). Furthermore, public engagement with science is vital to scientific progress as well, as the incorporation of new media technologies has increased the opportunities available for scientists to engage the general public in understanding their work (Holliman et al. 2009). Previous work has shown the use of mobile technology or tablets to access information and learning materials from anywhere and at any time is particularly effective for this type of engagement (Ford and Leinonen 2009). This allows for the user to learn at their own pace and time, following whatever information they find interesting and

therefore building a broader knowledge base (Traxler 2007). These research findings define the need for producing a means of visualising various surgical methods to treat aortic aneurysms to clinicians, patients, and the general public. Overall, the use of imaging modalities in healthcare to better treat and diagnose diseases shows the value that visualisations bring to understanding and educating about medical conditions. This displays the need for a public engagement tool for aortic aneurysms that are frequent in the elderly population. These research findings define the need for producing a means of visualising various surgical methods to treat aortic aneurysms to clinicians, patients, and the general public. This informative application, surrounding two different types of aneurysms, will generate awareness regarding surgical methods, customisable treatment options, clinical performance success statistics, risk factors, and awareness of innovative stent grafts for AAA and TAA cases.

1

Creating Interactive Three-Dimensional Applications to Visualise Novel. . .

1.3.1

7

Conceptual Development (Storyboard/Outline)

This stage commenced with conceptual ideation, which subsequently led to the creation of a storyboard and mood board. A contact at Terumo Aortic was consulted regarding exactly what they were looking for within the applications, and their specifications were used to create a storyboard and mood board in keeping with their desires and graphic style. The storyboard specifically allowed for a clear idea of the flow of the applications, as well as identifying the key features to be highlighted. The storyboards for both applications are depicted in Figs. 1.4 and 1.5.

1.3.2

Fig. 1.3 Imaging pipeline

1.3

Methods

The purpose of this project was to create clear, informative, and anatomically accurate interactive applications that communicate the important facts and figures concerning both aforementioned stent grafts. Specifically, 3D animations and anatomical models will be utilised to highlight the key features of the product, its deployment and its clinical performance. The below sections will demonstrate how the surgical techniques work for patients and will help ensure that clinicians have the best possible understanding of the potential uses of the devices. Therefore, these applications will enable clinicians to have an understanding of the alternative medical devices and exploit their key features to effectively broaden the treatable patient population.

Digital 3D Content Production

Once the flow of the applications was developed, 3D content production began for both applications. This was composed of segmentation, retopology, 3D modelling, and texturing. These assets were then used to create animated videos and integrated into the 3D applications within Unity.

1.3.2.1

Segmentation of the Aorta, Kidneys and Associated Vessels Open-source software 3D Slicer for medical image informatics was utilised to segment the aorta and surrounding anatomy that were generated into 3D models and incorporated into the 3D animations. Contrast and brightness were manipulated to allow visualisation of distinct anatomical structures; this allowed for precise segmentation from medical data. Segmentation was completed by using the editor module and utilising the paint brush tool with a threshold of 100 to 1000. Segmentation was repeated, moving through every slide of the dataset and manually segmenting the needed organs. Once the models were created, they were decimated by 20% to

8

Fig. 1.4 TAA application storyboard

Fig. 1.5 AAA application storyboard

S. Bakalchuk et al.

1

Creating Interactive Three-Dimensional Applications to Visualise Novel. . .

9

Fig. 1.6 Fast-Marching lung segmentation

reduce the polycount. Models were then exported in OBJ format for use across Autodesk platforms. The same segmentation process was repeated for the kidney, renals, and celiac vessels. The lung model for the TAA application followed a similar pipeline. It was first created in slicer based off the CT chest pre-set. The FastMarching tool within the editor module was utilised to segment the lungs, while the threshold paint tool was used to segment the trachea and bronchi. The Surface toolbox was then used to smooth the model and fill the holes. Figure 1.6 shows the segmentation process and the final model rendered within 3D Slicer.

1.3.2.2 1.3.2.2.1

Bifrost Visual Programming

Voxel Volume Remeshing Using Bifrost Graph Editor Bifrost Graph is a Maya Plugin with a new visual programming framework that generates voxel volumes from 3D graphics. This was used due to the complexity of the generated segmented models from 3D Slicer; the models were unable to be retopologised automatically with the high polygon count generated in 3D Slicer. The stepby-step process can be found in Fig. 1.7; the images describe the Bifrost workflow utilised to achieve the desired new Maya mesh. This process was applied to all 3D Slicer models, so that there were had manageable polycounts in Maya. The end result is a new model with clean Maya mesh; voxelating and rebuilding the mesh in Bifrost fixed the holes in the mesh, lamina faces, and

nonmanifold geometry. It was important to perform Bifrost visual programming framework to these models from 3D Slicer, so Autodesk would function properly, and the resulting animations would be able to render at quick speed.

1.3.2.3 Retopology and Sculpting Once the 3D Slicer models were converted to usable Maya mesh, they were retopologised to change the mesh from triangular to quadrilateral mesh and ensure full Maya functionality in the modelling and animation editors. See Fig. 1.8 for retopology and remesh workflow. Automatic retopology allowed a lower polygon count by 50–75%, which helped maintain a clean Maya workspace that did not crash. Retopologising also allowed the model to be sculpted, textured, and animated effectively using quadrilateral mesh. All 3D Slicer models were retopologised after undergoing the Bifrost process. After the structures were successfully retopologised, sculpting occurred on the models. This involved using Maya’s sculpt tools to smooth and adjust the models. Figure 1.9 shows the various sculpting tools and a selection of the models before and after being sculpted in Autodesk Maya. A similar process was carried out for the lungs in the TAA application. 1.3.2.4 Modelling of the Heart The heart was modelled manually in Maya with anatomical references. As a reference, the 3D heart model on BioDigital.com was used as this website is very reliable and is highly noted for its

10

S. Bakalchuk et al.

Fig. 1.7 Bifrost visual framework workflow

anatomical accuracy (Qualter et al. 2012). The model was created in Autodesk Maya, by first sculpting a primitive cylinder against the reference images and then extruding a portion of the mesh for the atriums and extruding the various blood vessels. The same process was used for modelling the thoracic aorta and the branch

vessels. The thoracic aorta was manually cut open in the area of the aneurysm to reveal the inner layer of the vessel and then subsequently attached to the abdominal aorta. See Fig. 1.10 for the untextured model and the polygon mesh of the heart and aorta.

1

Creating Interactive Three-Dimensional Applications to Visualise Novel. . .

11

Fig. 1.8 Remesh and retopology workflow

1.3.2.5 Modelling of Relay Endograft The Relay stent graft was modelled using a process of both photogrammetry and manual retopology. First, multiple photographs of the stent graft were taken and imported into Agisoft Metashape for automatic photogrammetry. The photogrammetry process is illustrated in Fig. 1.11. Through this process, a rough mesh was achieved that while unsuitable for automatic retopologising, contained an accurate representation of the stent, which was used as a model reference. To create the model within Maya, the surface of the photogrammetry mesh was made live, and the CV curve tool was used to trace along the outline of the imported mesh to create an accurate

representation of the springs comprising the structure of the stent. Once the springs were traced, duplicated, and aligned, the quad draw tool was utilised to model the fabric connecting each row of wires. Figure 1.12 depicts the process of using the photogrammetry mesh as a reference for creating the shape of the stent using the Maya retopology tools, as well as the resulting model from this process.

1.3.2.6

Modelling of Fenestrated Anaconda Endograft Terumo aortic provided a physical Fenestrated Anaconda stent to reference for modelling a four-fenestration stent graft (Inchinnan, Scotland, UK). Measurements of the physical stent were

12

Fig. 1.9 (a) Autodesk Maya 2020 Sculpting tools used. (b) Kidney before and after smoothing. (c) Superior mesenteric artery before and after smoothing. (d)

S. Bakalchuk et al.

Abdominal aortic aneurysm before and after smoothing. (e) Top aorta before and after smoothing

Fig. 1.10 Final heart mesh for use in animation

taken, and three cylinders created in Maya and taken into Illustrator and measured using the ruler tool to ensure accurate model dimensions; this process is seen in Fig. 1.13.

1.3.2.6.1 Wires and Stitching of Stent Graft After the stent base mesh was created, the CV curve tool was used to depict the wires on the stent. The wires and stitching of the Fenestrated Anaconda needed to be depicted as rolling curves

1

Creating Interactive Three-Dimensional Applications to Visualise Novel. . .

13

Fig. 1.11 Stent photogrammetry in Agisoft Metashape. This still depicts the automatic process by which the application converts a point cloud generated from image data to a polygonal mesh

Fig. 1.12 (a) Process of creating stent based off of photogrammetry mesh. (b) Final untextured model of Relay stent

that were perfectly smooth; this was extremely important to depict as wires that had a slight bend, or were not perfectly even, could cause fractures or occlusion in patients. Dimensions were provided by Terumo Aortic (Fig. 1.14) so

the peaks and valley features were accurately modelled as well as the wires. The valley and legs modelled off of the provided dimensions are seen in Figs. 1.14 and 1.15. Once the stent body and wires were recreated, stitching was

14

S. Bakalchuk et al.

Fig. 1.13 Process of measurements for stent graft starting with base cylinders

Fig. 1.15 Smooth rolling wire curve

Fig. 1.14 Dimensions provided and resulting modelled peak and valley

applied to the model. The stitching for the seams of the stent graft was completed using Maya Plugin, MASH; the process is seen in Fig. 1.16. 1.3.2.6.2 Stitches and Fine Details of Graft To replicate the seams, small cylinders, and torus shapes were added to the stent graft; this can be seen in Fig. 1.17. These were assigned aiStandardSurface materials. 1.3.2.6.3 Additional Stent Body Models Additional models were needed that are used during stent deployment. Figure 1.18 shows the

Iliac Leg model designed in Autodesk Maya by creating a cylinder and then selecting every other edge and inverting the model. Edge loops were applied on either side of this model to create a smooth look and allow for colouring of the wire to be applied to the model. 1.3.2.6.4 Deployment Devices Deploying the stent required the use of five additional models which can be found in Fig. 1.19. These were created on Maya through a series of modelling and referring to the surgical manual for FEVAR surgery.

1

Creating Interactive Three-Dimensional Applications to Visualise Novel. . .

15

Fig. 1.16 MASH process for creating the stent’s stitching

1.3.2.7 Texturing in Substance Painter The application Substance Painter was used in the case of the TAA application to paint realistic textures onto the 3D assets created. For the heart model, once the mesh was completed, it was exported into Substance Painter to automatically unwrap the UVs and paint on various texture maps with a Wacom tablet. This application was utilised to paint the texture of the heart muscle and all of the coronary arteries. Figure 1.20 shows the painting process, as well as the finished product created on substance painter. The resulting texture maps were then exported into Maya and

attached as bitmaps onto various channels on an Arnold aiStandard shader material. Figure 1.21 shows a map of the Arnold texture utilised. The lungs were also exported into Substance Painter for UV unwrapping and manual generation of texture maps. An assortment of the application’s default brush settings was used to paint a texture onto the surface of the model, as well as the bump map paint brushes to make the surface of the lungs appear more realistic. Figure 1.22 depicts the manual brushwork and texturing performed within Substance Painter and a

16

S. Bakalchuk et al.

Fig. 1.17 Stitching details: close-up and far away on rolling curve wires

Fig. 1.18 Anaconda™ Iliac Leg (a, b) created based on the Terumo Iliac Leg photograph (c)

final rendering of the lungs once exported back to Maya. Finally, the stent graft was brought into Substance Painter to add textures and fine details within the stent. This included stitching, wires and a number of other important markers featured on the device. Additionally, a fabric texture was created for the body of the stent and a metal

texture for the wires. Figure 1.23 depicts the process of texturing the model within Substance Painter, as well as the finished product created.

1.3.2.8 Informational Animations The following headlines outline the methodology implemented to generate a series of animations to visualise FEVAR and TEVAR surgical technique

1

Creating Interactive Three-Dimensional Applications to Visualise Novel. . .

Fig. 1.19 Deployment devices. (a) All deployment devices together. (b) Iliac Legs used after Fenestrated Anaconda deployed to connect to associated vessels. (c)

17

Sheath used for deployment of the stent. (d) Deployment tube. (e) Deployment wires for guiding the sheath. (f) Sheath aid tip

Fig. 1.20 Textured heart in Substance Painter: process (a) and final texture map (b)

18

S. Bakalchuk et al.

Fig. 1.21 Graph of aiStandard Arnold texture settings

Fig. 1.22 Lung texturing process in Substance Painter and rendering of the textured model in Maya

for AAA and TAA patients, respectively, and clinical successes after inserting the stents. Different animations per stent were produced for the three modules: deployment, features and clinical performance. 1.3.2.8.1

Animations for the Fenestrated Anaconda Stent Graft Manual sculpting and modelling adjustments to the aorta and associated vessels had to be made for different animations. The faces on the front side of the aorta and renal arteries were manually selected and detached from the main mesh. As a result, a realistic view of the aorta was created to allow for femoral access to be visualised.

All of the deployment tools were animated using the curve warp deformer in Maya. The deformer firstly allows for objects to be deformed along a curve path and enables manipulation of the length scale and offset of the 3D object; this made animating quite a challenge given the factors, such as scale, rotation and smoothed movement, considered when setting key frames. A lattice and cluster system was used to animate the deployment and unsheathing of the stent. Each cluster was animated, and keys set individually to control scaling, position and rotation of the stent unsheathing and expanding. A visualisation of the lattice, clusters and twist handle is seen in Fig. 1.24.

1

Creating Interactive Three-Dimensional Applications to Visualise Novel. . .

19

Fig. 1.23 Texturing the Relay stent in Substance Painter: process and final rendering

The proximal access animation for the feature module was created and animated by using the curve warp deformer. By creating an opaque stent, the camera was able to zoom in and out of the stent to show access from the upper aorta and down into the fenestration. The repositioning animation required the camera to be animated in various positions in order to depict a close-up visualisation of the stent moving, rescaling and inflating after unsheathing, one of the key competitive features of the Fenestrated Anaconda. The versatility animation was also created by animating a camera around the stent to show the various positions and sizes of the fenestrations and how they are uncompromised by stents or wires (Colgan et al. 2018; Dijkstra et al. 2014; Midy et al. 2017). Figure 1.25 shows resulting stills from these two animations. Terumo Aortic required four clinical performance statistics about this specific graft to be showcased in the animations. For this, the Maya

Plugin ‘MASH’ was utilised to animate red blood cells (RBCs) going through the stents and into the vessels. This was an easily accessible system to use and apply changes to animation speed, distance apart, randomness and rotation. The process of creating a MASH network for the RBC animation portion is discussed in more detail below. 1.3.2.8.2

Animations for the Proximal Relay Stent Graft The animation process began by animating the delivery system. To animate the stent lead wire, a cylinder was created, and the curve warp tool was utilised to animate the wire moving along a curve going up the length of the aorta. This was done so that the wire can be portrayed climbing through the aorta and into the aortic arch. Next, the sheath was animated by creating a larger cylinder that follows the same path of the lead wire and lands within the aortic arch. This animation was created by animating the max length of

20

S. Bakalchuk et al.

Fig. 1.24 Stent animation process by lattice, cluster and twist handle

the sheath with the curve warp deformer and repeated so that there was both an inner and outer sheath portrayed. In order to animate the stent deployment, the curve warp deformer was utilised to allow the stent to be deployed along the

curve, followed by a flare deformer, which allowed for the stent to be scaled up as it is deployed. A separate flare deformer was used for the top springs of the stent, to portray them detaching from the tip capture mechanism after

1

Creating Interactive Three-Dimensional Applications to Visualise Novel. . .

21

Fig. 1.25 Proximal and repositionable animation stills to visualise the benefit of these custom surgical techniques

Fig. 1.26 Depiction of stent deployment animation process for TAA animations

the stent is fully deployed. After this, the same curve warp method was used to reverse the direction of the wires and the sheath comprising the delivery system, in order to demonstrate how these components exit the aorta. Figure 1.26 shows the set-up for the delivery system, as well as the flare deformer and curve warp deformer settings utilised for this animation. In order to animate the heart beating, multiple lattice deformers were used on each part of the heart in a synchronised loop. This loop made it so

that three different parts of the heart were expanded and then shrunk in synchrony. Figure 1.27 portrays the lattice deformers and graph editor utilised for this animation within Maya. 1.3.2.8.3 Red Blood Cell Flow Animations To create the animation of the blood cells flowing through the aorta, the MASH tool was used on the previously modelled red blood cell to create a network of particles that flow along predetermined curves. A CV curve was fashioned

22

S. Bakalchuk et al.

Fig. 1.27 Lattice deformers utilised for heart beat animation

Fig. 1.28 MASH network set-up for TAA blood flow animations

for each potential RBC path, and a corresponding MASH network for each of the four paths was created. The paths and the MASH network settings can be seen in Fig. 1.28. A randomness node was also added to the MASH settings so that the rotation and orientation of the cells would be varied. 1.3.2.8.4 Post Processing The application Adobe After Effects was used to compile the multiple series of still images rendered into videos. In order to create the compositions in After Effects, the rendered PNG sequence was selected for each video and then imported as a composition into the software. Once all of the videos were compiled, various features of the application were used to add unique text and graphics.

1.3.2.9

Application Development

1.3.2.9.1 Home Screen Construction of the home screen required combining three relevant renderings in Photoshop with a standard background circle provided by Terumo Aortic to create buttons. These buttons lead the user to the three main sections of the applications: features, deployment and clinical performance. In addition to the home screen buttons, a number of other custom UI features were created that appear within the applications. All of these were designed to fit the Terumo Aortic graphic style while still remaining intuitive and simplistic. Figure 1.29 depicts the home screen for both applications.

1

Creating Interactive Three-Dimensional Applications to Visualise Novel. . .

23

Fig. 1.29 Home screen for TAA application (left) and AAA application (right)

Fig. 1.30 Features section of TAA application (left) and AAA application (right)

1.3.2.9.2 Features Section To create the interactive applications, all the 3D models required were imported into Unity. These were all exported as FBX’s with their corresponding textures and imported into the features section of the applications. A corresponding icon was added to a portion of the model for each feature, and the buttons were coded so that a specific video would come up when the user clicked on a feature. Figure 1.30 shows the ‘Features’ unity scene for both applications with the imported models and custom UI.

Additionally, a script for a progress bar was created with the help of supervisor Dr. Matthieu Poyade. This feature was created underneath the video so that the user can track the video progress and interact with the bar to fast forward or rewind the video. Figure 1.31 depicts the video interface created within Unity for both deployment modules, while Fig. 1.32 depicts the video interface created for both clinical performance modules.

1.3.2.9.3

1.4.1

Clinical Performance and Deployment Sections For the clinical performance and deployment modules, the rendered MP4s and relevant UI were imported into the scene. The videos were then placed onto a raw image with a render texture, so they played seamlessly on the screen. Simple scripts were added in order to create a play button, a pause button, and a replay button.

1.4

Results Outcomes from Evaluating the Finished Application with Clinical Professionals

Following application development, both products underwent a process of evaluation by clinical professionals at Terumo to assess their effectiveness in communicating the key features of the devices to a broader medical audience. For

24

S. Bakalchuk et al.

Fig. 1.31 Deployment section of TAA application (left) and AAA application (right)

Fig. 1.32 Clinical performance section of TAA application (left) and AAA application (right)

the most part, this evaluation process provided valuable feedback about the effectiveness of these applications. Both received high ratings for face validity and content validity across all three sections of the application. However, it was reported that the deployment models and animations within the TAA app could have been improved. Similarly, the Iliac Leg stent used for completing the FEVAR technique in the AAA app could have been improved regarding modelling and texturing techniques. Additionally, the blood flow simulations were reported to be distracting with the texturing of the aorta and red blood cells. There was an overall consensus that the applications were easy to use and very intuitive to figure out. In general, both applications effectively communicated the most important aspects of the device, while the deployment section of the TAA and AAA apps could benefit from a few improvements to enhance its accuracy. In the future, an updated version of these

applications could be created that contains a more detailed and accurate depiction of the stents. While future testing still needs to be conducted, these preliminary results offer a valuable starting point to indicate how these applications will be received by patients and professionals in the field. Moreover, the research and methodologies used to create these scientific animations and corresponding applications show a useful way of communicating advanced medical devices through cutting-edge, intuitive, technological methods.

1.5

Discussion

The aim of this project was to create two userfriendly interactive applications with various 3D visualisations of the two corresponding stent grafts. These visualisations would demonstrate to clinicians how these products function and

1

Creating Interactive Three-Dimensional Applications to Visualise Novel. . .

aid cardiac patients to understand the key features and surgical process of the stent deployment.

1.5.1

Discussion of Development Process

The development process of these applications was simplified by the fact that Terumo Aortic had a clear idea of what information to include in the applications. The ideation and co-design process with the company lead to the joint conclusion that both apps should be broken down into three modules: deployment, features and clinical performance. Once this became clear, designing the layout of the applications and determining the required UI and 3D assets came naturally. The data extraction, modelling, animation and application development each came with their own limitations. Slicer 3D was utilised only to segment the medical data, so this was a fairly quick process. On the other hand, the modelling, specifically of the stent, was a prolonged, capricious process due to multiple factors, including limitations to resources and time. This was especially challenging because not only did the structure of the stent have to be exactly like the Terumo Aortic model, but it also had to be constructed in a manner that allowed for it to be animated. However, modelling of the surrounding anatomy, specifically the heart, aorta, lungs and kidneys, was an enjoyable and exciting learning process. The texturing process for the TAA application was one of the most rewarding aspects of the whole development procedure, as the application Substance Painter made it quite easy to paint hyper realistic and detailed textures directly onto the models. This was especially necessary due to the small coronary arteries in the heart and the many small stiches on the stent. The animation process was the most challenging by far for both applications, particularly the deployment animations. In the case of the AAA app, the discovery of a technique to animate the stent unsheathing provided a successful final outcome, while the TAA application was slightly

25

less accurate, as the curved shape of the stent meant that the aforementioned stent unsheathing technique would not work. The blood flow animations were created seamlessly with the user-friendly and intuitive MASH toolkit within Maya. For the AAA app specifically, the MASH network created a nice outcome for the RBCs to be seen flowing through the Fenestrated Anaconda, Advanta V12 Balloon stent and Iliac Leg stents. In retrospect, this type of project could have been approached differently given the time constraints and feedback received. Nonetheless, the 3D models and animations concluded with a positive outcome. Troubleshooting had to occur to produce Arnold renders that were both high quality and reflective of the time allocated for rendering. Difficulties with the lighting and quality of the rendering versus time to render each frame were considered, and render sequences were divided up amongst computers. Rendering was a timeconsuming, but ultimately rewarding, process. The application creation process was rather simple on the other hand, as the deployment and clinical performance scenes consisted mainly of imported videos. Scripting the video interface started off reasonably simple as well, as the coding required for adding a start, stop, and replay button was fairly minimal. However, the process of adding the scroll bar was much more difficult. While more complex, the features scene was quite intuitive to set up as well. Once all the models were imported and textured, it was quite simple to import the rendered videos and integrate them into the application. The design of the app was a back-and-forth process with the graphics team at Terumo, which ended up being quite laborious. While the creation of these applications was a very long and difficult process at times, it was also extremely worthwhile.

1.5.2

Discussion of Application Feedback

The feedback received for these applications was predominantly positive, in that most participants

26

S. Bakalchuk et al.

agreed that the models were realistic and accurate, but the delivery system models could have used some adjustments. The clinical performance animation and the features animations got mostly positive feedback, while the feedback for the deployment animation was more neutral. The content validity was rated highly for all three modules of the applications, as participants generally agreed that the key facts and figures were communicated effectively. In regard to the applications as a whole, the participants found them very easy to use and very informative. It is important to keep in mind however that the cohort utilised for testing were all highly skilled professionals and extremely knowledgeable concerning this device, more so than the target audience would be. Therefore, any negative feedback received from this group would not necessarily indicate bad results. If there was more time, it would have been beneficial to test these applications on more participants, allowing for more reliable data.

1.5.3

Benefits and Drawbacks of the Application/3D Visualisation Technique

In addition to creating a valuable application for professionals in the surgical device industry, the methodology used for this application presents a novel 3D visualisation technique, which uses a combination of highly realistic anatomical model creation and simple, intuitive application design. This innovative methodology can be adapted for many different uses, within many different fields, and therefore marks a significant contribution to the realm of biomedical visualisation. The new techniques, however, do come with a set of benefits and drawbacks that must be considered before adapting this methodology to future projects. For instance, it was widely stated that certain aspects of the application could have been more realistic, most commonly the medical devices. Additionally, this technique can also be more time-consuming and expensive than a simpler 2D animation. Thus, there are significant benefits to such techniques as well. For instance,

the high level of detail and realism provided by a 3D application makes a complicated topic quite easy to understand and visualise for a broad audience. Additionally, the integration of these visualisations within an interactive application makes the animations widely accessible and more easily viewed by the subject.

1.5.4

Limitations

The most apparent limitations to this project were caused by the current COVID-19 pandemic. Most notably, all work had to be done remotely, and no work could be conducted which involved meeting people in person or working with clinicians. These factors made it so that the time for development was limited. For the AAA application specifically, it was understood that the stent model would be provided in an OBJ format, because the Fenestrated Anaconda is a highly intricate device with key features. Therefore, these applications were not completed to the full potential that they would have been if allotted more time or resources. The lack of time also made it so that we were unable to recruit as many participants as originally planned. Overall, despite all of these shortcomings, a great deal was learnt about creating intricate models, Autodesk plugins, animation networks and final renders to create high-quality animations.

1.5.5

Further Development

There are many ways in which this application could be advanced. Most notably, the applications could be further developed to incorporate some feedback from the participants. Regrettably, the time constraints of the project and COVID-19 precautions meant that testing at a medical conference could not occur. Fortunately, ample feedback was provided throughout collaboration by professionals of Terumo Aortic that ensured anatomical and stent graft accuracy during the development process. Once more refined, it could be very beneficial to create applications such as this for other Terumo

1

Creating Interactive Three-Dimensional Applications to Visualise Novel. . .

products, so that the company can effectively communicate the key features of their products to clinicians and patients in a more appealing way. The lack of engaging visualisations and medical interactive applications for clinical specialists, supported in the literature, show an obvious need for these applications to be shown to clinicians to positively impact their choices in treating complex aneurysms.

1.6

Conclusion

Advances in understanding stent graft surgical techniques have led to the need for interactive applications to engage healthcare workers so they can exploit its key features to effectively broaden the treatable patient population. Overall, the demonstrated methods in this project were successful in creating animations for interactive applications that showcased new features of novel stent grafts. The series of animations for both the Fenestrated Anaconda and the Relay device made up a practical interactive application that works to inform both the physician and patient of various surgical device options for treating rare, complex anatomical aortic aneurysm cases. These in turn provide resources for the physician to visually explain the surgical procedure and long-term benefits to a patient. Subsequently, this improves the discussion between patient and physician, providing a deeper understanding and patientcentred surgical approach in elderly patients who suffer from aortic aneurysms. Although these applications require additional testing, and still have a few areas that necessitate further refinement, these preliminary results show a great potential for the creation of an application that can utilise 3D visualisation techniques to successfully engage both physicians and patients. The interaction with a surgical device company to create the anatomical and surgical models, textures, animations and final application development provides an innovative approach to visualising surgical processes, generating awareness about medical conditions and facilitating treatment discussions between the physician and those with aortic aneurysms. Given the range of

27

positive interactions, as supported in the literature, of the general public with emerging visualisation technologies, the applications created provide a valid visualisation tool to engage clinicians, as well as the general public, about new medical devices, treatment alternatives, and research pertaining to healthcare an easily accessible visual format (Baker et al. 2002; Ford and Leinonen 2009; Traxler 2007). The creation of these applications also establishes a methodology for building informative surgical tools and positively influences the healthcare field through accurate dissemination and communication of surgical procedures and associated statistics.

References Abraha I, Romagnoli C, Montedori A et al (2009) Thoracic stent graft versus surgery for thoracic aneurysm. Cochrane Database Syst Rev 1:Cd006796. https://doi. org/10.1002/14651858.CD006796.pub2 Alsafi A, Bicknell CD, Rudarakanchana N et al (2014) Endovascular treatment of thoracic aortic aneurysms with a short proximal landing zone using scalloped endografts. J Vasc Surg 60(6):1499–1506 Ashton HA, Scott R, Bridgewater SG (2002) Randomized clinical trial of screening for abdominal aortic aneurysm in women. Br J Surg [online] 89(3):283–285 Autodesk, Inc. (2020) Maya. Available at: https:/autodesk. com/maya Avishay DM, Reimon JD (2020) Abdominal aortic repair. In: StatPearls. StatPearls Publishing, Treasure Island, FL Baker DW, Gazmararian JA, Williams MV et al (2002) Functional health literacy and the risk of hospital admission among Medicare managed care enrollees. Am J Public Health 92(8):1278–1283 Baker DW, Parker RM, Williams MV et al (1998) Health literacy and the risk of hospital admission. J Gen Intern Med 13(12):791–798 Ben Abdallah I, El Batti S, Sapoval M et al (2016) Proximal scallop in thoracic endovascular aortic aneurysm repair to overcome neck issues in the arch. Eur J Vasc Endovasc Surg 51(3):343–349 Bercovich E, Javitt MC (2018) Medical imaging: from roentgen to the digital revolution, and beyond. Rambam Maimonides Med J 9(4):e0034 British Heart Foundation (2020) Thoracic aortic aneurysms–Causes, symptoms & treatments. Accessed 28/05 Brown LC, Powell JT, Thompson SG, Epstein DM, Sculpher MJ, Greenhalgh RM (2012) The UK endovascular aneurysm repair (EVAR) trials: randomised trials of EVAR versus standard therapy.

28 Health Technol Assess 16:1–218. https://doi.org/10. 3310/hta16090 Bungay PM, Burfitt N, Sritharan K, Muir L, Khan SL, De Nunzio MC, Lingam K, Huw Davies A (2011) Initial experience with a new fenestrated stent graft. J Vasc Surg 54:1832–1838. https://doi.org/10.1016/j.jvs. 2011.05.115 Burt JR, Zimmerman SL, Kamel IR et al (2014) Myocardial T1 mapping: techniques and potential applications. Radiographics 34(2):377–395 Coelho-Filho OR, Rickers C, Kwong RY et al (2013) MR myocardial perfusion imaging. Radiology 266 (3):701–715 Colgan FE, Bungay PM, Burfitt N, Hatrick A, Clarke MJ, Davies AH, Jenkins M, Gerrard D, Quarmby JW, Williams R (2018) Operative and 1-year outcomes of the custom-made fenestrated anaconda aortic stent graft-a UK multicenter study. Ann Vasc Surg 46:257–264. https://doi.org/10.1016/j.avsg.2017.05. 027 Cosford PA, Leng GC (2007) Screening for abdominal aortic aneurysm. Cochrane Database Syst Rev: CD002945. https://doi.org/10.1002/14651858. CD002945.pub2 De Bruin JL, Baas AF, Buth J, Prinssen M, Verhoeven ELG, Cuypers PWM, van Sambeek MRHM, Balm R, Grobbee DE, Blankensteijn JD, DREAM Study Group (2010) Long-term outcome of open or endovascular repair of abdominal aortic aneurysm. N Engl J Med 362:1881–1889. https://doi.org/10.1056/ NEJMoa0909499 Dijkstra ML, Tielliu IFJ, Meerwaldt R, Pierie M, van Brussel J, Schurink GWH, Lardenoye J-W, Zeebregts CJ (2014) Dutch experience with the fenestrated Anaconda endograft for short-neck infrarenal and juxtarenal abdominal aortic aneurysm repair. J Vasc Surg 60:301–307. https://doi.org/10.1016/j.jvs.2014. 02.011 Dueck AD, Kucey DS, Johnston KW, Alter D, Laupacis A (2004) Survival after ruptured abdominal aortic aneurysm: effect of patient, surgeon, and hospital factors. J Vasc Surg 39:1253–1260. https://doi.org/10.1016/j. jvs.2004.02.006 Elefteriades JA (2002) Natural history of thoracic aortic aneurysms: indications for surgery, and surgical versus nonsurgical risks. Ann Thorac Surg 74(5):S1877– S1880. Discussion S1892–1878 Fernández-Alonso L, Fernández Alonso S, Martínez Aguilar E et al (2018) Endovascular treatment of aortic arch lesions using scalloped Endografts. Vasc Endovasc Surg 52(1):22–26 Ferrari E, Tozzi P, von Segesser L (2006) Thoracic aorta emergencies: is the endovascular treatment the new gold standard? Interact Cardiovasc Thorac Surg 5(6):730–734 Filardo G, Powell JT, Martinez MA-M, Ballard DJ (2015) Surgery for small asymptomatic abdominal aortic aneurysms. Cochrane Database Syst Rev

S. Bakalchuk et al. Ford M, Leinonen T (2009) Mobile learning: transforming the delivery of education and training, pp 195–214 Heikkinen M, Salenius J-P, Auvinen O (2002) Ruptured abdominal aortic aneurysm in a well-defined geographic area. J Vasc Surg 36:291–296. https://doi.org/ 10.1067/mva.2002.125479 Holliman R, Whitelegg L, Scanlon E, et al. (2009) Investigating science communication in the information age: implications for public engagement and popular media Howard DPJ, Banerjee A, Fairhead JF, Handa A, Silver LE, Rothwell PM (2015) Age-specific incidence, risk factors and outcome of acute abdominal aortic aneurysms in a defined population. Br J Surg 102:907–915. https://doi.org/10.1002/bjs.9838 Isselbacher EM (2005) Thoracic and abdominal aortic aneurysms. Circulation 111(6):816–828 Karmy-Jones R, Jackson N, Long W et al (2009) Current management of traumatic rupture of the descending thoracic aorta. Curr Cardiol Rev 5(3):187–195 Kasirajan K, Heffernan D, Langsfeld M (2003) Acute thoracic aortic trauma: a comparison of endoluminal stent grafts with open repair and nonoperative management. Ann Vasc Surg 17(6):589–595 Kato N, Dake MD, Miller DC et al (1997) Traumatic thoracic aortic aneurysm: treatment with endovascular stent-grafts. Radiology 205(3):657–662 Kuivaniemi H, Ryer EJ, Elmore JR, Tromp G (2015) Understanding the pathogenesis of abdominal aortic aneurysms. Expert Rev Cardiovasc Ther 13:975–987. https://doi.org/10.1586/14779072.2015.1074861 Liu J, Ma W, Liu F et al (2007) Study and application of medical image visualization technology. Springer, Berlin, Heidelberg, pp 668–677 Midy D, Becquemin J-P, Mialhe C, Frisch N, Martinez R, Caradu C, EFEFA Research Group (2017) Results of the French multicentric study of ANACONDATM fenestrated endografts in the treatment of complex aortic pathologies (EFEFA Registry). Ann Vasc Surg 43:151–165. https://doi.org/10.1016/j.avsg.2017.03. 171 Park JH, Chung JW, Choo IW, Kim SJ, Lee JY, Han MC (1996) Fenestrated stent-grafts for preserving visceral arterial branches in the treatment of abdominal aortic aneurysms: preliminary experience. J Vasc Interv Radiol 7(6):819–823. https://doi.org/10.1016/s10510443(96)70854-0. PMID: 8951748 Powell JT, Gotensparre SM, Sweeting MJ, Brown LC, Fowkes FGR, Thompson SG (2011) Rupture rates of small abdominal aortic aneurysms: a systematic review of the literature. Eur J Vasc Endovasc Surg 41(1):2–10 Qualter J, Sculli F, Oliker A et al (2012) The biodigital human: a web-based 3D platform for medical visualization and education. Stud Health Technol Inform 173:359–361 Saremi F (2017) Cardiac MR imaging in acute coronary syndrome: application and image interpretation. Radiology 282(1):17–32

1

Creating Interactive Three-Dimensional Applications to Visualise Novel. . .

Shaw PM, Loree J, Gibbons RC (2020) Abdominal aortic aneurysm (AAA). In: StatPearls. StatPearls Publishing, Treasure Island, FL Taylor AM, Wessels Q (2019) Using technology to engage the public in biomedical sciences. Adv Exp Med Biol 1171:127–135 Traxler J (2007) Defining, discussing and evaluating mobile learning: the moving finger writes and having writ. Int Rev Res Open Distr Learn 8(2):43–47

29

Vascutek Lmtd (2020) Newmains Avenue, Newmains Ave, Glasgow, Inchinnan, Renfrew PA4 9RR Wang LJ, Prabhakar AM, Kwolek CJ (2018) Current status of the treatment of infrarenal abdominal aortic aneurysms. Cardiovasc Diagn Ther [online] 8(S1):S191–S199

2

Using Confocal Microscopy to Generate an Accurate Vascular Model for Use in Patient Education Animation Angela Douglass, Gillian Moffat, and Craig Daly

Abstract

Hypertension is a condition requiring lifelong medication, where patients often feel well with or without treatment. Uncontrolled hypertension, however, can lead to permanent remodelling processes that occur to the vascular structure, which are seldom understood by the public. As a result, a significant burden is placed on healthcare systems globally as a result of the effects of hypertension and lack of adherence to prescribed treatment. Improving patient education through welldesigned interactive applications and animation is a known strategy that can improve adherence rates to medication. In the context of hypertension, little attention has been given to helping patients understand the unseen damage that occurs to vessels exposed to high blood pressure. However, generating an accurate representation of a vessel and the changes that occur can be challenging. Using microscopy data is one way for creating an anatomically correct model, but this often needs careful consideration as data cannot be directly imported. Here we describe methods

A. Douglass (*) · C. Daly School of Life Sciences, College of Medical, Veterinary Sciences, University of Glasgow, Glasgow, UK e-mail: [email protected] G. Moffat School of Simulation and Visualisation, Glasgow School of Art, The Hub, Glasgow, UK

for creating an accurate 3D model of a small artery using confocal microscopy data. This model can then be animated to demonstrate the substructures and pathological changes that occur in hypertensive conditions to better inform patients about the dangers of uncontrolled blood pressure. Keywords

3D modelling · Animation · Patient education · Hypertension · Vascular remodelling

2.1

Introduction

Creating computer-generated 3D anatomical models for animation or interactive applications for the purposes of education is often fraught with challenges. The artist must consider the audience at which the content will be aimed and tailor the detail accordingly. It is therefore an important consideration that the models do not contain an overwhelming amount of information whilst retaining scientific accuracy (Schwan and Papenmeier 2017). This can prove more difficult when considering the portrayal of microanatomy and structures at cellular and subcellular levels (Daly et al. 2014). The viewer must be able to appreciate the scale at which the objects are being observed. Furthermore, as many of the small cell structures are colourless, any colour and texture choices

# The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 P. M. Rea (ed.), Biomedical Visualisation, Advances in Experimental Medicine and Biology 1356, https://doi.org/10.1007/978-3-030-87779-8_2

31

32

applied to help the viewer interpret the models must not be misleading (Xiao 2013). It is therefore important that during the visual development phase, a balance is struck between artistic licence and realism. There are many strategies available to create 3D models, and with the increasing availability of cheaper computer power as well as open-source software, it is now even easier to build models from 3D datasets generated by scanning equipment (e.g. MRI, PET or CT) (Eid et al. 2017). The same is true for visualising smaller-scale structures such as tissues and cells using microscopy. Improved spatial and temporal resolution of acquired images means more accurate and detailed models can be constructed (Daly et al. 2014). This does not come without its challenges. The shear complexity of this data requires an expert eye to help tease out the key information, and models require significant processing prior to use in any apps, infographics or animations (McCrorie et al. 2016). The composition of any rendered images created from 3D models must also be a consideration; clear well-structured images with adequate dialogue help draw attention to features of interest and are crucial to delivering key messages to the viewer. Placing things well and giving suitable context can enhance comprehension of a difficult topic and increase audience retention (Schwan and Papenmeier 2017). Whilst engaging with individuals for the purposes of education is valuable, the ability to change an individual’s behaviour in a positive manner is ultimately the aim for many patientfocused animations and infographics (McCrorie et al. 2016). This is particularly true when considering hypertension, where lifestyle modifications and adherence to medication reduce a patient’s risk of potentially debilitating and life-threatening complications (Milani et al. 2017). In this chapter we discuss the creation of a model of a small artery that can be used to explain their structure and function in health and under hypertensive conditions. We focus on how this model can be used within the context of hypertension to explain the importance of adherence to medication to patients.

A. Douglass et al.

2.2

Blood Pressure

Determining an individual’s blood pressure involves the measurement of two distinct values, the systolic and diastolic pressures. The systolic pressure is the pressure at which the blood arrives at the arteries after being ejaculated from the heart. The diastolic pressure measures the pressure during the heart’s refilling period between beats (Vischer and Burkard 2017). Blood pressure is categorised into ranges that correlate with cardiovascular health which varies with a person’s age and sex (Fig. 2.1) (Unger et al. 2020). Arteries have a propensity to lose flexibility and stiffen as a part of the normal aging process which can elevate blood pressure, meaning pressure varies also depending upon age (Toda et al. 1980). There is also natural variation between the sexes as hormones such as oestrogen influence vascular tone (Wang et al. 2020). This is particularly notable in the post-menopausal years, where blood pressure can increase more rapidly in women than at any other time (Cifkova et al. 2008).

2.3

Blood Pressure Regulation

With the exception of the pulmonary arteries, the arterial system is responsible for transporting oxygenated blood from the heart to the tissues of the body. This network of vessels is highly responsive to external stimuli and adapt quickly to changing pressures by contraction and relaxation of the vessel walls, increasing or decreasing the luminal diameter. The length of the vessel also changes as part of this response (Mercadante and Raja 2020). The regulation of blood pressure is complex involving multiple organ systems primarily via the renin-angiotensin-aldosterone system (RAAS). It serves as a protective mechanism for tissues, helping to ensure that damaging high pressures are kept from the terminal capillaries within organs by constriction of the small arteries (Warnert et al. 2016). Arteries vary in their size and structure depending upon their location.

2

Using Confocal Microscopy to Generate an Accurate Vascular Model for Use. . .

33

Fig. 2.1 Variation of Mean Systolic and Diastolic Blood pressure between Males and Females with Age. Blood pressure increases with age and varies between the sexes in individuals not receiving antihypertensive medication. Typically, males have a higher systolic and diastolic

pressures compared with women, however this changes in the post-menopausal years, where females tend to exhibit higher average pressures. Image taken from (Beaney et al. 2018)

When blood leaves the heart, the first artery it comes into contact with is the aorta (Mercadante and Raja 2020). This large artery is highly elastic with a large muscular wall. Despite its size its role in blood pressure regulation is minimal. Instead, it is the small- to medium-sized arteries that are of greatest importance (Delong and Sharma 2019). These arteries, also known as resistance arteries, respond to internal stimuli such as transmural pressure and sheer stress within the vessel from blood flow but also from stimuli from the sympathetic nervous system via vascular nerves that form a network throughout the vascular system (Delong and Sharma 2019). Hormones via the RAAS also influence blood vessels to help maintain pressure homeostasis. Juxtaglomerular cells from the kidney release a hormone called renin into the blood in response to a drop in blood pressure (Patel et al. 2017). Renin acts on the freely circulating angiotensinogen produced by the liver to form angiotensin I. Angiotensin I is a weak vasoconstrictor that is then converted to a potent vasoconstrictor, angiotensin II by angiotensin-converting enzyme (ACE) (Fig. 2.2) (Patel et al. 2017).

The production of angiotensin II also influences production of aldosterone. This hormone acts on the distal tubules of the nephrons within the kidney to promote reabsorption of sodium and water and the excretion of potassium, which results in an increased blood volume and increased blood pressure (Elliott 2007).

2.4

Pathophysiology of Hypertension

Whilst cardiac output is important for generating the pressure within the arterial system, it is the resistance of the peripheral arteries that influences the overall pressure of the arterial blood (Mercadante and Raja 2020). Blood pressure varies naturally throughout the day, and short periods of hypertension in response to acute stresses are normal. However, prolonged exposure of the vasculature to high pressures becomes pathogenic and warrants a diagnosis of hypertension (Brown and Haydock 2000). Hypertension can be further classified into two groups, either ‘essential’ or ‘primary’ hypertension or

34

Fig. 2.2 Summary of the renin-angiotensin-aldosterone system. The renin-aldosterone system is the hormone system that plays a primary role in blood pressure as well as sodium, potassium, and fluid balance within the body

secondary hypertension. Essential hypertension is typically diagnosed in 90–95% of cases of hypertension (Brown and Haydock 2000). This relates to cases where there is not a definitive underlying cause but can be attributed to a lifestyle or metabolic element. Secondary hypertension is diagnosed where there is a specific cause, commonly a chronic kidney condition, that should be the focus of any treatment plan (Scala et al. 2008). The trend of blood pressure increases observed with age occurs most frequently in western populations, and it is believed to be related to factors such as higher sodium intake, lower potassium levels, higher likelihood of obesity, alcohol consumption and reduced physical activity. Sodium/potassium imbalance and obesity impact the regulatory mechanisms of blood pressure, having a negative impact (Scala et al. 2008).

A. Douglass et al.

Uncontrolled high blood pressure causes damage to the small arteries and ultimately has an impact on multiple organ systems. Most commonly, this affects the heart which increases the risk of myocardial infraction or cardiac failure (Zhou et al. 2018). There are also implications for the brain, as high blood pressures can lead to stroke and vascular dementia (Ben Nasr et al. 2018; Warnert et al. 2016). Hypertension also has a profound effect on the kidneys. The ability of the kidneys to filter blood efficiently and remove waste products is dependent upon maintenance of normotensive pressures (Mennuni et al. 2014). Persistent damage to the kidneys due to high blood pressures can lead to renal failure. Other disorders include peripheral artery disease (PAD), vision loss and sexual dysfunction (Mennuni et al. 2014). Also the effects on the body, hypertension also represents a significant global healthcare burden, contributing to over 18 million deaths each year. Around one third of all deaths can be attributed to hypertension (Unger et al. 2020). Alarmingly, hypertension is affecting younger individuals, and incidences globally appear to be increasing (Luo et al. 2020). In Scotland nearly 50% of all myocardial infarctions and strokes are linked to uncontrolled hypertension. The issue is further compounded when considering as many as 30% of adults have hypertension with only around half of those diagnosed and many of those are not in receipt of effective treatment (‘Scotland Factsheet fear of heart and circulatory diseases 2020).

2.5

Peripheral Resistance Artery Structure and Vascular Remodelling in Hypertension

The anatomy of arteries is related to their function depending upon what organ they supply and where they are found. Peripheral resistance arteries that are predominantly involved in the regulation of blood pressure have a layered structure consisting of different cell types (Mercadante and Raja 2020). The inner most layer is a single cell surface of endothelial cells that face directly into the lumen of the vessel. Behind this layer is a

2

Using Confocal Microscopy to Generate an Accurate Vascular Model for Use. . .

thin elastic lamina, followed by a layer of smooth muscle cells known as the tunica media (Mulvany and Aalkjaer 1990). Another elastic lamina encompasses the media, which is surrounded by the outer most layer, the adventitia. This layer— unlike the organised arrangement of cells observed in the other layers—is made up of a poorly characterised attachment of elastin and collagen fibres (Briones et al. 2003). It is also interspersed with a variety of cell types such as fibroblasts, macrophages and nerve cells, which are surrounded by adipose tissue (Mulvany and Aalkjaer 1990). The cells within these layers can communicate with each other and respond to internal and external signals to adapt the vessel shape in response to changing pressures (Intengan et al. 1999). When vessels are exposed to high blood pressures, even over a relatively short period, these resistance arteries undergo organisational and structural changes known as vascular remodelling (Martinez-Quinones et al. 2018). In the early stages of remodelling, the number of cells increases in the adventitial layer, and this is likely an increase in the number of fibroblasts in response to proinflammatory signals (MartinezQuinones et al. 2018). These cells can then migrate and differentiate to myofibroblasts further perpetuating the inflammatory response. The smooth muscle layer also becomes thickened resulting in a larger, less flexible artery (Touyz et al. 2018). Remodelled arteries have more collagen than healthy arteries and are less able to adapt to pressure changes. The remodelling process is also not a reversible one; although some recovery is possible, long-term exposure to high blood pressure means that it is unlikely to be undone using medication (Schiffrin 2010).

2.6

Treatment of Hypertension

Medications available to counteract hypertension are relatively inexpensive and are effective at lowering blood pressure (Katharina et al. 2004). When considering a treatment pathway, clinicians weigh up the risk of a patient’s cardiovascular

35

risk based on their lifestyle, BMI, age and sex in combination with multiple blood pressure measurements, before agreeing an appropriate course of action, rather than basing any decision solely on blood pressure measurement (NICE Guidelines, No 136). In some cases of essential hypertension, modification of lifestyle may be successful for lowering blood pressure and should be provided as an option prior to prescribing medication (Drevenhorn et al. 2007). Where blood pressure may be in the hypertensive range and lifestyle medication is either unsuccessful or not an option, antihypertensive drugs will be the first line of defence (Unger et al. 2020). There is an extensive variety of antihypertensive medications available. Typically more than one medication will be required, and a process of trial and error is adopted to find the most suitable medication, whilst being mindful of other diseases where there may be a contraindication (Brown and Haydock 2000). Most antihypertensive medications can be categorised into four subtypes depending upon their mechanism of action and are summarised in Table 2.1. In most cases, where suitable, beta-blockers are used to lower blood pressure as a primary course of action. These drugs act quickly to lower cardiac output; however, recent evidence has suggested that these drugs whilst beneficial fail to reduce the risk of cardiovascular disease and do not lower blood pressure as effectively as other more specific medications (Wiysonge et al. 2012). Angiotensin-converting enzyme inhibitors (ACEI) have been shown to be effective for lowering blood pressure. Drugs within this class prevent the conversion of angiotensin I to angiotensin II. They also reduce excretion of water and sodium by the kidneys, increasing blood volume. These drugs are associated with mild side effects in around 10% of patients but are generally considered safe (Li et al. 2014). Angiotensin receptor blockers (ARBs) also act on the renin-angiotensin system and are associated with less side effects. They have been developed as a second-generation antihypertensive and are also used in congestive heart failure and diabetic nephropathy (Brunner 2007).

36

A. Douglass et al.

Table 2.1 Antihypertensive medications categorised into four subgroups based on their mechanism of action Drug category Angiotensin-converting enzyme (ACE) inhibitors Angiotensin II receptor blockers (ARBs) Calcium channel blockers Diuretics Beta-blockers

Mode of action These drugs target and inhibit angiotensin formation, preventing its activation ARBs act directly on angiotensin receptors and block its action Reduce calcium entry and inhibiting cellular vasoconstrictive responses Reduce sodium and water content within blood, reducing blood volume Block the stimulatory effects of adrenaline on the heart, reducing cardia output

Diuretics are also considered widely as a frontline drug to treat hypertension. They work by binding to the thiazide receptor in the kidney and influence sodium uptake (Schiffrin 2010). Calcium channel blockers have a vasodilatory effect and are also considered a good initial choice for treatment (Elliott 2007).

2.7

Medication Adherence

Although there is a broad spectrum of medications available for the treatment of hypertension, and diagnosis is simple and non-invasive, mortality rates due to the complications of hypertension remain high (Poulter et al. 2020). The World Health Organization (WHO) acknowledges that hypertension is both the largest preventable and treatable issue affecting health (Zhou et al. 2018). This is in part due to an estimated four in seven individuals being undiagnosed, but also of those who are diagnosed, it is estimated that 50% fail to adhere to their medication (Burnier 2017). The reasons for which are multifactorial and complex but extensive research in the area has identified a few common issues. Firstly, patients with hypertension are often asymptomatic, particularly in the case of essential hypertension, and as the medications for controlling blood pressure are lifelong, patients may not have the motivation nor the understanding of their mechanism of action to stay committed in the long term (Gavrilova et al. 2019).

Drug examples Lisinopril, ramipril and enalapril Valsartan and losartan Amlodipine and nifedipine Chlorothiazide and amiloride Atenolol and propranolol

Another compounding factor is the pill burden placed on patients. Typically, medication for hypertension is taken daily and rarely requires one tablet. Patients are also likely to be prescribed statins to help lower blood cholesterol along with their medication. Patients taking one medication daily have a greater success rate at adhering to their medication in the longer term that those that are required to take multiple pills (Farrell et al. 2013). It is also challenging to accurately measure patient adherence. Often, as patient’s adherence is assessed based on consulting directly with the patient, this can be an inaccurate method as patients were found to either misinterpret their instructions or failed to remember or genuinely recall their medication-taking habits (Burnier 2017). The lack of accurate monitoring has led to false diagnoses of resistant hypertension, resulting in more medication being prescribed by GPs or in some cases surgical renal denervation being carried out (Hyman and Pavlik 2015).

2.8

Patient Education Can Improve Medication Adherence

The complex nature of non-adherence means it is unlikely that a one-size-fits-all solution will present itself. However, a recent review highlighted that good patient education that encompasses treatment and long-term complications in addition to solid patient engagement is most likely to be the most successful and cost-effective option to improve patient outcomes (Roldan et al. 2018).

2

Using Confocal Microscopy to Generate an Accurate Vascular Model for Use. . .

Simple strategies, such as increasing contact time with specialist nurses to provide information and promote healthy lifestyle strategies, have a positive outcome on the blood pressures of patients (Hacihasanoǧlu and Gözüm 2011). Although more patient contact time with healthcare professionals is effective in improving adherence rates, an individual’s lack of health literacy can be a barrier to success (Tavakoly Sany et al. 2018). Visual aids are a valuable educational resource. They can be used to help simplify complex information and can also avoid obstacles relating to language and cultural differences. They can also be distributed widely and limit variability of information between healthcare providers (Pratt and Searles 2017). Choosing the appropriate format for visual learning is also key, and 2D images have been criticised for not providing the breadth and depth information when trying to communicate anatomical structures in a teaching environment. Information can be lost or misinterpreted (Ballantyne 2011). With the increase in availability of cheap 3D printers, more anatomical structures are now being created and used within the context to teach, both for medical students, and to assist with surgical planning but also for patient education (Garcia et al. 2018). 3D datasets such as MRI, CT and PET imaging can be used to help generate these models. Although a degree of skill is required to assemble models digitally for 3D visualisation or printing, these models can be highly detailed and accurate (Xiao 2013). Digital 3D models also do not necessarily need to be reserved for 3D printing; rather they have wide-ranging applications in animation and interactive mobile applications. 3D models do not necessarily need to be confined to the large anatomical structures (Garcia et al. 2018; Bijani et al. 2020). Microscopy imaging can also generate 3D datasets that can be used to create 3D digital models providing more visual aids to enhance understanding of less frequently observed micro anatomical details (Daly et al. 2002). Within the context of hypertensive patients, little information is available as to whether

37

improving education can be achieved using 3D models as teaching tools around hypertension and vascular remodelling. Furthermore, there are limited studies on visual strategies to demonstrate the effects of medication on the arteries.

2.9

Generating Digital 3D Models Using Confocal Microscopy

In vitro imaging of the small resistance arteries has long been used for research into hypertension and cardiovascular disease, allowing the physiology of the cellular interactions that regulate normal vascular conditions to be observed (Martinez-Quinones et al. 2018). These methods also provide insights into the processes that occur during arterial remodelling and the changes vessels undergo when exposed to high blood pressures, providing valuable models for exploring the roles of drugs and other biological agents (Schiffrin 2010). Florescence-based laser scanning confocal microscopy is a powerful tool used for imaging individual cells and their substructures to whole tissue sections and organoids. The method differs from conventional light microscopy in that it eliminates noisy out of focus signals using a pinhole, giving clearer high-resolution images. It has the capability to acquire serial optical 2D image sections allowing for 3D reconstruction of images (Jonkman and Brown 2015). A light source in the form of lasers or LEDs is used to excite fluorophores and label specific cell structures, which emit light at a known wavelength, allowing for identification of key structures or cell types. Images for each fluorophore channel are captured sequentially and can be overlayed using imaging software such as Imaris and ImageJ (Hartig 2013; Gautier and Ginsberg 2021). Visualisation of whole vessels is challenging due to the limited size of the scan area of the microscope. Multiple images must be generated which requires complex stitching of the data. Acquisition of the data is also plagued with technical challenges such as phototoxicity and photobleaching, particularly when working with

38

live cell specimens (Hoebe et al. 2007). Capturing high-resolution images over several channels encompassing different fluorophores always comes at a cost. This typically means there is a trade-off between lower-resolution images and preserving live cells or higher-resolution images with distorted 3D datasets due to cell shrinkage during the fixation process (Knight et al. 2003). Whilst advances in microscopy and imaging software have improved the resolution of datasets, 3D models created by Imaris or ImageJ imaging software are still limited. Fine detail is usually lost even with image post-processing steps, and often there are artefacts that complicate the images meaning they may not be ideal for teaching where the viewer is not familiar with confocal datasets (Dobrucki 2004). In such cases, it can prove advantageous to import datasets into 3D modelling software such as Blender or Autodesk Maya to create a more refined mesh for the 3D object and remove erroneous data (Boshkovikj et al. 2014; Asadulina et al. 2015). Unfortunately, 3D datasets are not simply imported directly into 3D modelling software. Images must first be aligned using software such as ImageJ. This open-source software allows for correction of the number of images for all channels in each z-series. The images for each channel can also be exported to create a single series of tiff files. The image series can then be imported to a 3D visualisation software such as 3D Slicer, Cura or Simplify3D. These softwares are useful for creating the 3D isosurface required to create a model suitable for 3D modelling software or 3D printing. Commonly used in medical visualisation, DICOM image data format from MRI scans, CT, PET and MRA has also proven invaluable for creating interesting visualisation from microscopy data. The 3D visualisations created from confocal data can be useful for presenting observations to back up scientific data. However, using these models for multimedia applications may not be ideal in the context of patient or student education. It can also prove challenging to convey the changes that occur in disease states using unprocessed images, as they may not provide the visual cues to the untrained eye. To provide a clear

A. Douglass et al.

comparison of healthy and diseased systems, animation can provide a solution. 3D models constructed from unprocessed images may require further refinement, and simply creating a 3D mesh will not suffice. Models optimised using 3D software can be textured or have colour applied in the form of computer-generated shaders; this can allow for more realistic colours to be applied or for highlighting structures of interest. Here we describe methods for reconstructing z-series of 2D images created by confocal imaging of rodent mesenteric artery. Using ImageJ to align image sections, the resulting z-series can be imported directly into 3D Slicer to segment key structures and generate a 3D mesh that can be edited in Autodesk Maya. Using measurements and isolating small structures allow extrapolation of the data to generate whole tissue representations where only a partial image of tissue sections exists. The resulting vessel model was created as an open-source asset as a teaching or public engagement animation, and we evaluate its accuracy and usefulness for patient education.

2.10

Building a Complete Vessel 3D Model from a Partial Confocal Microscopy Dataset

A typical workflow for generation of a 3D model from an image stack is shown in Fig. 2.3. Ordinarily this consists firstly of image acquisition to create a series of still images through the specimen in the z-plane. These would then be imported and aligned in ImageJ or Imaris. In fluorescent microscopy different ‘channels’ are used to distinguish and capture images of fluorophores as they emit light at a known wavelength throughout the z-plane. These images can be merged, and a pseudo colour applied to visualise the overlapping fluorophores on the same image to create a single series of images. To build a digital model from these images that can be edited and manipulated, a mesh must be created. This can be performed in segmenting software such as 3D Slicer. This software provides the added benefit that structures of

2

Using Confocal Microscopy to Generate an Accurate Vascular Model for Use. . .

39

Fig. 2.3 Workflow for generating 3D digital models from confocal datasets. Images must first be acquired using a microscope and scanning samples at regular intervals through the z-plane. Images can then be adjusted in ImageJ, and fluorescent images from different channels can be merged here. Image series are combined into a 3D model in 3D Slicer and further refined importing into Maya for retopology, rendering and animation

interest can be separated and a model created directly from that or combined with other structures. However, importing 3D isosurface directly from segmenting software such as 3D Slicer typically will result in a mesh with very large numbers of polygon faces, which can be impractical to process. Meshes my also not be ‘watertight’ as gaps are a common feature and can be time-consuming to rectify. This can be solved using retopology, where the 3D isosurface model is rebuilt by mapping new geometry on top

of the existing mesh to create a new model. This serves two purposes as firstly the total number of polygons in the final model are reduced, but also the mesh will contain little to no triangle or ngon faces that can interfere with any texturing, rigging or deformation for animation and rendering that maybe required further down the production pipeline. Using microscopy data for generating 3D models is further challenged by the available resolution of the images used to build models. Here

40

we discuss a method for accurately modelling the structure of a peripheral resistance artery, by using confocal datasets to make measurements and key landmarks to act as a template for accurate modelling of the vascular structure. In this instance, images were obtained from a z-scan of rodent mesenteric artery that had been prelabelled with florescent markers to indicate cell nuclei, smooth muscle and elastin. Data was acquired by Dr. Craig Daly using a BioRad Radiance 2100 confocal microscope and images captured over three corresponding fluorophore channels. Images exported directly from the microscopy software were then imported to ImageJ for image correction and analysis. Once in ImageJ the images created from the three colour channels were aligned and merged to create a single image for each step in the z-series (Fig. 2.4). These images were saved as png files and then exported into 3D Slicer. The actual size of the mesenteric artery specimen image was small due to the small sample area limits that confocal microscopy can image (typically 512
512 pixel images and a sample depth of around 50 μm). This also limits the ability to view the full diameter of the vessel; however, the full thickness of the vessel wall can be observed as well as the vessel layers and key cell types. Using 3D Slicer’s segmenting tools, a filter was then applied using the adaptive histogram to reduce erroneous voxels from the dataset. To extract the layers of the vessel and create separate models, a combination of automatic and semiautomatic thresholding was used based on the Hounsfield values. The painting tools were used to manually select areas of interest before the model maker was applied to generate the 3D isosurface (mesh) (Fig. 2.5). This was then imported into Autodesk Maya 3D modelling software. The imported model only provided a small arc segment of the whole artery (Fig. 2.6). However, once the overall thickness and size of the vessel have been recreated, the proportions of the structures that make up the layers of the arterial wall were then be determined by extrapolating from this data.

A. Douglass et al.

Maya is not the 3D modelling software of choice for creation of accurately dimensioned models; however, it is perfectly placed for medical animation. The software does have some features that allow for basic measurement which is more than sufficient for creating proportioned models of biological structures. The actual length of the vessel would be around 5–10 μm, and it would not be practical to directly replicate these sizes in Maya. Instead using the Maya units of measurement meant that although the models were larger, they were directly proportional and would be better suited for lighting, rendering and animation. As we know that the shape of the artery would be roughly cylindrical, we used the measurement tools within to calculate the diameter of the lumen and outer circumference of the vessel. To do this, the arc length and angle of the arc were determined in Maya. This was then used in the formula below to determine the circumference of the vessel and ultimately the diameter: Arc length Angle of Arc ¼  Circumference ðcÞ 360 Diameter ¼ c=π: Once the overall thickness and size of the vessel were recreated, the proportions of the structures that make up the layers of the arterial wall were then determined.

2.11

Modelling the Tunica Intima

Pipe primitive shape from Maya’s built-in modelling tools was used to reconstruct the tunica intima. The inner most surface of the pipe was created to match the same diameter as the lumen, as determined above. The tunica intima is also punctuated with small circular pores which span the endothelium and internal elastic lamina, allowing communication with the smooth muscle cells of the tunica media, essential for the normal functioning of the vessel (Esser et al. 1998). These fenestrae were visible from the confocal imaging and were observed to

2

Using Confocal Microscopy to Generate an Accurate Vascular Model for Use. . .

41

Fig. 2.4 Images aligned and adjusted in ImageJ. Images captured by the confocal microscope were optimised in ImageJ by adjusting the brightness and contrast values. A de-noise filter was also applied to clean up images. Corresponding images from each colour channel were combined before exporting as a series of png files

vary in distribution and size. Endothelial wall fenestrae size and spacing through the internal elastic lamina were replicated by measuring the variation in sizes and spacing. By examining the distribution of fenestrae over multiple confocal images, it was observed that fenestrae occur in clusters and are not evenly distributed. The thickness of the pipe was made to match proportionally the area that this layer would typically occupy. To create the fenestrated surface, an individual vertex was selected and chamfered to create additional vertices; these could then be expanded to create a hole in the mesh and target welded to corresponding vertices on the outer wall of the pipe primitive. The mesh was duplicated to create an internal elastic lamina and an endothelial layer. Further geometry was added to the endothelial layer by using the smoothing functions, and this layer was

sculpted to create subtle protrusions into the lumen where the nuclei of the endothelial cells would create a greater depth in these areas (Fig. 2.7).

2.12

Tunica Media

The tunica media consists of one or more layers of smooth muscle cells. These cells are orientated in a particular helical arrangement with distinctive elongated nuclei. The smooth muscle cells also are likely to exist in functional bands that may facilitate its function for coordinated contraction that ultimately influences the lumen diameter and vessel length regulating blood pressure (Daly et al. 2002). Of the densely populated cells of the vessel, smooth muscle cells make up much of the

42

A. Douglass et al.

Fig. 2.5 Segmentation of vessel using 3D Slicer. The sequence of png image file exported from ImageJ was used to create a 3D model of the imaged sample in 3D Slicer. Using the segmentation tools, all the areas of

interest were selected carefully using a combination of automatic and manual methods. This was then exported as the 3D model obj file that can be used in Maya

thickness of the vessel. To recreate this layer, a pipe primitive was created and sculpted to show the elongated shape of the individual smooth muscle cells and how they are arranged within the vessel. To add further detail, 2D texture was created in Photoshop to mimic the nuclear arrangement observed after nuclear staining (Figure 2.8).

layer, branching though the elastin network and extending down to the smooth muscle cells of the tunica media (Weinstein 2005). Within the adventitia, adipocytes are interspersed and extend out significantly from the vessel. The perivascular adipose tissue that surrounds the vessel is also innervated and works in concert with the other vessel layers (Huang Cao et al. 2017). From the 3D confocal dataset, it is possible to identify the nerves, elastin and adipocytes from the scan data. Using the external wall as a reference, the size, distribution and number of each of these structures were replicated with separate meshes. These were then grouped by layers and duplicated around the whole vessel wall, adding in some variations to avoid repeating patterns but ensuring that the distribution of the cells was accurate (Fig. 2.9).

2.13

Tunica Externa

The tunica externa or tunica adventitia is the external most layer that surrounds the vessel. This layer exhibits a much-reduced organisational arrangement and contains a more varied selection of cell types (Halper 2018). The adventitia consists of fibroblasts, macrophages and stem cells dispersed throughout an array of elastin fibres. Adventitial nerves also permeate this

2

Using Confocal Microscopy to Generate an Accurate Vascular Model for Use. . .

43

Fig. 2.6 Imported 3D isomesh from 3D Slicer into Maya. The 3D isomesh created in 3D Slicer was imported in Maya for further adjustment. As can be observed from the polygon count on the top left region of the view port,

the number of polygon faces was too high to be used directly in any animation or images. Also, the portion of the sample imaged only contained a small section of the vessel wall

Although the aim of creating this model was to create something as close to the real vessel as possible, it was not practical for demonstrating the different layers in an animation. The advantage of creating a model this way, however, will allow other users to create their own custom shaders to tailor the model to their needs.

interactive application, whereby a user could explore the layers individually using a mobile device or desktop computer.

2.14

Simple Effects in Animation

To animate the blood vessel wall, several different built-in toolkits can be adopted in Maya. As the model was structured into layers, they can easily be separated from each other to show the various components and structures that comprise each layer. This was achieved simply by setting key frames on the transformation of each of the components, which were timed to fit around a prepared script. Alternatively, the model lends itself well for incorporation into a dynamic

2.15

Vascular Wall Remodelling Using Blend Shapes

The blend shape function is a powerful tool in Maya that is used extensively in medical animation. It provides the animator the ability to create various states for the model and seamlessly animate the transition between these states. It is used often to animate the heart beating or lungs expanding and contracting to mimic breathing. Using the blend shape deformer, an additional model was created by duplicating the original. The smooth muscle cell layer polygon mesh of this second model was then sculpted to appear thickened compared with the original. This also reduced the lumen diameter as is a common

44

Fig. 2.7 Modelling of the endothelial cell layer and internal elastic lamina. The endothelial cell layer (inner most) was created to match the dimensions determined earlier using a pipe primitive shape. More geometry was added Fig. 2.8 Creating a tunica media layer. A tunica media layer was created again using estimated proportions from the measurements taken of the original model. Again, a pipe primitive shape was use to block out the shape and thickness of this layer. More geometry was added using the smoothing tools, and ridges were sculpted along the surface to mimic the helical arrangement of the smooth muscle cells (a). The model was the UV-unwrapped, and a texture created in Photoshop was applied (b)

A. Douglass et al.

using the smooth functions and an uneven surface created using the sculpting tools. This layer was then duplicated to create an internal elastic lamina (shaded in grey)

2

Using Confocal Microscopy to Generate an Accurate Vascular Model for Use. . .

45

Fig. 2.9 Creating an adventitia layer. The adventitial layer was created by overlaying a series of low poly primitive shapes to build up a network of collagen and elastin fibres. Larger branching cylindrical shapes were

added to hint at the nerves that would be interwoven throughout this layer. Adipocytes were also created and added to the outer surface to represent part of the perivascular fat layer

feature of hypertrophic vascular remodelling. Although other forms of structural changes can occur, it was felt that animating a transition from a normal to hypertrophic state would be the most effective method for communicating the changes that occur in hypertension. After creating a suitable thickened and remodelled artery, the animation from a diseased state was built by setting key frames to show a gradual transition between healthy and diseased states (Fig. 2.10).

instances rather than geometry, it also reduces the computational power required, reducing render time and allowing for more complex animations to be created. In the field of medical animation, MASH is commonly adopted as intricate molecular scenes can be quickly created and controlled with minimal effort. One of the frequent uses for MASH is for the creation of blood flow. To assist viewers with understanding the size of a small- to medium-sized resistance artery, blood flow was animated passing through the vessel. To do this, a single red blood cell was modelled from a polygon primitive shape and a mash waiter applied. This generated multiple instances of the red blood cell, the number of which was controlled using the MASH distribution node. There are many ways to animate the movement of objects, but the simplest method arguably is animating objects along a curve. By placing a smooth curve using the EP curve tool through the vessel, the path of the red blood cells was controlled. The red blood cells were attached

2.16

Maya’s MASH Toolkit

The MASH toolkit is a built-in feature of Autodesk Maya which allows for the development of motion graphics and animations. MASH uses sophisticated algorithms that allow for multiple instances of 3D objects to be controlled using a node-based system. This gives greater control to the artist whilst reducing timeconsuming manual placement of objects within a scene. As the objects within a scene can be set to

46

A. Douglass et al.

Fig. 2.10 Animating vascular remodelling using blend shapes. Using the blend shape tool, the original model was duplicated and sculpted to mimic the thickening of the smooth much layer and narrowing of the lumen space

that typically is observed in remodelled vesicles. The transition was animated using key frames to show the difference between healthy (a) and diseased states (b)

to a curve simply by adding a curve node to the MASH network and assigning an EP curve (Fig. 2.11). Using a curve gives the added advantage of being able to adjust the motion path of the animation at any point to create the right movement of objects compared with manual placement. The distribution of the red blood cells can also be easily controlled via the MASH waiter. Altering the ‘step’ parameters via the curve node and x, y and z orientation of the red blood cells insures even distribution which can be further controlled when animating by adjusting velocity random and noise, helping to give a more natural, less uniformed flow to the cells. Good animating practice requires that objects move as the viewer would anticipate. For example, in the 12 basic principles of animation originally developed by Ollie Johnstone and Frank Thomas, animations must adhere to the basic laws of physics; otherwise the animation will appear manufactured and unrealistic. Even very subtle inaccurate movements by objects can immediately catch the viewers’ attention, distracting them and even reducing the perceived credibility of the content.

To enhance the realism of the blood flow, a random node can be added to the MASH network, this allows for small adjustments to the position and rotation of each object to be made, and the strength of which can be simply altered using sliders, until an organic looking flow is achieved.

2.17

Materials (Shaders)

Selecting the colour palette forms a pivotal part of the design choices that inform what material attributes and textures would be best placed in the scene. It is frequently overlooked; however, it is a fundamental part of the visual development process for medical visualisations. Depending upon the context in which the model is going to be used, it may be necessary to consider photorealistic textures or maybe just simple coloured materials. In this context, as a tool for patient education, a balance of light warm colours was used to distinguish vessel structures whilst avoiding creating a potentially off putting overly realistic model. In this case we also wanted to

2

Using Confocal Microscopy to Generate an Accurate Vascular Model for Use. . .

47

Fig. 2.11 Using MASH to create blood flow. The MASH motion graphics toolkit was used to create a blood flow though vessel. A red blood cell was created and instances

created using MASH distribution. Blood cell could then be animated along an EP curve (highlighted in blue) that is not rendered in any final images

highlight the anatomical substructures as this can be further developed to show the differences between a healthy artery and one that is remodelled after exposure to high blood pressure. For the tunica media, showing the organisation of the smooth muscle cells was required as this layer becomes thickened and the number of smooth muscle cells increase when remodelled. The orientation of these cells is also interesting as they typically present themselves in a helical structure with elongated nuclei. To hint at the

presence of the nuclei, a high-resolution texture was created in Photoshop and overlaid in the correct orientation and UV unwrapping the media layer. The texture was aligned with the small protrusions on the media surface where the cells would naturally be at their widest point due to the presence of a nucleus and endoplasmic reticulum. The hypershader is used in Maya to control the specularity, diffuse and metallic parameters as well as other properties of a material. Again,

48

A. Douglass et al.

using a node-based system, material attributes can be controlled as well as textures, bump, normal and displacement maps. Using the aiStandard (Arnold) surface shader to control these features, the various layers of the vessel were assigned materials in line with the chosen colour palette and retained a semi-realistic feeling by including subsurface scattering. This gave the model a ‘skin’ like quality as the light penetrated the surface and reflected giving a slight translucency to the layers.

2.18

Lighting

The importance of good lighting in any scene whether it is an animation, film or photograph cannot be overstated. Lighting in a scene not only can make images visually appealing but can help lay the groundwork by setting the appropriate mood. A well-lit scene can help strike an instructional and informative tone. Most of the 3D softwares available allow the artist freedom to create a variety of lighting options from studio to environmental, and Maya is no exception. The diversity of options for lighting a scene is beyond the scope of this text, but there are many useful resources available, for example, ‘Light for Visual Artists Second Edition: Understanding and Using Light in Art & Design’ by Richard Yot. To effectively light the vessel scene, a standard 3-point lighting rig was constructed using Maya area lights. Although area lights usually increase render time, they tend to result in better shadows and highlights, operating essentially as soft boxes. A key light was set to illuminate the opening of the vessel closest to the camera and to provide the main source of light into the luminal space. The key light was also used to create a drop shadow which was positioned below the vessel to help give the viewer a sense of the vessel floating in the 3D space. To highlight the adventitia and soften the shadows created by the key light, a fill light was used adjacent to the key light. A back light was also added with a lower exposure than the other lights in the scene, and this light was also tinted with a pale blue hue in contrast to the

white, warm lighting from the key and fill lights. This added depth to the overall image and created an interesting rim light on the edge of the vessel.

2.19

Rendering

Test renders were carried out during the production phases to test lighting and materials. In Maya, the Arnold render engine was used, which calculates a final image from the view of a camera positioned in the scene to create an impactful composition. For single images, the output image size was set to 1920
1080 in png format with an anti-aliasing value set to 6 and subsurface scattering set to 3. This produces high-quality images with limited noise that can be used in final post-production in Photoshop. For animated scenes, a series of png images can be created and composited in Adobe After Effects. After Effects is a commonly used software that allows for compositing and postprocessing effects to be added. When animating the explosion of the vessel layers, labels can be added here to each of the layers to guide the viewer. It also provides an opportunity to further refine animation timings to align with any voice over script as well as making final adjustments to the colour and lighting. In the final production phase, the sequence of pngs can then be further rendered and compressed into an MP4 format suitable for most media players and upload to video streaming sites such as YouTube or Vimeo.

2.20

Results

Final images of the vessel are shown in Fig. 2.12. Showing some of the various ways, the model could be displayed and animated. The changes that occur to the vessel can easily be observed in side-by-side comparison, creating easily interpretable images for use in education or for patient information documentation. For animation, the model provides flexibility for viewing in different states to help convey the transitions that occur from health to disease whilst also offering an insight into the anatomical structures and cell

2

Using Confocal Microscopy to Generate an Accurate Vascular Model for Use. . .

49

Fig. 2.12 Images of the final rendered vessel. Images show examples of the final rendered vessel. (a) Rendered with animated blood flow. (b) Exploded vessel with

annotated layers. (c) Side-by-side comparisons of health and remodelled vessels

types. Showing the vessel alongside red blood cells helps add context and scale to the scene, which can also be easily animated to show blood flow changes in relation to the changes that occur with contraction and relaxation of the vessel under normal or pathological conditions.

for prolonged periods of high blood, to protect downstream organ damage. Improving patient education and communicating good health practices can make a significant impact to patient outcomes and can also be a very cost-effective strategy in tackling the issues surrounding poor medication adherence (Calano et al. 2019). Creating clear and engaging visual materials is a good tried and tested method in other educational settings, as it can convey more information in a simpler format requiring less text. It can help give anatomical structure context and drive a better understanding of what happens in disease states as well as how medication works (Pratt and Searles 2017). There are many ways to graphically depict medical information; however, it can be challenging to balance realism in the imagery with over simplified representations. Too much information can be off putting or perhaps perceived as to graphic for a patient, conversely simply illustrations can result in misinterpretation and may miss the mark (McCrorie et al. 2016).

2.21

Discussion and Evaluation

Very few hypertensive patients fully understand the lasting effects of unmanaged blood pressure (Ben Nasr et al. 2018). Indeed, it is not uncommon for patients to request a reduction in medication or to stop altogether at specialist clinics, as there is a perception that a prolonged normotensive state means that their hypertension has been treated and does not require further intervention (Heisler 2008). This is likely the core reason for the poor adherence to medication observed in this condition (Burnier 2017). It is crucial therefore to communicate the irreversible changes that occur when blood vessels are remodelled to compensate

50

The wealth of 3D datasets and relatively simple methodology for creating digital 3D models mean that there are opportunities to not only create detailed images that can be tailored to suit the audience, but also these models can be used in animations and interactive applications, giving a broader range of platforms to deliver educational content (Daly et al. 2002). There are many ways in which models can be created, and often datasets taken directly from a microscope or scan can contain too much detail, 3D modelling can provide a solution to simplify this data and create more refined models suitable for many applications. Although modelling from primitive shapes requires a degree of artistic skill, it can be achieved relatively simply utilising 3D datasets as templates on which to build a model. These models can be low poly and flexible making them useful for gamification and animation. However, there is the potential that small details present in the true models are lost through the process of segmenting and retopology or reconstructing from primitive shapes. The artist must decide which is more valuable depending upon their end application.

References Asadulina A et al (2015) Object-based representation and analysis of light and electron microscopic volume data using blender. BMC Bioinformatics 16(1):229. https:// doi.org/10.1186/s12859-015-0652-7 Ballantyne L (2011) Comparing 2D and 3D imaging. J Vis Commun Med 34(3):138–141. https://doi.org/10.3109/ 17453054.2011.605057 Beaney T, Schutte AE, Tomaszewski M, Ariti C, Burrell LM, Castillo RR, Charchar FJ, Damasceno A, Kruger R, Lackland DT, Nilsson PM, Prabhakaran D, Ramirez AJ, Schlaich MP, Wang J, Weber MA, Poulter NR, Napiza-Granada C, Sevilla MRC, Atilano AA, Ona DID, More A, Jose AP, Maheshwari A, Kondal D, Yu W, Li W, Xu S, Yu J, Zhang H, Widyantoro B, Turana Y, Situmorang TD, Sofiatin Y, Barack R, Lin H-J, Wang T-D, Chen W-J, Sirenko Y, Evstigneeva O, Negresku E, Yousif ME, Medani SA, Beheiry HM, Ali IA, Zilberman JM, Marin MJ, Rodriguez PD, GarciaVasquez F, Kramoh KE, Ekoua D, Lopez-Jaramillo P, Otero J, Sanchez G, Narvaez C, Accini JL, HernandezHernandez R, Octavio JA, Morr I, Lopez-Rivera J, Ojji D, Arije A, Babatunte A, Wahab KW, Fernandes M, Pereira SV, Valentim M, Dzudie A, Kingue S,

A. Douglass et al. Ngongang DAD, Ogola EN, Barasa FA, Gitura B, Malik F-T-N, Choudhury SR, Al Mamun MA, Minh VH, Viet NL, Truong SC, Ferri C, Parati G, Torlasco C, Borghi C, Goma FM, Syatalimi C, Zelveian PH, Barbosa E, Barroso WS, Penaherrera E, Jarrin E, Yusufali A, Bazargani N, Tsinamdzgvrishvili B, Trapaidze D, Neupane D, Mishra SR, Jozwiak J, Malyszko J, Konradi A, Chazova I, Ishaq M, Memon F, Heagerty AM, Keitley J, Brady AJB, Cockcroft JR, McDonnell BJ, Lanas F, Chia Y-C, Ndhlovu H, Kiss I, Ruilope LM, Mbolla BFE, Milhailidou AS, Woodiwiss AJ, Perl S, Dolan E, Azevedo V, Garre L, Boggia JG, Lee VWY, Kowlessur S, Miglinas M, Sukackiene D, Wainford RD, Habonimana D, Masupe T, Ortellado J, Wuerzner G, Alcocer L, Burazeri G, Delgado ES, Lovic D, Mondo CK, Mostafa A, Nadar SK, Tiburcio OV, Leiba A, Dorobantu M, De Backer T, Chifamba J, Stergiou G, Nwokocha CR, Sokolovic S, Toure AI, Connell KL, Khan NA, Burger D, Rodrigues MDC, Kramer BK, Schmieder RE, Unger T, Wyss FS, Yameogo NV, Beistline H, Kenerson JG, Alfonso B, Olsen MH, Soares M (2018) May measurement month 2017: an analysis of blood pressure screening results worldwide. Lancet Glob Health 6 (7):e736–e743. https://doi.org/10.1016/S2214-109X (18)30259-6. ISSN 2214-109X Ben Nasr N et al (2018) Evaluation of a hypertensionbased patient education program in a stroke center. Int J Clin Pharm 40(6):1490–1500. https://doi.org/10. 1007/s11096-018-0745-2 Bijani M et al (2020) A comparison between multimedia and traditional education in encouraging adherence to treatment regimen in patients with hypertension. The Open Hypertension J 12(1):1–6. https://doi.org/10. 2174/1876526202012010001 Boshkovikj V et al (2014) Three-dimensional visualization of nanostructured surfaces and bacterial attachment using Autodesk® Maya®. Sci Rep 4(1):4228. https://doi.org/10.1038/srep04228 Briones AM et al (2003) Role of elastin in spontaneously hypertensive rat small mesenteric artery Remodelling. J Physiol 552(1):185–195. https://doi.org/10.1113/ jphysiol.2003.046904 Brown MJ, Haydock S (2000) Pathoaetiology, epidemiology and diagnosis of hypertension. Drugs 59(SUPPL. 2):1–12. https://d oi.o rg/10.216 5/00003495 200059002-00001 Brunner HR (2007) Angiotensin II receptor blockers. Comprehensive Hypertension 75246:1003–1017. https://doi.org/10.1016/B978-0-323-03961-1.50084-2 Burnier M (2017) Drug adherence in hypertension. Pharmacol Res 125:142–149. https://doi.org/10.1016/ j.phrs.2017.08.015 Calano BJD et al (2019) Effectiveness of a communitybased health programme on the blood pressure control, adherence and knowledge of adults with hypertension: a PRECEDE-PROCEED model approach. J Clin Nurs 28(9–10):1879–1888. https://doi.org/10.1111/jocn. 14787

2

Using Confocal Microscopy to Generate an Accurate Vascular Model for Use. . .

Cifkova R et al (2008) Blood pressure around the menopause: a population study. J Hypertens 26(10):1976–1982. Available at: https://journals.lww. com/jhypertension/Fulltext/2008/10000/Blood_pres sure_around_the_menopause__a_population.11.aspx Daly C, Clunie L, Ma M (2014) From microscope to movies: 3D animations for teaching physiology. Microsc Anal 28(6):7–10 Daly C et al (2002) Analysing the 3D structure of blood vessels using confocal microscopy. Microsc Anal 92: 5–8. Available at: internal-pdf://219.224.93.159/01 2002 Daly M and A Analysing the 3-D Structu.pdf Delong C, Sharma S (2019) Physiology, peripheral vascular resistance. StatPearls Publishing, Treasure Island (FL). Available at: http://europepmc.org/books/ NBK538308 Dobrucki JWB (2004) Confocal microscopy: quantitative analytical capabilities’, in cytometry, 4th edition: new developments. Academic Press, pp 41–72. https://doi. org/10.1016/S0091-679X(04)75003-7 Drevenhorn E, Kjellgren KI, Bengtson A (2007) Outcomes following a programme for lifestyle changes with people with hypertension. J Clin Nurs 16 (7b):144–151. https://doi.org/10.1111/j.1365-2702. 2005.01493.x Eid M et al (2017) Cinematic rendering in CT: a novel, lifelike 3D visualization technique. Am J Roentgenol 209(2):370–379. https://doi.org/10.2214/AJR.17. 17850 Elliott WJ (2007) Systemic hypertension. Curr Probl Cardiol 32(4):201–259. https://doi.org/10.1016/j. cpcardiol.2007.01.002 Esser S et al (1998) Vascular endothelial growth factor induces endothelial fenestrations in vitro. J Cell Biol 140(4):947–959. https://doi.org/10.1083/jcb.140. 4.947 Farrell B, French Merkley V, Ingar N (2013) Reducing pill burden and helping with medication awareness to improve adherence. Canadian Pharmacists J / Revue des Pharmaciens du Canada 146(5):262–269. https:// doi.org/10.1177/1715163513500208 Garcia J et al (2018) 3D printing materials and their use in medical education: a review of current technology and trends for the future. BMJ Simulation and Technology Enhanced Learning 4(1):27–40. https://doi.org/10. 1136/bmjstel-2017-000234 Gautier MK, Ginsberg SD (2021) A method for quantification of vesicular compartments within cells using 3D reconstructed confocal z-stacks: comparison of ImageJ and Imaris to count early endosomes within basal forebrain cholinergic neurons. J Neurosci Methods 350:109038. https://doi.org/10.1016/j.jneumeth.2020. 109038 Gavrilova A et al (2019) Knowledge about disease, medication therapy, and related medication adherence levels among patients with hypertension. Medicina (Kaunas) 55(11):1–12. https://doi.org/10.3390/ medicina55110715

51

Hacihasanoǧlu R, Gözüm S (2011) The effect of patient education and home monitoring on medication compliance, hypertension management, healthy lifestyle behaviours and BMI in a primary health care setting. J Clin Nurs 20(5–6):692–705. https://doi.org/10.1111/ j.1365-2702.2010.03534.x Halper J (2018) Chapter four–basic components of vascular connective tissue and extracellular matrix. In: Khalil RA (ed) Vascular pharmacology: cytoskeleton and extracellular matrix. Academic Press, pp 95–127. https://doi.org/10.1016/bs.apha.2017.08.012 Hartig SM (2013) Basic image analysis and manipulation in ImageJ. Curr Protoc Mol Biol 102(1):14.15.1–14.15.12. https://doi.org/10.1002/ 0471142727.mb1415s102 Heisler M (2008) Actively engaging patients in treatment decision making and monitoring as a strategy to improve hypertension outcomes in diabetes mellitus. Circulation 117(11):1355–1357. https://doi.org/10. 1161/CIRCULATIONAHA.108.764514 Hoebe RA et al (2007) Controlled light-exposure microscopy reduces photobleaching and phototoxicity in fluorescence live-cell imaging. Nat Biotechnol 25(2):249–253. https://doi.org/10.1038/nbt1278 Huang Cao ZF, Stoffel E, Cohen P (2017) Role of perivascular adipose tissue in vascular physiology and pathology. Hypertension 69(5):770–777. https:// doi.org/10.1161/HYPERTENSIONAHA.116.08451 Hyman DJ, Pavlik V (2015) Medication adherence and resistant hypertension. J Hum Hypertens 29(4):213–218. https://doi.org/10.1038/jhh.2014.73 Intengan HD et al (1999) Mechanics and composition of human subcutaneous resistance arteries in essential hypertension. Hypertension 33(1):569–574. https:// doi.org/10.1161/01.HYP.33.1.569 Jonkman J, Brown CM (2015) Any way you slice it–a comparison of confocal microscopy techniques. J Biomol Tech 26(2):54–65. https://doi.org/10.7171/jbt. 15-2602-003 Katharina W-M et al (2004) Hypertension treatment and control in five European countries, Canada, and the United States. Hypertension 43(1):10–17. https://doi. org/10.1161/01.HYP.0000103630.72812.10 Knight MM et al (2003) Live cell imaging using confocal microscopy induces intracellular calcium transients and cell death. Am J Phys Cell Phys 284(4):C1083– C1089. https://doi.org/10.1152/ajpcell.00276.2002 Li ECK, Heran BS, Wright JM (2014) Angiotensin converting enzyme (ACE) inhibitors versus angiotensin receptor blockers for primary hypertension. Cochrane Database Syst Rev 2017(12):CD009096. https://doi.org/10.1002/14651858.CD009096.pub2 Luo D et al (2020) Association between high blood pressure and long term cardiovascular events in young adults: systematic review and meta-analysis. The BMJ 370. https://doi.org/10.1136/bmj.m3222 Martinez-Quinones P et al (2018) Hypertension induced morphological and physiological changes in cells of

52 the arterial wall. Am J Hypertens 31(10):1067–1078. https://doi.org/10.1093/ajh/hpy083 McCrorie AD, Donnelly C, McGlade KJ (2016) Infographics: healthcare communication for the digital age. Ulster Med J 85(2):71–75 Mennuni S et al (2014) Hypertension and kidneys: unraveling complex molecular mechanisms underlying hypertensive renal damage. J Hum Hypertens 28(2):74–79. https://doi.org/10.1038/jhh.2013.55 Mercadante AA, Raja A (2020) ‘Anatomy, Arteries’, in. Treasure Island (FL) Milani RV et al (2017) Improving hypertension control and patient engagement using digital tools. Am J Med 130(1):14–20. https://doi.org/10.1016/j.amjmed.2016. 07.029 Mulvany MJ, Aalkjaer C (1990) Structure and function of small arteries. Physiol Rev 70(4):921–961. https://doi. org/10.1152/physrev.1990.70.4.921 Patel S et al (2017) Renin-angiotensin-aldosterone (RAAS): the ubiquitous system for homeostasis and pathologies. Biomed Pharmacother 94:317–325. https://doi.org/10.1016/j.biopha.2017.07.091 Poulter NR et al (2020) Medication adherence in hypertension. J Hypertens 38(4):579–587. https://doi.org/10. 1097/HJH.0000000000002294 Pratt M, Searles GE (2017) Using visual aids to enhance physician-patient discussions and increase health literacy. J Cutan Med Surg 21(6):497–501. https://doi.org/ 10.1177/1203475417715208 Roldan PC, Ho GY, Ho PM (2018) Updates to adherence to hypertension medications. Curr Hypertens Rep 20(4):34. https://doi.org/10.1007/s11906-018-0830-x Scala D et al (2008) Promotion of behavioural change in people with hypertension: an intervention study. Pharm World Sci 30(6):834–839. https://doi.org/10. 1007/s11096-008-9235-2 Schiffrin EL (2010) Circulatory therapeutics: use of antihypertensive agents and their effects on the vasculature. J Cell Mol Med 14(5):1018–1029. https://doi.org/ 10.1111/j.1582-4934.2010.01056.x Schwan S, Papenmeier F (2017) Learning from animations: from 2D to 3D? In: Lowe R, Ploetzner R (eds) Learning from dynamic visualization: innovations in research and application. Springer International Publishing, Cham, pp 31–49. https://doi.org/ 10.1007/978-3-319-56204-9_2

A. Douglass et al. Scotland Factsheet fear of heart and circulatory diseases (2020), (July) Tavakoly Sany SB et al (2018) Health providers’ communication skills training affects hypertension outcomes. Med Teach 40(2):154–163. https://doi.org/10.1080/ 0142159X.2017.1395002 Toda T et al (1980) Morphometrical analysis of the aging process in human arteries and aorta. Cells Tissues Organs 106(1):35–44. https://doi.org/10.1159/ 000145167 Touyz RM et al (2018) Vascular smooth muscle contraction in hypertension. Cardiovasc Res 114(4):529–539. https://doi.org/10.1093/cvr/cvy023 Unger T et al (2020) 2020 International Society of Hypertension global hypertension practice guidelines. J Hypertens 38(6):982–1004. https://doi.org/10.1097/ HJH.0000000000002453 Vischer AS, Burkard T (2017) Principles of blood pressure measurement–current techniques, office vs ambulatory blood pressure measurement. In: Islam MS (ed) Hypertension: from basic research to clinical practice. Springer International Publishing, Cham, pp 85–96. https://doi.org/10.1007/5584_2016_49 Wang S et al (2020) Sex differences in the structure and function of rat middle cerebral arteries. Am J Phys Heart Circ Phys 318(5):H1219–H1232. https://doi. org/10.1152/ajpheart.00722.2019 Warnert EAH et al (2016) Is high blood pressure selfprotection for the brain? Circ Res 119(12):e140– e151. https://doi.org/10.1161/CIRCRESAHA.116. 309493 Weinstein BM (2005) Vessels and nerves: marching to the same tune. Cell 120(3):299–302. https://doi.org/10. 1016/j.cell.2005.01.010 Wiysonge CS et al (2012) Beta-blockers for hypertension (review) Beta-blockers for hypertension. Cochrane Database Syst Rev 11(8):CD002003. https://doi.org/ 10.1002/14651858.CD002003.pub5.www. cochranelibrary.com Xiao L (2013) Animation trends in education. Int J Infor Edu Tech 3(3):286–289. https://doi.org/10.7763/ijiet. 2013.v3.282 Zhou D et al (2018) Uncontrolled hypertension increases risk of all-cause and cardiovascular disease mortality in US adults: the NHANES III linked mortality study. Sci Rep 8(1):1–7. https://doi.org/10.1038/s41598-01827377-2

3

Methods and Applications of 3D Patient-Specific Virtual Reconstructions in Surgery Jordan Fletcher

Abstract

3D modelling has been highlighted as one of the key digital technologies likely to impact surgical practice in the next decade. 3D virtual models are reconstructed using traditional 2D imaging data through either direct volume or indirect surface rendering. One of the principal benefits of 3D visualisation in surgery relates to improved anatomical understanding—particularly in cases involving highly variable complex structures or where precision is required. Workflows begin with imaging segmentation which is a key step in 3D reconstruction and is defined as the process of identifying and delineating structures of interest. Fully automated segmentation will be essential if 3D visualisation is to be feasibly incorporated into routine clinical workflows; however, most algorithmic solutions remain incomplete. 3D models must undergo a range of processing steps prior to visualisation, which typically include smoothing, decimation and colourization. Models used for illustrative purposes may undergo more advanced processing such as UV unwrapping, retopology and PBR texture mapping. Clinical applications are wide ranging and vary significantly between specialities. J. Fletcher (*) St Mark’s Hospital, Harrow, UK e-mail: jordan.fl[email protected]

Beyond pure anatomical visualisation, 3D modelling offers new methods of interacting with imaging data; enabling patient-specific simulations/rehearsal, Computer-Aided Design (CAD) of custom implants/cutting guides and serves as the substrate for augmented reality (AR) enhanced navigation. 3D may enable faster, safer surgery with reduced errors and complications, ultimately resulting in improved patient outcomes. However, the relative effectiveness of 3D visualisation remains poorly understood. Future research is needed to not only define the ideal application, specific user and optimal interface/platform for interacting with models but also identify means by which we can systematically evaluate the efficacy of 3D modelling in surgery. Keywords

3D-modelling · Segmentation · Surgery

3.1

Introduction

3D visualisation and printing techniques have been highlighted as one of the key technologies likely to impact surgery in the next decade by the 2018 Royal College of Surgery ‘Future of surgery’ review (RCS 2018). Patient-specific 3D models derived from imaging data offer the prospect of personalised medicine. There are a wide

# The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 P. M. Rea (ed.), Biomedical Visualisation, Advances in Experimental Medicine and Biology 1356, https://doi.org/10.1007/978-3-030-87779-8_3

53

54

J. Fletcher

range of reconstruction techniques and visualisation methods available with a similarly diverse array of potential application including pre-operative planning, intraoperative navigation, surgical simulation, anatomical and patient education (Bücking et al. 2017). Early research suggests 3D modelling may improve a surgeon’s anatomical understanding; enable faster, safer surgery; and ultimately help improve patient outcomes. However, despite such early optimism, the technology remains largely experimental with significant developments needed if models can be routinely utilised in patient workflows. The ideal applications, user and interface have yet to be established. 3D modelling forms a part of a wider tapestry of interrelated emerging technologies such as virtual/augmented reality, telementoring, robotics, artificial intelligence and advanced data analytics that constitute the emerging field of digital surgery. This review provides an overview of reconstruction techniques and how 3D patient-specific modelling has been utilised in surgical specialties.

3.2 3.2.1

Methods of 3D Virtual Reconstructions Segmentation

Segmentation of Digital Imaging and Communications in Medicine (DICOM) data sets is a pre-requisite for medical visualisation. Segmentation is the task of delineating image data into meaningful structures relevant to a particular task. Practically, unique labels are assigned to each voxel designating its membership to a given anatomical structure. Segmentation is composed of two key aspects: • Identification—Anatomical structures of interest should be recognised. • Delineation—The borders of structures should be precisely defined. In the context of 3D visualisation, segmentation is necessary to selectively show objects of interest (Paragios and Duncan 2015). A wide

range of imaging segmentation software solutions and techniques are available. The process can be fully manual or automated to varying degrees. The degree of automation is highly dependent on the quality and type of imaging involved. For example, automatically segmenting bone is relatively easy owing to the high contrast with respect to surrounding structures in comparison to intra-abdominal imaging with its structures of homogenous Hounsfield intensities (the Hounsfield scale is a semiquantitative method of measuring x-ray attenuation and corresponds to the subsequent greyscale CT/MRI image).

3.2.1.1 Manual Segmentation Manual is the simplest and most robust method of segmentation and is widely employed owing the ease of implementation and availability of multiple open-source software solutions (see Fig. 3.1 and Table 3.1). An operator (radiologist/surgeon/trained technician) will manually label a scan slice by slice giving different anatomical structures unique segmentation labels. The main advantage of this approach is the flexibility offered—models can be generated even with suboptimal imaging or with indistinct structures with low contrasts and unusual shapes (e.g., small blood vessels or tumours). However, this method is timeconsuming, limiting the applicability in routine clinical care pathways. Furthermore, manual segmentation lacks reproducibility and is entirely operator dependent. 3.2.1.2

Algorithmic Approaches to Segmentation Reliable algorithms are required for delineation of anatomical structures. Computer-aided segmentation should aim to automate the process facilitating fast, accurate and reproducible results. Algorithmic automatic segmentation techniques must factor in problems common to all imaging modalities such as signal noise, partial volume effect (loss of signal in small structures due to limited resolution of imaging system), imaging artefact (e.g., motion, ring,

3

Methods and Applications of 3D Patient-Specific Virtual Reconstructions in Surgery

55

Fig. 3.1 Manual segmentation in ITK-SNAP of mesenteric blood vessels for planning complete mesocolic excision

Table 3.1 Summary of available segmentation software packages Segmentation applications Materialise mimics Materialise pro plan 3D Slicer ITK-SNAP MI-3DVS Synapse Vincent Hisense CAS OsiriX IQQA-Liver Ziostation MITK ReLiver Amira 4.1 M3D MedGraphics ZedTrauma iNtellect Cranial Navigation Imascap

intensity inhomogeneity). We will briefly review some of the current available methods. Thresholding Users set a global or an interval lower and upper threshold in order to generating a binary image. Voxels are either classified as belonging to the

IntelliSpace Portal HepaVision MEDICAL Imaging Three Divisional Visualisation System iPlan CMF MI-3DVS Innersight Labs VirSSPA Leonardo InSpace SuperImage Orthopaedics edition M3DICS HipPlan (Symbiosis) 3D Plus Body Visible System Visible Patient Maxilim FreeForm Modelling System

target structure or else marked as background. This is mostly used for segmenting CT data for bone (Sharma and Aggarwal 2010). Thresholding will typically generate an incomplete segmentation owing to partial volume effect (averaging of image intensities leads to lower-intensity values for voxels only partially representing a structure).

56

Region-Based Segmentation Region-based segmentation is based on the concept of homogeneity. A target structure is assumed to possess similar pixels clustered together. It is like threshold-based segmentation in that a threshold determines which voxels belong to the segmentation. However, it differs in that one connected component is considered. A growing process is initiated from user-selected points (often termed seeds). Each neighbouring voxel is recursively aggregated until a prespecified user criteria is met (typically threshold parameters). Region-growing approaches are typically used for contrast-enhanced vascular structures (Paragios and Duncan 2015). If the aim is to trace vascular structures as far into their periphery as possible, simple regiongrowing approaches yield incomplete results as the process is broken if a voxel fails to satisfy the homogeneity criteria due to partial-volume effects. Edge-Based Segmentation Edge-based segmentation relies on discontinuities in the image data, usually signified by rapid changes in pixel signal intensity between two different structures (Sharma and Aggarwal 2010). Atlas-Based Segmentation In atlas-based segmentation, the geometry and features of organs, blood vessels and soft tissues are compiled. A large database of images forms the basis of a statistical atlas which represents the anatomical variations found in a patient population. Statistical shape models (SSMs) are iteratively deformed to fit the target of new structures with shapes that are derived from the atlas training set of labelled data (Paragios and Duncan 2015; Kaur and Kaur 2014). Artificial Intelligence and Deep Learning Medical imaging interpretation represents a unique challenge in the field of computer vision. Convolutional neural networks (a class of artificial neural networks, itself a subset of artificial intelligence) are the predominant algorithm

J. Fletcher

employed in computer vision and have been successfully utilised for segmentation of 3D medical imaging data (Hu et al. 2017; Trebeschi et al. 2017). Taking inspiration from the visual cortex, convolutional neural networks (CNNs) process data in a grid pattern to adaptively learn spatial patterns in a hierarchical fashion from low- to high-level features (Yamashita et al. 2018). CNNs have been successful to multiple forms of 3D imaging data sets including brain and abdominal segmentation tasks with performances approaching that of a radiologist (Hu et al. 2017; Trebeschi et al. 2017). Deep learning techniques may help fully automate imaging segmentation and dramatically reduce the time needed for 3D reconstructions—a necessary requirement if 3D models are to be used in routine clinical care.

3.2.2

Rendering Methods for 3D Virtual Models

We can broadly divide two rendering approaches to 3D medical visualisation: 1. Volumetrically (direct) rendered models 2. Surface (indirect) rendered models

3.2.2.1 Volumetric Rendering Volumetric rendering represents the original data set without the requirement of an intermediate representation and is therefore also termed direct rendering. Volume rendering encompasses a set of techniques that display sampled functions of the 3D volume data onto a 2D viewing plane. The rendering technique uses a simplified model of how light interacts with matter. Optical models generally describe the net gain of loss of radiance as light passes through a volume. To render an image, a camera is defined with respect to the space relative to the 3D volume. Colour and transparency values are assigned to each voxel as a ray of light passes through the volume via a RGBA (red, green, blue, alpha—the alpha channel denotes the transparency) transfer function. The basic volume rendering equation considers only emission and absorption. More advanced techniques model light interactions with a volume

3

Methods and Applications of 3D Patient-Specific Virtual Reconstructions in Surgery

more accurately generating more realistic rendered images. For a detailed summary of volumetric rendering, I would refer the reader to other dedicated resources such as Kim et al. (Kim et al. 2020).

3.2.2.2 Surface Rendering Techniques Surface or indirect rendered models are composed of a polygon mesh approximation of the target anatomy’s surface structure. A polygon mesh is extracted from the segmentation data using Visualisation Toolkit (VTK—is an objectoriented based open-source software for computer graphics, image processing and visualisation) and Insight Segmentation and Registration Toolkit (ITK—an open-source, crossplatform library that provides an extensive suite of software tools for image analysis) libraries (Yushkevich et al. 2006). The initial advantages of lower memory requirements compared to the equivalent direct volume rendering have become less relevant with the improvements and widespread availability of GPU that support real-time rendering of even complex volume data sets. However, other developments have ensured the continued importance of surface rendering techniques, principally: • Biophysical simulations requiring surface mesh data for volume grids. • Surface rendering remains the predominant method used by web, mobile and virtual/augmented reality platforms. Although direct volume rendering is possible, surface rendering prevails owing lower memory requirements and manipulates and optimises with 3D modelling software (especially relevant in an educational context). Given the much lower storage footprint, surface meshes will remain the preferred rendering technique for web-based applications. • Mesh data is a requirement for 3D printing. Life-sized anatomical prints are increasingly being employed in certain fields such as maxillofacial and orthopaedic fields to run patient-specific simulations for complex reconstructive surgery (Robb 1999).

57

A polygon mesh is a collection of vertices, edges and faces which form the polyhedral three-dimensional object. Vertices describe a position in three-dimensional Euclidian space. Two vertices joined by a straight line form an edge. A polygon is defined by three (triangle) or four (quadrangle or ‘quads’) vertices joined by the corresponding number of edges. As previously discussed, surface extraction methods rely on binary decisions whether a given voxel belongs to the surface of a structure or not. This can yield inaccurate results when considering anatomical structures with indistinct boundaries or poor-quality imaging with significant artefact as found in abdominal/pelvic CT and MRI imaging. A stepping or ‘staircase’ artefact is seen as the segmented contours of adjacent imaging slices do not overlap, which becomes more pronounced as the distance between imaging slices increases. Incomplete segmentations can result in holes in the resulting surface mesh or give rise to multiple disconnected regions. Similarities of isovalues between adjacent distinct anatomical structures can result in erroneously exceeding the intended pixel classification if using threshold-based techniques.

3.2.3

Post-Processing of Surface Polygon Mesh

Post-processing aims to improve the functionality and appearance of surface-rendered models. The aesthetic appearance of the depicted anatomy is an important consideration of 3D visualisation and will have a key influence on user understanding. At the most basic level, application of colour will help distinguish different structures. However, processing also is important from a functional and performance perspective—models need to be optimised for their intended user interface platform. Decimation and smoothing represent two of the most important post-processing steps typically employed.

3.2.3.1 Decimation Decimation refers to the algorithmic reduction of the 3D mesh polygon count, generating a lower-

58

J. Fletcher

Fig. 3.2 Demonstration of effect of decimation

resolution version of the surface mesh while maintaining the overall geometry. In computer graphics, whenever a frame is rendered, the graphic card must translate the co-ordinate in three-dimensional space of every vertex to the two-dimensional space on the monitor. Decimation is essential to reduce the memory requirements of rendering. This is especially important when the user interface will be deployed on web- or mobile-based platforms in order to prevent any noticeable latency between user input and model transformations (movement, rotation or scaling). A target performance of 60 frames per second (fps) ensures smooth, fluid model movements in 3D space, but anything over 30fps is acceptable and not likely detectable to the human eye (Sarkar 2014; Technologies n.d.). Taking a target number of polygons or percentage reduction as an argument, geometry is removed until this target is reached. The overall topology of the mesh is altered as vertices and edges are assessed for relevance and subsequently removed (Paragios and Duncan 2015). There is a trade-off between the mesh size and overall accuracy of the model with the aim to remove only geometrical detail not contributing significantly to the overall shape (see Fig. 3.2). The degree of decimation will vary between models. Generally, intricate lattice like structures

depicting complex blood vessels will not tolerate excessive decimation without significant distortion/degradation of the geometry.

3.2.3.2 Smoothing Smoothing flattens the angles between adjacent faces and is necessary to remove the ‘stepping artefact’ evident in the results of the raw segmentation and is aimed primarily at improving the aesthetic appearance to facilitate comprehension. The Laplacian smoothing operator is widely utilised by modelling software and toolkits (Preim and Botha 2014). Each vertex is iteratively moved in the direction of its geometric centre with respect to its surrounding neighbouring vertices. As all vertices are treated equally, this method achieves smoothness at the expense of accuracy—a significant issue in medical applications where accurate depictions of the patient’s anatomy are required. Deformations of the surface are produced where areas of smaller edges lie next to areas with larger edges. Oversmoothing can result in volume shrinkage of the 3D mesh and a ‘dotted line’ artefact as areas of mesh are lost entirely (see Fig. 3.3). More advanced smoothing techniques seek to mitigate some of these limitations. Constrained mesh smoothing and context-aware methods seek to restrict smoothing to certain regions and reduce

3

Methods and Applications of 3D Patient-Specific Virtual Reconstructions in Surgery

59

Fig. 3.3 Polygon mesh smoothing

the amount of smoothing to preserve features (Moench et al. 2011; Moench et al. 2010).

3.2.4

Advanced 3D Modelling Techniques

Automated algorithmic approaches to postprocessing are more suited to clinical applications such as pre-operative planning and intraoperative navigation where speed and the fidelity of the models are of higher importance. However, when considering utilising 3D reconstructions for illustrative or educational purposes (e.g., student and patient educations), we can opt to use advanced 3D modelling techniques employed extensively in the visual effects and computer games industries. There is a myriad of software solutions available with

distinct and often overlapping functionality as summarised in Table 3.2. Most modelling workflows will employ multiple applications to take advantage of their unique features.

3.2.4.1

Complex 3D Modelling and Digital Sculpture 3D modelling applications give the user powerful control over all aspects of the polygon mesh which can be manipulated and altered using a wide range of available tools. Useful features pertinent to anatomical modelling include the ability to divide meshes into separate components, solidification/extrusion of surfaces to add wall thickness (e.g., in vessels and abdominal viscera) and animation techniques. Furthermore, we can augment the model by modelling structures such as the

60

J. Fletcher

Table 3.2 Summary of 3D modelling applications General 3D modelling applications AutoCAD 3ds Max Autodesk Maya Cinema 4D Blender MODO 3d sculpture ZBrush (Pixologic) ZBrush core/Sculptris (Pixologic) Mudbox (Autodesk) Blender (Blender Foundation) 3D texturing Substance Painter/Substance Designer/Substance Alchemist Mari Marmoset Toolbag

Subscription £1968/annum Subscription £1968/annum Variable subscription rates—£55.10/months Free £1596 perpetual licence; subscription £539/year £652 perpetual licence £131.34 Subscription: £12/month Free Indie subscription—£14.53/month £1619 perpetual licence Subscription—£10/month

mesentery that we cannot currently segment and visualise. Digital sculpture is primarily used for high polygon organic modelling (especially relevant for anatomy which consists of curved and irregular surfaces). Sculpting allows the introduction of detail that would be otherwise difficult with traditional polygon modelling techniques. We have employed digital sculpture to add striations to muscle and other fine surface details to reconstructed anatomy. One of the most useful features is the ability to model at different subdivisions (low to high polygon). The high polygon model detail can be projected onto the low polygon model and exported as normal map (see below). Normal maps fake the lighting of surface detail enabling the performance advantage of a low polygon mesh combined with the superior visual appearance of high polygon model.

3.2.4.2 Retopology 3D objects derived from segmentation are composed of a triangle mesh. However, most advanced 3D modelling techniques are better suited to a mesh composed of quadrangles (commonly termed ‘quads’), especially if UV unwrapping and texturing or animation is to be employed (see later). Most 3D modelling software have built-in algorithmic tools to facilitate rapid quad remeshing as demonstrated in Fig. 3.4.

Fig. 3.4 Quad remeshing of triangle polygon mesh of superior mesenteric vein and tributaries

3.2.4.3 UV Unwrapping UV unwrapping describes how a 2D image is projected onto a 3D model and is a pre-requisite texture mapping. UV space is based on a 0 to 1 grid, and a UV map consists of the 3D model ‘XYZ’ co-ordinates flattened into a 2D ‘UVW’ space/tile.

3

Methods and Applications of 3D Patient-Specific Virtual Reconstructions in Surgery

61

Fig. 3.5 UV-unwrapped 3D model of mesenteric vessels and associated organs with UV check grid applied as a texture

The letters ‘U’ and ‘V’ denote the axes of the 2D texture map (as ‘X’, ‘Y’ and ‘Z’ are used to describe the axes of the model in 3D space). Seams are created on the model which inform how the mesh should be cut and laid flat in the UV grid. Auto UV unwrapping can be used, but manually placed seams by the user will yield better results for complex structures. For an optimal result, UV maps should have a consistent scale and avoid visible seams and stretching and ensuring an even texel density. Most 3D applications have UV checker grids to ensure a model has been unwrapped correctly (Fig. 3.5).

3.2.4.4

Texture Maps and Physically Based Rendering Texture mapping describes the method of applying 2D images to 3D objects in order to alter their physical appearance. Texturing is one of the best means of achieving photorealistic rendered models. Rendering in a computer graphics context can be defined as a process producing an image from the information in a three-dimensional scene.

Physically based rendering (PBR) is a method of rendering that aims to simulate reality by modelling how light interacts with a material and can be used to produce photorealistic models (Pharr et al. 2010). There are multiple map types in PBR rendering, which we outline below: • Normal maps are used to simulate surface details. It is a RGB colour map where each component corresponds to the X, Y and Z co-ordinates of a surface normal (the vector perpendicular to the tangent plane of a surface—in computer graphics it determines the orientation to the light source). Its main purpose is to add detail without adding additional geometry. Normal maps can therefore greatly enhance the appearance of low polygon models. Normal maps do not alter the position of the model’s vertices; therefore, the model silhouette is unaltered. • Bump maps are greyscale maps that add simulated height to the model surface (black representing the lowest and white the highest

62

elevation). Unlike normal maps, bump maps display only height differences and do not contain angle information. Bump maps can be used for shallow fine detail. • Displacement maps are greyscale and unlike normal and bump maps change the position of the vertices to alter the height of the surface of the mesh along its normal, altering the underlying geometry. PBR has two principal workflows: metal/ roughness and specular/displacement. Maps specific to metal/roughness workflows include base colour, metallic and roughness. Base colour (albedo) maps denote the raw colour with no lighting information. Photographic images can be used to give a photorealistic quality. Metallic maps are black and white images that acts as a mask that defines areas that denote raw metal (characterised by high reflectance of light) or non-metals also known as dielectrics. The metallic map informs the shader how to interpret information found in the base colour. The roughness map is a similarly greyscale image and denotes the surface irregularities that cause light diffusion with smooth surfaces reflect light in a uniform manner, while rougher surfaces scatter light. Specular/glossiness workflows use unique texture maps—diffuse colour, specular and glossiness maps. The reflectance values for metal and non-metal materials are placed in the specular maps. The glossiness map functions in a similar capacity to the roughness map. Other maps utilised in both workflows include transparency, translucency, emission, refraction, ambient occlusion and subsurface scattering. Subsurface scattering describes how light penetrates the surface of translucent objects (such as the skin or an organ surface) and is scattered after being reflected multiple times at irregular angles within the material and is an essential component of photorealism (McDermott 2018) An example reconstruction of the colon with PBR texture maps can be seen in Fig. 3.6.

J. Fletcher

3.3 3.3.1

Applications of 3D Models in Surgical Practice 3D Models in Surgical Planning

Pre-operative planning can be defined as any activity aimed at understanding a patient’s anatomy or pathology in order to inform decisionmaking and/or determine an appropriate surgical treatment (Salb et al. 1999). 3D models have been used in a wide array of planning applications which can be categorised as follows: 1. Improving understanding of anatomy/ pathology 2. Patient-specific simulation 3. Resection planning (usually in the context of oncological surgery) 4. Reconstruction planning

3.3.1.1 Anatomical Understanding One of the principal benefits often cited of 3D visualisation is an improved understanding, with the implicit assumption that this will facilitate subsequent operative performance with the ultimate benefit to patient outcomes. The evidence remains contradictory. Awan et al. found the use of 3D-printed acetabular fracture models improved the ability to identify fracture subtypes in a group of orthopaedic trainees (Awan et al. 2018). Similarly, objective understanding of renal tumours was improved for both student and consultant urologists when 3D models were used in comparison to conventional CT imaging (Lee et al. 2018). Other authors have failed to find any benefits in using 3D. Yang et al. found no benefit of 3D virtual or printed models in comparison to CT when surgeons were asked to identify three retroperitoneal vessels. However, the consultant urologists were examined only on basic anatomical structures (vena cava, renal vein, etc.) with 3D likely being of most benefit with complex variable anatomy (Yang et al. 2018a). Azer et al. performed a systematic review on the

3

Methods and Applications of 3D Patient-Specific Virtual Reconstructions in Surgery

63

Fig. 3.6 PBR maps and final render for 3D reconstruction of the colon, pancreas, duodenum and mesenteric vessels

impact of 3D anatomy models on learning and demonstrated the heterogeneous nature of the literature in terms of study design and outcome measures which makes meaningful meta-analysis difficult. Sixty percent of the 30 studies were randomised control trials and with remaining 40% non-randomised comparative studies. Sixty percent utilised objective outcome measures with subjective ratings were used in the rest (Azer and Azer 2016). Students generally had a preference for 3D visualisation techniques over traditional teaching methods (Azer and Azer 2016). Crucially, it was recognised that multiple factors interact to influence the effectiveness of 3D models on learning—3D model appearance,

interface design and most importantly integration in a wider curriculum.

3.3.1.2 Patient-Specific Simulation Patient-specific simulation can be subdivided into process and outcome simulation. In process simulation, the model is used to recreate aspects of the surgery as a form of rehearsal. Outcome simulation attempts to predict operative results and can include aesthetic outcomes post reconstruction, blood flow or organ function (Preim and Botha 2014). Simulation-based training has assumed increasing importance in surgical education in

64

recent decades owing to reduced operative volume and patient safety concerns. 3D modelling has opened the possibility of patient-specific rehearsal and overcomes some of the limitations of existing simulation techniques that rely on generic models lacking real-world anatomical variation (Kneebone 2005; Reznick 2005). Scale models of bony anatomy have been widely employed by trauma and maxillofacial surgeons to perform osteotomies and pre-shape and application of osteosynthesis plates. 3D models may be especially beneficial in complex facial reconstructions (Fan et al. 2017; Ciocca et al. 2012; Zheng et al. 2018). Multiple comparative studies report reduced operative times, improved accuracy and superior aesthetic results (Crafts et al. 2017). Advances in 3D printing technology have enabled printing of soft flexible materials. Initial feasibility studies for simulating nephrectomies with patient-specific 3D-printed models have been undertaken. Glybochko et al. evaluated patient-specific silicone models for five patients with renal cell carcinoma. Models had high face validity, with surgeons rating the subjective utility of the models (Glybochko et al. 2018). von Rundstedt et al. generated soft material models for patient-specific rehearsal. Similar enucleation times and resected tissue volumes between the model and actual tumours demonstrated construct validity (von Rundstedt et al. 2017). Although promising, such these early studies lack any clear objective outcome measures for the impact on operative performance. Further drawbacks relate the time and expense incurred. Generating virtual patient-specific realistic procedural simulations based on imaging-derived 3D models remains a significant challenge. In addition to the complex model processing required to optimise for use in a game engine, high-level programming skills are required to applications able to simulate complex physical interactions based on user input (Zhang et al. 2017). Currently, only a limited number of early technical feasibility studies are available that have not assessed the effect on surgical performance or

J. Fletcher

patient outcomes (Rai et al. 2018; Won et al. 2018).

3.3.1.3 Resection Planning Achieving an R0 tumour-free resection margins is one of the most important predictors of long-term disease-free survival (Shaikh et al. 2016). Hepatobiliary surgeons have used 3D reconstructions for planning liver resections for hepatobiliary carcinomas (Xiang et al. 2015). Surgeons must balance ensuring tumour-free margins while preserving enough tissue to avoid post-resection liver failure. Identification of vascular tributaries is a fundamental aspect of surgical planning in order to preserve healthy liver. Multiple studies have compared and assessed the effect of 3D visualisation in comparison to the standard 2D CT and have found significant reductions in blood loss and operative times. 3D virtual simulated resections have been shown to accurately predict values for the specimen volume and surgical margins using the technique (Tian et al. 2015). In one comparative cohort study comparing 3D virtual planning with 2D CT in 305 consecutive patients undergoing hepatectomy, the surgical plan was altered in 49/131 of complex cases with 15 patients deemed previously unresectable based on 2D DICOM data re-considered for curative intervention (Wang et al. 2017). Furthermore, by combining perfusion data with virtual resections, surgeons can automatically be provided with resection volumes, functional liver reserve and dysfunction volumes, all critical information when planning the feasibility of hepatectomy. The impact on oncological outcome remains to be seen with conflicting results observed in the non-randomised comparative studies available. No difference between the R0 resection rates was found between 2D and 3D groups of patients undergoing hepatectomy for hilar cholangiocarcinoma (Andert et al. 2017). 3D models have similarly been utilised for planning minimally invasive partial nephrectomy for renal cell carcinoma. This procedure can be used with certain tumours to enable effective local tumour control while preserving maximum

3

Methods and Applications of 3D Patient-Specific Virtual Reconstructions in Surgery

renal function (Russo et al. 2002) . 3D has been employed as surgeons must possess a detailed understanding of the renal hilar vessels in order to preserve perfusion to the remaining kidney. In one randomised control trial by Shirk et al., the use of the 3D virtual reconstruction when performing robotic partial nephrectomy resulted in a significant reduction in operative time, estimate blood loss and vessel clamp time (Russo et al. 2002). Porpiglia et al. and Wang et al. similarly reported significant reductions in operative times and intraoperative blood loss (Russo et al. 2002; Zheng et al. 2018). Other preliminary works have been conducted in planning complete mesocolic excision for colonic cancer (Shirk et al. 2019) and complex sarcoma resections (Porpiglia et al. 2018).

3.3.1.4 Reconstruction Oral maxillofacial surgery (OMFS) primarily deals with the reconstruction of the bones of the facial area following trauma or to correct congenital malformations (Luzon et al. 2018). Craniofacial anatomy is not only geometrically complex, but the aesthetic outcomes are of critical importance as facial deformities are highly visible, carry significant stigma and can have a devastating impact on a patient’s mental health. 3D visualisation has been shown to improve the speed and accuracy of reconstruction. 3D virtual planning enables surgeons to plan osteotomies, rehearse reconstructive techniques (Luzon et al. 2018) and accurately model postoperative appearances (Jentzsch et al. 2016). Furthermore, using computer-assisted design (CAD) applications with 3D printing has allowed pre-operative planning to be transferred into the operating theatre through personalised cutting guides and custom implants (Herlin et al. 2013). In facial trauma the spatial localisation of bone fragments is essential in restoring contours of the face. Several studies have demonstrated the value of 3D CT visualisation in the treatment of displaced complex mid-facial and mandibular fractures (Herlin et al. 2013; Van Hemelen et al. 2015; Day et al. 2018). One review by Lin et al. identified 78 studies in the past decade that have employed 3D virtual planning with 3D-printed cutting guides used to

65

transfer virtual planning to actual orthognathic reconstructions (Lo Casto et al. 2012). While the majority of studies were prospective using historical controls, the majority reported improved accuracy and significantly reduced operative times (Hanasono et al. 2010).

3.3.2

Intraoperative Navigation

3D models have also been utilised for intraoperative navigation. The method and complexity of 3D model-guided surgery can range from simply displaying the model on a separate screen within the theatre up to advanced augmented reality (AR)-based navigation techniques. Our unit has utilised the built-in Tile Pro function of the da Vinci surgical robot to utilise the 3D reconstruction to aid intraoperative decisionmaking in complete mesocolic excision (see Fig. 3.7). The concept of computer-assisted surgical navigation with real-time tracking of the patient and instruments in relation to imaging data is long-standing having been initially developed in neurosurgery (Lin et al. 2018). In stereotactic navigation, imaging data is uploaded to a dedicated processing unit, and the surgeon can verify their instrument position on the imaging in the axial, coronal and sagittal planes (Ayoub et al. 2014). More recently the idea of utilising 3D reconstructions as augmented reality overlays on the operative view has gained traction. The appeal of AR rests on the potential to visualise structures that cannot be immediately seen by the surgeon (e.g., internal structures of the liver). Augmented reality exists on the reality-virtuality continuum and is characterised by overlaying digital elements on a real-world scene (Gildenberg 1983). AR must seamlessly integrate pre-operative reconstructions with intraoperative imaging, registration and surgical tracking within a common framework (Wijsmuller et al. 2018). The ability to accurately register a 3D model to the operative view and subsequently synchronise with the dissection and tissue manipulations of the surgeon remains a significant technical challenge.

66

J. Fletcher

Fig. 3.7 Live intraoperative view from da Vinci console with Tile Pro used to display 3D reconstruction of mesenteric blood vessels. Complete mesocolic excision involves dissection of the complex variable central

mesenteric tributary vessels; therefore 3D models can help improve anatomical understanding and decisionmaking

AR can be divided into video-based AR and optical see-through AR achieved via headmounted displays such as the Microsoft HoloLens™. Minimally invasive surgery, particularly using the da Vinci surgical robotics platform, has dominated video AR. Key technical components of AR surgical navigation include:

device to create a map of the room to create a 3D co-ordinate system in which the virtual model is placed (Milgram et al. 1994). • Registration refers to aligning the 3D patientspecific model with intraoperative acquired data. Registration can be: – Manual (in which a human scales, transforms and rotates the virtual model until it is a good match with the intraoperative view). – Point-based—a set of natural (e.g., bone) or artificial (e.g., fiducial markers glued to the skin). – Shape-based—a surface mesh/point cloud of the intraoperative anatomy is registered

• Optical calibration is key to enable transformation between the model and the environment. In video-based systems, the camera is calibrated so the surgical site corresponds to the video image derived from the camera sensor. Calibration of optical see-through devices is more complex and utilises sensors in the

3

Methods and Applications of 3D Patient-Specific Virtual Reconstructions in Surgery

to the pre-operative model based on the shape information. – Volume-based—3D geometric information is acquired intraoperatively and mapped to the 3D pre-operative planning image (Linte et al. 2010). • Tracking is essential to ensure the virtual elements correspond to the live operative view as the surgical view of the scene changes. In robotic systems the built-in kinematics could be used to track the instruments in relation to the camera view. Corrective measures are generally needed owing to error propagation between the co-ordinate encoder and camera frame (Frantz et al. 2018). Optical tracking uses videometric or infraredbased systems. Mono- and single-camera tracking systems have been utilised. In multicamera setups, fixed fiducial markers used during registration are used to determine the dynamic reference frame by triangulating their position in 3D space. Mono-camera systems calculate the position of markers by homography. The accuracy of tracking is affected by the distance between the tracker and camera. More recent approaches seek to improve the accuracy of tracking utilising mapping of the surgical scene from the endoscope view or simultaneous localisation and mapping approaches (Maier-Hein et al. 2019; Pachtrachai et al. 2019). The clinical utility of AR overlays remains to be seen with the majority of the literature focusing on technical development/feasibility (Mahmoud et al. 2019; Lamarca et al. 2021). In one recently published systematic review of AR navigation in robotic surgery, the authors reviewed 93 studies over 19 years of research and concluded the technology has yet to reach maturity (Lamarca et al. 2021). There is evidence to suggest that AR overlays can induce user fatigue with headset-based solutions and be distracting through visual clutter in the case of video overlays (Bertolo et al. 2020; Qian et al. 2019). In the immediate future, research will likely focus on engineering improvements before clinical utility can be truly assessed.

3.3.3

67

3D Models in Surgical Patient Education

Patient education is a pre-requisite of informed consent and shared decision-making (Lambooij et al. 2009). However, existing processes may be inadequate with results from multiple studies suggesting patient comprehension of their operation is poor (Hughes-Hallett et al. 2015; Marteau et al. 2001; Mark and Spiro 1990; Lavelle-jones et al. 2009). The way information is presented clearly has a dramatic influence on patient understanding and recall (Lemaire 2006). 3D models have been utilised as patient education and communication tools. One of the principle-reported advantages of 3D visualisation is improved anatomical perception (Fink et al. 2010; Bui et al. 2012; Azer and Azer 2016; Marconi et al. 2017; Brazina et al. 2014). 3D models may have an advantage over traditional medical illustration in helping patients understand their disease. This may be especially important for complex highrisk operations. Our unit has utilised personalised 3D models for patients undergoing pelvic exenteration surgery for advanced or recurrent rectal cancer. For patients with advanced rectal cancer, there is a consideration between an extended survival and the risk of significant morbidity with radical surgery. 3D models can facilitate understanding of their disease and the rationale for such radical surgery (see Fig. 3.8). Patients cite improved understanding of their disease, better communication with medical professionals and a positive emotional impact as the main benefits. Other studies examining the use of 3D models in patient education have been encouraging. Yoon et al. found personalised printed models improved patient self-reported knowledge and satisfaction during consent for surgical resection for stage 1 lung cancer (Wang et al. 2018). Similarly improved satisfaction was observed with 3D-printed models used for pre-operative education for patients undergoing robot-assisted partial nephrectomy (Yang et al. 2018b). 3D has also been shown to improve patient knowledge objectively. Bernhard et al. assessed the effect of personalised models on patient understanding of their renal tumour using a custom questionnaire.

68

J. Fletcher

Fig. 3.8 (a–c) 3D reconstructions based on CT/MRI data of three advanced rectal cancer cases. Models were used during the consent process at the St Mark’s Hospital Complex Cancer Clinic (London, UK) as part of an ongoing study evaluating the effectiveness of 3D models in patient consent. Segmentations were performed manually using ITK-SNAP

Those given a personalised 3D kidney model demonstrated an improved understanding of renal physiology by 16.7%, anatomy by 50%, tumour characteristics by 39% and planned procedure by 44.6% (Yoon et al. 2018). Parents of children undergoing hepatectomy demonstrated improved objective understanding when 3D-printed models were employed during the consent process (Teishima et al. 2018). Such studies only compare 3D against current standard practice and utilise variable non-validated outcome measures.

3.4

Conclusion

3D modelling is an emerging technology that will likely be increasingly integrated into surgical workflows corresponding with rapid advances in

technology and ubiquitous computing. Patientspecific modelling has the potential to improve anatomical understanding of pre-operative planning and facilitate patient communication. However, the effectiveness of 3D modelling in surgery has yet to be established. Research is needed to help elucidate the ideal user (novice versus expert) and specific indications. Intuitively, 3D seems most applicable to procedures involving complex anatomy or precision. The literature is extremely heterogeneous in terms of what procedure, study design and outcome measures. Relatively few studies include any form of comparator making conclusions of efficacy difficult. Advances in the technical aspects of model production will be necessary, with increasing automation and quality assurance measures a priority. Demonstrating the added value of 3D must be offset against the added

3

Methods and Applications of 3D Patient-Specific Virtual Reconstructions in Surgery

time and expenditure incurred. Consideration must also be given to the end user and platform design if 3D models are to achieve their potential. The rapidly proliferating means of interacting with 3D model (desktop, mobile device, virtual and augmented reality platforms and 3D-printed physical models) adds further complexity to design and implementation. With an increasing focus on precision in surgery, the hope in 3D visualisation will find suitable applications and ultimately help improve patient outcomes. Collaboration between surgeons, radiologists, engineers, computer scientist and most importantly patients will be crucial to develop solutions that help achieve this highly desirable goal.

References Andert A et al (2017) Impact of preoperative threedimensional computed tomography cholangiography on postoperative resection margin status in patients operated due to hilar cholangiocarcinoma. Gastroenterol Res Pract 2017:1947023 Awan OA et al (2018) Efficacy of 3D printed models on resident learning and understanding of common acetabular Fracturers. Acad Radiol 26:130 Ayoub N et al (2014) Evaluation of computer-assisted mandibular reconstruction with vascularized iliac crest bone graft compared to conventional surgery: a randomized prospective clinical trial. Trials 15 Azer SA, Azer S (2016) 3D anatomy models and impact on learning: a review of the quality of the literature. Health Professions Edu 2:80–98 Bertolo R et al (2020) Systematic review of augmented reality in urological interventions: the evidences of an impact on surgical outcomes are yet to come. World J Urol 38:2167–2176 Brazina D, Fojtik R, Rombova Z (2014) 3D visualization in teaching anatomy. Procedia Soc Behav Sci 143: 367–371 Bücking TM et al (2017) From medical imaging data to 3D printed anatomical models. PLoS One 12:1–10 Bui D, Nakamura C, Bray BE, Zeng-Treitler Q (2012) Automated illustration of patients instructions. AMIA Symposium pp. 1158–67 Ciocca L et al (2012) CAD/CAM guided secondary mandibular reconstruction of a discontinuity defect after ablative cancer surgery. J Cranio-Maxillofac Surg 40: e511–e515 Crafts TD et al (2017) Three-dimensional printing and its applications in otorhinolaryngology–head and neck surgery. Otolaryngol Head Neck Surg 156:999–1010

69

Day KM, Gabrick KS, Sargent LA (2018) Applications of computer technology in complex craniofacial reconstruction. Plast Reconstr Surg Glob Open 6:e1655 Fan B et al (2017) Clinical effects of 3-D printing-assisted personalized reconstructive surgery for blowout orbital fractures. Graefes Arch Clin Exp Ophthalmol 255: 2051–2057 Fink AS et al (2010) Predictors of comprehension during surgical informed consent. ACS 210:919–926 Frantz T, Jansen B, Duerinck J, Vandemeulebroucke J (2018) Augmenting Microsoft’s HoloLens with vuforia tracking for neuronavigation. Healthc Technol Lett 5:221–225 Gildenberg PL (1983) Stereotactic neurosurgery and computerized tomographic scanning. Appl Neurophysiol 46:170–179 Glybochko PV et al (2018) Multiple application of threedimensional soft kidney models with localized kidney cancer: a pilot study. Urologia 85:99–105 Hanasono MM, Jacob RF, Bidaut L, Robb GL, Skoracki RJ (2010) Midfacial reconstruction using virtual planning, rapid prototype modeling, and stereotactic navigation. Plast Reconstr Surg 126:2002–2006 Herlin C, Charles J, Bigorre M, Cheikh H, Captier G (2013) Computer-assisted midface reconstruction in Treacher Collins syndrome part 1 : skeletal reconstruction. 41:670–675 Hu P et al (2017) Automatic abdominal multi-organ segmentation using deep convolutional neural network and time-implicit level sets. Int J Comput Assist Radiol Surg 12:399–411 Hughes-Hallett A et al (2015) Inattention blindness in surgery. Surg Endosc 29:3184–3189 Jentzsch T, Vlachopoulos L, Fürnstahl P, Müller DA, Fuchs B (2016) Tumor resection at the pelvis using three-dimensional planning and patient-specific instruments: a case series. World J Surg Oncol 14:1–12 Kaur D, Kaur Y (2014) Various image segmentation techniques: a review. Int J Comput Sci Mob Comput 3:809–814 Kim J, Jung Y, Feng DD, Fulham MJ (2020) Chapter seventeen–biomedical image visualization and display technologies. In: Feng DD (ed) Biomedical information technology (second edition). Academic Press, pp 561–583. https://doi.org/10. 1016/B978-0-12-816034-3.00017-1 Kneebone R (2005) Evaluating clinical simulations for learning procedural skills: a theory-based approach. Acad Med 80:549–553 Lamarca J, Parashar S, Bartoli A, Montiel JMM (2021) DefSLAM: tracking and mapping of deforming scenes from monocular sequences. IEEE Trans Robot 37: 291–303 Lambooij MM, IJsselsteijn W, Fortuin MM, Heynderickx II (2009) Visual discomfort and visual fatigue of stereoscopic displays: a review. J Imag Sci Tec 53:1–14 Lavelle-jones C, Byrne DJ, Rice P, Cuschieri A (2009) Factors affecting quality of informed consent. BMJ 306:885–890

70 Lee H et al (2018) Personalized 3D kidney model produced by rapid prototyping method and its usefulness in clinical applications. Int Braz J Urol 44:952–957 Lemaire R (2006) Informed consent–a contemporary myth ? J Bone Joint Surg 88:1–7 Lin HH, Lonic D, Lo LJ (2018) 3D printing in orthognathic surgery–a literature review. J Formos Med Assoc 117:547–558 Linte CA, White J, Eagleson R, Guiraudon GM, Peters TM (2010) Virtual and augmented medical imaging environments: enabling technology for minimally invasive cardiac interventional guidance. IEEE Rev Biomed Eng 3:25–47 Lo Casto A et al (2012) Imaging evaluation of facial complex strut fractures. Seminars in Ultrasound, CT and MRI 33:396–409 Luzon JA et al (2018) Implementation of 3D printed superior mesenteric vascular models for surgical planning and/or navigation in right colectomy with extended D3 mesenterectomy: comparison of virtual and physical models to the anatomy found at surgery. Surg Endosc 32:567–575 Mahmoud N et al (2019) Live tracking and dense reconstruction for handheld monocular endoscopy. IEEE Trans Med Imaging 38:79–89 Maier-Hein L, Speidel S, Stenau E, Chen E, Ma B (2019) Chapter 3: registration. In: Mixed and augmented reality in medicine. CRC Press, Taylor and Francis Group Marconi S et al (2017) Value of 3D printing for the comprehension of surgical anatomy. Surg Endosc Other Interv Tech 31:4102–4110 Mark JS, Spiro H (1990) Informed consent for colonoscopy. Arch Intern Med 150:777 Marteau T, Dormandy E, Michie E (2001) A measure of informed choice. Health Expect 4:99–108 McDermott, W. (2018). The PBR guide Milgram P, Takemura H, Utsumi A, Kishino F (1994) Augmented reality: a class of displays on the realityvirtuality continuum, pp 282–292 Moench T, Adler S, Preim B (2010) Staircase-aware smoothing of medical surface meshes. In Proceedings of the 2nd eurographics conference on Visual Computing for Biology and Medicine. Eurographics Association, pp 83–90 Moench T, Gasteiger R, Janiga G, Theisel H, Preim B (2011) Context-aware mesh smoothing for biomedical applications. Comput Graph 35:755–767 Pachtrachai K, Vasconcelos F, Dwyer G, Hailes S, Stoyanov D (2019) Hand-eye calibration with a remote centre of motion. IEEE Robot Autom Lett 4:3121– 3128 Paragios N, Duncan J (2015) Handbook of biomedical imaging. Handbook of biomedical imaging. https:// doi.org/10.1007/978-0-387-09749-7 Pharr M, Humphreys G, Hanrahan P (2010) Physically based rendering: from theory to implementation. Elsevier Science & Technology Porpiglia F, Fiori C, Checcucci E, Amparore D, Bertolo R (2018) Hyperaccuracy three-dimensional

J. Fletcher reconstruction is able to maximize the efficacy of selective clamping during robot-assisted partial nephrectomy for complex renal masses. Eur Urol 74: 651–660 Preim B, Botha CP (2014) Visual computing for medicine. Morgan Kaufmann Qian L, Wu JY, Maio S, Navab N, Kazanzides P (2019) A review of augmented reality in robotic-assisted surgery. IEEE Transactions on Medical Robotics and Bionics pp, 1–1 Rai A et al (2018) Patient-specific virtual simulation–a state of the art approach to teach renal tumor localization. Urology 120:42–48 RCS (2018) Future of surgery Reznick RK (2005) Surgical simulation. Ann Surg 242: 640–641 Robb RA (1999) 3-D visualization in biomedical applications. Annu Rev Biomed Eng 1:377–399 Russo P et al (2002) Partial nephrectomy: the rationale for expanding the indications. Ann Surg Oncol 9:8 Salb T, Weyrich T, Dillmann R (1999) Preoperative planning and trainng simulation for risk reducing surgery. In: Proceedings of international training and education conference, pp 1–8 Sarkar S (2014). Why frame rate and resolution matter: a graphics primer. Polygon https://www.polygon.com/ 2014/6/5/5761780/frame-rate-resolution-graphicsprimer-ps4-xbox-one Shaikh I et al (2016) High subcortical sacrectomy: a novel approach to facilitate complete resection of locally advanced and recurrent rectal cancer with high (S1-S2) sacral extension. Color Dis 18:386–392 Sharma N, Aggarwal LM (2010) Automated medical image segmentation techniques. J Med Phys 35:3–14 Shirk JD et al (2019) Effect of 3-dimensional virtual reality models for surgical planning of robotic-assisted partial nephrectomy on surgical outcomes: a randomized clinical trial. JAMA Netw Open 2:e1911598–e1911598 Technologies U (n.d.) Unity–manual: optimizing graphics performance. https://docs.unity3d.com/Manual/ OptimizingGraphicsPerformance.html Teishima J et al (2018) Usefulness of personalized threedimensional printed model on the satisfaction of preoperative education for patients undergoing robotassisted partial nephrectomy and their families. Int Urol Nephrol. https://doi.org/10.1007/s11255-0181881-2 Tian F et al (2015) Three-dimensional morphometric analysis for hepatectomy of centrally located hepatocellular carcinoma: a pilot study. World J Gastroenterol 21: 4607–4619 Trebeschi S et al (2017) Deep learning for fully-automated localization and segmentation of rectal cancer on multiparametric MR. Sci Rep 7:1–9 Van Hemelen G et al (2015) Three-dimensional virtual planning in orthognathic surgery enhances the accuracy of soft tissue prediction. J Craniomaxillofac Surg 43:918–925

3

Methods and Applications of 3D Patient-Specific Virtual Reconstructions in Surgery

von Rundstedt FC, Scovell JM, Agrawal S, Zaneveld J, Link RE (2017) Utility of patient-specific silicone renal models for planning and rehearsal of complex tumour resections prior to robot-assisted laparoscopic partial nephrectomy. BJU Int 119:598–604 Wang X-D et al (2017) Traditional surgical planning of liver surgery is modified by 3D interactive quantitative surgical planning approach: a single-center experience with 305 patients. Hepatobiliary Pancreat Dis Int 16: 271–278 Wang L et al (2018) Three-dimensional printing PEEK implant: a novel choice for the reconstruction of chest wall defect. Ann Thorac Surg Wijsmuller AR et al (2018) A step towards stereotactic navigation during pelvic surgery: 3D nerve topography. Surg Endosc Other Interv Tech 32:3582–3591 Won TB et al (2018) Early experience with a patientspecific virtual surgical simulation for rehearsal of endoscopic skull-base surgery. Int Forum Allergy Rhinol 8:54–63 Xiang N et al (2015) Application of liver threedimensional printing in hepatectomy for complex massive hepatocarcinoma with rare variations of portal vein: preliminary experience. Int J Clin Exp Med 8:18873–18878

71

Yamashita R, Nishio M, Do RKG, Togashi K (2018) Convolutional neural networks: an overview and application in radiology. Insights Imaging 9:611–629 Yang T et al (2018a) Impact of 3D printing technology on comprehension of surgical anatomy of retroperitoneal tumor. World J Surg 42:2339–2343 Yang T et al (2018b) The impact of using threedimensional printed liver models for patient education. J Int Med Res 46:1570–1578 Yoon SH et al (2018) Personalized 3D-printed model for informed consent for stage I lung cancer: a randomized pilot trial. Semin Thorac Cardiovasc Surg: 3–5. https:// doi.org/10.1053/j.semtcvs.2018.10.017 Yushkevich PA et al (2006) User-guided 3D active contour segmentation of anatomical structures: significantly improved efficiency and reliability. NeuroImage 31:1116–1128 Zhang J, Chang J, Yang X, Zhang JJ (2017) Virtual reality surgery simulation: a survey on patient specific solution. Lect Notes Comput Sci:220–233. https://doi.org/ 10.1007/978-3-319-69487-0_16 Zheng W et al (2018) Application of 3D-printing technology in the treatment of humeral intercondylar fractures. Orthopaedics Traumatol: Surg Res 104:83–88

4

Proof of Concept for the Use of Immersive Virtual Reality in Upper Limb Rehabilitation of Multiple Sclerosis Patients Rachel-Anne Hollywood, Matthieu Poyade, Lorna Paul, and Amy Webster

Abstract

Multiple sclerosis (MS) is a debilitating disease which gradually reduces motor function and mobility. Virtual reality (VR) has been successfully utilised in support of existing therapeutic approaches for many different conditions, and new innovative and experimental features could be the future of VR rehabilitation. The Quest is a new headset by Oculus, with its built-in tracking, relatively low cost, portability and lack of reliance on expensive processing heavy PCs to power it, and could be an ideal system to facilitate at-home or clinic-based upper limb rehabilitation. A hand-tracking-based rehabilitation game aimed at people with MS was developed for Oculus Quest using Unity. Two distinct games were made to replicate different types of hand exercises, piano playing for isolated finger flexion and maze tracking for coordination and arm flexion. This pilot study assesses the value of such approach along with evaluating intrinsic and extrinsic methods of providing feedback, namely, positive scoring, negative scoring and audio response. One physiotherapist and two individuals with MS were surveyed. Participant response was

R.-A. Hollywood (*) · M. Poyade The Glasgow School of Art, Glasgow, Scotland, UK

positive although small sample size impacts the user testing validity of the results. Future research is recommended to build off the data gathered as a pilot study and increase sample size to collect richer feedback. Keywords

Multiple sclerosis · Virtual reality rehabilitation · Hand-tracking · Oculus Quest · Upper limb rehabilitation · Intrinsic and extrinsic feedback

4.1

Rationale

Multiple sclerosis (MS) is a degenerative disease particularly prevalent in Scotland that gradually reduces mobility and dexterity (Kearns et al. 2019). This makes performing everyday tasks increasingly difficult for the individual and can therefore be detrimental to quality of life (QOL). However, this can be partially managed through regular exercise (Latimer-Cheung et al. 2013). While rehabilitation games aimed at stroke patients follow many of the same base concepts as those for MS patients, they rarely consider aspects like muscle weakness and fatigue that affect people with MS. These studies also typically look at the improvements in motor skills or how gamification of therapy affects adherence (Burke et al. 2009; Alankus et al. 2010).

L. Paul · A. Webster Glasgow Caledonian University, Glasgow, Scotland, UK # The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 P. M. Rea (ed.), Biomedical Visualisation, Advances in Experimental Medicine and Biology 1356, https://doi.org/10.1007/978-3-030-87779-8_4

73

74

R.-A. Hollywood et al.

Expanding on previous work with the Leap Motion (LM) tracking system examining its suitability for use in the rehabilitation of MS patients (Webster et al. 2019; Soomal et al. 2020), the Oculus Quest offers built-in hand-tracking eliminating the need for further equipment such as a computer set-up necessary for the LM. This project examined if immersive virtual reality (VR) systems would be as suitable for upper limb rehabilitation as its non-immersive VR and lower limb counterparts (Choi 2014). This project will look at the existing literature on conventional and technology-based rehabilitation to facilitate the design and development of an application for Oculus Quest to act as a proof of concept for the systems used in MS rehabilitation and to examine the role of feedback in rehabilitation. This research aims to fill the gap of examining the feasibility of using the Oculus Quest hardware in the field of MS rehabilitation as previous studies highlighted the need for an immersive element in MS rehabilitation and Soomal et al. (2020) proposed examining innovative head-mounted devices (HMDs).

4.2

Multiple Sclerosis and Conventional Physiotherapy

The three main categories of MS are primaryprogressive (PPMS), relapsing-remitting (RRMS) and secondary-progressive (SPMS), with RRMS being the most common and in 70% of cases advancing to SPMS (National Multiple Sclerosis Society 2020). The disease can be highly debilitating, affecting an individual in many ways, such as causing a loss of mobility and dexterity, as well as fatigue (Müri et al. 2015; Rohrig 2018). This can contribute to depression and a decreased QOL as an individual gradually loses their ability to perform basic everyday tasks (Janardhan and Bakshi 2002). MS varies in symptoms and severity, and it has no definitive cause, but the disease has been linked to many factors including gender,

environment and lifestyle choices with the disease being more common in females, those who live further from the equator in temperate climates and smokers (Ascherio and Munger 2007; Olsson and Alfredsson 2017; Causes of MS 2020; MS: the facts 2020). Symptoms are managed to maintain as high a QOL as possible for the individual affected and to slow progression of the disease. This can be achieved via physical therapy, occupational therapy or disease-modifying medication (MS: the facts 2020). Conventional physical therapy has shown a positive correlation in slowing the progression of MS symptoms, in both remitting and progressive forms of MS (Latimer-Cheung et al. 2013). Physiotherapy is usually designed around the individual, adapting their goals to their level of impairment. By ensuring that these rehabilitative goals are specific, measurable, attainable, realistic and timely (SMART) to the individual, it is possible to optimise recovery (Rohrig 2018). Physiotherapy largely works due to leveraging the plasticity of the brain by having an individual perform repetitive actions to retrain the brain to adapt to the damage that caused the symptoms, while it is possible to achieve significant restoration of mobility there are limitations to its efficacy (Rohrig 2018). Arm ability training (AAT) is one proposed method of rehabilitation (Platz et al. 2001). AAT was designed to improve manual dexterity post traumatic brain injury and focuses on movements necessary in everyday activities. The set of exercises designed by a clinician for Platz et al.’s study contained movements that were also highlighted by a focus group of individuals with MS and a study by Webster et al. (2019), with exercises focusing on tapping, pinching and twisting being particularly relevant for everyday activities. Home-based or telerehabilitation has risen in use, allowing an individual to participate in therapy in their own home to help remove some barriers to treatment (Peretti et al. 2017), namely, removing the need to travel to a facility which can be a temporal and physical drain to the individual as well as being a cost-effective alternative for

4

Proof of Concept for the Use of Immersive Virtual Reality in Upper Limb. . .

health services (Housley et al. 2016). This is particularly important in current circumstances with Covid-19 which prevents attendance to such services (Mantovani et al. 2020). Despite this home-based rehabilitation also has its limits, primarily due to low adherence rates (Carter et al. 2003; Engström and Öberg 2005). While the repetitive actions are ideal for utilising plasticity, participants often find the exercises boring and have low levels of motivation for maintaining practice (Lohse et al. 2013). This problem could be where VR and gaming technology could be beneficial.

4.3

Virtual Reality-Based Rehabilitation

While usually associated with the commercially available HMDs, virtual reality is a combination of aspects including interaction and visualisation used to immerse a user entirely in a virtual environment. There is much debate about the use of virtual reality in the rehabilitation of motor skills. While the consensus for treatment compliance is positive but debated (Rose et al. 2018) and pain relief is positive (Hayden et al. 2005; Jones et al. 2016; Chan et al. 2006), there remains an insight to be gained about the capabilities of immersive VR in rehabilitation. Effectively, one review reported that when stroke patients used solely virtual reality rehabilitative therapy (VRRT), their recovery was noticeably inferior to those who used solely conventional physiotherapy but marked an improvement in recovery when VRRT was used in conjunction with conventional therapy to augment or bolster therapy time (Laver et al. 2017). This contradicts Laver et al. (2017) who suggested that VRRT was more effective than conventional therapies although not in a statistically significant manner. This could be due to how emergent the technology was, the studies examined in Laver et al. (2017) analysis as all reported low participant numbers and familiarity with the technology which could potentially affect systems’ use and perception (Standen et al. 2015).

4.3.1

75

Interaction

To fully leverage the benefits of VR in the field of rehabilitation, it becomes necessary to combine head-mounted displays (HMDs) with peripheral technology, whether it be controllers, tracking or haptics. These peripherals aid in immersion (Witmer and Singer 1998) and psychological involvement (Slater and Wilbur 1997) and allow for a more natural interaction model that mimics the everyday actions a patient may need to relearn. Sucar et al. (2013) study of gesture therapy uses controllers paired with a specially designed support for the patient’s hands and wrists. This study compared the variance in kinematic traces and hence showed that across several sessions using the virtual reality system, patient’s hand control improved while participating in the game-based exercises they were set. Meanwhile other studies examined tracking peripherals, and one such peripheral is the Kinect™ device. Kinect uses depth sensors and red-green-blue cameras to track motion. Originating in the gaming sector, it has since been adopted by the clinical sector for use in full-body and lower limb rehabilitation. The low cost and accessibility of Kinect has made it popular in the field of VR rehabilitation research as it is ideal for telerehabilitation (Mousavi Hondori and Khademi 2014). However, it suffers from accuracy issues that make it less effective when tracking precise movements such as those found in hand rehabilitation (Zhou and Hu 2007). However, another hand-tracking device known as the Leap Motion (LM) utilises infrared light-emitting diodes and cameras to track hand movement and allows for a more natural interaction model, showing promising results in preliminary research for potential use for rehabilitation. Soomal et al. (2020) found that people with MS were generally optimistic about the use of LM for home-based rehabilitation; however, they cited latency issues, a lag between the actions they performed and the reaction on screen as being off-putting. While participants in Webster et al. (2019) mostly found that it was difficult to understand how interactions worked with LM, this

76

R.-A. Hollywood et al.

could be attributed to lack of practice with an unfamiliar technology. LM has also previously been used together with HMDs in the rehabilitation field to create a fully immersive interactive system (Dias et al. 2019; Sulimanov and Olano 2019). With the release of the Oculus Quest HMD which has built-in hand-tracking, a similar untethered and cost-effective all-in-one system is now available. The tracking offered by both LM and Kinect also has limitations. While Kinect has been used for hand-tracking (Cordella et al. 2012), it is more suitable for full-body tracking (Hondori and Khademi 2014). Meanwhile, the LM suffers from limited tracking range, distortion during bimanual manipulation and occlusion (Tao et al. 2013; Webster et al. 2019). Another possible method to augment a system dealing with the improvement of motor skills would be the inclusion of haptic feedback. Many MS patients lose grip strength as a part of their condition. Including force feedback or resistance when a patient picks up a virtual object within a virtual environment could aid in both the improvement of motor function (Adamovich et al. 2005) and immersion (Kim et al. 2017), the latter of which plays a part in adherence and pain relief as previously discussed. This could be achieved by using a cyber-glove or similar technology (Polygerinos et al. 2015).

4.3.2

Visualisation

Presence and immersion are other factors that can influence the effects of VR in rehabilitation. Presence consists of a psychological state which can be defined as feeling you are in a place but you are not physically inhabiting that space, while immersion is reliant on technology for sensory inducement within an experience (Berkman and Akan 2019; Slater 2018). While originally VR systems had a costbenefit trade-off in recent years, new systems have been released with lower starting costs and greater specs like field of view and refresh rate (Borrego et al. 2018). These new, leaner and more cost-effective systems could be a beneficial aid in home-based rehabilitation. Research suggests that

a sense of immersion and perceived presence in a virtual environment has many effects including increasing participant motivation (Ijsselsteijn et al. 2004). Virtual reality often ‘gamifies’ treatment for patients. Some studies examined the role of gamification in therapy and how it can result in heightened levels of patient motivation and exercise adherence due to giving patients a feeling of control or agency over their own recovery (Burke et al. 2009; Lau et al. 2017). Furthermore, factors such as task and performance feedback and intensity to be changed to suit the patient’s needs, ensuring the exercises are never too easy or too challenging for the patient. This keeps them in a state of flow, which minimises feelings of boredom or frustration that might usually cause a patient to abandon their treatment (Chen 2007).

4.3.3

HMDs in MS Rehabilitation

The specific use of HMDs in rehabilitative treatment of MS is a growing field of study, with many researchers examining how they affect cognitive load and user’s sense of immersion (Ozkul et al. 2020). Headsets allow for a greater sense of presence and perceived immersion in a virtual environment (Slater 2018), which can be beneficial for rehabilitation in several ways. Particularly in gait and lower limb rehabilitation, HMDs have been widely studied with Kern et al. (2019) designing a study using a treadmill and a HMD experiment to examine the impacts on gait rehabilitation. The study results suggest that patients with MS found using HMDs in gait rehabilitation less physically taxing while maintaining a similar level of cognitive load and it increased their perceived performance and competence. Participants also found the virtual training to be less frustrating and on average spent longer in a session than they did with their conventional therapy which was attributed to increased motivation supporting several other studies that link VR rehabilitation with increased motivation. While Guo and Quarles (2012) study found that participants with MS when asked to score how natural walking through a virtual

4

Proof of Concept for the Use of Immersive Virtual Reality in Upper Limb. . .

environment, using a 7-Likert scale, they scored higher than the healthy control group. On a scale where 1 is the lowest and 7 is the highest sense of naturalness, participants with MS scored 5.3 as opposed to the healthy participants who scored the interaction method as 4.4 on average. This suggests that how people with MS experience presence in VR is different and therefore should be taken into account when designing VR experiences for people with MS.

4.4

Treatment Adherence and Motivation

There have been numerous studies into how exercise improves dexterity and upper limb movement in MS (Lamers 2016; Latimer-Cheung et al. 2013), showing a positive correlation between time spent participating in rehabilitative activities and regained limb functionality (Kwakkel et al. 2004; Schneider et al. 2016). Due to the falling cost of virtual reality technology, it has become possible for patients to have a system in their home and engage in home-based rehabilitative exercise (Jack et al. 2001). However, home-based rehabilitative exercise has low adherence rates with one study finding that 50% to 55% of those prescribed home-based rehabilitative exercises were non-adherent (Carter et al. 2003). Some studies suggested that utilising virtual reality during treatment can lead to increased patient motivation, an important factor in treatment adherence (Jack et al. 2001; Saunders 2015; Elor et al. 2018). A review found that most studies reported high levels of adherence when using virtual reality in a rehabilitation environment, suggesting that the immersion brought by the system and the gamification of exercises played a part in increased patient adherence (Rose et al. 2018). However, Rose et al. (2018) also suggested that small sample sizes prevent these findings from being statistically significant. Contrary to these findings, Standen et al. (2015) reported low levels of adherence with participants using a VR system citing unfamiliarity with the technology as a barrier to use. Despite this, the

77

participants acknowledged the motivating effects of the technology as well as its flexibility.

4.4.1

Feedback

There are two main forms of feedback, intrinsic and extrinsic, which can be further defined as either concurrent or terminal. Intrinsic feedback is sensory and innate, while extrinsic feedback is supplementary, such as commentary on form from a clinician/therapist (Magill and Anderson 2014). While it is difficult to emulate intrinsic feedback, extrinsic feedback has been used in many different fields to provide information on performance and task result (Magill 2001; Van Vliet and Wulf 2006), and motivate the feedback recipient (Hartveld and Hegarty 1996). Extrinsic feedback can be further broken down into two subcategories, knowledge of results (KR) and knowledge of performance (KP), with the latter being attributed to increased motor learning retention (Mcnevin et al. 2011). KR is binary, usually indicating whether a task has been performed adequately or not, while KP is more descriptive, giving an indication as to how performance can be improved. Another factor to consider is the time at which feedback is administered due to the current debate around when the optimal time is. While concurrent feedback shows increased performance during administration, these benefits are short term and can adversely affect an individual’s innate ability for intrinsic feedback (Mcnevin et al. 2011). It is also suggested that after the initial stages of learning concurrent feedback becomes redundant and instead applying terminal feedback in decreasing frequency is effective (Sigrist et al. 2013). Feedback is not ‘one size fits all’, for example, while a low score may inspire determination; in some the same score may cause deflation in others. Parker et al. found that while stroke patients found feedback a motivating factor, others found that feedback indicating poor performance made them reluctant to use the system (Parker et al. 2014). One participant highlighted

78

R.-A. Hollywood et al.

scores as being a motivating factor, with his carer suggesting that the praise he received out with the exercise itself acts as positive reinforcement encouraging further participation. It is important to note that these studies examine stroke patients rather than MS; there are very few papers that focus specifically on feedback for people with MS. While stroke and MS have many similar symptoms, MS patients can suffer from additional symptoms such as fatigue and cognitive impairment that could change the affect that specific forms of feedback may have.

4.5

Aims and Objectives

The primary aim of this project was to design and develop a suite of upper limb rehabilitation games for an immersive VR system utilising existing rehabilitation strategies for MS. The secondary aim was to examine the role performance and task feedback play in VR-based upper limb rehabilitation. To achieve these aims, the following objectives were created: • Build a body of knowledge around MS and the current methods of conventional therapy used in upper limb treatment for the condition. • Design and develop a gamified application for the Oculus Quest based upon findings from formative literature in the topics of MS rehabilitation, VR rehabilitation and upper limb rehabilitation. • Assess the content validity of the development through appraisal from people with MS and physiotherapists specialising in neurological rehabilitation. • Discuss whether these findings indicate the games are fit for purpose and reflect on the role of feedback.

4.6 4.6.1

Methods Workflow (Fig. 4.1)

4.6.1.1 Materials The game was developed and built in Unity for use with an Oculus Quest headset.

Adobe Photoshop CC was used during the design process to create prototype interface designs. To create 3D model assets, 3ds Max was used, while Substance Painter and Adobe Photoshop CC were used to create textures. These assets were imported to Unity which acted as the game’s development environment. In order to create an immersive VR experience, the Oculus Plugin for Unity was used to set up the environment for use with a HMD and handtracking. Finally, Visual Studio Code was used for scripting. Third-party 3D and audio assets were utilized to cut development time and due to limited experience creating these mediums. 3D assets were sourced from Sketchfab, TurboSquid, Unity Asset Store and OpenGameArt. The audio assets were sourced from OpenGameArt, Berklee College of Music Sampling Archive and Ministerio de Educación y Formación Profesional.

4.6.2

Design and Development Process

The first step in the design phase was to look at existing literature on exercises used in the rehabilitation of MS and identify what type of movements these exercises trained. These movements could then be mapped to real-life activities that would provide a base narrative for each serious game. The proposed exercises also take inspiration from real-life activities and existing non-rehabilitative games; these were then ranked by development priority. A storyboard detailing the menu flow of the game was created outlining the user interface (UI) design (Fig. 4.2). The UI was designed with usability in mind leveraging Nielsen’s usability heuristics (Nielsen 2005). Figure 4.3 is an additional storyboard of the individual game designs, detailing their intended outcomes and mechanics. Colour can be important for creating the ideal environment for the game. Certain colours can be used to evoke specific feelings and moods within a user; in this case the aim is to create a calming neutral environment that evokes focus (Hidayetoğlu and Ozkan 2011). To this end

4

Proof of Concept for the Use of Immersive Virtual Reality in Upper Limb. . .

79

Research

Game Design

Storyboarding

Asset Acquisition

Environment Design

Alpha Implementation

Implementation

Survey Design

Evaluation

Fig. 4.1 Workflow of the project

Fig. 4.2 Researcher storyboards part 1

blues and greens were selected in pastel shades, and these are tones often used in hospital waiting rooms to create a calming atmosphere (Hidayetoğlu and Ozkan 2011). A cream colour was selected instead of white, to bring

warmth and ensure the environment was not too ‘sterile’ and ‘cold’, while brown was chosen to bring contrast and act as a grounding influence.

80

R.-A. Hollywood et al.

Fig. 4.3 Researcher storyboards part 2

4.7

Developmental Outcomes

Three scenes were made all set within one perpetual 3D environment with the viewpoint of the player being changed by switching to cameras situated in different areas of the virtual environment. One scene was created as a menu to allow the player to choose which game they wished to play and to swap between the two games; two scenes were created as games that would require the hand and finger movements to replicate the exercises in Table 4.1, the piano scene and the maze scene.

4.7.1

Menu Scene

The first scene in the game is the menu scene, and this scene displays the game’s title along with

buttons that allow the user to interact with and navigate through multiple menus. The menu is integrated as an object within the virtual environment, in this case a TV screen, rather than as a head-up display (HUD) (Fig. 4.4). The player can turn around and view the whole environment at their leisure; they can read about why the game was created by pressing the About button, swap to the game selection menu by pressing the Games button or quit the game entirely. Interactions take place at three key points in the scene as shown in Fig. 4.5. The Xs signify the camera’s starting position, with the colour indicating which scene it is referring to. Red for menu, blue for piano and yellow for maze. The game selection menu replaces the initial menu and gives the player the choice between

4

Proof of Concept for the Use of Immersive Virtual Reality in Upper Limb. . .

81

Table 4.1 Proposed exercises and their inspiration Proposed exercise Piano playing

Movement Coordinated and isolated finger extension

Priority Compulsory

Bubble popping game Ping pong

Interphalangeal flexion of the thumb and index finger (pinching) Wrist flexion/extension

Optional

Circuit maze

Arm flexion and medial rotation (aiming/maze tracking)

Compulsory

Optional

Description The game would emulate typing by having the player hit set keys to produce a tune. Would utilize mechanics like Guitar Hero™, (System 2005) i.e. the key should be hit in time with some sort of indicator Bubbles will appear around the scene in various positions, and the player must pinch them to pop them The player places their hands at either side of a virtual pinball machine where they must press the buttons on the sides to operate paddles to prevent the ball from leaving play. Virtual rods will be in place to encourage the correct hand movement Recreating the circuit/electric maze board game. Player has to follow the path of the maze using their index finger. Feedback will be given based on how many times they touched the wall of the maze; the lower the number, the better

Based on Webster et al. (2019), Soomal et al. (2020) Also inspired by (Lang 2013) Webster et al. (2019), Soomal et al. (2020) Platz et al. (2001), Müri et al. (2015)

Platz et al. (2001)

Fig. 4.4 Initial view of the scene

two games, the piano game and the maze game, or to return to the initial menu.

4.7.2

Piano Scene

To aid immersion, measures were taken to ensure the environment was consistent across all three scenes the environment was loaded in only once and was not destroyed between scenes to ensure continuity; only the starting position of the camera is changed.

The aim was to gamify rehabilitation to achieve this; the mechanics of Guitar Hero™ (System 2005) were replicated using falling notes to indicate when keys should be pressed (Fig. 4.6). Rather than having set levels, the player could change settings to fit their stage of rehabilitation, swapping between different songs and speeds as necessary as shown in Fig. 4.7. The literature suggests that scoring is a motivating factor and indicator of progress to some patients. Scores can be compared by patients resulting in external motivation in the

82

Fig. 4.5 Player can turn around to view the rest of the environment

Fig. 4.6 Piano game in play

Fig. 4.7 Settings for the piano game could be changed

R.-A. Hollywood et al.

4

Proof of Concept for the Use of Immersive Virtual Reality in Upper Limb. . .

83

Fig. 4.8 Maze 1, Maze 2 and Maze 3 (pictured from left to right)

Fig. 4.9 Maze 3 (inverted). The starting and end points have swapped places. Also pictured to the right is the maze switching interface

form of competing with peers. By providing a score breakdown of perfect, great, good and missed notes it also acts as an internal form of motivation giving players an achievable goal, for example, getting all perfect notes or never missing a note.

4.7.3

Maze Scene

Three mazes were constructed in order to provide a difficulty curve, one simple and two more complex (Fig. 4.8). The UI in this scene was very simple with the scoring system offset to the left as can be seen in Fig. 4.8 and the controls to the right as seen in Fig. 4.9.

The player can invert these mazes, the idea behind this being to make it easier to swap which hand is in play and in the case of asymmetric mazes give the sensation of playing a new maze (Fig. 4.9). The player simply traces the shape of the maze with their finger, audio feedback in the form of a buzzer sound and visual feedback in the form of an incrementing error counter that will activate if the player’s finger leaves the maze before reaching the end. Development progressed steadily with relatively few issues, one major issue being that in the piano game the right hand could not reliably press the piano keys to trigger sound. Due to the small developer base and limited support as mentioned before, it was difficult to troubleshoot this issue. It was eventually resolved by removing the contact tests from the ButtonController script on the piano keys. Being unable to reliably access information on the causes of such issues contributed to lengthening both implementation and testing times due to time spent troubleshooting and diagnosing the problem.

4.7.4

Evaluation

4.7.4.1 Participants Three participants were recruited, one physiotherapist and two people with MS. Ages

84

R.-A. Hollywood et al.

Table 4.2 Participant population Participant A B C

Occupation Volunteer worker Medically retired Physiotherapist

varied between 25 and 60 years old, and gender was split: two males and one female (Table 4.2). These participants were recruited via a gatekeeper with prior permission to contact them about participation in future studies.

4.7.4.2

Experimental Set-Up and Procedure Due to health and safety concerns and lockdown restrictions relating to Covid-19, it was not possible to have the participants test the game directly. An alternate method was conceived sharing a screen capture recording which was taken from the game in use. This recording was then embedded into a survey in the form of an unlisted YouTube video only accessible via link (https:// youtu.be/CGRic1IGM7s) that would be sent out to the participants. This allowed the participants to watch the video in their own time to abide by lockdown legislation and guidelines. A survey was created asking 18 questions that would generate mostly qualitative data regarding what factors of the game would motivate and encourage treatment adherence for people with MS as well as to gain information on whether the games were suitable for their purpose. The surveys were sent out simultaneously to all participants, and no time limit for participation was given. An estimated completion time of 25 minutes was given including the time required to watch the video and answer the questions in the survey. 4.7.4.3 Ethics This experiment received ethical approval from the Glasgow School of Art learning and teaching office and follows the ethical guidelines put in place by this institution. 4.7.4.4 Data Analysis The survey contained 18 questions split into three main sections, 3 pre-screening questions,

Age 51 35 26

Gender Male Female Male

4 questions on motivation and 14 questions on content validity. The questions were mostly qualitative in nature as motivation and content validity are difficult concepts to quantify and qualitative data would provide a rich insight into the factors affecting motivation and validity. There were also a few quantitative questions in the form of Likert scales to gauge likelihood of use and usefulness of certain factors. These questions were: • How likely do you believe an individual with MS would be to use such a system? • How do you believe the presented approach would be likely to help through physical rehabilitation for upper limb? • How useful would you find these difficulty changes in supporting progressive physical rehabilitation? • How useful do you think hand-tracking technology like the system demonstrated could be in rehabilitation? The Likert scales were based on likelihood and on usefulness and used a range of 1 to 5. For questions 1 and 2, the scale denotes likelihood of use and likelihood of there being a benefit to use, with 1 being highly unlikely, 2 being somewhat unlikely, 3 being equally likely and unlikely, 4 being somewhat likely and 5 being very likely. For questions 3 and 4, the scale denotes general usefulness, with 1 being not useful, 2 being limited usefulness, 3 being some usefulness, 4 being useful and 5 being very useful. It is highly variable with what is deemed motivating varying from person to person; therefore, to get a general idea of what the target audience may find motivating, participants were asked to provide qualitative data on what features of the game they liked and disliked and to expand upon why this was the case. Content validity ensures that the game accomplishes its goal, in this case testing whether

4

Proof of Concept for the Use of Immersive Virtual Reality in Upper Limb. . .

the games and system proposed would provide appropriate exercise for rehabilitation and whether the target audience would even use the system. To achieve this, questions relating to likelihood of use and appropriateness of the games were given. The results of the survey were analysed using theme-based content analysis (Neale and Nichols 2001) to categorise responses into similar themes and sentiments to gain greater understanding of the general perception of the application and the common reasons or features that affect content validity and motivation (Table 4.3). It can also highlight trends and themes that are not initially apparent. Factors such as usability and presence were not evaluated due to inability to perform user testing of the application.

4.8

Results

The game was generally well received receiving mostly neutral and positive responses; there was however a distinct split in the responses from the participants with MS and the physiotherapist participant in the long-form answers. The first Likert question asked how likely participants would be to use such a system at home. Participants scored the game a 3.6 on average with participants with MS scoring it a 4 on average and physiotherapist scoring it a 3 (Fig. 4.10). The second Likert question asked if the participants believed the presented game would be useful in helping rehabilitate the upper limbs. Participants scored the game a 3.6 on average with participants with MS scoring it a 4 on average and physiotherapist scoring it a 3 (Fig. 4.11). The third Likert question asked the participants about the usefulness of adaptive difficulty in progressing through physical rehabilitation. On average this was scored a 4.6 with participants with MS scoring it a 5 on average and physiotherapist scoring it a 4 (Fig. 4.12). And the final Likert question asked about the usefulness of hand-tracking technology in general regarding rehabilitation. On average this was

85

scored a 4.6 with participants with MS scoring it a 5 on average and physiotherapist scoring it a 4 (Fig. 4.13).

4.9

Discussion

In summary, two distinct games were created for the Oculus Quest utilising its built-in hand-tracking technology to create an application aimed at rehabilitating hand movement in people with MS. Two people with MS and one physiotherapist were shown a video of the games in use and were surveyed on their opinions on motivating factors and content validity of the application. General comments on the project were positive with participants finding the games a novel method of rehabilitation and stating that they believed hand-tracking would be of use in rehabilitation. The games themselves were somewhat well received with participants with an average of 3.6 on the Likert scale suggesting that they were slightly useful for rehabilitation (Fig. 4.11). Theme-based analysis of participants’ answers to the qualitative questions in the questionnaire found that feelings towards scoring were mixed with the participants with MS feeling it had no effect on their motivation. This contrasts starkly with the findings of several studies which found that scoring would have a motivating or demoralising effect depending on the participant but that it was generally enjoyed as a method to track progress (Parker et al. 2014; Standen et al. 2015; Webster et al. 2019; Soomal et al. 2020). Meanwhile the physiotherapist believed it could potentially have motivating or demoralising effects depending on how it was implemented and hence should be presented as an option rather than being mandatory. This supports the existing literature discussed, in which Parker et al. (2014) found that low scores would inspire determination in some of their participants but for others it discouraged the use of the system. This discrepancy could be due to the participants being disconnected or otherwise uninvolved with the action on screen, i.e. an onlooker rather than being an active participant. A game is a piece of

86

R.-A. Hollywood et al.

Table 4.3 Analysed survey responses Response ‘[scoring had..] no effect’ ‘[scoring was. . .] not at all [effective]’ ‘The wording of the scoring system in the second (maze) game could also be revised to enhance motivation. For example, “errors” could be changed to something like “attempts”—this would move away from the negative associations of “errors”’ a ‘Might be worthwhile giving the patient the option of viewing/recording their scores’ a ‘. . .in the earlier stages of rehab with greater levels of impairment may be demotivated if their scores are low or do not progress’ a ‘I think the audio feedback provided a clear indication of success for both tasks’ a ‘It would be quite motivational to be a success as to be a failure’ ‘[audio feedback was. . .] not at all [effective]’ ‘By continuing to practice the VR, the user should be able to see steady improvements and this will encourage the user to keep going’ ‘It might help with coordination and rewire the brain to find an alternative way of problem solving’ ‘I think VR would be incredibly useful in strengthening the upper limbs and improving movement’ ‘Something that would be beneficial to my health then I would be more encouraged to participate’ ‘As this is a novel intervention, it will likely interest patients’ a ‘Alternative methods are always welcome of learning new ways to do things’ ‘Learning new skills’ ‘More games could be included to encourage long-term engagement from patients’ a ‘And perhaps encourage adherence’ ‘By starting morning easy setting and then increasing difficulty as upper limb movement improved, it offers a satisfying and visual scale to show the progress made’ a ‘Providing varying levels of task difficulty is essential to providing treatment’ ‘Through integrating visual and motor stimuli, hand-tracking systems have the potential to enhance upper limb motor recovery in people with neurological conditions’ a ‘The games presented will be appropriate for some upper limb impairments—for example, they are focused mainly on fine motor control of the wrist/hand’ a ‘Perhaps other games could be developed which incorporates global upper limb movement (including shoulder, elbow) within functional tasks/movements. In addition, impairments of coordination could also be considered’ a ‘I think this will vary from patient to patient based on factors such as self-efficacy, motivation, and impairments (e.g. cognition, visual impairments)’ a ‘The younger fraternity might find it easier than an older person’ ‘I would have liked to have participated myself’ ‘I think a key challenge of future research will be working out the characteristics of patients who will use this (and benefit from this) and why this is the case’ a ‘I think the music played during the games might be distracting—particularly for people with cognitive impairments’ a

Common theme Scoring (5)

Higher-order theme Motivation (12)

Audio feedback (3)

Health benefits (4)

Novelty (2)

Content Validity (7)

Treatment Adherence (2) Progression (2)

Hand-tracking Potential (1) Future Work (2)

Scalability (2)

Target audience (3)

Benefactors (4)

Identifying key users (1) Music (2)

Usability (4) (continued)

4

Proof of Concept for the Use of Immersive Virtual Reality in Upper Limb. . .

87

Table 4.3 (continued) Response ‘To enhance to usefulness of the audio feedback, perhaps the background music could only play in the main menu so that the only audio stimuli within the games are related to feedback’ a ‘The goal of each game was clear after the instructions were read. However, as the instructions were accessed after pressing a button, the goal of the games may not be initially clear to patients who do not access the instructions. Perhaps the instructions could appear automatically at the start of each game to ensure that all patients understand’ a ‘Perhaps the visual feedback within the first (piano) game could be enhanced to make it clearer which key the patient has to press’ a

Common theme

Higher-order theme

UI (2)

Answers given by physiotherapist participant

2

Physiotherapist

2 (66.7%)

Participant with MS

1 1 (33.3%)

0

0 (0%)

0 (0%)

1 Very Unlikely

2

0 (0%) 3

4

5 Very Likely

Fig. 4.10 Responses to question ‘How do you believe individuals with MS would be likely to use such a system at home?’

Fig. 4.11 Responses to question ‘How do you believe the presented approach would be likely to help through physical rehabilitation for upper limb?’

88

R.-A. Hollywood et al. 2

2 (66.7%)

Physiotherapist Participant with MS

1 1 (33.3%)

0

0 (0%)

0 (0%)

0 (0%)

1 Not Useful

2

3

4

5 Very Useful

Fig. 4.12 Responses to question ‘How useful would you find these difficulty changes in supporting progressive physical rehabilitation?’

interactive media so often the ‘feel’ or ‘impact’ cannot be fully experienced by just watching. Alternatively, a reason for this split between participants could be due to the small number of participants and the uneven split between physiotherapist participants and participants with MS. A large sample size is needed to research this split further. The wording of the scoring system in the maze game was also thought to be demoralising, and it was suggested that an alternative to ‘errors’ be used. All participants felt that the adaptable difficulty of the games was somewhat very useful in supporting progression through physical rehabilitation. On average adaptable difficulty scored a 4.6 on usefulness suggesting that participants felt that it was a very important feature to have for rehabilitative games. The physiotherapist highlighted how it could be tailored to an individual’s needs and stage of recovery over a longer timeline, while one of the participants with MS stated it would be helpful even over the short term of a session to be able to adapt the difficulty as their movement improved throughout the day. While there was a split in response tone in the long-form answers between physiotherapist and participants with MS, scoring in the Likert questions was consistent with the physiotherapist generally scoring the system lower. However, there was a difference of opinion within the participants with MS contingent in which the younger participant consistently scored the system higher. This is echoed by a statement that

there was a concern that older or people with less experience with the technology may find it more difficult to engage with a concept that was also highlighted in Standen et al. (2015) which stated that unfamiliarity with the technology could adversely affect adherence. It is also supported by a statement made by the physiotherapist that likelihood of usage will vary from patient to patient and that patients with specific characteristics will benefit from the system more. However, due to the circumstances which have forced into the aforementioned experimental procedure design, these results might lack subjectivity as it could be difficult to properly assess a fully immersive and interactive system without firsthand experience of using that system. A further study allowing participants to gain this experience by using the system may produce different results. Issues were highlighted regarding visual and audio feedback. For the former, which key the player had to press was often unclear until the note hit the key. It was also felt that the goal of each game, while clear once the information panel was revealed, should have been shown automatically rather than hidden behind a button press. This suggests that usability may currently be an issue and that future work may require improving upon this aspect. Audio feedback was well received with participants enjoying the instant task-based feedback with a preference to the positive reinforcement of the piano game over the negative reinforcement of the maze.

4

Proof of Concept for the Use of Immersive Virtual Reality in Upper Limb. . .

89

Fig. 4.13 Responses to question ‘How useful do you think hand-tracking technology like the system demonstrated could be in rehabilitation?’

However, participants felt the background music was a distraction and could overload people with cognitive impairments. This could be due to the volume of the music and this game’s heavy use of audio feedback to indicate performance. This finding contrasts with responses to music in Webster et al. (2019) in which the music was not noticed by most participants and the one who did found it quite pleasant. As music preference is subjective, it would be advisable to make it optional, especially in case it contributes to cognitive overloading. Most participants felt that people with MS might use the system at home, scoring it a 3.6, with one participant stating a desire to have been able to use the application for the evaluation. This could indicate a desire for immersive VR rehabilitation; however it may be necessary to rerun the study with user testing in order to get an accurate gauge of whether there is an audience for the system. Only one participant, the physiotherapist, gave a long-form comment on the overall potential for hand-tracking in rehabilitation stating that they believed it could be used to enhance recovery for those with neurological conditions. However, all participants felt that it could be useful with it scoring a 4.6 on average on the Likert scale. This supports the findings in both Webster et al. (2019) and Soomal et al. (2020) that hand-tracking could be an effective and novel tool for rehabilitation. Additional features were suggested with the most common being the inclusion of more games in order to add more variety, exercise

more of the upper limb and encourage long-term adherence. By combining serious game design patterns with emergent virtual reality technology, it may become possible to create an immersive experience that follows the principles of rehabilitative exercise: repetition, feedback, motivation and task-oriented training. The ability to adapt difficulty also allows for the technology to be used throughout the patient’s journey to recovery and give each session of play an achievable goal, further motivating the patient. This was highlighted in the survey responses which placed an emphasis on having a feeling of steady progress as being beneficial to treatment. No technical issues were highlighted by participants in the additional comments section of the questionnaire. The Quest seems well suited for the application due to its lightweight build and built-in hand-tracking; however, this was tested by a person without MS so further testing to study suitability for MS patients is required. No technical issues were detected when the system was in use; however implementation was challenging due to excessive encapsulation and limited documentation and developer support. It cannot be disputed that ‘out-of-the-box’ virtual reality is often unsuited to the needs of the patient and often require additional tracking peripherals such as Leap Motion for upper limb rehabilitation or Kinect for lower limb adding to cost. It also adds unnecessary weight and bulk to a system that can be uncomfortable and discourage use.

90

R.-A. Hollywood et al.

Additionally as an all-in-one system the Oculus Quest does not require a PC to run, PCs capable of powering VR are often very expensive meaning in a lot of cases the Quest may be the most cost-efficient alternative for VR treatment when factoring in the cost of the headset, PC and Leap Motion necessary for the same results. With the Quest’s tracking being comparable to Leap Motion, it suffers from the same problems in regard to occlusion as well as an additional issue in the form of a blind spot in the direct centre of the headset which would often result in lost tracking. However, it could be argued that the convenience of an all-in-one system makes the Quest an improvement on previous interaction models. The downside to the Quest lies largely in its experimental nature, and it is a relatively new device and is yet to generate a large developer base in the same way other technologies have and as hand-tracking is still deemed an experimental feature support for developing with it is extremely limited. This can make the implementation of even simple interaction methods using handtracking difficult especially as heavy use of encapsulation in the code base hinders access and manipulation of specific game objects such as only tracking the collisions of a single finger or only tracking a single hand. For this reason, many of the original features outlined in the design phase such as colour coding the fingers on the hand with the notes associated with them were not possible.

4.9.1

Future Works

In terms of the application itself, several noteworthy improvements were made by participants during the study, some were made post-evaluation such as only having the background music play in the menu scene rather than throughout the game to lessen the strain on cognitively impaired users and to highlight the audio feedback and stimuli the games provide. Other suggestions were the implementation of more games. Four games had been included in the initial plan. However this was later scaled back to two due to time constraints; hence future work

may include scaling the project back up as well as patching existing bugs present in the project such as menu buttons not resetting to their default state. The survey highlighted several concepts that require further study. Only one participant reported it highly likely that the system would be useful at home with another highlighting that it would vary from patient to patient who would benefit from its use. A possible gap in research may be the identification of what characteristics patients who would use this have and how these systems could be optimized based on these characteristics. Another consideration is that VR is a very practical technology, and hence it is difficult to assess aspects such as motivation from evaluating a video, due to infection risk and concerns that at this time it was impossible to have the participants use the system themselves. One participant commented on this stating that they would have liked to have participated themselves. In the future it may be beneficial to rerun this study with user testing, observation, and the target audience gaining real-life experience of using the application may provide a richer and more indicative idea of its features and faults. A larger contingent of participants with a roughly even split to observe if this trend continues would be beneficial.

4.10

Conclusion

Overall while the results of this study are promising in regard to the feasibility of Oculus Quest-based rehabilitation for MS, further study with more participants is required to better adapt immersive VR rehabilitation to the specific needs of people with MS. While some interesting points have been highlighted regarding the target audience for the system and the benefit of adaptable difficulty, some directly contradict findings in other studies such as the inclusion of background audio and the motivational effect of scoring. It may be necessary to conduct further research using user testing to validate these findings. Due to the small sample size, the validity of these findings is questionable; however, the results

4

Proof of Concept for the Use of Immersive Virtual Reality in Upper Limb. . .

could be used as a starting point for further study as they highlighted the varying opinions between the physiotherapist and the participants with MS. Also due to the technology’s emergent status, there is not extensive research on the use of the Oculus Quest as a tool for rehabilitation in MS.

References Adamovich SV et al (2005) A virtual reality-based exercise system for hand rehabilitation post-stroke. Presence, New Jersey Alankus, G. et al (2010) Towards customizable games for stroke rehabilitation Ascherio A, Munger KL (2007) Environmental risk factors for multiple sclerosis. Part I: the role of infection. Ann Neurol 61(4):288–299. https://doi.org/10. 1002/ana.21117 Berkman MI, Akan E (2019) Presence and immersion in virtual reality. In: Encyclopedia of computer graphics and games. Springer International Publishing, pp 1–10. https://doi.org/10.1007/978-3-319-08234-9_162-1 Borrego A et al (2018) Comparison of oculus rift and HTC vive: feasibility for virtual reality-based exploration, navigation, exergaming, and rehabilitation. Games for Health J Mary Ann Liebert Inc 7(3):151–156. https:// doi.org/10.1089/g4h.2017.0114 Burke JW et al (2009) Optimising engagement for stroke rehabilitation using serious games. Vis Comput 25 (12):1085–1099. https://doi.org/10.1007/s003710090387-4 Carter S, Taylor D, Levenson R (2003) A question of choice: compliance in medicine taking, London Causes of MS (2020). Accessed Jun 24, 2020, from https:// www.mstrust.org.uk/about-ms/what-ms/causesms Chan EA et al (2006) Application of a virtual reality prototype for pain relief of pediatric burn in Taiwan. https://doi.org/10.1111/j.1365-2702.2006.01719.x Chen J (2007) Flow in games (and everything else) Choi JH et al (2014) Effectiveness of commercial gamingbased virtual reality movement therapy on functional recovery of upper extremity in subacute stroke patients. Ann Rehabilitation Med. Korean Academy of Rehabilitation Medicine 38(4):485–493. https:// doi.org/10.5535/arm.2014.38.4.485 Cordella, F. et al (2012). Patient performance evaluation using Kinect and Monte CarloBased finger tracking Dias P et al (2019). Using virtual reality to increase motivation in post-stroke rehabilitation. Accessed Jun 25, 2020, from www.oculus.com/rift Elor A, Teodorescu M, Kurniawan S (2018) Project star catcher: a novel immersive virtual reality experience for upper limb rehabilitation. ACM Trans Access Comput 11. https://doi.org/10.1145/3265755

91

Engström LO, Öberg B (2005) Patient adherence in an individualized rehabilitation programme: a clinical follow-up. Scand J Public Health 33(1):11–18. https://doi.org/10.1080/14034940410028299 Enrique Sucar L et al (2013) Gesture therapy: an upper limb virtual reality-based motor rehabilitation platform. IEEE Transactions on Neural Systems and Rehabilitation Engineering Guo R, Quarles J (2012, March) Differences in presence between healthy users and users with multiple sclerosis. In 2012 IEEE VR Workshop on Perceptual Illusions in Virtual Environments (pp. 1–6). IEEE Hartveld A, Hegarty JR (1996) Augmented feedback and physiotherapy practice motor programme. https://doi. org/10.1016/S0031-9406(05)66414-0 Hayden JA, Van Tulder MW, Tomlinson G (2005) Systematic review: strategies for using exercise therapy to improve outcomes in chronic low back pain. Ann Intern Med:776–785. https://doi.org/10.7326/00034819-142-9-200505030-00014 Hidayetoğlu ML, Ozkan A (2011) Effects of interior colors on mood and preference: comparisons of two living rooms article in perceptual and motor skills. https://doi.org/10.2466/24.27.PMS.112.2.509-524 Hondori HM, Khademi M (2014) A review on technical and clinical impact of Microsoft Kinect on physical therapy and rehabilitation. https://doi.org/10.1155/ 2014/846514 Housley SN et al (2016) ‘Increasing access to cost effective home-based rehabilitation for rural veteran stroke survivors.’, Austin J Cerebrovasc Dis Stroke, 3(2), pp. 1–11. Accessed Jun 17, 2020, from http://www. ncbi.nlm.nih.gov/pubmed/28018979 Ijsselsteijn WA et al (2004) Virtual cycling: effects of immersion and a virtual coach on motivation and presence in a home fitness application Jack D et al (2001) Virtual reality-enhanced stroke rehabilitation, IEEE Transactions on Neural Systems and Rehabilitation Engineering Janardhan V, Bakshi R (2002) Quality of life in patients with multiple sclerosis: the impact of fatigue and depression. J Neurol Sci 205(1):51–58. https://doi. org/10.1016/S0022-510X(02)00312-X Jones T, Moore T, Choo J (2016) The impact of virtual reality on chronic pain. https://doi.org/10.1371/journal. pone.0167523 Kearns PKA et al (2019) Regional variation in the incidence rate and sex ratio of multiple sclerosis in Scotland 2010–2017: findings from the Scottish multiple sclerosis register. J Neurol 266(10):2376–2386. https://doi.org/10.1007/s00415-019-09413-x Kern F, Winter C, Gall D, Käthner I, Pauli P, Latoschik ME, (2019, March). Immersive virtual reality and gamification within procedurally generated environments to increase motivation during gait rehabilitation. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (pp. 500–509). IEEE Kim M, Jeon C, Kim J (2017) A study on immersion and presence of a portable hand haptic system for

92 immersive virtual reality. Sensors 17(5):1141. https:// doi.org/10.3390/s17051141 Kwakkel G et al (2004) Effects of augmented exercise therapy time after stroke: a metaanalysis. Stroke 35 (11):2529–2539. https://doi.org/10.1161/01.STR. 0000143153.76460.7d Lamers I (2016) Anneleen Maris, Deborah Severijns, Wouter Dielkens, Sander Geurts, Bart van Wijmeersch, and Peter Feys. "upper limb rehabilitation in people with multiple sclerosis: a systematic review.". Neurorehabil Neural Repair 30(8):773–793 Lang B (2013) Road to VR. https://www.roadtovr.com/ mixed-reality-pianotrainer-makes-learning-pianoeasy/ Latimer-Cheung AE et al (2013) Effects of exercise training on fitness, mobility, fatigue, and health-related quality of life among adults with multiple sclerosis: a systematic review to inform guideline development. Arch Phys Med Rehabil:1800–1828.e3. https://doi. org/10.1016/j.apmr.2013.04.020 Lau HM et al (2017) Serious games for mental health: are they accessible, feasible, and effective? A systematic review and meta-analysis. Front Psych. https://doi.org/ 10.3389/fpsyt.2016.00209 Laver KE et al (2017) Virtual reality for stroke rehabilitation. Cochrane Database Syst Rev. https://doi.org/10. 1002/14651858.CD008349.pub4 Lohse K et al (2013) Video games and rehabilitation: using design principles to enhance engagement in physical therapy. https://doi.org/10.1097/NPT. 0000000000000017 Magill RA (2001) Motor learning as a function of KR schedule and characteristics of task-intrinsic feedback the effects of erroneous information on performance and learning view project. https://doi.org/10.1080/ 00222890109601903 Magill RA, Anderson DI (2014) Motor learning and control concepts and applications. Accessed Jun 24, 2020, from www.mhhe.com Mantovani E, Zucchella C, Bottiroli S, Federico A, Giugno R, Sandrini G, Chiamulera C, Tamburin S (2020) Telemedicine and virtual reality for cognitive rehabilitation: a roadmap for the COVID-19 pandemic. Front Neurol 11:926 Mcnevin NH et al (2011) An examination of feedback use in rehabilitation settings supra-postural task constraints on optimizing postural control view project the effects of erroneous information on performance and learning view project an examination of feedback use in rehabilitation settings. Critical Rev Phys Rehabilitation Med 23(4):147–160. https://doi.org/10.1615/ CritRevPhysRehabilMed.v23.i1-4.110 Mousavi Hondori H, Khademi M (2014) A review on technical and clinical impact of microsoft kinect on physical therapy and rehabilitation. J Med Eng 2014 MS: the facts (2020) Accessed Jun 24, 2020, from https:// www.mstrust.org.uk/about-ms/what-ms/msfacts

R.-A. Hollywood et al. Müri R et al (2015) Mult Scler-2015-Kamm multiple sclerosis Msj Journal. https://doi.org/10.1177/ 1352458514565959 National Multiple Sclerosis Society (2020) Types of MS, New York. Accessed Jun 15, 2020, from http://www. nationalmssociety.org/What-is-MS/Types-of-MS Neale H, Nichols S (2001) Theme-based content analysis: a flexible method for virtual environment evaluation. Int J Human Comp Stud 55(2):167–189. https://doi. org/10.1006/ijhc.2001.0475 Nielsen J (2005) Ten usability heuristics. Accessed July 4, 2020, from http://www.useit.com/papers/heuristic/ heuristic_list.html Olsson T, Alfredsson L (2017) Interactions between genetic, lifestyle and environmental risk factors for multiple sclerosis individual-participant data metaanalysis in working populations view project multiple sclerosis research view project. https://doi.org/10. 1038/nrneurol.2016.187 Ozkul C, Guclu-Gunduz A, Yazici G, Guzel NA, Irkec C (2020) Effect of immersive virtual reality on balance, mobility, and fatigue in patients with multiple sclerosis: a single-blinded randomized controlled trial. Eur J Integr Med 35:101092 Parker J et al (2014) Stroke patients’ utilisation of extrinsic feedback from computerbased technology in the home: a multiple case study realistic evaluation. BMC Med Inform Decis Mak 14(1):46. https://doi.org/10.1186/ 14726947-14-46 Peretti A, Amenta F, Tayebati SK, Nittari G, Mahdi SS (2017) Telerehabilitation: review of the state-of-the-art and areas of application. JMIR Rehabilitation and Assistive Technologies 4(2):e7 Platz T et al (2001) Arm ability training for stroke and traumatic brain injury patients with mild arm paresis: a single-blind, randomized, controlled trial. Arch Phys Med Rehabil 82(7):961–968. https://doi.org/10.1053/ apmr.2001.23982 Polygerinos P et al (2015) Soft robotic glove for hand rehabilitation and task specific training Rohrig M (2018) A resource for healthcare professionals physical therapy in multiple sclerosis Rose T et al (2018) Immersion of virtual reality for rehabilitation–review. https://doi.org/10.1016/j. apergo.2018.01.009 Saunders W (2015) Effectiveness, usability, and costbenefit of a virtual reality-based telerehabilitation program for balance recovery after stroke: a randomized controlled trial. Arch Phys Med Rehabil 96 (3):418–425. https://doi.org/10.1016/j.apmr.2014.10. 019 Schneider EJ et al (2016) Increasing the amount of usual rehabilitation improves activity after stroke: a systematic review. J Physiother 62(4):182–187. https://doi. org/10.1016/j.jphys.2016.08.006 Sigrist R et al (2013) Augmented visual, auditory, haptic, and multimodal feedback in motor learning: a review. Psychon Bull Rev:21–53. https://doi.org/10.3758/ s13423-012-0333-8

4

Proof of Concept for the Use of Immersive Virtual Reality in Upper Limb. . .

Slater M (2018) Immersion and the illusion of presence in virtual reality. Br J Psychol 109(3):431–433 Slater M, Wilbur S (1997) A framework for immersive virtual environments (FIVE): Speculations on the role of presence in virtual environments. Accessed Jun 24, 200, from http://www.cs.ucl.ac.uk/staff/M.Slater Soomal HK et al (2020) Enabling more accessible MS rehabilitation training using virtual reality, pp 95–114. https://doi.org/10.1007/978-3-030-43961-3_5 Standen PJ et al (2015) Patients’ use of a home-based virtual reality system to provide rehabilitation of the upper limb following stroke. Phys Ther 95 (3):350–359. https://doi.org/10.2522/ptj.20130564 Sulimanov L, Olano M (2019) Virtual reality Mirror therapy rehabilitation for post-stroke patients. https://doi. org/10.1145/3306214 Systems HM (2005) Guitar Hero

93

Tao G et al (2013) Evaluation of Kinect skeletal tracking in a virtual reality rehabilitation system for upper limb hemiparesis. https://doi.org/10.1109/ICVR.2013. 6662084 Van Vliet PM, Wulf G (2006) Extrinsic feedback for motor learning after stroke: what is the evidence? https://doi.org/10.1080/09638280500534937 Webster A, Poyade M, Rea P, Paul L (2019) The co-design of hand rehabilitation exercises for multiple sclerosis using hand tracking system. In: Biomedical Visualisation. Springer, Cham, pp 83–96 Witmer BG, Singer MJ (1998) Measuring presence in virtual environments: a presence questionnaire., Presence Zhou H, Hu H (2007) Human motion tracking for rehabilitation-a survey. https://doi.org/10.1016/j.bspc. 2007.09.001

5

Virtual Wards: A Rapid Adaptation to Clinical Attachments in MBChB During the COVID-19 Pandemic Camille Huser, Kerra Templeton, Michael Stewart, Safiya Dhanani, Martin Hughes, and James G. Boyle

Abstract

When the COVID-19 pandemic suddenly prevented medical students from attending their clinical attachments, the faculty involved in the third year of medical school (MBChB3) at the University of Glasgow created Virtual Wards. The focus of the Virtual Wards was to continue teaching of clinical reasoning remotely whilst COVID-19 restrictions were in place. Virtual Wards were mapped to the common and important presentations and conditions and provided opportunity for history-taking, clinical examination skills, requesting investigations, interpreting results, diagnosis and management. The Virtual Wards were successful, and further wards were developed the following academic year for MBChB4 students. This chapter describes the theoretical underpinnings of the Virtual Wards and the technological considerations, followed by a description of the Wards themselves. We then analyse an evaluation of the Virtual Wards and provide both a faculty and student perspective. Throughout the chapter,

C. Huser (*) University of Glasgow, School of Medicine, Glasgow, UK e-mail: [email protected] K. Templeton · M. Stewart · S. Dhanani · M. Hughes · J. G. Boyle University of Glasgow, School of Medicine, Glasgow, UK NHS Greater Glasgow and Clyde, Glasgow, UK

we provide tips for educators developing Virtual Ward environments. Keywords

Virtual Ward · Clinical reasoning · COVID19 · Online distance learning · Medicine · Surgery

5.1

Introduction

Medical students in MBChB3 at the University of Glasgow were withdrawn from our clinical campus during the first wave of the COVID-19 pandemic (early in the second semester of the academic year from March 2020 to June 2020). For MBChB3 students, this meant the loss of two 5-week clinical attachments in Junior Medicine and Junior Surgery and an urgent need to rapidly adapt face-to-face case-based teaching that mapped to the students ‘top presentations’ and intended learning outcomes. In response, Junior Virtual Wards were rapidly developed for Junior Medicine and Junior Surgery. Whilst it was not possible to replicate the experience of embedding into a clinical team, as is the case in face-to-face attachments, we aimed to facilitate the students’ longitudinal development of core knowledge and clinical reasoning in readiness for their return. When students returned in MBChB4 Semester 1 (August 2020), further Senior Virtual Wards were developed to support the delivery of longer

# The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 P. M. Rea (ed.), Biomedical Visualisation, Advances in Experimental Medicine and Biology 1356, https://doi.org/10.1007/978-3-030-87779-8_5

95

96

C. Huser et al.

clinical attachments in Senior Medicine and Senior Surgery. At that time, face-to-face small group teaching was limited (due to faculty availability and social distancing), and students were at risk of self-isolation (due to being infected with COVID-19 or having been in contact with a COVID-19-infected person). This chapter will outline some of the theoretical underpinnings and technological considerations of the Virtual Wards. We then provide a detailed description of the Virtual Wards before reflecting from a faculty and student perspective. We conclude by looking forward to future work and suggesting some tips for implementation in your local context.

5.2

Theoretical Underpinnings

Virtual Wards were designed to develop medical students’ core knowledge and clinical reasoning. Clinical reasoning can be defined as: the cognitive processes by which clinicians integrate clinical information (history, exam findings, and test results), preferences, medical knowledge, and contextual (situational) factors to make decisions about the care of an individual patient. (Cook et al. 2019, p. 1310)

Clinical reasoning has not been explicitly taught in most medical schools until recently, but there is a growing consensus developing amongst medical educators that health profession curricula should explicitly include clinical reasoning deliberate practice (Cooper et al. 2021). Faulty clinical reasoning has been attributed as the most frequent cause of diagnostic error (Graber et al. 2005). The rate of diagnostic error remains unacceptably high worldwide (World Health Organization 2016), and efforts to reduce this source of harm to patients are gaining momentum (National Academy of Medicine 2015). It is anticipated that explicitly teaching clinical reasoning will improve performance in the workplace, though evidence supporting this view has yet to emerge. There has been some resistance to this development from senior clinicians who are unfamiliar with the theory and language around

clinical reasoning. They feel they managed to develop good clinical reasoning without this formal instruction—and they are often right. In this way clinical reasoning is analogous to communication skills—although many doctors developed good communication skills without explicit teaching sessions, this was not universally true. The formal and explicit teaching of communication skills using the comskil model, for example (Brown and Bylund 2008), has improved the overall performance of younger doctors compared to their older colleagues (Bylund et al. 2009). Good clinical reasoning requires the development of a number of domains: clinical skills including communication and examination skills, use and interpretation of diagnostic tests, understanding cognitive and affective biases, understanding human factors, critical thinking, patient-centred evidence-based medicine and shared decision-making (Cooper et al. 2021). Teaching this complex set of inter-related skills is difficult in the best of circumstances and impossible without innovative solutions in the face of the COVID-19 pandemic. Virtual Wards were therefore designed with all components of clinical reasoning: information gathering, hypothesis generation, forming a problem representation, generating a differential diagnosis, selecting a leading or working diagnosis, providing a diagnostic justification and developing a management or treatment plan (Daniel et al. 2019). A number of theories have informed research on clinical reasoning—for example, dual-process (Pelaccia et al. 2011), script (Charlin et al. 2007), cognitive load (Durning et al. 2011) and situated cognition (McBee et al. 2018). These multiple theories provide the foundation for our work on the Virtual Wards.

5.2.1

Dual-Process Theory

Since the time of Plato and Aristotle, there has been a distinction made between intuitive and analytical thinking (Croskerry 2009a, b). In more recent decades, a variety of cognitive psychologists have adapted and developed dual-

5

Virtual Wards: A Rapid Adaptation to Clinical Attachments in MBChB During. . .

process theory in different fields. Daniel Kahneman was a long-standing leader in the field and popularised the theory in his bestselling book (Kahneman 2011). The theory defines two cognitive systems which interact to generate reason: the intuitive, tacit or experiential system (also coined system 1) and the analytical, deliberate or rational system (also coined system 2) (Stanovich and West 2000; Pelaccia et al. 2011). The intuitive system (system 1) is a vital component of clinicians’ day-to-day reasoning and comprises the majority of clinical thinking. It uses mental shortcuts (heuristics) and is rapid, intuitive and context-sensitive. The intuitive system (system 1) relies on pattern recognition (Elstein and Schwartz 2002) and is sometimes equated to ‘gut feelings’ (Stolper et al. 2009). Experts use the intuitive system (system 1) often and with great success. However it is prone to conscious and unconscious influences which can cause error. These include patient characteristics, illness characteristics, clinical workload, distractions, interruptions and resource issues and are an unavoidable aspect of the context within which clinicians make decisions (Stanovich and West 2000). The deliberate system (system 2) is used to make decisions made using purposely collected information, which is followed by a conscious series of cognitive learning activities (Pelaccia et al. 2011). The deliberate system (system 2) is analytical and systematic. It takes more time and effort than the intuitive system (system 1) but permits abstract reasoning and hypothetical thinking. Most experts in the field conclude it is less likely to be erroneous, although there is not universal agreement on this matter (Norman et al. 2017). With experience and subject mastery, deliberate system (system 2) processes may devolve to the intuitive system (system 1) processes, and clinicians toggle between the two systems (Stanovich and West 2000). This is the basis for the Universal Model for Diagnostic Reasoning developed by Croskerry (Croskerry 2009a, b). According to the Universal Model for Diagnostic Reasoning, both intuitive and deliberate systems are important in the development of

97

clinical reasoning, and our Virtual Wards aimed to develop both systems progressively and in parallel. Necessarily, students use more deliberate (system 2) thinking as they start to learn clinical medicine. Intuitive (system 1) thinking requires background knowledge and experience and hence develops with practice. In the Virtual Wards, the intuitive system (system 1) was engaged through repeated exposure to patient vignettes, images and patient history, whilst the deliberate system was reinforced by using similar sequential steps in all patient cases, as will be described later in this chapter.

5.2.2

Script Theory

Script theory explains how humans can relatively effortlessly understand and make sense of very complex situations, by using cognitive structures of memory referred to as schemas or scripts (Charlin et al. 2007). These scripts are constructed through repeated experience and retrieved when a familiar situation is encountered. They include the epidemiology (who gets this illness), the time course of the presentation (how it presents with respect to time) and the clinical features of the disease (key aspects found in history and examination). When a clinician encounters a patient, if this information matches a known script, that illness script is moved into the working memory, which will lead to a subsequent diagnostic hypothesis (Charlin et al. 2007). It is important to note that illness scripts arise from repeated exposure and therefore include information about typical presentation as well as normal variation for a particular illness or condition. If the presentation is typical and there are no or minimal facts which do not fit the script, diagnosis may be made quickly and easily. If there are discordant pieces of information, such as when more than two illness scripts might apply to a particular scenario or when there is a significant mismatch between a retrieved illness script and the scenario, for example, then deeper conscious analysis is prompted to avoid diagnostic error. Therefore, it is important the medical students establish illness scripts but also engage

98

C. Huser et al.

deeper analytical skills when mismatch occurs (Rikers et al. 2002). This toggling between illness scripts and deeper conscious thinking is important to avoid diagnostic errors. In clinical practice it is likely that both occur in parallel along a continuum and that toggling reflects the admixture at any point in time (Custers 2013). Virtual Wards aimed to expose students to typical presentations with the aim of developing basic illness scripts. In addition, each initial case presentation was also followed by deliberate sequential clinical reasoning steps to simulate and stimulate deeper cognitive analysis, eventually leading to correct diagnosis formation and management plan generation.

5.2.3

Cognitive Load Theory

Cognitive load theory is an established learning theory and addresses how humans learn, with a particular emphasis on the limitations of the working memory (Sweller 2011). Essentially, information is processed in the working memory, which has a limited capacity in terms of the amount of information held and the length of time the information can be held there for. Information can be encoded and passed on from the working memory to the long-term memory, which can hold a potential limitless amount of information for a lifetime (Young et al. 2014). This process is what we refer to as learning. According to cognitive load theory, there are three types of information or cognitive load which are associated with any learning task (Sweller 2011). Firstly, intrinsic cognitive load is the information directly linked to the learning task or, in other words, the information to be learnt (Sweller 2011). Germane cognitive load refers to the essential effort required by the working memory to sort and organise the information presented to allow it to be integrated into pre-existing schemas in the long-term memory (Sweller 2011). Both intrinsic and germane loads are therefore essential for learning. Extrinsic cognitive load on the other hand is additional information presented to the learner in the environment or context of the learning event (Sweller

2011). This can be background noise, slides with non-essential images or information presented in a complex manner (Young et al. 2014). This extrinsic cognitive load impairs learning and should be reduced as much as possible for efficient learning. In addition, the cognitive load learning theory predicts that splitting complex information into smaller chunks of information will allow learners to process new information within the capacity of their individual working memory, allowing better retention and integration of the new information into cognitive schemas (Durning et al. 2011). Mancinetti et al.’s (2019) review has many tips for using cognitive load theory to improve clinical teaching. To inform clinical education design based on current cognitive load theory principles, Leppink and van den Heuvel (2015) have proposed a three-dimensional model with steps or levels towards proficiency in tasks. At level I, high support is given to students performing low-fidelity and low-complexity tasks. In level II, tasks remain low fidelity but increase in complexity, and in level III, the fidelity is increased incrementally. In our Virtual Wards, for the complex skill of clinical reasoning in particular, we followed the first dimension outlined by Leppink and van den Heuvel (2015). Namely, high support was provided to students on low-fidelity and low-complexity tasks, with the aim of moving to dimensions II and III in later clinical years (Leppink and van den Heuvel 2015). To comply with dimension I, the steps in the clinical reasoning process were presented individually and sequentially to reduce intrinsic cognitive load. The order of the steps remained identical in all cases, and we used the pre-existing virtual learning environment as detailed below to reduce extrinsic cognitive load.

5.2.4

Situated Cognition

Situated cognition theory aims to explain the situation-dependent essence of clinical reasoning, taking into account the many interactions which take place during a clinical encounter (Durning

5

Virtual Wards: A Rapid Adaptation to Clinical Attachments in MBChB During. . .

et al. 2011). Situated cognition is a student- or clinician-centred theory which proposes that clinical reasoning is not a linear process, but rather a multi-dimensional outcome, and places importance on the context of the encounter. Contextual factors affecting clinical reasoning can include clinician factors (e.g. knowledge, sleep deprivation, burnout), patient factors (diagnostic suggestion, anxiety) and encounter factors (time is short, noisy environment). Clinical reasoning emerges and evolves from dynamic interactions between social actors (patients, relatives, other clinicians) and artefacts (electronic health records, written notes/charts). Importantly, McBee et al. (2018) found that the performance of medical students was impacted by a wider variety of contextual factors than expert clinicians, an effect attributed to less robust illness scripts. The Virtual Wards were therefore aimed at developing students’ deliberate practice and illness scripts in different contexts, which may have reduced the impact of those contextual factors on clinical reasoning performance when students returned to face-to-face clinical attachments.

environment had the advantage of providing access and completion data, allowing us to monitor student engagement. Again, this was critical during the start of the COVID-19 pandemic but would be an advantage to any educator creating Virtual Wards. Having decided to use Moodle, the functionality most fitting to the development of Virtual Wards and to the support of clinical reasoning was the Moodle lesson, in combination with embedded H5G interactive content. H5G is an embedded Moodle functionality which allows complex interactive activities to be developed. The favourable characteristics of Moodle lesson were: – The flexibility in content which could be included – The inclusion of automatically marked question pages – The control over non-linear lesson flow – The ability to include a large amount and variety of content in a single click

5.3.1

5.3

Technological Considerations

In the context of the COVID-19 pandemic, a pragmatic approach to designing the Virtual Wards was taken. However, our choice of using the existing virtual learning environment (Moodle) has advantages in both emergency and planned development. In our emergency context, it avoided time delays to sort out new contracts, and training time for faculty and students was reduced. In addition, at a time when we were all adapting to new ways of living, learning and working, using a platform which students and faculty were familiar with was important to avoid causing additional burdens. In planned development of Virtual Wards, hosting the ward on the students’ usual virtual learning environment would also be of benefit to avoid extrinsic load (Orru and Longo 2019) as mentioned above. In addition, using our existing virtual learning

99

Flexibility of Content

Being able to include a variety of content modalities was important to maintain student engagement, to provide content suitable to various learning preferences (Liew et al. 2015; Rose and Meyer 2002) and to suit the content being delivered in each case study. To maintain student engagement, it was important to be able to include text, as well as embedded images, videos, interactive slides, web links and quizzes. Text is formatted with headings allowing the use of screen readers, and videos can be embedded with transcripts. In Moodle lessons, the information can easily be presented to students in a variety of ways without the need to separately download PDFs or other documents. The Virtual Wards therefore presented information in as similar a way as possible to how it would be presented to students on a real ward, maintaining the benefits of contextual learning as much as possible (Schrewe et al. 2018).

100

5.3.2

C. Huser et al.

Inclusion of Automatically Marked Questions

Being able to include question pages directly in Moodle lesson was an asset both in terms of interactivity and monitoring student progress. This ability was one of the main drivers for selecting this functionality rather than the ‘book’ functionality, for example, which only allows information presentation. Automatically marked question pages were added for interactivity to maintain student engagement and to provide feedback on performance. It also allows students to check their knowledge and understanding as they progress through a case. Individual feedback for each answer option allowed students to get an immediate explanation for their wrong answers and forced students to think carefully about each step in the clinical reasoning process. Furthermore, being able to monitor student performance rather than simply engagement with the Virtual Wards allowed the faculty team to identify and support engaged but struggling students, as well as those not engaging. Finally, Moodle lessons were set up so that they could be undertaken repeatedly by students, and incorporating automatically marked questions allowed students to check for improvement in their performance over time.

5.3.3

Control over Non-linear Lesson Flow

The ability to have a non-linear lesson flow is a great asset in making Virtual Wards as realistic as possible and maintaining benefits of contextual learning. To develop their clinical reasoning, students must learn to prioritise and order their processes or actions. Having a non-linear lesson flow allowed us to give limited clues as to what the next step in the case should be and instead allow students to select from several possibilities. Another benefit of non-linear lesson flow is the ability to provide students with individualised Virtual Ward experiences, which will vary based on their own choices, as well as on their answers

to given questions. This can lead to a more efficient use of the students’ time, as they are directed to revision targeted at material they need to focus on, whilst other students can be directed to further reading if they have clearly understood a particular topic, thereby creating a personalised adaptive learning environment (Peng et al. 2019).

5.3.4

Large Amount of Information in a Single Click

One of the disadvantages of Moodle and other virtual learning environments is that they can become a repository for a long list of files or links, leading to an environment which is difficult to navigate. In e-learning design, it is a good practice to avoid creating such long lists of links, as it can become overwhelming for students and difficult to find information. The use of the Moodle lesson allowed us to keep the studentfacing page clean, with only a single link per case, and all the embedded material accessible through that link. At the time of the COVID-19 emergency pivot to online learning, it was particularly important to avoid overwhelming students and to keep learning easy to access. However, this would be good online distance learning practice for any educator developing Virtual Wards.

5.3.5

Embedding H5G Interactive Content

Despite the benefits of Moodle lesson mentioned above, it felt necessary to embed additional interactive content from the H5G functionality. These were created in H5G and embedded directly in the Moodle lessons, but not displayed on the course page, thereby maintaining our uncluttered student view. H5G allowed us to create interactive content which aligned more closely with the actual ward environment than Moodle lesson, such as creating a virtual urine dipstick test. It also allowed us to include images of patients, giving students the opportunity to virtually examine the body system of their choice.

5

Virtual Wards: A Rapid Adaptation to Clinical Attachments in MBChB During. . .

5.3.6

Tips for Virtual Ward Developers

As the creation of Virtual Wards is timeconsuming, using a system with which faculty and students are familiar is recommended. Choose a functionality which allows you to monitor student progress, allows you to include a large variety of content modalities and if possible allows for non-linear flow. The technology chosen should be focussed on helping students develop clinical reasoning and higher-order thinking skills, rather than leading students through a case and asking for fact recall.

5.4 5.4.1

Description of the Virtual Wards The Content Covered by the Virtual Wards

The content of both junior and senior Virtual Wards was designed around the ‘top presentations’ in medicine and surgery. For the year 3 students, this consisted of ten presenting complaints in each Virtual Ward, as seen in Fig. 5.1. The year 4 Virtual Wards followed a similar structure and covered a further 20 presenting complaints. Each presenting complaint formed a module, which was the focus of each week’s teaching.

important conditions. Each module would typically involve a range of learning materials—this could be video tutorials, audio tutorials, PowerPoint presentations, quizzes and the mainstay of the clinical reasoning aspect of the course, interactive cases created with Moodle lesson. Typically each module would involve an introductory tutorial covering how to take a focused history and examine a patient with the key presenting complaint for the module and one to three interactive cases. Providing several cases of the same presentation allowed students to integrate normal variation within their developing illness scripts (Charlin et al. 2007). In addition, one or two further tutorials covering key points raised in the interactive case(s) or providing further material on an additional topic relevant to the presenting complaint were included. These stimulated further conscious analysis in line with the deliberate system of dual-process theory (Pelaccia et al. 2011). Finally, a set of singlebased answers to multiple-choice questions to allow students to test their learning and staff to monitor progress and understanding were also included, as seen in Fig. 5.2. Online asynchronous tutorials (video, audio or PowerPoint) and quizzes are likely to be familiar teaching strategies to many readers, and the novel aspect of the Virtual Wards was the development of the online interactive cases, which are detailed below.

5.4.3 5.4.2

The Format of the Modules

Week 1 of the Junior Ward provided students with a grounding in the interpretation of common investigations they would be required to review in subsequent weeks. For this stand-alone week, material was presented as video tutorials to allow students to become familiar with interpretation of ECGs, basic radiology and blood investigations. In subsequent weeks students were expected to work through material based around one top presentation and covering several common or

101

The Interactive Cases

The aim of the interactive cases is to allow situational learning as far as possible and attempt to replicate the experience for the student of clerking in a patient (McBee et al. 2018). The interactive cases therefore presented students with information about the patient (history, examination, investigations) in the sequence that they would typically perform these tasks in real life. Each case contains a number of interactive elements (typically 5–10 per case) where the student is asked to apply their knowledge to decide how to manage their patient.

102

C. Huser et al.

Fig. 5.1 Example topics covered by the Junior Medicine Virtual Ward

Fig. 5.2 An example of the Moodle menu for the ‘Nausea and Vomiting’ module

Examples taken from the Senior Medicine Virtual Ward are shown below to help readers visualise the concept.

5.4.3.1 Setting the Scene Each case typically starts with a brief introduction to the scenario. The student is told what role they

5

Virtual Wards: A Rapid Adaptation to Clinical Attachments in MBChB During. . .

103

Fig. 5.3 Setting the scene: an example from the Abdominal Pain module

are playing (usually an FY1 doctor) and given a very brief description of the background to the case. To maintain contextual learning (Schrewe et al. 2018), this was presented in as close a way to a real ward, such as by using a referral letter, for example, as seen in Fig. 5.3. Following a constructivist learning model (Torre et al. 2006), we then typically aim to activate the student’s prior learning with an activity. The examples in Fig. 5.4 use ‘drag-and-drop’ activities in slightly different ways to encourage the student to think about initial differential diagnoses. Once the interactive activity is completed, the student receives an immediate score and can move onto the next page where the answer and explanation is presented. As this is a formative exercise rather than summative, students are able to re-attempt the activity as many times as they wish until they are satisfied with their level of knowledge and understanding. By providing the answers on the next page, we provide students with instant feedback, which has a substantial impact on learning (Nutbrown et al. 2016). All pages provide a ‘Back’ button which

allows the student to return to the previous activity for a further attempt if they wish, allowing students to progress at their own speed and revise specific areas as often as they like. This deviates from the normal flow of events in a ward but provides learners with an opportunity for an individual learning experience leading to more complete understanding (Peng et al. 2019). Once the student has completed the initial activity, they move on to the next part of the clerk which conventionally is history-taking. In the early modules, interactive elements were used to reinforce the accepted order of activities in a clerk in, as shown in Fig. 5.5.

5.4.3.2 Interactive History-Taking Interactive elements are often used to allow the students to consider what specific questions relating to the presenting complaint should be asked, as shown in Fig. 5.6. This helps develop clinical reasoning as well as their knowledge of the presenting complaint under study. Aside from interactive elements, other formats can be used to provide or add to the history, including audio recordings, visual representations

104 Fig. 5.4 Activities based on initial differential diagnoses can be used to activate prior knowledge. (a) Initial thoughts about more or less likely differential diagnoses, taken from the Abdominal Pain module. (b) Classifying differentials as an alternative strategy to activate prior learning, taken from the Diarrhoea module

Fig. 5.5 How to assess a patient: arranging the tasks in the correct order, taken from the Abdominal Pain module

C. Huser et al.

5

Virtual Wards: A Rapid Adaptation to Clinical Attachments in MBChB During. . .

105

Fig. 5.6 An example of a matching activity to explore the history of a presenting complaint, taken from the Weight Loss module

of conversations, plain text notes on patient history or documents found in the clinical environment, which add a realistic aspect to the case, right down to interpreting handwriting.

In some cases the correct identification of specific signs or clinical features is further tested by the inclusion of questions (multiple choice or free text) or interactive elements, as seen in Fig. 5.9.

5.4.3.3 Observations and Examination After completing the history-taking component of the case, the student will move on to tasks centred around observations and examination findings. To encourage active learning (Graffam 2007), many of the observation tasks involve interactive elements, giving students the opportunity to interpret data and test their understanding of the information presented, as shown in Fig. 5.7. Virtual patient examination in the modules frequently uses the ‘hotspot’ H5G Moodle functionality to allow students to explore the clinical findings. Any background image (diagram or photo) can represent the patient, and a variety of clinical signs can be added, as shown in Fig. 5.8. In addition, embedded videos can also allow students to observe recorded examinations to either consolidate previous knowledge or develop new skills and are used as a surrogate for actual clinical examinations.

5.4.3.4

Investigations: Selection and Interpretation A key aspect of developing clinical reasoning through the interactive cases is to develop the students’ ability to identify which tests should be performed or prioritised and then to interpret the investigation results. Interactive elements were therefore designed to allow students to select their priority investigations, as shown in Fig. 5.10. Students’ interpretation of investigations are developed using a variety of formats, depending on the investigation in question. The range of possible modalities allowed us to mimic real ward artefacts as closely as possible, as seen in Fig. 5.11: It is also important to note that although questions requiring free text answers can be used, our experience suggests that students quickly become aware that these can be bypassed by

106

Fig. 5.7 Interpreting data is made more realistic by the use of clinical documents. (a) Interpretation of a completed NEWS chart, taken from the Confusion

C. Huser et al.

module. (b) Inclusion of multiple-choice questions allows students to test their understanding of the information presented, example taken from the Confusion module

5

Virtual Wards: A Rapid Adaptation to Clinical Attachments in MBChB During. . .

Fig. 5.8 A range of backgrounds can be used for ‘click on the hotspot’ activities to represent examination of the patient. (a) A diagrammatic background is used in this

107

example from a Nausea and Vomiting case. (b) Hotspots can consist of text or image information as seen in this example from a Nausea and Vomiting case

Fig. 5.9 Drag-and-drop elements can incorporate visual aspects too, as shown in this example from the Headache module

108

C. Huser et al.

Fig. 5.10 Can the student identify which investigations to carry out? A drag-and-drop activity to test knowledge of the correct investigations taken from the Jaundice module

writing any text such as X in the box, allowing them to move directly onto the answer page. As a result we limited our use of free text short-answer questions and preferred to use modified drag-anddrop functionality to check student understanding of the material presented. For example, instead of asking students to describe their interpretation of an ECG as a free text short answer, they had to select the correct description(s) using drag-and-drop functionality from a selection of 16 possibilities.

5.4.3.5 Refining the Differential Having obtained the history, examined the patient and been presented with relevant investigation findings, the next step in developing clinical reasoning is to ask students to refine their differential diagnosis further to arrive at a likely diagnosis. In the Virtual Wards, this has typically been done either using a multiple-choice question (with the remaining differentials as the possible answers) or a drag and drop with a single correct element. Again, immediate feedback, and an ability to try the question as many times as necessary,

simulates realistic supervisor feedback in a ward environment.

5.4.3.6 Management Having reached a diagnosis, students continue to develop their clinical reasoning skills by identifying the correct management options for the virtual patients. Again, this was achieved mostly through drag-and-drop exercises, giving students a wide range of likely management strategies and asking them to select the correct one(s), as shown in Fig. 5.12. To develop their clinical reasoning, students must learn to think about the appropriate management, as well as consider broader management issues such as medication side effects, complications or members of the multidisciplinary team which need to be involved. Our Virtual Wards tested these aspects in an interactive fashion, again using drag-and-drop functionality. As well as identifying the correct options, the Virtual Wards used interactive functionality to allow students to carry out online equivalents of paper-based tasks that would be expected in the clinical setting, such as filling in drug or intra-

5

Virtual Wards: A Rapid Adaptation to Clinical Attachments in MBChB During. . .

109

Fig. 5.11 A range of interactive elements and questions were used to test students’ understanding and interpretation of investigation results. (a) The ‘find the hotspot’ feature allows students to plot graphs based on results they are provided with—this H5P can identify if the student has placed their mark within the designated ‘correct zone’. Having ‘drawn’ the graph, correct interpretation can then be assessed with further questions. (b) ‘Find the hotspot’ can also be modified to test students’ ability to

identify abnormalities on radiographic images, example from the Falls module. (c) A ‘highlight the keyword or number’ activity ensures students take time to review test results and identify abnormal findings; example is taken from the Weight Loss module. (d) Alternatively drag-anddrop activities can be used to allow the student to assess their interpretation of the results presented, example from the Confusion module

venous fluid prescriptions. The flexibility of the H5P interactive software allowed us to upload the forms used in the local context, thereby simulating situational learning. Finally, other core clinical skills such as performing a structured handover were developed in an interactive manner. By using a variety of

interactive functionalities available in Moodle, it is possible for students to develop their clinical reasoning and data interpretation skills, as well as become more familiar with skills and behaviours essential for junior doctors. The Virtual Wards therefore prepare students for junior clinical attachments and complement later clinical

110

C. Huser et al.

Fig. 5.12 Interactive elements to drive learning about management of the condition of interest. (a) Selecting appropriate management strategies from a number of options, example taken from the Nausea and Vomiting module. (b) Classifying management strategies for management according to severity of presentation, example from the Falls module

attachments. Furthermore, the Virtual Wards help standardise the teaching provision across various clinical sites, as all students receive the same basic teaching on the top presentations.

5.5

Evaluation and Future

Whilst the Virtual Wards seemed popular with the students, and developed in an iterative process

5

Virtual Wards: A Rapid Adaptation to Clinical Attachments in MBChB During. . .

111

incorporating student feedback, we performed an evaluation of these learning modules to tease out the important lessons from this experience.

weeks. In a more planned approach, we could have spent more time on achieving familiarity with software ahead of content creation.

5.5.1

5.5.3

Asynchronous Engagement with Virtual Wards

It quickly became apparent that without compulsory deadlines, students were engaging with the Virtual Wards at varied rates. Some students were keeping up with each week as they were made available, whilst others were working 1, 2 or more weeks behind, in an asynchronous manner. This was due to a number of reasons which can be grouped into intrinsic and extrinsic factors. Intrinsic factors were student motivation and screen fatigue. Extrinsic factors included other work and family duties, geographical location and Internet connection. The result of this desynchrony of students and educators was a difficulty in supporting students as they progressed through earlier weeks, and faculty had to monitor all forums daily, for weeks after the module in question had passed. This issue has been reported in other asynchronous online learning formats such as Massive Open Online Courses (MOOCs), post-graduate taught masters’ programmes and microcredential courses (Huser et al. 2021).

5.5.2

Issues Working with Multiple New Technologies

Technology was both a powerful tool which made interactive Virtual Wards possible and a source of difficulty during the process. The faculty involved had to rapidly learn to use a variety of software, which led to a number of technological hiccups during the early stages of the Virtual Ward creation. These included sound quality issues, pictures appearing too large or too small, clinical case progression logic issues and difficulty in anonymising forum posts. These were frustrating to those involved in the content creation and also to the students for whom it posed a barrier to learning during the first couple of

Clinician Time Involved to Create Content

Another issue which we encountered was that of requiring active clinicians to create the content. Whilst this is typical for medical education, at the onset of the COVID-19 pandemic, there was a great deal of upheaval for the clinicians involved, and many of them did not have time to prioritise teaching. Whilst the students were aware and understanding of the pressures that clinicians were experiencing at the time, angst and frustration were expressed when content was delayed. With a more planned approach to Virtual Ward creation, this would not be an anticipated issue. In fact, preparing recordings for asynchronous delivery affords clinicians more flexibility to fit teaching around their clinical commitments.

5.5.4

Simultaneous Virtual Wards

In normal circumstances, the surgery and medicine placements take place one after the other. However, due to the circumstances and to allow clinicians more flexibility in contributing content for the wards, the decision was made to run each block concurrently for 10 weeks, rather than 5 weeks each. With hindsight, this was the correct decision in the context of a pandemic, but it resulted in certain frustrations for the students. With the increased planning time available for the following academic year, this has now changed back to two consecutive 5-week blocks.

5.5.5

Quality Control of Benevolent Contributor Content

As mentioned above, we relied on clinician time to develop ward modules, clinicians who were facing a pandemic. Fortunately for the Surgery

112

C. Huser et al.

Virtual Ward, many of the elective operating lists had been cancelled, and as a trainee body, the surgeons are generally very ‘CV-motivated’. This gave a fairly large pool of voluntary contributors to create materials. Unfortunately, however, there was a degree of difficulty in ensuring quality control of the materials which were submitted. A large regret is that many of the contributors failed to provide questions to complement their presentations. With a more planned approach to Virtual Ward creation, more training and feedback can be given to contributors.

5.5.6

A Reflection on the Faculty Experience

This was an exciting and challenging time to be involved with teaching and one which posed many unexpected difficulties. The steep learning curve and desire to provide a quality experience for the students, coupled with the upheaval of the pandemic, resulted in a range of difficulties, eventually giving way to some degree of satisfaction with the final Virtual Wards. The initial stages of learning new skills, whilst trying to coordinate a body of contributors and also interact with the students, were deeply motivating but also filled with concern over not being able to deliver within the tight time lines. These feelings morphed over the course of the 10-week block as feedback from the various involved parties came in. Student feedback was largely positive at the start as they appreciated the pace with which things had had to be set up; however, as the course progressed, the students became increasingly familiar with the materials and began to give more constructive feedback. The combination of receiving constructive and honest feedback in this situation was sometimes difficult to hear as there was little time to change course and a large amount of work was being done each day on the project. It should be said, however, that the faculty were particularly keen that the students receive quality teaching and thus the feedback was taken on-board and acted upon. Towards the end of the Virtual Ward development time, as feedback had resulted in changes,

and along with the satisfaction of ‘the end being in sight’, feelings of relief and satisfaction with a ‘job well done’ came to the fore. As student feedback came in, it caused a slight blow to confidence; however it did help to guide change. The situation we were in of producing the materials at pace meant that we were frequently finding ourselves wishing we could spend more time in development before publishing content for the students to interact with. As such, when the students gave feedback, it was often identifying problems which we would have loved to have time to address ahead of time. These themes will be explored more in the section on student feedback. Latterly in the process, as we had responded to feedback and as students could look back and see what they had learnt, we began to see an improvement in both the quality of the materials which we were producing and also in the engagement from the students. This gave a large boost to morale and really spurred us on in the final weeks. This is not to say that the content was perfect but more that we could, at that point, recognise our position on the incline of learning.

5.5.7

The Students’ Perspective

At the time of the inception of the Virtual Wards, students were in a position of upheaval, uncertain of what was to become of their studies and unsure of their future as a whole. There were concerns even amongst the early year groups that they would be recruited into the hospital as staff and any studying would be pushed aside. Although it is clear to see with hindsight that these concerns would never have become a reality, one cannot express just how heavily such worries played on the minds of students at the time. The initial reaction of the student body to the announcement of the Virtual Wards was one of reliefs. There were messages of thanks and also of encouragement for those of us who were working in clinical areas as well as in teaching. There was enthusiasm to make the most of the new learning experience and to really engage with what was being put online. Sadly, however, the reality of

5

Virtual Wards: A Rapid Adaptation to Clinical Attachments in MBChB During. . .

online learning, at home and far from friends without the pre-COVID-19 luxury of in-person patient contact, began to take hold. The main themes of feedback which we received during the Virtual Wards were regarding technical issues or questions relating to specific topics. This was largely a trouble-shooting exercise for those of us involved in Virtual Ward creation and didn’t pose too many challenges other than keeping up with the now asynchronous nature of teaching where students could interact with any part of the course any time after publication. During the final week of the Virtual Ward, we solicited formal feedback from students, which fell into the following main categories: the Virtual Ward format, the specific content, some technical difficulties and problems due to loss of clinical contact.

5.5.7.1 The Virtual Ward Format Generally speaking there was a positive response to the new format of teaching online. The commonest theme to come from the feedback was that it allowed students to approach the materials at their own pace. This was due to the asynchronous online delivery, particularly important as many students had returned home and were based in different time zones. The students also appreciated that they could review materials at their leisure, something which many remarked was a problem with ‘normal’ teaching strategies. Students also identified that the Virtual Ward format standardised the ‘clinical’ experience compared to ward-based teaching, where there is a degree of chance involved with what pathologies are encountered during a particular attachment. Students felt that this new format allowed all students to get the same minimum depth of experience for the top presentations. A common criticism was that running Medicine and Surgery Virtual Wards concurrently led to a degree of confusion and overburdening with workload. However, a few students enjoyed the overlap, as it encouraged blending the two realms and consolidating knowledge of the whole rather than keeping the two discrete.

113

5.5.7.2 Feedback on Content The materials which the students appreciated the most were the ones which were most interactive, and this was frequently mentioned in the feedback. In particular, students appreciated the interactive tutorials which paused to ask questions every few minutes, and the interactive clinical cases which took a more in depth look at a certain pathology or clinical course. Students especially liked when complex concepts were broken down into bitesize chunks and presented in relatively simple forms, rather than as one lecture which explained an entire concept (Humphries and Clark 2021). As the Virtual Wards were replacing time in person on the wards, there was a great deal of enthusiasm, for instance, when things were put in the context of actual hospital work. For example, requesting bloods and having to make other management decisions whilst awaiting the results or having to think about the practicalities of sending a patient for an x-ray were particularly praised. Lastly, a significant number of students remarked specifically about appreciating when clinicians were honest and frank. One such example regarded a doctor who shared their experience of speaking to a family following the unexpected death of their relative. These frank discussions are normally an informal part of ward attachment experiences, and we would recommend consciously including them in Virtual Wards. 5.5.7.3 Amount of Content The main criticisms of the content related to a desire for more of the content which the students felt to be beneficial are more cases, more questions, more simulations, etc. Students would have liked to see several cases of the same condition to appreciate the variety of presentations, as well as more conditions. Occasionally there was a feeling that the materials were overladen with information and did not contain enough interaction, and the students felt that this led to fatigue and less efficient use of the learning opportunities. Although there was a formal, formative end of block assessment, the students also strongly felt that more formal formative assessments would have been useful and would

114

have increased motivation as well as providing valuable feedback on progress. In a more planned approach, more content, more interactivity and more formative assessment would be added.

5.5.7.4 Technical Difficulties Moodle, our virtual learning environment, received high praise from the students, despite some initial technical difficulties. For a few students, the quality of their Internet connection posed the largest difficulty, as much of the video content required a high bandwidth to stream. This was probably the most difficult technical issue to fix from a development point of view, as compressing the files too much would result in poor-quality presentations and impedes the learning experience. Other specific technical issues were faster to fix, such as some interactions not working properly when made available to students or unexpected quirks of the functionalities we used. For example, one type of matching question required students to get all answers correct before allowing them to move on. A common theme in this feedback was that many of the students liked to be able to manage their time and would have appreciated either a progress bar or an estimated duration of each segment; as we progressed in the course, we added progress bars; however estimated completion times proved harder to establish. Another issue was that there was no option to view many of the materials’ full screen, and this resulted in a lot of time spent looking at a window in a window, increasing screen fatigue. 5.5.7.5 Loss of Clinical Contact Finally, students felt that their learning was impacted by the loss of a physical presence on the wards. A significant proportion of the students felt that many of the finer points of clinical decision-making and the nuance from interaction with patients were lost by the course being delivered online. The Virtual Wards has always be seen as supporting ward-based experience rather than replacing it, and as students are able to go back to ward attachment experiences, it will be interesting to see the impact of the Virtual Ward on their attachment experiences.

C. Huser et al.

5.5.8

Lessons Learnt

The student feedback indicates a number of learning points from the Virtual Wards. Firstly, whilst it cannot be denied that students must have clinical contact during their undergraduate studies, the Virtual Wards have helped fill some gaps in the students’ education. Particularly, the Virtual Wards have given students an introduction to clinical understanding of topics for which they previously only had textbook knowledge. As one of our main goals was to encourage a shift from comprehension to application levels of Bloom’s taxonomy (Krathwohl 2002), reading that students themselves recognised this shift was welcome. It is clear that students have recognised the importance of interactive materials, especially those which push their understanding to application, and they have requested more of this type of material. Whilst timing issues prevented us from creating more in the first run of Virtual Wards, we will continue to develop this resource as a supplement to ward attachments. The use of our normal virtual learning environment (Moodle) proved successful. Staff and students were familiar with the platform, which reduced some of the extrinsic cognitive load (Orru and Longo 2019). It also ensures that all students can access the Virtual Wards without additional IT requirements. As we move towards a blended learning environment, we must ensure that the provision of online content does not exacerbate the inequalities already present in our society.

5.6

Tips for Setting Up Virtual Wards

– Use a pre-existing virtual learning environment if possible. This will reduce workload as well as extrinsic cognitive load and will provide student engagement and/or student performance data. – Split cases into smaller chunks which students can do at a pace which suits them. This allows

5

–

– –

–

–

– –

Virtual Wards: A Rapid Adaptation to Clinical Attachments in MBChB During. . .

students to work within their germane working memory capacity, provides flexibility and improves accessibility. To facilitate development of clinical reasoning, use sequential questioning to encourage formation of illness scripts and deliberate practice. Use interactive functionalities to improve student engagement. Present content in a variety of different ways to address learning preferences and maintain student engagement. Variety will also stimulate more varied illness script development. Use frequent formative self-assessment with instant feedback, such as multiple-choice questions to allow students and faculty to assess their understanding. Present students with multiple cases of a presentation to ensure illness scripts generated are authentic and not confined to a single text book presentation. Present realistic information to students, such as handover notes and real investigation reports. The material should remain as low-tech as possible. This will ensure easy access by all students (Fawns et al. 2020).

5.7

The Future of Virtual Wards

Whilst we all hope that the restrictions caused by the pandemic will not be repeated, it is evident that necessity has bred innovation, and the Virtual Wards have filled a need in student learning not restricted to the context of a pandemic. We believe that the Virtual Wards are a powerful support tool for students to start to develop clinical application of their knowledge and clinical reasoning with hard scaffolding (Braun et al. 2019). We will continue to provide Virtual Wards as a powerful adjunct to clinical learning, which will provide an element of standardisation of the attachment experience of students regardless of which patients are seen in wards, and to provide a safe platform for students to deliberately practise their clinical reasoning with

115

immediate feedback in the components they find most challenging. With further improvements in response to student feedback, the future of the Virtual Wards looks very bright.

References Braun L, Borrmann KF, Lottspeich C, Heinrich DA, Kiesewetter J, Fisher MR, Schmidmaier R (2019) Scaffolding clinical reasoning of medical students with virtual patients: effects on diagnostic accuracy, efficiency, and errors. Diagnosi 6(2):137–149. https:// doi.org/10.1515/dx-2018-0090 Brown RF, Bylund CL (2008) Communication skills training: describing a new conceptual model. Acad Med 83(1):37–4 4. https:// doi .org/ 10.1097/AC M. 0b013e31815c631e Bylund CL, Brown R, Gueguen JA, Diamond C, Bianculli J, Kissane DW (2009) The implementation and assessment of a comprehensive communication skills training curriculum for oncologists. PsychoOncology 19(6):583–593. https://doi.org/10.1002/ pon.1585 Charlin B, Boshuizen HP, Custers EJ, Feltovich PJ (2007) Scripts and clinical reasoning. Med Educ 41(12):1178–1184. https://doi.org/10.1111/j. 1365-2923.2007.02924.x Cook DA, Durning SJ, Sherbino J, Gruppen LD (2019) Management reasoning: implications for health professions educators and a research agenda. Acad Med 94(9):1310–1316. https://doi.org/10.1097/ACM. 0000000000002768 Cooper N, Bartlett M, Gay S, Hammond A, Lillicrap M, Matthan J, Singh M (2021) Consensus statement on the content of clinical reasoning curricula in undergraduate medical education. Med Teach 43(2):152–159. https:// doi.org/10.1080/0142159X.2020.1842343 Croskerry P (2009a) Clinical cognition and diagnostic error: applications of a dual process model of reasoning. Adv Health Sci Educ 14(Suppl 1):27–35 Croskerry P (2009b) A universal model of diagnostic reasoning. Acad Med 84(8):1022–1028 Custers EJ (2013) Medical education and cognitive continuum theory: an alternative perspective on medical problem solving and clinical reasoning. Acad Med 88(8):1074–1080. https://doi.org/10.1097/ACM. 0b013e31829a3b10 Daniel M, Rencic J, Durning SJ, Holmboe E, Santen SA, Lang V, Ratcliffe T, Gordon D, Heist B, Lubarsky S, Estrada CA, Ballard T, Artino AR Jr, Sergio Da Silva A, Cleary T, Stojan J, Gruppen LD (2019) Clinical reasoning assessment methods: a scoping review and practical guidance. Acad Med 94(6):902–912. https://doi.org/10.1097/ACM.0000000000002618 Durning S, Artino AR Jr, Pangaro L, van der Vleuten CPM, Schuwirth L (2011) Context and clinical reasoning: understanding the perspective of the

116 expert’s voice. Med Educ 45(9):927–938. https://doi. org/10.1111/j.1365-2923.2011.04053.x Elstein AS, Schwartz A (2002) Clinical problem solving and diagnostic decision making: selective review of the cognitive literature. BMJ 324:729–732 Fawns T, Jones D, Aitken G (2020) Challenging assumptions about “moving online” in response to COVID-19, and some practical advice. MedEdPublish 9(1):83. https://doi.org/10.15694/mep.2020.000083.1 Graber ML, Franklin N, Gordon R (2005) Diagnostic error in internal medicine. Arch Intern Med 165(13):1493–1499 Graffam B (2007) Active learning in medical education: strategies for beginning implementation. Med Teach 29(1):38–42. https://doi.org/10.1080/ 01421590601176398 Humphries B, Clark D (2021) An examination of student preference for traditional didactic or chunking teaching strategies in an online learning environment. Research in learning. Technology 29. https://doi.org/10.25304/ rlt.v29.2405 Huser C, Campbell S, Fontaine S, Jamieson S, Marks L, Singer J, Young R (2021) Lessons learned from early adopters of blended and online learning. J Perspect Appl Academic Prac 9(2):137–145. https://doi.org/10. 14297/jpaap.v9i2.488 Kahneman D (2011) Thinking, fast and slow, 1st edn. Farrar, Straus and Giroux, New York Krathwohl DR (2002) A revision of Bloom's taxonomy: an overview. Theory Pract 41(4):212–218. https://doi.org/ 10.1207/s15430421tip4104_2 Leppink J, van den Heuvel A (2015) The evolution of cognitive load theory and its application to medical education. Perspectives Med Edu 4:119–127 Liew SC, Sidhu J, Barua A (2015) The relationship between learning preferences (styles and approaches) and learning outcomes among pre-clinical undergraduate medical students. BMC Med Educ 15:44. https:// doi.org/10.1186/s12909-015-0327-0 Mancinetti M, Guttormsen S, Berendonk C (2019) Cognitive load in internal medicine: what every clinical teacher should know about cognitive load theory. Eur J Intern Med 60:4–8. https://doi.org/10.1016/j.ejim. 2018.08.013 McBee E, Ratcliffe T, Schuwirth L, O’Neill D, Meyer H, Madden SJ, Durning S (2018) Context and clinical reasoning. Perspectives Med Edu 7:256–263 National Academy of Medicine (2015) Improving diagnosis in health care. The National Academies Press, Washington (DC). https://doi.org/10.17226/21794 Norman GR, Monteiro SD, Sherbino J, Ilgen JS, Schmidt HG, Mamede S (2017) The causes of errors in clinical reasoning: cognitive biases, knowledge deficits, and dual process thinking. Acad Med 92:23–30 Nutbrown S, Higgins C, Beesley S (2016) Measuring the impact of high quality instant feedback on learning.

C. Huser et al. Practitioner Research in Higher Education 10(1) http:// ojs.cumbria.ac.uk/index.php/prhe/article/view/318 Orru G, Longo L (2019) The evolution of cognitive load theory and the measurement of its intrinsic, extraneous and germane loads: a review. In: Longo L, Leva M (eds) Human mental workload: models and applications. H-WORKLOAD 2018. Communications in Computer and Information Science, vol 1012. Springer, Cham. https://doi.org/10.1007/978-3-03014273-5_3 Pelaccia T, Tardif J, Triby E, Charlin B (2011) An analysis of clinical reasoning through a recent and comprehensive approach: the dual-process theory. Med Educ Online 16(1):5890. https://doi.org/10.3402/meo. v16i0.5890 Peng H, Ma S, Spector JM (2019) Personalized adaptive learning: an emerging pedagogical approach enabled by a smart learning environment. Smart Learn Environ 6(9). https://doi.org/10.1186/s40561-019-0089-y Rikers RMJP, Schmidt HG, Boshuizen HPA, Linssen GCM, Wesseling G, Paas FGWC (2002) The robustness of medical expertise: clinical case processing by medical experts and subexperts. Am J Psychol 115(4):609–629 Rose DH, Meyer A (2002) Teaching every student in the digital age: universal Design for Learning. Association for Supervision and Curriculum development. Alexandria, VA Schrewe B, Ellaway RH, Watling C, Bates J (2018) The contextual curriculum: learning in the matrix, learning from the matrix. Acad Med 93(11):1645–1651. https:// doi.org/10.1097/ACM.0000000000002345 Stanovich KE, West RF (2000) Individual difference in reasoning: implications for the rationality debate? Behav Brain Sci 23(5):645–726. https://doi.org/10. 1017/s0140525x00003435 Stolper E, Van Royen P, Van de Wiel M, Van Bokhoven M, Houben P, Van der Weijden T, Dinant GJ (2009) Consensus on gut feelings in general practice. BMC Fam Pract 10:66 Sweller J (2011) Chapter two–cognitive load theory. Psychol Learn Motiv 55:37–76. https://doi.org/10. 1016/B978-0-12-387691-1.00002-8 Torre DM, Daley BJ, Sebastian JL, Elnicki DM (2006) Overview of current learning theories for medical educators. Am J Med 119(10):903–907. https://doi. org/10.1016/j.amjmed.2006.06.037 World Health Organization (2016) Diagnostic errors. Available from: http://www.who.int/iris/handle/ 10665/252410 Young JQ, Van Merrienboer J, Durning S, Ten Cate O (2014) Cognitive load theory: implications for medical education: AMEE guide no. 86. Med Teach 36(5):371–384

6

Artificial Intelligence: Innovation to Assist in the Identification of Sono-anatomy for Ultrasound-Guided Regional Anaesthesia James Lloyd, Robert Morse, Alasdair Taylor, David Phillips, Helen Higham, David Burckett-St. Laurent, and James Bowness Abstract

Ultrasound-guided regional anaesthesia (UGRA) involves the targeted deposition of local anaesthesia to inhibit the function of peripheral nerves. Ultrasound allows the visualisation of nerves and the surrounding structures, to guide needle insertion to a perineural or fascial plane end point for injection. However, it is challenging to develop the

J. Lloyd · D. Phillips Department of Anaesthesia, Royal Gwent Hospital, Aneurin Bevan University Health Board, Newport, UK R. Morse Machine Learning Software Engineer, Intelligent Ultrasound Limited, Cardiff, UK A. Taylor NHS Tayside, Dundee, UK H. Higham Nuffield Division of Anaesthesia, Oxford University Hospitals NHS Foundation Trust, Oxford, UK OxSTaR Centre, Nuffield Department of Clinical Neurosciences, University of Oxford, John Radcliffe Hospital, Oxford, UK D. Burckett-St. Laurent Department of Anaesthesia, Royal Cornwall Hospitals NHS Trust, Truro, UK J. Bowness (*) Department of Anaesthesia, Royal Gwent Hospital, Aneurin Bevan University Health Board, Newport, UK OxSTaR Centre, Nuffield Department of Clinical Neurosciences, University of Oxford, John Radcliffe Hospital, Oxford, UK e-mail: [email protected]

necessary skills to acquire and interpret optimal ultrasound images. Sound anatomical knowledge is required and human image analysis is fallible, limited by heuristic behaviours and fatigue, while its subjectivity leads to varied interpretation even amongst experts. Therefore, to maximise the potential benefit of ultrasound guidance, innovation in sonoanatomical identification is required. Artificial intelligence (AI) is rapidly infiltrating many aspects of everyday life. Advances related to medicine have been slower, in part because of the regulatory approval process needing to thoroughly evaluate the risk-benefit ratio of new devices. One area of AI to show significant promise is computer vision (a branch of AI dealing with how computers interpret the visual world), which is particularly relevant to medical image interpretation. AI includes the subfields of machine learning and deep learning, techniques used to interpret or label images. Deep learning systems may hold potential to support ultrasound image interpretation in UGRA but must be trained and validated on data prior to clinical use. Review of the current UGRA literature compares the success and generalisability of deep learning and non-deep learning approaches to image segmentation and explains how computers are able to track structures such as nerves through image frames. We conclude this review with a case study from industry (ScanNav Anatomy

# The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 P. M. Rea (ed.), Biomedical Visualisation, Advances in Experimental Medicine and Biology 1356, https://doi.org/10.1007/978-3-030-87779-8_6

117

118

J. Lloyd et al.

Peripheral Nerve Block; Intelligent Ultrasound Limited). This includes a more detailed discussion of the AI approach involved in this system and reviews current evidence of the system performance. The authors discuss how this technology may be best used to assist anaesthetists and what effects this may have on the future of learning and practice of UGRA. Finally, we discuss possible avenues for AI within UGRA and the associated implications. Keywords

Anatomy · Artificial intelligence · Blocks · Computer vision · Convolutional neural network · Machine learning · Regional anaesthesia · Ultrasound · Sono-anatomy

practice. One such area of innovation is the application of artificial intelligence (AI), specifically computer vision, to support US image analysis. The first part of this chapter reviews the difficulties associated with human US image analysis and how they may affect our ability to perform consistently high-quality UGRA. An introduction to the field of AI follows and then a review of relevant published material on the concept of AI-assisted ultrasound image analysis in UGRA. The first commercial system to achieve regulatory approval, ScanNav Anatomy Peripheral Nerve Block, is briefly discussed. The chapter then concludes with a perspective on how these developments may impact clinical practice and what further developments may bring to UGRA in future.

6.2 Abbreviations AI AR CNN DL LA LDL ML RAUK RCoA UGRA US VR

6.1

Artificial intelligence Augmented reality Convolutional neural network Deep learning Local anaesthesia/anaesthetic Low-density lipoprotein Machine learning Regional Anaesthesia UK Royal College of Anaesthetists Ultrasound-guided regional anaesthesia Ultrasound Virtual reality

Introduction

Acquiring and interpreting ultrasound (US) images is a key skill in ultrasound-guided regional anaesthesia (UGRA). It is complex, comprises multiple elements and takes years to reach the expert level. Due to the increasing number and complexity of procedures, novel strategies are needed to support training and

6.2.1

Part 1: Challenges in Ultrasound Image Interpretation and Ultrasound-Guided Regional Anaesthesia What Is Ultrasound-Guided Regional Anaesthesia?

Ultrasound-guided regional anaesthesia necessitates the targeted deposition of local anaesthetic (LA) in close proximity to peripheral nerves or along a fascial plane in which nerves are known to course (Bowness and Taylor 2020). The resulting loss of neural function can provide a sole means of anaesthesia, analgesia as an adjunct to general anaesthesia or analgesia in isolation such as in acute or chronic pain. Techniques first relied on the use of anatomical surface landmarks, sometimes in combination with paraesthesia on injection, to guide needle insertion and injection. Electrical nerve stimulation was then incorporated to aid target/nerve identification by generation of a motor or sensory (paraesthesia) response. Anaesthetists now almost exclusively use US to guide needle placement. Ultrasound machines generate a 2D image of anatomical structures by analysing the reflection of high-frequency sound waves. Technological

6

Artificial Intelligence: Innovation to Assist in the Identification of. . .

advances, such as high-frequency probes, enhanced image processing, and echogenic needles have improved the performance and utility of US imaging in regional anaesthesia immensely. This progress in image quality has occurred in parallel with a growing body of evidence demonstrating improved outcomes associated with US guidance (Neal et al. 2016). However, one part of the image processing system remains conspicuously similar to that present at the conception of regional anaesthesia—the anaesthetist at the end of the needle.

6.2.2

Why Is Regional Anaesthesia Difficult?

To understand this, we need to break down the process of UGRA and explore the challenges presented by each step. Essential components of successful UGRA are: • Selection of the right block • Acquiring and interpreting an optimised ultrasound image • Planning a safe needle path and visualising the needle tip • Ensuring accurate deposition of LA at the target site • Post-procedure monitoring In the context of this series, Biomedical Visualisation, focus will be predominantly drawn to the difficulties associated with acquiring and interpreting an optimised ultrasound image. The other components will be briefly mentioned for completeness.

6.2.2.1 Selection of the Right Block Correspondence in UGRA literature provides conflicting accounts on matters as basic as the anatomy of the adductor canal and femoral triangle and the name of the corresponding block (Cowlishaw and Kotze 2015). There is a tendency for UGRA publications to focus on novel techniques, and novel applications of existing techniques, over research into optimising existing techniques (Turbitt et al. 2020). While new ideas show what is possible, they may not show what is

119

best and can give the impression that UGRA is overly complicated. Improving outcomes in, and increasing the use of, UGRA requires the adoption of standardised, evidence-based techniques (El-Boghdadly et al. 2021). Recognition of this problem by Regional Anaesthesia UK (RA-UK) has led to the concept of ‘Plan A blocks’. These are seven ‘high value’ blocks that all anaesthetists should become proficient in (Turbitt et al. 2020). The techniques chosen have broad utility in clinical practice, enabling anaesthetists to provide anaesthesia and analgesia for a wide range of situations. Work is in progress to standardise the ultrasound scanning practice for these blocks (Bowness et al. 2021— work currently unpublished).

6.2.2.2

Acquiring and Interpreting an Optimised Ultrasound Image The premise of this topic is also covered in a previous chapter of this series by Bowness and Taylor (2020), which provides further information. 6.2.2.2.1 Operator Dependence Ultrasound is highly operator dependent. Even experts may need additional help, especially in complex patients such as the obese or those with distorted anatomy (e.g. trauma or previous surgery). Image optimisation requires adjustment of the probe pressure, alignment, rotation and angle of insonation (as well as altering the gain and depth settings of the image itself). This must be performed continuously, as confident interpretation of the anatomy requires dynamic scanning rather than a single US image. Image interpretation is also subjective, meaning that different anaesthetists may reach different conclusions on viewing the same image. However, it is vital in planning the needle path (to prevent trauma to nerves, blood vessels and serous membranes) and identifying the site to deposit LA for optimum effect. 6.2.2.2.2 Anatomical Variation Sir William Osler, the famed nineteenth-century physician, once wrote: ‘Variability is the law of life, and as no two faces are the same, so no two

120

bodies are alike, and no two individuals react alike and behave alike under the abnormal conditions which we know as disease’ (Osler 1903). Christophe et al. (2009) demonstrated such structural dissimilitude in the nerves seen during an axillary-level brachial plexus block. The authors demonstrated 10 variations across 153 patients, with only 65% of patients exhibiting the arrangement most commonly shown in anatomical texts. There is a range of quality in anaesthetic educational resources pertaining to anatomy (Bowness and Taylor 2019), and clinicians performing UGRA are not always cognisant of such anatomical variation (Bowness et al. 2019a, b). 6.2.2.2.3

Learning Materials Depict Ideal Versions of Sono-anatomy Learning materials typically show sonographic images of young, slim, healthy volunteers. These individuals allow the clearest imaging of anatomical structures, allowing the author to demonstrate the salient anatomical points. The value of clarity is further enhanced when the image is subjected to the graphical constraints of a print publication or the digital constraints of limiting the file size. However, this demographic is not where UGRA can be of most clinical benefit and the practising clinician may be faced with challenging sonographic images which are markedly different. 6.2.2.2.4 Comorbidity Image interpretation can be more difficult in the presence of conditions such as obesity or oedema. Such patients often stand to benefit most from UGRA in order to avoid the associated cardiorespiratory risks of general anaesthesia but pose significant challenges for UGRA. Oedema increases the depth to the target, which increases attenuation of the US beam with subsequent image degradation, and can compress or displace other structures used for image recognition (Henderson and Dolan 2016). The depth is similarly increased in obesity, and the properties of adipose tissue worsen the image degradation as the irregularly shaped layers of tissue generate an uneven speed of sound (Fiegler et al. 1985) and a

J. Lloyd et al.

speckling of the image. Additionally, the tissue surrounding nerves in obese patients has higher echogenicity than non-obese patients, creating a reduction in image contrast between nerve and tissue (Marhofer et al. 2014). Imaging can be difficult after trauma or surgery, causing distortion of anatomy, or air in the tissues, which forms an impenetrable barrier for US and so obscures structures deep to it. The presence of wounds or dressings may also limit probe position. 6.2.2.2.5 Inattentional Blindness When attentional focus is centred on a particular task, humans often fail to recognise even obvious visual stimuli. This was notoriously demonstrated by Simons and Chabris (1999), where subjects were asked to count the number of ball passes in a basketball game. Most failed to notice the gorilla that walks through the middle of the court during the game. Drew et al. (2013), in a homage to this work, showed that expert radiologists failed to identify a picture of a gorilla hidden in a CT scan in over 80% of cases. Inattentional blindness has been shown to worsen with age (Graham and Burke 2011). This is relevant to anaesthetists as the anaesthetic workforce is getting proportionally older; 7% of consultants are now over 60, up from 5% 5 years ago (RCoA 2020). This trend is likely to continue with the increasing age of retirement for clinicians in the NHS. 6.2.2.2.6 Satisfaction of Search The identification of one abnormality within an image reduces the likelihood of identifying a second (Croskerry 2002), which may provide another explanation for anaesthetists failing to identify structures on ultrasound images. This phenomenon has been used to explain case reports in medical imaging where diagnoses have been missed, such as a retained central venous catheter placement wire (Lum et al. 2005). 6.2.2.2.7 Fatigability Physical fatigue is evident in UGRA when operators exhibit characteristic signs such as the need to switch hands or tremors (Sites et al. 2007). These symptoms will hamper image

6

Artificial Intelligence: Innovation to Assist in the Identification of. . .

production and optimisation. It is more pronounced amongst less experienced operators, explained by their tendency to be less efficient with their movements, leading to longer procedural time, and increased physical strain. Experienced operators may also exhibit fatigue, particularly during longer procedures, such as patients requiring multiple blocks, or cases where greater probe pressure or manipulation is required such as deep blocks or obese patients. The vigilance decrement is a term used to describe the loss of vigilance humans exhibiting over time during a prolonged task, first noticed in radar operators during the Second World War (Mackworth 1950). More recently a retrospective analysis of 2.9 million radiology reports by Hanna et al. (2018) showed that errors in medical image recognition are far more likely to occur later in a shift. Within the field of radiology, there is clear evidence that mental fatigue contributes to errors in image analysis (TaylorPhillips and Stinton 2019). Although the same studies have not been conducted for regional anaesthesia, it would be surprising if anaesthetists did not exhibit similar characteristics.

circumferential spread around the nerve (Brull et al. 2011) and to avoid intravascular injection. There are some proponents of intraneural injection (Cappelleri et al. 2016), though most advise against it over concerns of mechanical or chemical trauma (Short et al. 2016). With experience, US can detect even low volumes of intraneural injection by observing increase in the crosssectional area of the nerve (Moayeri et al. 2012). Inconsistency regarding what defines successful LA spread and what defines hazardous or unsuccessful deposition is a barrier to standardised practice.

6.2.2.5

Post-Procedure Monitoring Both to Ensure Effect and to Monitor for any Complications US guidance in regional anaesthesia has aimed to reduce the risk of RA complications such as LA toxicity, inadequate block and trauma (Neal et al. 2016). However, post-procedure monitoring remains crucially important for early recognition, treatment and limitation of their effects.

6.2.3 6.2.2.3

Planning a Safe Needle Path and Visualising the Needle Tip A safe needle path from skin entry point to the target nerve ensures proximate structures are not damaged by needle advancement. This requires understanding the anatomy surrounding the target, as well as the target itself. For example, for the interscalene block, the dorsal scapular nerve and the supraclavicular nerve both may lie along the potential needle path (Fig. 6.1). Continuous visualisation of the needle tip is an important safety consideration during UGRA, but operators show errors such as inserting the needle on the wrong side of the probe (Sites et al. 2007) and poor maintenance of needle tip visibility (Neal et al. 2016). 6.2.2.4

Ensuring Accurate Deposition of Local Anaesthetic Around the Target Structure Local anaesthetic injection is visualised under US to help improve block reliability through

121

Education in Ultrasound-Guided Regional Anaesthesia

In the UK unsupervised, independent practice of regional anaesthesia has previously been an optional competency within the Royal College of Anaesthetists’ curriculum (Turbitt et al. 2020). The 2021 curriculum makes independent UGRA a core competency, proposing all anaesthetists should be able to deliver ‘a range of safe and effective regional anaesthetic techniques to cover the upper and lower limb, chest and abdominal wall’ (RCoA 2021). This promotion of UGRA within the curriculum will likely lead to an increase in UGRA practice, driving the need for novel approaches to learning and performing UGRA. Clinical training to attain expertise in UGRA takes years, and the end points are not clear: suggestions on the number of cases needed to achieve competence range from 28 to 80 blocks (Barrington et al. 2012; Sites et al. 2007). Sites

122

Fig. 6.1 Ultrasound image of the interscalene-level brachial plexus nerve roots with dorsal scapular nerve highlighted. AS anterior scalene, C5 C5 nerve root, C6

et al. (2007) also recorded novice operators misjudging their needle tip position, causing intraneural puncture, and intramuscular injection, highlighting the risk to patients from inexperienced clinicians. Simulation in UGRA is currently of modest value, though there is evidence that educational tools such as phantoms and simulators improve competency (Abdallah et al. 2016). Cadaveric learning is a helpful but expensive, tightly regulated and limited resource. In addition, the preservation process can distort the appearance of soft tissue on ultrasound. Immersive technology such as augmented reality (AR) and virtual reality (VR) hold promise that is not yet realised. In AR, where elements of the virtual world are brought into the physical world, image overlays could improve image recognition, while simulated needle trajectory may permit practice of needle probe alignment with no risk to the patient. Virtual reality, which involves representation of the physical world in a virtual space, may allow the learning of UGRA with no patient involvement at all. Virtual reality is already used for learning laparoscopic surgical techniques, where studies show improved learning outcomes (Rangarajan et al. 2020). However, these techniques are not yet in widespread use.

J. Lloyd et al.

C6 nerve root, DSN dorsal scapular nerve, MS middle scalene muscle, PN phrenic nerve, ScN supraclavicular nerve

6.3

6.3.1

Part 2: An Introduction to Artificial Intelligence for Clinicians What Is Artificial Intelligence?

Artificial intelligence (AI) is any computer system that appears to demonstrate human-like intelligence. There are a multitude of computational techniques that can be used to create an intelligent system. The term AI is sometimes used interchangeably with other terms such as machine learning (ML) and deep learning (DL), which can be confusing. Artificial intelligence includes the field of ML, which in turn includes the field of DL (Fig. 6.2). Machine learning is a field associated with the implementation of algorithms that allow mathematical functions to ‘learn’ from data in order to perform a task. The data could be one or several of a multitude of formats (including but not limited to audio, text and image). There are a variety of tasks that can be completed, for example, the classification of bird type based on birdsong (Kahl et al. 2020), the detection and monitoring of hate speech found on Twitter (Pereira-Kohatsu et al. 2019) or the identification of anatomical structures in ultrasound images (Bowness et al. 2021a, b, c).

6

Artificial Intelligence: Innovation to Assist in the Identification of. . .

123

Fig. 6.2 Context model showing how key terms in the field of AI inter-relate

Deep learning is a subfield of ML where a specific set of functions are arranged in such a way that the structure mimics the architecture and interactions of biological neurones and the brain (in a much-simplified manner). These functions are arranged in specific structures, called architectures, that are best suited to the associated task. Convolutional neural networks (CNNs) are a specialised type of DL model that are well suited for image data. DL has increased in popularity in recent years due to advances in computational power. In addition, an increase in the volume of data in today’s digital world has increased the demand for algorithms that work effectively with such large volumes of data. DL models are particularly well suited to this.

6.3.2

Machine Learning Categories

Machine learning algorithms can be divided into categories by the format of the data used to train them. These categories are listed below: • Supervised • Semi-supervised • Unsupervised In supervised ML, the model is trained on labelled data, where the label is the attribute to be predicted by the model. For example, if a model was predicting whether an image contained a cat or dog, all the training data for a supervised model would need to learn from

images labelled as either cat or dog. The task of segmenting medical images using a model falls under the category of supervised learning. Semi-supervised ML is when some of the observations are labelled and some are not. Often, the model is initially trained on unlabelled data and then fine-tuned on labelled data. These techniques can be useful when the ratio of labelled to unlabelled data is low. Unsupervised machine learning is where no data is labelled and so the categories for prediction are decided by the model itself by using the intrinsic structure of the data.

6.3.3

The Computational Problem

In order to identify anatomical structures within ultrasound images, an algorithm/model must be able to classify every pixel in each image as belonging to a specific class, with each class representing an anatomical region. This task is called semantic image segmentation (the term segmentation shall be used from now on for the sake of brevity). In order to be useful during UGRA, the classification of each pixel must be achieved in real time in order to augment the video stream from the ultrasound machine. This means that the system must be fast enough to receive a video frame, process the frame and classify each pixel before displaying the output. This typically needs to be done in 30 frames per second in order to keep up

124

J. Lloyd et al.

with real-time video display. Videos are consecutive frames, with each frame consisting of pixel data represented by a value between 0 and 255 (0 is a completely black pixel and 255 is completely white). The data required for training UGRA models is therefore large. To appreciate the size of the data, let’s work through an example of a short cine loop. If we assume a cine loop is 6 seconds long at 30 frames per second, and each frame is 900 by 600 pixels, this means there are 6
30
900
600 ¼ 97,200,000 data points in 6 seconds of video! It is for this reason that the progressively increasing computational power, along with more readily available large datasets, has coincided with a rise in the use of DL for computer vision in medical images.

6.3.4

Rule-Based vs Model-Based Techniques

There is also large biological and technical variation. In the former, variations include different regions of the body, anatomical structures and inter-patient variation (e.g. body mass index (BMI)). Technical variation includes US machine settings (e.g. contrast, gain, depth), different probes and different ultrasound machines (with their different image processing strategies). Finally, noise (unwanted signal) further complicates this task. In order to train generalisable DL models, the models must be trained on data that represents this variation by including cine loops from patients that show reasonable inter-patient variation as well as different views of the same anatomical structure. Some of this variation does not need to be accommodated for as long as this is taken into account during model deployment. For example, if it is known the model is to be deployed on a specific ultrasound machine, then cine loops from that machine only can be included in the training data. Image segmentation can be achieved using methods that fall into one of two categories: rule-based and model-based techniques or a combination of the two.

6.3.4.1 Rule-Based Techniques When using a rule-based approach, a set of hardcoded rules are designed. These rules define whether a pixel, in a given context, belongs to a specific class. Examples include modelling arteries as ellipses in a brute-force manner (Smistad and Lindseth 2016; Smistad et al. 2017). In a ‘brute-force’ approach, all different possible options are considered by the model, and the option with the highest chance of success is chosen. This is the approach used by IBM’s Deep Blue, the first computer to win a chess match against a reigning world champion under standard conditions (famously beating Garry Kasparov in 1996). An advantage of such methods is the speed of execution. A disadvantage is the difficulty to generalise either to different views of the same structure or between individuals. For example, if arteries were searched for using a templating method (e.g. characterised by an ellipse), some arteries may be missed due to the fact that they are transected at different angles by the ultrasound beam, giving rise to differently shaped arteries. For example, an artery running parallel to the skin surface, with the probe held at 90 degrees to the skin, will appear circular in the short axis view— whereas when the probe is tilted, or if the artery does not run parallel to the skin, the artery will appear as an ellipse. This variation could reduce the effectiveness of a rule-based approach. Another disadvantage is the inability for one set of rules to work for different anatomical structures. 6.3.4.2 Model-Based Techniques In the modelling approach, a function or combination of functions is defined. These functions contain learnable parameters, the values of which are initially random or all zero. During model training, the parameters are modified in such a way as to minimise the loss function (a measure of how well the model is performing given a set of parameters). Through optimisation, the parameters are updated until the function reflects the relationship between the pixels and regions of interest. The model can then be shown

6

Artificial Intelligence: Innovation to Assist in the Identification of. . .

previously unseen ultrasound images and predict segmentation masks for the regions of interest. An advantage of this method is that the models have the potential to be generalisable and so accommodate for noise and biological or technical variation. A disadvantage is that the size of the models (number of parameters) can be potentially very large, and so it can be difficult to optimise models in order to infer at a sufficiently fast rate to keep up with real-time video streams. This constraint puts limits on the size of the model trained and the number of pre-processing steps, which consequently limits model performance.

6.3.5

Convolutional Neural Networks

Deep learning is a model-based technique. Deep learning differs from other ML techniques due to the fact the models are based on a neural network Fig. 6.3 Schematic showing high-level depiction of a fully connected neural network. The data flows from one layer to the next. Each neurone transforms the data, and this transformation uses model parameters. The first layer is the input data, and the last layer is the output. The layers in between are called hidden layers. The number of neurones in the last layer corresponds to the number of predictions. For example, in ultrasound image segmentation models, the last layer would contain at least the total number of pixels in one channel fed into the model in the original ultrasound image

Input Layer

125

structure. Each ‘neurone’ within the network contains multiple parameters that can be updated during training, and sets of neurones are arranged into layers (Fig. 6.3). There can be thousands of neurones in a given network and so millions of parameters. This gives neural networks the ability to model large volumes of data very well. The convolutional operation utilises a sliding window over an image or feature map array to provide inputs to the neurones in the next layer (Fig. 6.4). Each window of data from the preceding layer is inputted into a function (neurone) which calculates a linear combination of the variables with a set of learnable parameters followed by what is known as an activation function. The window shifts along the feature map by a defined number of units called the stride (in Fig. 6.4, stride ¼ 3 because the window shifts along by 3 values each time). Each neurone in a convolutional layer receives a vector of values

Hidden Layer

Hidden Layer

Output Layer

Key Neurone

flow of data

Layer

Fig. 6.4 Schematic showing how a layer in a convolutional neural network generates the next feature map. The image contains pixels with values ranging from 0 to 255. A sliding window selects grids of pixel values that are fed into a neurone in the next layer.

The neurone then transforms the data and outputs a single value that occupies the corresponding location in the outputted feature map. Each neurone is represented as the function f(x, wi) where wi are the learnable parameters for that neurone

126 J. Lloyd et al.

6

Artificial Intelligence: Innovation to Assist in the Identification of. . .

from specific locations in the feature map/image array, and this process captures spatial information well. ‘Max pooling’ layers also utilise a sliding window over an image or feature map array, but in this case, the array is just filtered for the maximum value. The maximum value from each window is passed to the next layer. These types of layers do not utilise learnable parameters and therefore are not optimised during model training. In a ‘fully connected’ neural network, all neurones of one layer are connected to all neurones of the next layer. Different convolutional neural networks (CNNs) generally utilise the layers described above as well as other layers in different combinations and architectures. In the field of computer vision, CNNs give the best performance. This is partly due to the nature of the operations that capture spatial information. CNNs are usually composed of several types of layers: convolutional, pooling and fully connected layers.

6.3.6

127

The U-Net Architecture

The U-Net model (Ronneberger et al. 2015) is a type of CNN well suited to image segmentation tasks and has been shown to be very successful in biomedical image segmentation in particular. The architecture consists of an initial contracting/ encoding section (decreases feature map size) followed by a second expanding/decoding section (increases feature map size). The schematic in Fig. 6.5 shows the U-shaped flow of the feature maps, hence the name of ‘U’ Net. The contracting/encoding section utilises a series of convolutional operations with max pooling operations. The expanding/decoding section then utilises a series of transposed convolutions (an operation similar to convolutions, but with the result of increasing the size of the feature array). This has the effect of increasing the feature map height and width, and this is necessary as the output needs to be the same size as the input images.

Fig. 6.5 Schematic showing U-Net architecture (reproduced with permission, Bowness et al. 2021b). The U-Net model utilises a series of encoding and decoding layers which decrease and increase the size of the feature map, respectively

128

J. Lloyd et al.

At the end of each set of convolutions in the encoding path, there are skip connections to the corresponding locations in the decoding path. These skip connections copy data from the encoding path and concatenate (stack) them with the feature maps of the decoding path. This maintains spatial information lost during encoding. The output from the model has the same dimensions as the input image where each ‘pixel’ is a value between 0 and 1 and represents the probability of that pixel belonging to a specific class (e.g. muscle, bone, artery, nerve).

6.3.7

How Models Train

To gain an intuition of what the process of ‘model training’ involves, let’s say that we want to predict a patient’s low-density lipoprotein (LDL) levels. Imagine we do not have blood testing equipment available so all we know about the patient are characteristics that can be measured with simple equipment, we know: • • • • •

Age (years) Body mass index (BMI) Waist skinfold thickness (mm) Waist circumference (mm) Diet quality (values calculated from diet questionnaire)

To be able to predict LDL levels, we would want to define a function that accepts the input (independent) variables and outputs a prediction for the concentration of LDL in that patient’s blood. Input variables are each associated with a parameter (or weight), which are initially not defined. With repeated exposure to ‘labelled’ training data (i.e. input/independent variables paired with the output/dependent variable—the LDL value), the parameters are iteratively adjusted (Fig. 6.6). Ultimately, the model settles on the best values for each parameter so that, for every patient, the combination of input variable and parameter gives a predicted value of the output (LDL level). If the model parameters have

been fitted well, then this predicted value should be close to the truth. A loss function is used during training to check how often the model predicts correctly: the lower the value of the loss function, the better the model has performed.

6.3.8

Model Evaluation

Model evaluation is the process of assessing how a model would perform on unseen data. In the case of segmentation models, the data will be partitioned into train, test and validation data (Fig. 6.7). Training utilises the train data, and then the model is evaluated on the test data at the end of each round of training (also known as an epoch). Each subset of data (e.g. patient age) in the train dataset will either be allocated to the train set or the test set (but never both). Likewise, data from the same patient will only appear in either the train or test set. If this wasn’t the case, then performance of the model may be artificially increased as testing data would too closely resemble training data. For example, imagine a scenario where ultrasound images from a patient called Sam were included in both train and test data. The model may learn the Sam-specific features of the data that make prediction on the Sam images in the test data a lot easier. During training, the model is evaluated on the test data. After training, the model is then used to predict on unseen videos (validation data). If the data is drawn from a sample with the same characteristics (e.g. data collected at the same time +/ place +/ by the same people +/ with the same machines, etc.), this is called internal validation. The validation data may contain data from different times/places, collected by different people, and from different geographical locations. This is called external validation. It is very important as it assesses the generalisability of the models (e.g. does a model that labels anatomical structures on ultrasound work with different ultrasound machines?).

6

Artificial Intelligence: Innovation to Assist in the Identification of. . .

Fig. 6.6 Schematic showing outline for model training and prediction for LDL prediction example. Schematic showing the model training and prediction process. During model training the model is shown examples of the

129

independent variables and dependent variables for a set of patients. Once the model has learnt from this data, it can be used to predict the dependent variable given only the independent variables

Fig. 6.7 Schematic showing how data is partitioned. The data is partitioned into train, test and validation sets (note that some sources may use the terms test and validation in the opposite manner)

130

J. Lloyd et al.

Fig. 6.8 Schematic showing how intersection over union (IOU) metric is calculated

Metrics used to assess the performance of a model typically take into account the degree of overlap between the predicted and true segmentation maps. One example of such a metric is the intersection-over-union (IOU) metric (Fig. 6.8). This takes the number of pixels overlapping between the annotations and predictions (intersection) and divides the number by the total area of the two masks combined (union). It remains to be seen, however, how such quantitative assessments of performance relate to benefit in clinical practice. Further work on this area is required to undertake clinical evaluation of the utility of such models and better understand what the quantitative assessments imply.

6.4

6.4.1

Part 3: The Current State of AI in Ultrasound Image Interpretation for Ultrasound-Guided Regional Anaesthesia How Can Technology Be Used to Augment UGRA?

Learning and practising UGRA presents many challenges, but recent developments in AI could bring wide-ranging benefits. Automated assistive technology could be used to help the identification of key sono-anatomical structures, such as

nerves, arteries and muscle (Bowness et al. 2021c). Systems could also aid in identifying the optimal ultrasound view before introduction of the needle (Smistad et al. 2017; Bowness et al. 2021a). Several groups have been evaluating methods of ultrasound segmentation and nerve tracking. This section reviews the current literature associated with computational techniques used for achieving the optimum ultrasound view during UGRA.

6.4.2

Summary of Different Approaches

There are several methods to identify structures on US, which include image segmentation, image classification and tracking methods. Each of these categories contains both DL- and non-DL-based techniques. As already mentioned in this chapter, segmentation models are models that delineate a region of interest within an image. The most prominent deep learning method for image segmentation in UGRA is the U-Net architecture (Ronneberger et al. 2015). Image classification involves passing a sliding window over an image, to deliver sections of an image to a model which classifies each window. This approach has been used in combination with a simple CNN plus spatiotemporal features

6

Artificial Intelligence: Innovation to Assist in the Identification of. . .

(Hafiane et al. 2017) and in combination with support vector machines, a type of machine learning model (Hadjerci et al. 2015, 2016). These methods simply classify subsets of an ultrasound image as containing a structure or not, and they must be combined with a secondary step that segments the region of interest in order to identify the edges of the structure. These methods are called contouring methods—they may not necessarily be able to segment a structure with an entire image, but if the bounding box of the object is identified, they can then delineate the outer edge of the structure. The most successful non-deep learning method for nerve delineation is the gradient vector flow + phase + probability map (PGVF) technique (Hafiane et al. 2014). Tracking can be used to follow an already identified nerve within an ultrasound video through time. This is different from image segmentation because there needs to be an initialisation step whereby the structure is first segmented, and this segmentation map is then used to seed the tracking algorithm which can then take over and follow the structure based on textual features. There have been 13 deep learning approaches and one non-deep learning approach implemented and evaluated for this task (Alkhatib et al. 2019).

6.4.3

Segmentation

6.4.3.1 Deep Learning Approaches There are several examples of DL-based segmentation methods. Smistad et al. (2018), Huang et al. (2019) and (Abraham et al. 2019) all used the U-Net architecture to segment anatomical features. Huang et al. (2019) and Abraham et al. (2019) trained models to identify a single structure in each case (the femoral nerve and brachial plexus nerves, respectively). Smistad et al. (2018) went beyond a single class to segment six classes of structures relevant to the axillary-level brachial plexus block: background, blood vessel, musculocutaneous nerve, median nerve, ulnar nerve and radial nerve. Abraham et al. (2019) also trained an M-Net model to identify the brachial plexus nerves. The M-Net is a modified

131

version of the U-Net architecture, where appropriately scaled US images are concatenated (stacked) at each level of the encoder part of the U-Net. Output feature maps from every decoding level are then fed into the loss function. A key factor in a robust evaluation of these models is the quality of the data used to train and test them and the evaluation scheme used. To evaluate the generalisability of the model, the data must contain suitable amounts of variation and must be a suitable size. Abraham et al. (2019) used data hosted by Kaggle (https://www.kaggle. com/c/ultrasound-nerve-segmentation); Kaggle is a platform for hosting machine learning competitions (https://www.kaggle.com/). Five thousand, six hundred and thirty-five (5635) images were used: 65% for training, 10% for validation and 25% for testing (note the transposition of the terms validation and testing compared to Fig. 6.7). Little information is provided about this dataset so the generalisability of the model is difficult to assess. It is therefore difficult to comment on whether the M-Net is a suitable model for segmentation of anatomical features in the context of UGRA. However, the M-Net model did outperform the U-Net when evaluated on the same data. The U-Net achieved a Dice similarity score (equivalent to IOU discussed earlier) of 0.599, whereas the M-net achieved 0.882. The M-Net therefore outperformed the U-Net on this particular dataset. Huang et al. (2019) trained their U-Net model on significantly less data; 562 images were used with 500 images in the development set and 62 images in the test set. The data were sourced from multiple US machines increasing the variation within the dataset. A more robust validation schema of tenfold cross-validation was also utilised which showed a median of 0.656 IOU. For the development and test sets, the IOU were 0.633 and 0.638, respectively. The performance of this model was worse than that of the M-Net by Abraham et al. (2019); however, the datasets used were different between the two studies, so it is difficult to compare the models directly. Smistad et al. (2018) used data from 49 different patients. A total of 462 images were annotated, taken from 123 videos recorded on

132

multiple US machines. A robust validation schema was used: leave-one-subject-out crossvalidation (where the data corresponding to a single patient is assigned to testing, while the rest is used for training; this is repeated until all patients have been allocated to testing once). Interestingly, the images were annotated with bounding boxes as opposed to freehand outlines despite the model outputting segmentation maps. A true positive was defined if 25% of the bounding box was classified as correct. Blood vessels showed the best performance with a recall and precision greater than 0.8. The median, ulnar and radial nerves were detected with f1 scores (also similar to IOU) of 0.73, 0.62 and 0.39, respectively. It was shown that the use of image augmentations increased the f1 score by 0.13. The performance of the models may have been improved if more data were collected and used for training. However, the evaluation scheme used was strong.

6.4.3.2 Non-deep Learning Approaches There are also prominent examples of non-DL approaches for segmentation, including Smistad and Lindseth (2016) and Smistad et al. (2018). In these papers the authors introduce and evaluate another method of identifying arteries in US images. The artery is modelled as an ellipsis, and a brute-force method is employed to look for specific black ellipses in the US image. A benefit of such an approach is the fact that a large amount of training data isn’t required. However, the system is only suitable for identifying arteries, which is possible due to the predictable shape of the artery in ultrasound images (depending upon the angle the US probe transects the artery). The same approach would be less effective in identifying more morphologically complex structures. The authors also identify the fascia lata and fascia iliaca by using a rule-based method that considers features of each pixel such as distance from the skin surface, distance from the femoral artery and the presence of bright edges. The edges are located using an algorithm which accounts for these factors in order to classify each pixel.

J. Lloyd et al.

Hafiane et al. (2014) and Hadjerci et al. (2015) implemented and evaluated a pipeline for segmenting specific nerve structures in US images. Hafiane et al. (2014) evaluated an algorithm for segmenting the sciatic nerve of the popliteal region that combines gradient vector flow + phase + probability map (PGVF). The technique relies on the nerve sections being of a unique texture to surrounding areas. Two of 15 images from different patients were used to train the algorithm, and the remaining 13 images were used for testing. The Dice metric was high at 0.9 (which assesses the overlap of segmented areas), and Hausdorff metric was 11.1 (which assesses the distance between boundaries of the segmented areas). However, the dataset for testing was small compared to other evaluation schema seen in the literature. Furthermore, the types of US machines used were not stated. It is possible that the test dataset did not contain enough variation to show if the algorithm was generalisable and therefore practically useful. Hadjerci et al. (2015) developed a system capable of segmenting the median nerve in different positions in the forearm: the elbow, proximal and distal forearm median nerves. During training, the technique extracts 37 texture-related features. Feature selection is used in order to filter for the most useful features. Three separate support vector machines are trained, one for each position of median nerve. During prediction, the three support vector machines are used to predict the presence of nerves, and the strongest prediction candidate was chosen. Subsets of images from a sliding window for each US image are used for prediction by the three support vector machines. In this study two datasets were collected at different times; the first contained images from eight patients and the second contained ultrasound images from five patients (the datasets were collected at different times). Data from three randomly selected patients from Dataset 1 were used for training and the remaining five patients from Dataset 1, and all patients from Dataset 2 were used for testing. The training data were annotated with bounding boxes, and so prediction was in the form of classifications of subset squares of the image. F1

6

Artificial Intelligence: Innovation to Assist in the Identification of. . .

score was used as the evaluation metric with overlap of bounding boxes of prediction and truth being a true positive if greater than 50%. The results show that predictions on Dataset 1 and Dataset 2 achieved F1 scores of 82% and 62%, respectively. Dataset 2 was collected at a later time so the decrease in performance could be due to the fact that the data were collected under different conditions. For example, a different US machine could have been used to collect the images in Dataset 2. These results suggest limitations in the model generalisability. Hadjerci et al. (2016) expanded on their earlier work (Hadjerci et al. 2015) to propose a generic framework for median nerve classification. In this paper they formalise a pipeline of several stages (despeckling filter, feature extraction, feature selection, classification and segmentation). They assessed a selection of different algorithms for each stage. The results showed that localisation using support vector machines combined with PGVF gave the best results with 0.82 and 10.4 for Dice and Hausdorff for segmentation, respectively. Interestingly, it was noted in the paper that segmentation by the PGVF method was successful even when the nerve structure wasn’t completely encompassed by the localisation region of interest.

6.4.4 6.4.4.1

Tracking Methods

How Does Tracking Fit in with Segmentation? Tracking methods are methods that segment a structure within an image based on the knowledge of where that structure is at a previous time point. This is highly applicable in UGRA because the US images are temporal. There is a relationship between where a structure is in a previous frame and where it will be in the next frame. However, tracking algorithms require an initialisation stage. This is where an algorithm segments the structure to be tracked, and this segmentation is then fed into the tracking algorithm which uses information extracted from the segmentation image to track the structure. This is useful in UGRA when the practitioner moves the probe to identify

133

the optimum US view prior to performing the block.

6.4.4.2 Approaches There are only a few examples of tracking algorithms used in the context of UGRA in the literature. A comparative study evaluated 13 deep learning tracking models applied to the task of median and sciatic nerve tracking (Alkhatib et al. 2019). The dataset used for evaluation of the 14 tracking techniques consisted of 25 videos of the median nerve (with an average of 335 images per video) and 17 videos of the sciatic nerve (with an average of 120 images per video). Overall, the efficient convolution operators (ECO) DL network showed the best tracking performance and showed promising results. There are many more papers on methods for segmentation of anatomical structures within ultrasound compared to tracking. This is surprising given the temporal relationship between images in ultrasound video. There is likely a lot of potential in this area.

6.4.5

Summary and Future Directions

The literature contains only a handful of approaches under investigation in the context of AI-assisted US image interpretation for UGRA. The most common approaches are DL (consisting of U-Net networks or modified U-Net networks) and localisation plus segmentation (an algorithm identifies a bounding box that encompasses the structure, and then a secondary algorithm such as PGVF (Hafiane et al. 2014) is used for delineation of the structure contained within the bounding box). In the latter approach, hand-crafted features must be generated from the US images, whereas in the DL approach, this is achieved automatically within the learnt function of the networks. There is only one paper that evaluates tracking techniques as applied to UGRA (Alkhatib et al. 2019). The DL approach, overall, is more generalisable both in terms of the good performances on test datasets and the fact that one technique can be used independently of the structure needing segmentation. However, the

134

J. Lloyd et al.

technique requires a vast amount of labelled data. Not as much data is needed for the non-DL algorithms that show similarly good results. It is difficult to compare models against each other due to the use of different evaluation metrics and different datasets between research groups. In the future, some effort to standardise these aspects of the evaluation process would enable more meaningful comparisons between different models.

6.5

Part 4: A Case Study: ScanNav Anatomy Peripheral Nerve Block

ScanNav Anatomy Peripheral Nerve Block (also known as ScanNav Anatomy PNB, formerly known as Anatomy Guide) is a system used to assist in structure identification on US during UGRA. The system utilises DL to identify anatomical structures in B-mode ultrasound images during real-time US scanning. The software overlays coloured segmentation predictions for key sonoanatomical structures to help identification during specific peripheral nerve block procedures (Fig. 6.9). It runs on a panel PC which plugs into the HDMI output of an US machine—it thus relies on the US machine to generate the images and then analyses them. As such, it is reliant on the operator acquiring images that display the structures in question and on the US machine settings being optimised. The system utilises a selection of different trained U-Net models relevant to specific peripheral block regions (further information on the model can be found in Bowness et al. 2021b). The supported regions include: – Brachial plexus of the neck (from interscalene groove to supraclavicular fossa) – Axillary-level brachial plexus – Rectus sheath plane – Erector spinae plane – Supra-inguinal fascia iliaca – Adductor canal – Popliteal-level sciatic nerve

Large volumes of data were used during training to increase the variation in data seen by the models and to mimic the types of image noise and artefacts seen in US images. Over 800,000 images were used for training, with a typical training set for a given peripheral nerve block region containing 115,000 pairs of ultrasound images and hand-annotated segmentation masks. For each region, images from patients were split into train (90%) and test (10%) datasets. The models were trained on the train dataset and evaluated against the test dataset. Validation consisted of videos of model predictions that were also evaluated by a panel of clinical experts to assess both the accuracy and usefulness of the model predictions (Bowness et al. 2021a). The assessment panel consisted of three independent experts (two from the USA and one from the UK). These experts identified the model predictions as helpful for identifying anatomical structures in 1330/1334 cases (99.7%) and for confirming the correct ultrasound view in 273/275 ultrasound scans (99.3%) (Bowness et al. 2021a). Further studies are underway to conduct an external validation of the system (in the USA) and perform a quantitative assessment using metrics such as the Dice correlation coefficient/IOU. The device gained regulatory approval for clinical use in Europe in April 2021—the first US AI device for UGRA in the world. Further data will be submitted for regulatory approval in the USA (via the US Food and Drug Administration). Subsequent studies, performed by this team and others, will undoubtedly continue to evaluate the device and its utility in clinical practice.

6.6

6.6.1

Part 5: The Future: Artificial Intelligence and Ultrasound-Guided Regional Anaesthesia Supporting Practice

The first generation of AI in UGRA has arrived to assist with image interpretation. ScanNav Anatomy PNB has been developed to identify anatomical structures on ultrasound images and

6

Artificial Intelligence: Innovation to Assist in the Identification of. . .

135

Fig. 6.9 Examples of colour overlay produced by ScanNav Anatomy PNB (reproduced with permission; Bowness et al. 2021a). (a) Supraclavicular-level brachial plexus: subclavian artery (red), brachial plexus nerves (yellow), first rib (blue), pleura (purple). (b) Erector spinae plane (thoracic region): trapezius/rhomboid/erector spinae (group) muscles (green), vertebral transverse process (yellow—not shown in this image), rib (blue), pleura (purple). (c) Rectus sheath: rectus abdominis muscle (green), rectus sheath (orange), peritoneal contents (brown). (e) Adductor canal: femoral artery (red), saphenous nerve complex (yellow), sartorius/adductor longus (green), femur (blue)

overlay a colour highlight to label key structures in real time. In time this system will become more refined—perhaps by increasing the accuracy of labelling, supporting block performance in different anatomical regions or developing the ability

to identify atypical anatomy (e.g. in obese patients or those with previous trauma/surgery). New systems will also emerge, which either perform a similar function or support practice in a different manner. For example, Smistad et al.

136

J. Lloyd et al.

(2017) have presented work on automated probe guidance to direct US scanning for UGRA, and Gungor et al. (2021) demonstrated the Nerveblox AI software labelling UGRA images in real time.

6.6.2

Changing How We Learn

The next revolution could be in how we assess our performance in UGRA. The ability to autonomously quantify performance parameters such as image optimisation, structure recognition and needle visualisation could be of use in assessment during training, for example, to demonstrate competence during simulation, prior to patient interaction. More experienced clinicians may utilise these same data to perform self-testing, for example, as part of appraisal and revalidation. Immersive technology could revolutionise learning UGRA both in terms of image recognition, needle probe alignment and advancement. High-fidelity simulation has the potential to build knowledge, competence and performance (Carey and Rossler 2021). AI systems could reduce or even bypass the need for an expert to be present for teaching. The introduction of haptic feedback would increase fidelity within the AR environment. Current needle simulators allow input from the practitioner to be represented in AR—the next generation may allow feedback from virtual needle position back to the practitioner, creating a form of mixed reality. Feedback could include visual, audible or tactile haptic warnings, such as when the needle tip is not visible on screen or when it is close to a recognised hazard. Further development could see simulated loss of resistance for piercing fascia or hard resistance for structures like bone.

6.6.3

The Extra Dimension

At present, AI in UGRA is based on recognition of sono-anatomical structures on a 2D image. Future AI work could seek to build 3D models of the subject by integrating data on probe position and orientation to the images generated. This

raises the possibility of improving structure recognition even when a suboptimal angle of insonation for the target structure is necessary. For example, in the axillary-level brachial plexus block, when the radial nerve moves posteriorly to pass through the lower triangular space of the arm, it lies in an oblique orientation to the other nerves targeted at that level. 3D imaging has the potential to improve how we judge adequate deposition of local anaesthetic. It has already been shown that circumferential spread of LA, as visualised in 2D US, decreases time to block onset and improves block reliability (Brull et al. 2011). What is not yet fully understood is the effect of longitudinal spread of LA along the nerve. There is data to suggest a greater longitudinal distance in spread has more of an effect on duration of block than LA dose (Madsen et al. 2020), and the ability to visualise both circumferential and longitudinal spread in realtime 3D may be beneficial. Development of 3D modelling for the anatomy or needle position may expose the limits of 2D screens. New methods of visually representing the model to the operator, such as an immersive 3D environment, may need development.

6.6.4

The Future of Clinical Practice

Widespread adoption of AI for image recognition in UGRA would likely have a significant impact on how people learn and practise UGRA. By providing automated assistance for real-time US scanning, AI may lessen the cognitive burden of performing UGRA. Humans have a finite cognitive capacity; reducing the burden of US scanning and image interpretation may allow practitioners to focus more on needle probe orientation and ensure adequate spread of LA. This assistance may support individuals to use UGRA more frequently and give confidence in using a wider range of techniques. Reducing cognitive load may also reduce the accumulation of fatigue and increase the performance of fatigued practitioners. In the UK, emergency out-of-hours medical services are often provided by doctors still in training programmes. These individuals

6

Artificial Intelligence: Innovation to Assist in the Identification of. . .

are typically less experienced and performing procedures while they are more likely to be fatigued, both of which are independent risk factors for errors. Assistive AI systems could therefore be doubly advantageous. The focus of AI within UGRA is currently on training and supporting the practice of the non-expert. In other fields of medical imaging, we have seen AI systems surpass even expert operators in their ability to recognise disease in modalities such as mammography (McKinney et al. 2020) and optical coherence tomography (De Fauw et al. 2018). As AI systems evolve, we may see the same results in UGRA, and the practical role of the expert may shift to needle probe orientation, needle insertion and injection. Even in such a reduced role, however, experts may not always be at the pinnacle of performance. One end point for AI in RA may be to integrate the image interpretation of AI with the precision of robotics to deliver the injection. This sort of autonomous system would be capable of both recognising the key sono-anatomical structures, performing safe needle insertion and administering LA injection to achieve a safe and efficacious block. Such an idea has already been trialled for venepuncture in humans, with success rates of 87% (Leipheimer et al. 2019). The successful integration of 3D ultrasound, AI image interpretation and robotic injection could mark a watershed moment in UGRA and image-guided interventional procedures in anaesthesia and many other fields of medicine.

Term Cine loop Convolution

Convolutional neural network

Feature map Image annotation

Loss function

Machine learning

Max pooling

Glossary of Terms Term Activation function Algorithm Array

Batch

Definition Function applied to the output of a neurone A set of rules to be followed, usually executed by a computer An arrangement of data points (composed of numbers, in this case), usually with a specified number of rows, columns and given depth A set of observations of data used during model training that are used to update the model’s parameters

Model parameter

Model training

137 Definition A temporally consecutive set of ultrasound images A convolutional operation is a type of mathematical operation found in convolutional neural networks. In a convolutional layer, subsets of the previous layer undergo a mathematical function. The output is then fed into the subsequent layer A type of neural network that can receive images as input and combines a succession of neural (convolutional) layers to perform learnt tasks Output of values from a preceding neurone layer The process of demarcating structures in ultrasound images by an expert, typically these annotated images are then used for model training and validation A function that evaluates how well a model is fitted to training data. It is updated during model training. The lower the value, the better the model fits to the training data. If model training is successful, the loss function will decrease with time. The value of the loss function is changed by model parameters changing (model training is the process of updating the model parameters in such a way that the loss function decreases) A field where algorithms utilise mathematical functions to ‘learn’ from data during training in order to perform a task Max pooling is a function utilised in some layers of convolutional neural networks. The function takes the maximum value of a grid of values from the previous layer. This value is used in subsequent operations/layers of the network A numerical value in a model function that is changed during training in order to increase performance of the model at a specific task. Also known as a ‘weight’ A process where a model is exposed to data. During the process, parameters in the model (continued)

138 Term

Neural network layer

Neurone

Observation

Region of interest Semantic image segmentation

Semi-supervised Sliding window

Supervised Unsupervised

J. Lloyd et al. Definition are updated in a way that optimises performance of a task. A layer in a neural network consisting of neurones. Each neurone receives input from the previous layer, and outputs from the neurones are given to the next layer A mathematical function in a layer that receives input from the previous layer and outputs a single number to the next layer A single example of data that can be trained on or inferred from. In the context of image segmentation, an observation would be an image A region in an image that is intended to be identified The demarcation of structures in images (to form a filled-in ‘mask’). Each pixel in the image is classified as belonging to a specified class Model trains on labelled and unlabelled data A square array (window) of numbers is extracted from a larger array. The window is then translated (slid) by a specified number of units before extracting another square array of numbers. This is usually repeated until the whole array has been visited Model trains on labelled data No data is labelled so the model learns from the structure within the data

References Abdallah FW, Macfarlane AJR, Brull R (2016) The requisites of needle-to-nerve proximity for ultrasound-guided regional anesthesia: a scoping review of the evidence. Reg Anesth Pain Med 41: 221–228 Abraham N, Illanko K, Khan N, Androutsos D (2019) Deep learning for semantic segmentation of brachial plexus nerves in ultrasound images using U-net and M-net. In: 2019 3rd International Conference on Imaging, Signal Processing and Communication (ICISPC), pp 85–89 Alkhatib M, Hafiane A, Vieyres P, Delbos A (2019) Deep visual tracking in ultrasound images. Comput Med Imaging Graph 76:101639 Barrington MJ, Wong DM, Slater B (2012) Ultrasoundguided regional anesthesia: how much practice do

novices require before achieving competency in ultrasound needle visualization using a cadaver model. Reg Anesth Pain Med 37:334–339 Bowness J, El-Boghdadly K, Burckett-St Laurent D (2021c) Artificial intelligence for image interpretation in ultrasound-guided regional anaesthesia. Anaesthesia 76:602–607 Bowness J, Macfarlane AJR, Noble A, Higham H, Burckett-St Laurent (2021b) Anaesthesia, nerve blocks & artificial intelligence. Anaesthesia News (in press) Bowness J, Taylor A (2019) Anatomy for the FRCA. Cambridge University Press, Cambridhe, UK Bowness J, Taylor A (2020) Ultrasound-guided regional Anaesthesia: Visualising the nerve and needle. In: Rea PM (ed) Biomedical Visualisation, Advances in experimental medicine and biology, vol 6. Springer, Switzerland, pp 19–34 Bowness J, Turnbull K, Taylor A, Halcrow J, Chisholm F, Grant C, Varsou O (2019a) Identifying the emergence of the superficial peroneal nerve through deep fascia on ultrasound and by dissection: implications for regional anesthesia in foot and ankle surgery. Clin Anat 32: 390–395 Bowness J, Turnbull K, Taylor A, Halcrow J, Raju P, Mustafa A, Chisholm F, Varsou O, Grant C (2019b) Identifying variant anatomy during ultrasound-guided regional anaesthesia: opportunities for clinical improvement. Br J Anaesth 122(5):75–77 Bowness J, Varsou O, Turbitt L, Burckett-St Laurent D (2021a) Identifying anatomical structures on ultrasound: assistive artificial intelligence in ultrasoundguided regional Anaesthesia. Clin Anat 34 (5):802–809. https://doi.org/10.1002/ca.23742 Bowness JS, Pawa A, Turbitt L et al (2021) International consensus on anatomical structures to identify on ultrasound for the performance of basic blocks in ultrasound-guided regional anesthesia. Reg Anesth Pain Med. https://doi.org/10.1136/rapm-2021-103004 Brull R, Macfarlane AJR, Parrington SJ et al (2011) Is circumferential injection advantageous for ultrasoundguided popliteal sciatic nerve block?: a proof-of-concept study. Reg Anesth Pain Med 36:266–270 Cappelleri G, Cedrati VLE, Fedele LL et al (2016) Effects of the Intraneural and Subparaneural ultrasoundguided popliteal sciatic nerve block: a prospective, randomized, double-blind clinical and electrophysiological comparison. Reg Anesth Pain Med 41:430–437 Carey JM, Rossler K (2021) The how when why of high Fidelity simulation In: StatPearls [internet]. Treasure Island (FL): StatPearls Publishing; Accessed Mar, 2021, from https://www.ncbi.nlm.nih.gov/books/ NBK559313/ Christophe JL, Berthier F, Boillot A, Tatu L, Viennet A, Boichut N, Samain E (2009) Assessment of topographic brachial plexus nerves variations at the axilla using ultrasonography. Br J Anaesth 103(4):606–612 Cowlishaw P, Kotze P (2015) Adductor canal block–or subsartorial canal block? Reg Anes Pain Med 40:175– 176

6

Artificial Intelligence: Innovation to Assist in the Identification of. . .

Croskerry P (2002) Achieving quality in clinical decision making: cognitive strategies and detection of bias. Acad Emerg Med 9(11):1184–1204 De Fauw J, Ledsam JR, Romera-Paredes B et al (2018) Clinically applicable deep learning for diagnosis and referral in retinal disease. Nat Med 24:1342–1350 Drew T, Vo MLH, Wolfe JM (2013) The invisible gorilla strikes again: sustained inattention blindness in expert observers. Psychol Sci 24:1848–1853 El-Boghdadly K, Wolmarans M, Stengel AD, Albrecht E, Chin KJ, Elsharkawy H, Kopp S et al (2021) Standardizing nomenclature in regional anesthesia: an ASRA-ESRA Delphi consensus study of abdominal wall, paraspinal, and chest blocks. Reg Anes Pain Med 46(7):571–580 Fiegler W, Felix R, Langer M, Schultz E (1985) Fat as a factor affecting resolution in diagnostic ultrasound: possibilities for improving picture quality. Eur J Radiol 5:304–309 Graham ER, Burke DM (2011) Aging increases inattentional blindness to the gorilla in our midst. Psychol Aging 26(1):162–166 Gungor I, Gunaydin B, Oktar SO, Buyukgebiz B, Bagcaz S, Ozdemir MG, Inan G (2021) A real-time anatomy {dentification via tool based on artificial {ntelligence for ultrasound-guided peripheral nerve block procedures: an accuracy study. J Anesth 19:1–4 Hadjerci O, Hafiane A, Conte D, Makris P, Vieyres P, Delbos A (2016) Computer-aided detection system for nerve identification using ultrasound images: a comparative study. Inform Med Unlocked 3:29–43 Hadjerci O, Hafiane A, Makris P, Conte D, Vieyres P, Delbos A (2015) Nerve localization by machine learning framework with new feature selection algorithm. In: Murino V, Puppo E (eds) Image analysis and processing–ICIAP 2015. Lecture notes in computer science, vol 9279. Springer, Cham Hafiane A, Vieyres P, Delbos A (2014) Phase-based probabilistic active contour for nerve detection in ultrasound images for regional anesthesia. Comput Biol Med 52:88–95 Hafiane A, Vieyres P, Delbos A (2017) Deep learning with spatiotemporal consistency for nerve segmentation in ultrasound images. arXiv:170605870 [cs] Hanna TN, Lamoureux C, Krupinski EA, Weber S, Johnson J-O (2018) Effect of shift, schedule, and volume on interpretive accuracy: a retrospective analysis of 2.9 million radiologic examinations. Radiology 287:205– 212 Henderson M, Dolan J (2016) Challenges, solutions, and advances in ultrasound-guided regional anaesthesia. BJA Education 16(11):374–380 Huang C, Zhou Y, Tan W et al (2019) Applying deep learning in recognizing the femoral nerve block region on ultrasound images. Ann Transl Med 7(18):453 Kahl S, Clapp M, Hopping WA, Goëau H, Glotin H, Planqué R, & Vellinga W-P, et al (2020) Overview of BirdCLEF 2020: bird sound recognition in complex

139

acoustic environments. CLEF 2020–11th International Conference of the Cross-Language Evaluation Forum for European Languages, Thessaloniki, Greece Leipheimer JM, Balter ML, Chen AI, Pantin EJ, Davidovich AE, Labazzo KS, Yarmush ML (2019) First-in-human evaluation of a hand-held automated venipuncture device for rapid venous blood draws. Technology (Singap World Sci) 7(3–4):98–107 Lum TE, Fairbanks RJ, Pennington EC, Zwemer FL (2005) Profiles in patient safety: misplaced femoral line guidewire and multiple failures to detect the foreign body on chest radiography. Acad Emerg Med 12 (7):658–662 Mackworth N (1950) Researches on the measurement of human performance. Med Res Council Spec Report Series 268:156 Madsen MH, Christiansen CB, Rothe C et al (2020) Longitudinal neural exposure to local anesthetic and nerve block duration: a retrospective analysis of experimental data from healthy volunteer trials. Reg Anes Pain Med 45:192–197 Marhofer P, Pilz-Lubsczyk B, Lönnqvist PA et al (2014) Ultrasound-guided peripheral regional anaesthesia: a feasibility study in obese versus normal-weight women. Int J Obes 38:451–455 McKinney SM, Sieniek M, Godbole V et al (2020) International evaluation of an AI system for breast cancer screening. Nature 577:89–94 Moayeri N, Krediet AC, Welleweerd JC, Bleys RLAW, Groen GJ (2012) Early ultrasonographic detection of low-volume intraneural injection. Br J Anaesth 109: 432–438 Neal JM, Brull R, Horn JL, Liu SS, McCartney CJ, Perlas A, Salinas FV, Tsui BC (2016) The second American Society of Regional Anesthesia and Pain Medicine evidence-based medicine assessment of ultrasound-guided regional anesthesia: executive summary. Reg Anesth Pain Med 41(2):181–194 Osler W (1903) On the educational value of the medical society. Boston Med Surg J 11:275–279 Pereira-Kohatsu JC, Quijano-Sánchez L, Liberatore F, Camacho-Collados M (2019) Detecting and monitoring hate speech in twitter. Sensors 19(21):4654 Rangarajan K, Davis H, Pucher PH (2020) Systematic review of virtual haptics in surgical simulation: a valid educational tool? J Surg Educ 77(2):337–347 Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional neural networks for biomedical image segmentation. arXiv:1505.04597 Royal College of Anaesthetists (2020) Medical workforce census report 2020. Accessed Mar, 2021, from http:// www.rcoa.ac.uk/sites/default/files/documents/202011/Medical-Workforce-Census-Report-2020.pdf Royal College of Anaesthetists (2021) Curriculum learning syllabus stage 3 (unapproved). Accessed Apr 23, 2019, from https://rcoa.ac.uk/documents/2021curriculum-learning-syllabus-stage-3-unapproved/ introduction

140 Short A, Chan VWS, Perlas A (2016) Is deliberate Intraneural injection a case of “false economy”? Reg Anes Pain Med 41:421–423 Simons DJ, Chabris CF (1999) Gorillas in our midst: sustained inattentional blindness for dynamic events. Perception 28(9):1059–1074 Sites BD, Spence BC, Gallagher J et al (2007) Characterizing novice behavior associated with learning ultrasound-guided peripheral regional anesthesia. Reg Anesth Pain Med 32:107–115 Smistad E, Iversen DH, Leidig L et al (2017) Automatic segmentation and probe guidance for real-time assistance of ultrasound-guided femoral nerve blocks. Ultrasound Med Biol 43:218–226

J. Lloyd et al. Smistad E, Johansen KF, Iversen DH, Reinertsen I (2018) Highlighting nerves and blood vessels for ultrasoundguided axillary nerve block procedures using neural networks. J Med Imaging (Bellingham) 5:044004 Smistad E, Lindseth F (2016) Real-time automatic artery segmentation, reconstruction and registration for ultrasound-guided regional Anaesthesia of the femoral nerve. IEEE Trans Med Imaging 35:752–761 Taylor-Phillips S, Stinton C (2019) Fatigue in radiology: a fertile area for future research. Br J Radiol 92:1099 Turbitt LR, Mariano ER, El-Boghdadly K (2020) Future directions in regional anaesthesia: not just for the cognoscenti. Anaesthesia 75:293–297

7

A Systematic Review of Randomised Control Trials Evaluating the Efficacy and Safety of Open and Endoscopic Carpal Tunnel Release Eilidh MacDonald and Paul M. Rea

Abstract

Introduction: Carpal tunnel syndrome is the most prevalent form of nerve compression syndrome of the upper limb; therefore, it is of clinical significance to critique treatment methods. There is an ongoing debate amongst clinicians as to which surgical method—open or endoscopic carpal tunnel release—provides better overall symptom relief and faster recovery time. This systematic review aimed to investigate the evidence from randomised control trials to evaluate the effectiveness and safety of open and endoscopic carpal tunnel release surgery. Methods: Database searches were carried out to identify literature. An inclusion and exclusion criteria was applied to only include randomised control trials which compared open and endoscopic surgery. Publications were then selected according to PRISMA guidelines, risk of bias was assessed and patient outcome was assessed. Results: Twenty-three studies were selected for this systematic review. It was found that for improvement to grip strength and symptom severity, the endoscopic group had more significant improvement in the short term, resulting in a quicker return to work time compared to the open group. The complication rate E. MacDonald · P. M. Rea (*) Anatomy Facility, School of Life Sciences, College of Medical, Veterinary and Life Sciences, University of Glasgow, Glasgow, UK e-mail: [email protected]

for both intervention groups was low despite more severe and irreversible complications such as prolonged pain and wound infections being observed in the open group; however, the endoscopic group reported a higher risk of needing repeat surgery. Conclusion: The quicker recovery time, improved cosmetic result and less severe complications observed with the endoscopic technique suggest that it should be used more often. However, this review found no convincing evidence of a significantly superior technique in the long term. Keywords

Carpal tunnel syndrome · Open carpal tunnel release · Endoscopic carpal tunnel release · Minimally invasive surgery · Patient outcomes · Open versus endoscopic surgery

7.1

Introduction

Carpal tunnel syndrome (CTS) is the most common nerve compression-related neuropathy in the upper limb, with it being reported as one of the top leading causes for long-lasting work absence in the USA (Sevy and Varacallo 2020). It can occur at any age; however, it is most prevalent in people aged 40–60 years old and is more common in females than it is in males (Burton et al. 2014). CTS is idiopathic and is usually a result of

# The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 P. M. Rea (ed.), Biomedical Visualisation, Advances in Experimental Medicine and Biology 1356, https://doi.org/10.1007/978-3-030-87779-8_7

141

142

E. MacDonald and P. M. Rea

environmental, social and occupational risk factors. Studies have found that hormonal changes during menopause and pregnancy, hypothyroidism, diabetes, rheumatoid arthritis and obesity are amongst the risk factors associated with CTS (Ibrahim et al. 2012). Additionally, it has been found that those in occupations which involve repetitive hand and wrist movements, manual labour and forced exertion on the wrist lead to an increased risk of CTS (Violante et al. 2016).

7.1.1

Carpal Tunnel Syndrome

The carpal tunnel is a duct-like structure situated within the anterior part of the wrist. The carpal bones form the base of the carpal tunnel, and a sheath of thick connective tissue known as the transverse carpal ligament (TCL) creates the superior border. The median nerve and flexor tendons of the hand travel within the carpal tunnel into the palm to supply motor and sensory function (Zamborsky et al. 2017). The median nerve subdivides into smaller nerve branches to supply specific areas of the hand; these are the recurrent branch, the palmar digital nerve, the palmar cutaneous nerve and the digital cutaneous nerve. The recurrent branch supplies motor innervation to the thenar muscles responsible for movement of the thumb. The palmar digital nerve supplies motor innervation to the lateral two lumbricals which are responsible for the movement of the index and middle fingers. The palmar cutaneous nerve provides sensory innervation to the lateral region of the palm, and the digital cutaneous nerve provides sensory innervation to the skin over the palmar surfaces of the thumb, index finger, middle finger and lateral half of the ring finger. Compression of the median nerve is what ultimately causes the onset of CTS; blood vessel damage, increased pressure within the carpal tunnel and mechanical trauma are all contributing factors leading to median nerve compression (Werner and Andary 2002). Mechanical trauma leads to inflammation and thickening of the flexor tendons, increasing the volume of the tissue within the carpal tunnel and, thus, increases the

interstitial fluid pressure; mechanical trauma can arise from repetitive hand movements or prolonged extension/flexion of the wrist (Ibrahim et al. 2012). The connective tissue surrounding the median nerve swells due to the venous flow being compromised as a result of this increase in pressure, causing ischaemic injury and compression to the median nerve (Sevy and Varacallo 2020). It is thought that the extent of the nerve damage is linked to the duration and severity of the increase in pressure (Zamborsky et al. 2017). The clinical presentation of CTS can vary from patient to patient, but typically individuals will display symptoms such like burning, pain, tingling and/or numbness in the fingers which seems to be worse at night-time. In more severe cases, patients will likely suffer from weakness in the thenar muscles which can make simple daily tasks that require fine motor skills of the thumb and fingers, such like, typing or closing buttons on a shirt nearly impossible. This usually results in individuals requiring time off work and a reduction in the ability to be able to carry out activities of daily living (Wipperman and Goerl 2016). Typically, a clinical diagnosis of CTS will involve nerve conduction testing alongside a physical examination and assessment of the patient’s clinical history. Nerve conduction testing is able to determine the severity of median nerve damage by sending a small shock through the nerve to determine how the electrical impulses travel through it; the slower the impulses travel through the median nerve, the more severe the damage is. Physical examination is completed by carrying out three functional tests: Tinel percussion test, Durkan compression test and Phalen wrist test (Hermiz and Kalliainen 2017). A patient’s history involves assessing information like nature of symptoms and any activities that exacerbate them, occupation, pain location, severity of symptoms and any possible injuries (Burton et al. 2014). The type of treatment a patient receives depends on the severity of the condition. Mild to moderate CTS is usually treated with nonsteroidal anti-inflammatory drugs (NSAIDs), resting, steroid injections and splinting. These types of

7

A Systematic Review of Randomised Control Trials Evaluating the Efficacy. . .

treatments have shown to be effective in the short term; however, in a lot of cases, symptoms will persist and become more severe. For severe CTS, treatment is usually carried out surgically which is known as carpal tunnel release (CTR) (Sayegh and Strauch 2014). There are multiple methods of CTR, and the efficacy and safety of each are still greatly debated.

7.1.2

The Surgical Interventions

Surgery for CTS can either be done by open surgery (open carpal tunnel release (OCTR)) or by endoscopic surgery (endoscopic carpal tunnel release (ECTR)). There are many variations of both surgical intervention methods; however, they all aim to decompress the median nerve. CTR is achieved by a transverse cut through the TCL, and this increases the volume of the carpal tunnel which in turn reduces pressure and thus decompresses the median nerve (Rodner and Katarincic 2008). The incidence of CTR surgery in NHS England is 1 in 1000 per year according to the findings of a recent nationwide observational study (Lane et al. 2021). Despite minimally invasive surgery being believed to provide quicker recovery time for patients and reduced risk of surgical complications, endoscopic carpal tunnel release is rarely carried out in the UK. This may be due to endoscopic surgery requiring greater technical skill and specialised training which can put financial strain on the NHS (Lane et al. 2021). Open carpal tunnel release surgery can be carried out by a standard open technique or a limited open technique. Standard open carpal tunnel release surgery involves a 3–4 cm deep incision from Kaplan’s line to the distal wrist crease through the subcutaneous fat and the palmar tissue to allow for full visualisation of the TCL. This technique reduced the risk of nerve damage as it ensures that there is sufficient visualisation of the palmar cutaneous branch. The TCL is dissected in a distal to proximal direction which consequently decompressed the median nerve by increasing the space within the carpal tunnel (Gurpinar et al. 2019). The incision is then closed, leaving a

143

large scar; the length of the scar is often a reason why complications may arise from this technique. Limited OCTR surgery adopts the same technique as standard OCTR; however, it involves a smaller skin incision. The incision extends for 1.5–2 cm from Kaplan’s line to allow for full visualisation of the TCL; this method results in a smaller scar which improves cosmetic outcomes. Endoscopic carpal tunnel release surgery can be performed through a single- or double-portal technique. Double-portal ECTR involves two 1-cm transverse incisions and was first described by James C. Y. Chow (Chow 1989). One incision is made just proximal to the distal wrist crease where a surgical instrument known as a dissector is inserted; this dissects a route through the carpal tunnel. The dissector exits through a second incision which is made in Kaplan’s line; this entry and exit method is what creates the two portals which are used operating. A trocar containing a cannula is inserted through the proximal portal which exits the distal portal whilst simultaneously depositing the cannula in the carpal tunnel underneath the TCL. The surgical tools are inserted and controlled through the distal portal, and an endoscopic camera is inserted through the proximal portal, allowing for surgeons to sufficiently visualise the TCL which is then cut through in a distal to proximal direction (Oertel et al. 2006; Chow 1989). The single-portal technique was first described by Agee and colleagues to reduce scarring (Agee et al. 1992). Once a 2 cm incision is made in the distal wrist crease where both the surgical instruments and endoscopic are inserted and controlled from. This allows full visualisation of the TCL and carpal tunnel to allow for CTR (Agee et al. 1992). The efficacy and safety of OCTR have already been proven successful, although complications can still arise. It has been found that OCTR is connected to an increased incidence of wound infections, prolonged thenar pain and scar tenderness (Sayegh and Strauch 2014). Despite this, OCTR remains the golden standard method of CTR surgery; this may be because ECTR surgery requires greater surgical skill. However, ECTR has been associated with quicker short-term

144

E. MacDonald and P. M. Rea

improvement due to the minimally invasive technique. There is a reduced risk of scar complications and pain as the overlying tissue is protected and kept intact. A meta-analysis by Thoma and colleagues found that those who underwent ECTR surgery experienced quicker recovery time and greater functionality improvement when compared to the OCTR technique (Thoma et al. 2004). Conversely, ECTR can carry a risk of short-term nerve damage and complication as it requires greater technical ability. A meta-analysis by Vasiliadis and colleagues suggests that ECTR may also lead to incomplete release of the TCL, leading to recurring CTS and a need for repeat surgery (Vasiliadis et al. 2015). Nonetheless, endoscopic surgery for carpal tunnel release has become increasingly popular, particularly in the USA (Michelotti et al. 2018).

7.1.3

Aims and Objectives

There is much debate between clinicians and surgeons as to which surgical technique is superior for the release of CTS. There is the possibility that technical ability and expertise may be a reason why studies comparing endoscopic and open carpal tunnel release surgery report varying patient outcomes. For this reason, it is of clinical significance to investigate patient outcomes following surgical CTR in randomised control trials (RCTs) to fully understand the advantages and disadvantages of each intervention. This information will aid clinicians in determining whether there is an intervention which provides patients with a quicker recovery time, reduced complication risk and better symptom relief to ensure that treatment remains in the patient’s best interest. It is important to review the most recent publications to determine this, and as such to the best of the author’s knowledge, this systematic review comparing OCTR to ECTR is the most recent since Vasiliadis et al. (2015). This systematic review aimed to evaluate patient outcomes following open and endoscopic carpal tunnel release surgery in RCTs to gain an overall understanding of the efficacy and safety of

each intervention. There was a particular focus on which intervention provides patients with the faster recovery time, greatest improvement to symptom severity and grip strength and less severe complications.

7.2 7.2.1

Methods Study Identification

Eligible literature was identified and screened for inclusion in this systematic review. Electronic searches on the WebOfScience (January 1864– October 2020) and the Cochrane Central Register of Controlled Trials (CENTRAL) (The Cochrane Library 2020, issue 10) databases were carried out in October 2020 to identify potential literature. The search terms used on the databases were as follows: ‘carpal tunnel’ and (‘open or endoscopic’ or ‘endoscopic versus open’) and ‘carpal tunnel’ and (open or endoscopic) and with a combination of the specific searches ‘versus’ and ‘randomised’; limitation to publication date and participant demographics, such as age or occupation, were not applied. There was no limitation to publication date as this study aimed to gain understanding from all control trials looking at open and endoscopic CTR since the surgical techniques were first used. The reference lists from other review articles looking at both the surgical techniques were manually crossreferenced to identify any potential publications which may have been missed in the electronic database searching.

7.2.2

Study Screening and Selection

When screening the titles and abstracts on the publications found from the database searches and manual cross-referencing, an inclusion and exclusion criteria was applied to ensure that only suitable literature was included in this systematic review. Any studies that did not comply with the inclusion criteria were excluded at this stage in the study selection process. The following inclusion criteria were used:

7

A Systematic Review of Randomised Control Trials Evaluating the Efficacy. . .

• Studies had to compare ECTR (double- or single-portal technique) and OCTR (limited or standard technique), as these were the standard variations of each technique, in a RCT design. This allowed for standardisation and equal methodological comparison between studies. • Studies had to be written in the English language and in the form of a full report for equal comparison and sufficient evaluation of the quality of the studies. • The studies had to include participants who had a clinical diagnosis of CTS; however it was not necessary if this was achieved by nerve conduction studies.

145

outcomes in the short term (
12 weeks followup) and in the long term (>12 weeks follow-up). Consequently, these were the intervals that were used in this systematic review to evaluate patient outcomes. Outcomes Assessed The primary outcome that was assessed in this systematic review was the overall improvement to the CTS symptom severity that patients were experiencing prior to CTR surgery. The secondary outcomes that were assessed in this review were regain of grip strength, time taken to return to work (RTW) or activities of daily living (ADL) and incidence rate and severity of complications following CTR surgery.

The following exclusion criteria were used: • As this study aimed to evaluate the evidence from primary literature, review articles were excluded. • Publications needed to be written as a full text, so abstract, letter, commentary and meetingtype articles were excluded as to allow for equal comparison. • Studies that compared OCTR/ECTR with another surgical technique such as the KnifeLight® technique or a non-surgical technique were excluded as patient outcomes could not be accurately compared. There was no restriction on the publication date as this study aimed to carry out a comprehensive investigation on all literature comparing the interventions since they were first used. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flowchart displays the screening and selection process for included studies (Fig. 6) (Moher et al. 2009).

7.2.3

Assessment of Patient Outcomes

Several patient outcomes were pre-specified and evaluated in order to gain an overall understanding of the efficacy and safety of each intervention. The majority of studies investigated patient

Assessing these outcomes allowed for interpretation of the results from the included studies to reach an overall conclusion as to which CTR technique was best in terms of symptom severity reduction, grip strength regain, faster recovery time and less severe complications.

7.2.4

Risk of Bias Assessment

To evaluate the quality of the included studies, several domains were assessed to identify any areas where a study may have introduced bias. The risk of bias was assessed in accordance with the Revised Cochrane risk-of-bias tool for randomised trials as described in Chap. 8 of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins et al. 2020). The domains were as follows: Allocation sequence bias was assessing whether the allocation sequence was designed to allow for sufficient randomisation. Low risk of bias was determined if the allocation sequence was adequately outlined and was considered to be unpredictable to both participants and surgeons as to which group participants would be assigned to. Unclear risk of bias was determined if there was no mention of the allocation sequence or if it was not sufficiently outlined. High risk of bias was determined if the allocation sequence did not allow for sufficient

146

randomisation or was considered predictable as to which intervention group participants would be assigned to. Allocation concealment bias was assessing if the allocation sequence was adequately concealed once participants had been assigned to each intervention group. Low risk of bias was determined if the allocation concealment method was sufficient and clearly stated to ensure participants and investigators did not know the group allocation. Unclear risk of bias was determined if there was no mention of the method used to conceal allocation. High risk of bias was determined if allocation concealment was not sufficient, and participants were aware of what group they had been assigned to. Participant group characteristics bias was assessing if the intervention groups were of equal characteristics and symptoms at baseline. Low risk of bias was determined if the intervention groups were of equal characteristics at baseline. Unclear risk of bias was determined if the characteristics of the intervention groups were not reported. High risk of bias was determined if the groups were not of equal characteristics at baseline and if any differences were not compatible with chance. Blinding bias was assessing whether the outcome assessor was blinded to which intervention group each participant was allocated to. Low risk of bias was determined if the outcome assessor was blinded. Unclear risk of bias was determined if there was no mention of whether the outcome assessor was blinded or not. High risk of bias was determined if the outcome assessor was not blinded and was aware of the technique each participant received. Blinding of both the surgeons and participants was not possible as surgeons were aware of which surgical technique they were performing and the participants were aware of the incision size meaning that the intervention technique which they received may have been obvious. Missing participant outcome data bias was assessing whether there was any participant data missing for any of the outcomes measured. Low risk of bias was determined if all, or nearly all, participant data from each outcome was reported;

E. MacDonald and P. M. Rea

any missing data should not have a significant effect on the results. Unclear risk of bias was determined if there was no mention or explanation of any missing participant data. High risk of bias was a lot of missing participant data missing, and there was not any evidence to prove that this missing data was unbiased to the overall results. Selective reporting bias was assessing whether there was selective reporting of the pre-specified outcomes that were outlined in the study methods. Low risk of bias was determined if measurements of all the pre-specified outcomes were reported at the pre-specified timepoints. Unclear risk of bias was determined if the methods did not pre-specify the outcomes that were to be measured and if the timepoints were not measured. High risk of bias was determined if there was missing data for most or all of the pre-specified outcomes at the pre-specified timepoints. Conflict of interest and funding bias was assessing whether the study funding came from a commercial entity that may have led to biased results. Low risk of bias was determined if the author declared there was no conflict of interests or if the trial funding was not from a commercial entity. Unclear risk of bias was determined if there was no mention of any conflict of interests. High risk of bias was determined if the authors declared there was conflict of interest and if the trial funding was from a commercial entity that may have led to biased results. The domains were determined as either low risk of bias (+), unclear risk of bias (?) or high risk of bias ().

7.2.5

Data Analysis

When reported in studies, information relating to participant demographics and individual study characteristics was extracted and presented in Table 7.1 (Table of Individual Participant and Study Characteristics). Information about participant demographics included factors such as average age, sex, whether nerve conduction testing was used to achieve a clinical diagnosis and the number of CTR procedures that were carried out

7

A Systematic Review of Randomised Control Trials Evaluating the Efficacy. . .

in each study. Information about the individual study characteristics included the follow-up intervals and how many participants were assigned to each group. When reported in studies, information about the patient short- and longterm outcomes was extracted and presented in Table 7.2 (Table of Participant Outcome Assessment). Data regarding overall symptom severity, grip strength regain, time taken to RTW/ADL and complications was assessed to gain an overall understanding of the aim of this study. Information regarding the assessment of the risk of bias in each individual study was extracted and presented in Table 7.3 (Table of Individual Study Bias Assessment).

7.3 7.3.1

Results Study Identification, Screening and Inclusion

The identification, screening and inclusion process is presented in the PRISMA flowchart (Fig. 7.1) depicted below: The electronic searches identified publications which could be screened for potential inclusion in this systematic review. The searches resulted in 177 publications identified from the CENTRAL database, 240 from the WebOfScience database and an additional seven publications from manually searching the reference lists of eligible articles and reviews, resulting in a total of 424 publications which titles seemed eligible for this review. Duplications were then removed leaving a total of 229 publications remaining for the screening process; it is at this point where the inclusion and exclusion criteria was applied. Eighty-two publications that were in the form of commentary, review, letter or abstract only were excluded. Further six cadaveric/laboratory studies were excluded along with 55 publications that investigated non-surgical treatments for CTS. Twenty-three publications were excluded for not being written in the English language, leaving a total of 65 publications for further eligibility screening.

147

The titles and abstracts of these publications were further screened to ensure they met the inclusion criteria. Thirty-one publications were excluded for not being in a RCT design leaving 34 publications for potential inclusion. After carefully considering these articles, five were excluded for comparing OCTR/ECTR to alternative surgical techniques. A further six were excluded for only comparing OCTR techniques without including ECTR technique. Finally, a total of 23 studies were eligible for inclusion in this systematic review.

7.3.2

Study Characteristics

Included Studies Information about the participant demographics and individual study characteristics is presented in Appendix 1 (Table of Individual Participant and Study Characteristics). Twenty-three studies that compared open and endoscopic carpal tunnel release were included in this systematic review. There were a total of 2147 CTR procedures: 1050 of these were OCTR surgery and 1097 were ECTR surgery. Nine studies assessed participants who only had unilateral CTS, four studies only included participants with bilateral CTS, and ten of the studies with some participants had bilateral CTS but not all. Eighteen studies accepted participants with CTS that had been clinically confirmed by nerve conduction testing, and five studies accepted participants with CTS which had been clinically confirmed with or without nerve conduction testing. Excluded Studies Eleven studies were excluded after comprehensive consideration and assessment of the titles and abstracts. Six studies were excluded as only variations of OCTR were investigated without comparing to ECTR. Five studies were also excluded due to comparing OCTR or ECTR to alternative surgical techniques: three studies compared standard OCTR with KnifeLight® surgery, one study compared standard OCTR to ulnar-L incision OCTR and one study compared transverse mini-incisions to standard OCTR.

148

E. MacDonald and P. M. Rea

177 publications identified from CENTRAL database.

240 publications identified from WebOfScience database.

7 publications identified through manual cross referencing.

229 publications remaining after duplicates were removed.

Inclusion/Exclusion Criteria Applied

166 publications excluded after title/abstract screening: 82 due to being commentary, abstract, letter, or review; 6 due to laboratory or cadaveric study design; 55 for comparing non-surgical techniques; 23 because the original text was not written in English.

65 full-text articles remaining to be screened for potential eligibility.

Inclusion/Exclusion Criteria Applied

42 publications excluded after further analysis: 31 due to non-randomised study design; 5 due to inclusion of alternative surgical techniques; 6 for not including ECTR.

23 studies included in systematic review.

Fig. 7.1 PRISMA Flow diagram showing the study selection process. This flow diagram shows how eligible studies were identified, screened and selected for inclusion in this systematic review. (Figure adapted from Moher et al. 2009)

7.3.3

Patient Outcomes

The patient outcomes in each individual study are presented in Appendix 2 (Table of Participant Outcome Assessment). Symptom Severity Improvement Symptom severity improvement took into consideration any improvement relating to pain, numbness and/or functionality. In the 17 studies that investigated improvement to symptom severity in the short term, all reported that there was a reduction in overall symptom severity compared to the participants’ preoperative status in both the

ECTR and OCTR groups. Nine of these studies reported that there was a greater and quicker reduction to symptom severity in the ECTR group compared to the OCTR group. In seven of the studies, both the ECTR and OCTR groups showed significant improvement in the short term, and there was no significant difference between the groups. Only in one study did the OCTR have greater reduction to symptom severity when compared to the ECTR group. Long-term symptom severity reduction was investigated in 15 of the studies. Similarly to the short-term investigation, both groups showed

7

A Systematic Review of Randomised Control Trials Evaluating the Efficacy. . .

improvement to symptom severity in all the studies. Eleven of the studies reported that, by the time of final follow-up, there were no significant differences between the symptom severity improvement in both groups. Three studies showed that the ECTR group had significantly greater improvement to the symptom severity compared to the OCTR group. One study reported that OCTR had greater improvement to symptom severity when compared to the ECTR group. Grip Strength Regain Fourteen studies investigated short-term grip strength regain, all of which showed that there was a significant improvement to grip strength compared to the participants’ preoperative strength in both intervention groups. Thirteen of the studies found that the ECTR intervention group regained grip strength at a quicker rate and returned to their preoperative grip strength faster than the OCTR group. Only one study found the OCTR group to have quicker regain of grip strength compared to the ECTR group. The long-term regain of grip strength was investigated in nine of the studies. In all nine of the studies, significant improvement to grip strength was reported in both intervention groups. In four of the studies, the ECTR group was found to still have significantly greater grip strength compared to the OCTR group at the time of final follow-up. In five of the studies, there were no significant differences between the two intervention groups regarding grip strength at the time of final follow-up. Time Taken to RTW/ADL Twelve of the studies investigated the average time taken to RTW/ADL. Overall, the ECTR group had a quicker RTW/ADL time compared to the OCTR group. The ECTR group had an average RTW/ADL time of 17 days with a range on 7–28 days. The OCTR group had an average RTW/ADL time of 25 days with a range of 16–46 days.

149

Complication Incidence Rate and Severity The OCTR group had a complication rate of 4.1% (43/1050 procedures). The most common complications that were found in the OCTR group were neurovascular damage, serious wound infections, reflex sympathetic dystrophy1 and pillar pain.2 The ECTR group had a complication rate of 5% (55/1097 procedures). The most common complications observed in the ECTR group was transient nerve damage and persistent CTS symptoms which required repeat surgery via OCTR.

7.3.4

Risk of Bias Assessment

The assessment of the risk of bias in each individual study can be seen in Appendix 3 (Table of Individual Study Bias Assessment). Allocation Sequence Bias Seventeen of the studies had a low risk of allocation sequence bias as it was adequately described and believed to provide studies with sufficient randomisation. None of the studies had a high risk of allocation sequence bias, and six of the studies had an unclear risk of allocation sequence as the allocation sequence used for randomisation was not described so it remained unclear how successful randomisation was. Allocation Concealment Bias Six of the studies had a low risk of allocation concealment bias, in that the allocation concealment was adequately designed in a way that participants could not predict which intervention group they were assigned to. Six of the studies had a high risk of allocation concealment bias in that the allocation concealment was designed in a way that participants could predict which groups they were assigned to, and 11 of the studies had an unclear risk of allocation concealment bias in 1 2

Exacerbated and prolonged pain. Pain in the thenar and hypothenar eminence.

150

that the method of allocation concealment was not described sufficiently. Intervention Group Characteristics Bias Twenty-one of the studies had a low risk of bias regarding the intervention group characteristics, in that the participant groups were not significantly different and had equal baseline demographics. None of the studies had a high risk of intervention group characteristics bias, and two of the studies had an unclear risk of bias regarding the intervention group characteristics as the demographics of the participant groups were not reported in the study so significant differences could not be determined. Blinding of the Outcome Assessor Bias The outcome assessor was blinded to which intervention each participant received in ten of the studies, resulting in a low risk of bias. In two of the studies, the outcome assessor was not blinded, resulting in a high risk of bias. Finally, in 11 of the studies, blinding of the outcome assessor was not mentioned, resulting in an unclear risk of bias. Missing Participant Outcome Data Bias In seven of the studies, the authors reported that any missing outcome data was not significant to the overall study results, resulting in a low risk of bias. One of the studies had a high risk of bias regarding missing participant data as this significantly affected the overall study results. An unclear risk of bias was determined in 15 of the studies as there was no mention if any missing participant data was significant to the overall results or not. Selective Reporting Bias Twenty of the studies reported results for all pre-specified outcomes and timepoints, resulting in a low risk of bias. Three of the studies had high risk of selective reporting bias in that not all the pre-selected outcomes had measurements reported for each timepoint mentioned in the study outline, and none of the studies had unclear risk of selective reporting bias.

E. MacDonald and P. M. Rea

Conflict of Interest and Funding In 14 of the studies, the authors declared there was no conflict of interest present regarding the study funding or any other factor, resulting in a low risk of bias. One of the studies had high risk of conflict-of-interest bias in that the authors declared conflict of interest but did not specify any information about the study funding. Finally in eight of the studies, there was no information regarding funding or conflict of interest, meaning that an accurate assessment could not be made, resulting in an unclear risk of bias.

7.4

Discussion

For those suffering with severe CTS, surgical release of the TCL is an effective treatment method to relieve symptoms and improve functionality in the affected hand. CTR continues to be one of the most performed surgeries today; however, OCTR remains the gold standard technique as the efficacy and safety have already been proven (Venouziou and Kerasnoudis 2020). Since ECTR was first introduced by Chow in 1989, there has been a debate as to if this technique provides better patient outcomes compared to OCTR (Chow 1989). It remains unclear of a definitive answer as to which technique should be preferred and many meta-analyses and reviews have reached varying conclusions after comparing patient outcomes.

7.4.1

Main Findings

Symptom Severity Improvement Overall improvement to symptom severity was determined as any improvement to numbness, functionality and pain. It was found that ECTR provides patients to greater and quicker improvement to symptom severity in the short-term follow-up when compared to the OCTR intervention group. One study included in this systematic review found that at a 2-week follow-up, the ECTR group had significant reduction to the severity of symptoms compared to the OCTR

7

A Systematic Review of Randomised Control Trials Evaluating the Efficacy. . .

group; however, by a 4-week follow-up, there was no significant difference between the two groups (Mackenzie et al. 2000). Similarly, another study reported that 88% of the participants in the ECTR group had improvement to symptom severity, compared to the 27% of the participants in the OCTR group. However, the same study also found that there was no significant difference between the two groups at time of the final follow-up in relation to symptom severity improvement (Sennwald and Benedetti 1995). In this systematic review, it was also found that overall, both groups had shown significant improvement to symptom severity from a 12-week follow-up, and by the time of the final follow-up, there was no significant difference between the two groups. Only few studies found the ECTR intervention group to have greater strength at the time of the final follow-up rather than both groups being of equal improvement. A meta-analysis by Li et al. (2020) found that the quicker and more significant short-term improvement to symptom severity in the endoscopic group was due to the reduced scar tenderness and greater patient satisfaction. The endoscopic method means that the overlying palmar fascia, muscles and tissue remain intact which results in minimal scarring and reduced scar tenderness when compared to the open intervention group (Malhotra et al. 2007). The shortterm quicker improvement shown in the endoscopic group correlates with the quicker return to work time also observed in the endoscopic group; greater improvement to pain and functionality allows participants to have a faster recovery and thus return to work quicker than the open group. Grip Strength Regain Similar to the improvement to symptom severity, the ECTR group showed greater improvement to grip strength regain compared to the OCTR group in the short-term. Despite both groups showing significant improvement to grip strength regain in the short-term, the ECTR group returned to the preoperative grip strength value or stronger

151

quicker than the OCTR group. One study found the endoscopic group to return to the preoperative grip strength value at twenty-eight days followup, whereas the open group did not return to the preoperative value until ninety days follow-up (Erdmann et al. 1994). Long-term grip strength regain measurements showed that overall, both groups were significantly stronger after 12 weeks follow-up and by the time of final follow-up both groups had returned to their preoperative grip-strength value. The greater improvement to grip strength observed in the endoscopic group was expected as the palmar muscles and fascia remain intact unlike the open technique where they are divided (Palmer et al. 1993). Similarly, short-term grip strength regain was greater in the endoscopic group than the open group in a meta-analysis carried out by Thoma et al. (2004). This quicker improvement to grip strength in the short-term in the endoscopic group may be a reason why participants show quicker improvement to functionality and consequently return to work quicker than the open group. Time Taken to RTW/ADL Twelve studies investigated the time taken to RTW/ADL following CTR surgery. Overall they showed that on average, the endoscopic group returned quicker than the open group. The average RTW time for the OCTR group was 25 days (range 19–46 days) and the average return to work time for the ECTR group was 17 days (range 7–28 days). The overlap in the range of data suggests that there may have been some cases in the OCTR group that returned to work quicker than the ECTR group. However, overall, the open group returned to work/ADL on an average of 8 days slower than the endoscopic group. In both intervention groups, all participants returned to work/ADL in the shortterm follow-up and by 7 weeks follow-up, all participants measured in the studies had sufficient improvement to RTW/ADL. RTW/ADL time can be affected by socioeconomic, psychological, clinical, and work-related factors which may

152

explain why there is a wide range of average days taken to return to work found in this review; further investigation into the participant demographics and average time taken to RTW/ADL would be needed to consolidate this (Sanati et al. 2011). Many studies have investigated the effects of CTR surgery on the time taken for participants to RTW/ADL. A systematic review comparing OCTR and ECTR also found that the endoscopic group returned to work/ADL 8 days quicker than the open group (Vasiliadis et al. 2015). Similarly, Sanati et al. (2011) also found ECTR surgery to provide patients with quicker RTW times compared to open surgery; these findings may be significantly favourable for patients who may be self-employed or would lose out on income due to being absent from work. Conversely, Thoma et al. (2004) and Gerritsen et al. (2001) carried out a meta-analysis and systematic review, respectively, and found that there was no significant evidence to suggest that ECTR surgery provided patients with a quicker RTW/ADL time. Interestingly, one study found that the time taken to RTW was associated with the occupation the participant did along with the working conditions. Those in occupations that required repetitive wrist movements such like typing or heavy manual labour such like a construction worker, returned to work much later than those who did not have these type of occupations (De Kesel, Donceel and De Smet 2008). Complication Incidence Rate and Severity Rate of complication was considerably low in both groups with OCTR having a marginally lower complication rate. When only taking complication rate into account, OCTR could be considered safer; however, the open group had a higher risk of complications that were prolonged over thr3ee months or were only reversible through treatment. The OCTR group showed an increased risk of severe wound and scar infections which, in some studies, could only be treated

E. MacDonald and P. M. Rea

through debridement or IV antibiotics (Gurpinar et al. 2019). The open intervention group reported an increased risk of reflex sympathetic dystrophy in the thenar and hypothenar muscles. It is suggested that the large incision through the palmar and thumb muscles is a reason for there to be increased pain in these regions (Malhotra et al. 2007). Another complication that was observed to be more severe in the open group was pillar pain; one study reported that it took 6 months for the pillar pain to completely disappear in several participants (Dumontier et al. 1995). Overall, it was found that the ECTR intervention group reported less severe complications despite showing an increased need for repeat surgery due to the persistent CTS symptoms. One of the most frequent complication to be reported in the ECTR group was reversible nerve damage which resulted in pain and numbness; in most cases experiencing this, the damage resolved spontaneously by 2 months follow-up. It was found in this systematic review that there were four cases in the ECTR group that had persistent symptoms which required repeat surgery via OCTR; whilst on the other hand, the OCTR group had zero cases requiring repeat surgery. Scholten et al. (2007) carried out a metaanalysis which analysed the data from RCTs which compared patient outcomes from OCTR and other variations of CTR surgery; similar to this systematic review, they found both OCTR and ECTR intervention techniques to have low complication rates. They also found OCTR to leave patients with more severe complications such like severe wound infections and reflex sympathetic dystrophy. Scholten et al. (2007) also found that there was a higher risk of repeat surgery needed in participants who underwent ECTR surgery. A systematic review by Thoma et al. (2004) found that there was a higher incidence rate of reversible nerve damage in the ECTR group; this was similar to the findings in this systematic review where any nerve damage was reversible and resolved by 2 months follow-up.

7

A Systematic Review of Randomised Control Trials Evaluating the Efficacy. . .

7.4.2

Study Quality

None of the domains assessing study quality resulted in a majority high risk of bias, three of the domains had a majority unclear risk of bias, and four of the domains has a majority low risk of bias. Consequently, the overall risk of bias was low meaning that the quality of the studies included in this review was moderate. The studies were considered to have an overall low risk of bias regarding allocation sequence, as this was adequately described in the majority of studies. Intervention groups were also found to have equal baseline characteristics and symptoms overall, also resulting in an overall risk of bias regarding the intervention groups characteristics. In contrast, a clear judgement could not be made regarding allocation concealment bias due to the majority of studies not mentioning the method of allocation concealment. As scars were obvious following the surgery and surgeons had to be aware of the intervention technique they were applying, blinding of the participants and surgeons could not be done. However, scars and details could be concealed from the individual assessing the patient outcomes, meaning that blinding of the outcome assessor could be achieved. This, however, was not clearly mentioned in the majority of studies, meaning that there was an unclear risk of bias overall regarding outcome assessor blinding bias. Despite all studies having missing participant data, the majority of them did not mention the significance and implications of missing data to the overall study results. This meant that there was an overall unclear risk of bias associated with missing participant outcome data. The majority of studies reported outcome results for each pre-specified outcome at the pre-specified timepoints which were outlined in the study methods. This resulted in an overall low risk of bias associated with selective reporting of outcome data. Finally, the majority of study authors declared there were no conflicts of interest in terms of funding or any other factor meaning that there

153

was an overall low risk of bias regarding conflicts of interest.

7.4.3

Limitations

There are several limitations to this systematic review. Firstly, for this systematic review, it was not possible to gain access to literature that required payment leading to incomplete analysis of all restricted studies which may have met the inclusion/exclusion criteria. Secondly, there was no analysis carried out on the sub-groups of the variations to each intervention techniques, i.e. no comparisons were made between standard and limited OCTR or between single- and doubleportal ECTR. This was due to the small number of studies which investigated the effects of limited OCTR meaning that it was more beneficial to compare the effects OCTR and ECTR as whole groups. Nevertheless, this remains a limitation as investigating the effects of each variation could have reported different results compared to investigating the effects of the intervention groups as a whole. This provides potential for further research into comparing the outcomes between the variations of OCTR and ECTR. Thirdly, not all the studies evaluated the same outcomes, thus, leading to a reduced number of participants evaluate for each outcome compared to if all studies evaluated the same outcomes. As well as this, even when studies did measure the same outcomes, it was not necessarily at the same timepoints. Investigating the effects of both techniques on the same outcomes at the same timepoints, would ensure that studies were more comparable and there would be stronger evidence to support the results from the outcomes measured. Finally, there was no statistical analysis carried out on the overall data in this review; a meta-analysis on the data could have determined if the overall findings were statistically significant.

154

7.4.4

E. MacDonald and P. M. Rea

Conclusions

There is a quicker recovery time and less severe complications associate with minimally invasive surgery. The quickened recovery time and greater improvement to outcomes in the short-term observed in the endoscopic group is due to the reduced scar/palmar tenderness and minimal pain; suggesting there is an association between improved cosmetic results and quickened recovery time (Malhotra et al. 2007). The quickened recovery time, greater short-term improvement to grip strength, and less severe complications observed in the ECTR group suggests that it would be worthwhile for surgeons to adopt the ECTR technique more often; although, the ECTR technique was found to have an increased risk of needing repeat surgery. However, ECTR does require greater technical ability and specialised training to ensure that surgeons have sufficient surgical skill to operate the complex surgical instruments. On the other hand, OCTR does not require this specialised training and can be carried out by most qualified surgeons. There are financial factors associated with ECTR that need to be considered before implementing the technique; there is an increase cost due to the need for repeat surgery, speciality training, and specialised endoscopic surgical equipment. There is also an increased cost required for OCTR as it was found that there is an increased risk of complications requiring prolonged treatment (Brown et al. 1993). Cost implications, however, are out with the scope of this review and further research taking cost

effectiveness when comparing ECTR and OCTR is recommended. It has been found that there is a link between the length of time a patient suffers from CTS and the severity of nerve damage; the longer a patient suffers, the greater the nerve damage (Zamborsky et al. 2017). This suggests that it would be worthwhile to investigate the length of time a patient suffers from CTS and the participant outcomes following CTR surgery; there could be a surgical method which proves to be optimal for patients suffering from severe nerve damage. This would create a more comprehensive analysis to allow clinicians to understand all aspects of patient outcomes and satisfaction when deciding on the best CTR technique. Overall, the quickened recovery time and less severe complications observed in the ECTR group suggests that endoscopic carpal tunnel release surgery should be adopted more often in the UK. However, this review found no convincing evidence of a significantly superior technique in terms of grip strength regain and improvement to symptom severity in the long-term. Thus, the intervention used for CTR should be ultimately decided according to patient assessment and technical ability of the surgeon. These findings do allow clinicians to have a wider understanding of the potential patient outcomes and possible complications following CTR surgery. However, further investigation into CTR surgery in routine clinical practice would provide a more accurate understanding of the patient outcomes and complication rates within a clinical setting.

7

A Systematic Review of Randomised Control Trials Evaluating the Efficacy. . .

155

Appendices Appendix 1. Table of Individual Participant and Study Characteristics

Table 7.1 Table summarising the participant demographics, study characteristics, and follow-up intervals—1050 participants underwent OCTR, and 1097 participants underwent ECTR

Author(s) Agee et al. (1992)

Atroshi et al. (2006)

Atroshi et al. (2015)

Brown et al. (1993)

Dumontier et al. (1995)

Ejiri et al. (2012)

Erdmann (1994)

Ferdinand and Maclean (2002)

Gumustas et al. (2015)

Gurpinar et al. (2019)

Participant cohort characteristics • 122 participants and 147 CTR procedures. • Clinical diagnosis was confirmed with nerve conduction testing. • 128 participants. • Average age was 44 years old. • Clinical diagnosis was confirmed with nerve conduction testing. • 124 participants. • Average age was 57 years old. • Clinical diagnosis was confirmed with nerve conduction testing. • 145 participants and 169 CTR procedures. • The average age was 57 years old. • 99 females and 46 males. • Clinical diagnosis was confirmed with nerve conduction testing. • 69 participants. • 85 females and 11 males. • The average age was 52 years old. • Clinical diagnosis was confirmed with nerve conduction testing. • 79 participants and 101 CTR procedures. • 71 females and 8 males. • The average age was 59 years old. • Clinical diagnosis was confirmed with nerve conduction testing. • 71 participants and 105 CTR procedures. • The average age was 52 years old. • Clinical diagnosis was confirmed with nerve conduction testing. • 25 patients with bilateral CTS leading to 50 CTR procedures. • 20 females and 5 males. • The average age was 55 years old. • Clinical diagnosis was confirmed with nerve conduction testing. • 41 participants. • 39 females and 2 males. • The average age was 45 years old. • Clinical diagnosis was confirmed with nerve conduction testing. • 104 participants. • 72 females and 32 males. • The average age was 51 years old.

Number of participants in each group OCTR ¼ 82 ECTR ¼ 65

Follow-up intervals 1, 2, 3, 6, 9, 13, and 26 weeks.

OCTR ¼ 65 ECTR ¼ 63

3 and 6 weeks; 3 and 6 moths.

OCTR ¼ 61 ECTR ¼ 63

1 and 9 years.

OCTR ¼ 85 ECTR ¼ 84

3 and 6 weeks and 3 months.

OCTR ¼ 40 ECTR ¼ 56

2 weeks and 1, 3, and 6 months.

OCTR ¼ 50 ECTR ¼ 51

4 and 12 weeks.

OCTR ¼ 52 ECTR ¼ 53

1 and 2 weeks; 1, 3, and 6 months; and 1 year.

OCTR ¼ 25 ECTR ¼ 25

6, 12, and 26 weeks and 1 year.

OCTR ¼ 20 ECTR ¼ 21

6 months.

OCTR ¼ 50 ECTR ¼ 54

2, 6, and 12 weeks and 1 year. (continued)

156

E. MacDonald and P. M. Rea

Table 7.1 (continued)

Author(s)

Jacobsen and Rahme (1996)

Kang et al. (2013)

Larsen et al. (2013)

MacDermid et al. (2003)

Mackenzie et al. (2000) Malhotra et al. (2007)

Michelotti et al. (2018)

Oh et al. (2017)

Saw et al. (2003)

Sennwald and Benedetti, (1995)

Tian, Zhao, and Wang (2007)

Participant cohort characteristics • Clinical diagnosis was confirmed with nerve conduction testing. • 29 participants and 32 CTR procedures. • 21 females and 8 males. • Clinical diagnosis was confirmed with nerve conduction testing. • 52 participants all of which had bilateral CTS. • 48 females and 4 males. • The average age was 55 years old. • Clinical diagnosis was confirmed with nerve conduction testing. • 90 participants. • 64 females and 26 males. • The average age was 51 years old. • Clinical diagnosis was confirmed with nerve conduction testing. • 123 participants. • The average age was 49 years old. • Clinical diagnosis was confirmed with nerve conduction testing. • 26 participants and 36 CTR procedures. • Clinical diagnosis was confirmed with nerve conduction testing. • 60 participants and 61 CTR procedures. • 35 females and 25 males. • The average age was 45 years old. • Clinical diagnosis was confirmed with nerve conduction testing. • 30 participants with bilateral CTS, leading to 60 CTR procedures. • 25 females and 5 males. • The average age was 54 years old. • Clinical diagnosis was confirmed with or without nerve conduction testing. • 67 participants. • 57 females and 10 males. • The average age was 52 years old. • Clinical diagnosis was confirmed with nerve conduction testing. • 150 participants. • 110 females and 40 males. • The average age was 52 years old. • Nerve conduction testing was not necessary for clinical diagnosis. • 47 participants. • 37 females and 10 males. • The average age was 55 years old. • Clinical diagnosis was confirmed with never conduction testing. • 62 participants and 70 CTR procedures. • 46 females and 16 males. • The average age was 52 years old.

Number of participants in each group

Follow-up intervals

OCTR ¼ 16 ECTR ¼ 16

2 and 6 weeks and 6 months.

OCTR ¼ 52 ECTR ¼ 52

3 months.

OCTR ¼ 60 ECTR ¼ 30

1, 2, 3, 6, 12, and 24 weeks.

OCTR ¼ 32 ECTR ¼ 91

1, 6, and 12 weeks.

OCTR ¼ 22 ECTR ¼ 14

1, 2, and 4 weeks.

OCTR ¼ 31 ECTR ¼ 30

1 and 6 months.

OCTR ¼ 30 ECTR ¼ 30

2, 4, 8, 12, and 24 weeks.

OCTR ¼ 35 ECTR ¼ 32

24 weeks.

OCTR ¼ 76 ECTR ¼ 74

1, 3, 6, and 12 weeks.

OCTR ¼ 22 ECTR ¼ 25

4, 8, and 12 weeks.

OCTR ¼ 36 ECTR ¼ 34

18 and 48 months.

(continued)

7

A Systematic Review of Randomised Control Trials Evaluating the Efficacy. . .

157

Table 7.1 (continued)

Author(s)

Trumble et al. (2001)

Wong et al. (2003)

Participant cohort characteristics • Clinical diagnosis was confirmed with nerve conduction testing. • There were 147 participants and 192 CTR procedures. • 96 females and 51 males. • The average age was 56 years old. • Clinical diagnosis was confirmed with nerve conduction testing. • 30 participants with bilateral CTS, leading to 60 CTR procedures. • 28 females and 2 males. • The average age was 47 years old. • Clinical diagnosis was confirmed with nerve conduction testing.

Number of participants in each group

Follow-up intervals

OCTR ¼ 95 ECTR ¼ 97

2, 4, 8, 12, 26, and 52 weeks.

OCTR ¼ 30 ECTR ¼ 30

2, 3, 8, and 16 weeks and 6 and 12 months.

Appendix 2. Table of Participant Outcome Assessment Table 7.2 Table summarising assessment of the pre-specified outcomes from the 23 publications included in this systematic review—the table shows the evaluation of overall improvement to symptom severity, regain of grip strength, time taken to RTW/ADL, and complication incidence and severity

Author (s) Agee et al. (1992)

Atroshi et al. (2006)

Short-term participant outcomes (
12 weeks follow up) Overall symptom severity: Both intervention groups showed significant improvement. Grip strength regain: Strength was regained quicker in the endoscopic group compared to the open group.

Overall symptom severity: Functionality was significantly improved by 3-weeks follow-up in the endoscopic compared to the open group. However, by 3-months

Long-term participant outcomes (>12 weeks follow-up) Overall symptom severity: The endoscopic group had a significantly greater percentage of participants who had improvement by 26-weeks follow-up. Grip strength regain: Both groups had significantly improved grip strength compared to their preoperative status by 26-weeks follow-up, however, the ECTR group was significantly stronger than the OCTR group. Overall symptom severity: By 6-months follow-up, both intervention groups showed significant decrease in symptom severity.

Time taken to return to work/activities of daily living OCTR mean time: 46 days ECTR mean time: 17 days

OCTR mean time: 33 days ECTR mean time: 28 days

Complications OCTR: Two participants had severe nerve damage and two participants had wound infections. ECTR: Two participants had persistent symptoms of CTS which required repeat surgery.

There was no serious wound infections or neurovascular damage in any intervention groups. OCTR: Repeat surgery was required due to persistent symptoms (continued)

158

E. MacDonald and P. M. Rea

Table 7.2 (continued)

Author (s)

Atroshi et al. (2015)

Brown et al. (1993)

Dumontier et al. (1995)

Short-term participant outcomes (
12 weeks follow up) follow-up, 71% of participants in the endoscopic group reported the absence of pain and numbness compared to 72% in the open group. Grip strength regain: The endoscopic group showed greater and faster improvement to strength regain. However, both groups had returned to their preoperative strength by 3-months follow-up. No data provided.

Overall symptom severity: Both groups had seen significant reduction to symptom severity at 3-weeks follow-up. This improvement continued significantly in the endoscopic group. Grip strength regain: The endoscopic group had significantly faster and greater improvement to grip strength compared to the open group at all the follow-up intervals. Overall symptom severity: By 3-months follow-up, 38% of the endoscopic group presented with pillar pain, compared to the 43% in the open group. Grip strength regain: The endoscopic group had significantly quicker

Long-term participant outcomes (>12 weeks follow-up)

Time taken to return to work/activities of daily living

Complications was required in one participant. ECTR: Repeat surgery was required in two participants due to persistent CTS symptoms.

Overall symptom severity: There was significant improvement to symptom severity in both groups from baseline to 1-year follow-up. However, there was no significant improvement between 1-year and 9-year follow-up. No data provided.

No data provided.

No data provided.

OCTR mean time: 28 days. ECTR mean time: 14 days.

OCTR: One participant experienced severe nerve damage which required ongoing treatment and resulted in reflex sympathetic dystrophy. ECTR: Four participants experienced minor neurovascular damage; however, this was not serious and resolved itself by the short-term follow-up interval.

Overall symptom severity: Pillar pain completely resolved by 6 months follow-up. Grip strength regain: By the time final followup, the endoscopic group had greater grip strength and both groups

Participants in the OCTR group took longer to return to work compared to the endoscopic group.

OCTR: Seventeen participants experienced pillar pain and two participants developed reflex sympathetic dystrophy. ECTR: Twenty-one participants experienced pillar pain and two participants (continued)

7

A Systematic Review of Randomised Control Trials Evaluating the Efficacy. . .

159

Table 7.2 (continued)

Author (s)

Ejiri et al. (2012)

Erdmann (1994)

Ferdinand and Maclean. (2002)

Gumustas et al. (2015)

Gurpinar et al. (2019)

Short-term participant outcomes (
12 weeks follow up)

Long-term participant outcomes (>12 weeks follow-up)

regain to grip strength compared to the open group. Overall symptom severity: There was significant improvement to symptom severity in both groups at 12-weeks follow-up, however there was greater improvement in the open group compared to the endoscopic group. Grip strength regain: The endoscopic group had significantly greater improvement to grip strength and by 28 days, the group returned to their preoperative strength.

had significantly improved.

Overall symptom severity: The endoscopic group had greater improvement to symptom severity compared to the open group. Grip strength regain: The endoscopic group had greater and quicker improvement to grip strength up until 12-weeks follow-up. No data provided.

Overall symptom severity: Both groups had significant

Time taken to return to work/activities of daily living

Complications developed reflex sympathetic dystrophy.

No data provided.

No data provided.

ECTR: Two participants developed long-term nerve damage which was still present after 1-year post operation.

Grip strength regain: By 90-days follow-up, the open group had returned to their preoperative value. By the time of final follow up, both groups had significant improvement to grip strength, however, the endoscopic group had stronger grip strength.

OCTR mean time: 39 days. ECTR mean time: 14 days.

Overall symptom severity: At the time of final follow-up, the endoscopic group still had greater improvement to symptom severity compared to the open group. Grip strength regain: There was no significant difference between the two groups. Overall symptom severity: At the time of final follow-up, both groups had significant improvement to symptom severity, however there was no significant difference between the two groups. Overall symptom severity: At 6-months follow-up, the OCTR

No data provided.

OCTR: One participant experienced nerve damage and one participant experienced a severe wound infection. Scar tenderness was more frequent in the open group compared to the endoscopic group. 13.5% of participants experienced complications. ECTR: 3.7% of participants experience complications. OCTR: One participant experienced persistent CTS symptoms and two participants experienced wound infections. ECTR: Persistent wound infections were observed in two participants.

No data provided.

OCTR: One participant developed a wound infection. ECTR: One participant experienced complications during surgery which required open surgery.

OCTR mean time: 22 days

OCTR: One participant experienced a severe wound complication (continued)

160

E. MacDonald and P. M. Rea

Table 7.2 (continued) Short-term participant outcomes (
12 weeks follow up)

Long-term participant outcomes (>12 weeks follow-up)

Time taken to return to work/activities of daily living

improvement to grip strength, however, there was no significant difference between the two groups.

group had greater pain rates than the ECTR group.

ECTR mean time: 18 days

Jacobsen and Rahme (1996)

Overall symptom severity: Both groups showed significant improvement to symptom severity, however, there was no significant differences between the two groups.

Overall symptom severity: By the time of final follow-up there was no significant differences between the two groups.

OCTR mean time: 19 days ECTR mean time: 17 days

Kang et al. (2013)

Overall symptom severity: The endoscopic technique was preferred by 65% of participants by 12-weeks follow-up. Overall symptom severity: Both groups had improvement to symptom severity at a similar rate. Grip strength regain: At 1-, 2-, and 3-weeks follow-up, the endoscopic group had quicker return to grip strength compared to the open group. By 6- and 12- weeks follow-up both groups showed significant improvement to grip strength. Overall symptom severity: Both intervention groups seen steady and similar improvement to symptom severity. Grip strength regain: At 1- and 6-weeks follow-up, the endoscopic group had

No data provided.

No data provided.

Grip strength regain: There was no significant difference between the two groups at the time of final follow-up.

OCTR mean time: 20 days ECTR mean time: 7 days

No data provided.

No data provided.

Author (s)

Larsen et al. (2013)

MacDermid et al. (2003)

Complications which required debridement and intravenous antibiotics. ECTR: Three participants experienced nerve complications, one of which resulted in prolonged and exacerbated pain where they required OCTR after 3-months post operation. OCTR: One participant experienced a wound infection. ECTR: Three participants developed temporary numbness and compromised nerve function; this healed at 2-weeks follow-up. OCTR: Participants experienced scar and pillar pain in the hand that was treated with open surgery. OCTR: Eleven participants developed prolonged pillar pain. Two participants developed superficial wound infections. ECTR: Four participants developed prolonged pillar pain. Two participants experienced transient ulnar nerve damage; however, this was not permanent and had resolved by the final follow-up. ECTR: CTS symptoms were persistent in four participants which required repeat surgery.

(continued)

7

A Systematic Review of Randomised Control Trials Evaluating the Efficacy. . .

161

Table 7.2 (continued)

Author (s)

Mackenzie et al. (2000)

Malhotra et al. 2007

Michelotti et al. (2018)

Short-term participant outcomes (
12 weeks follow up) significantly stronger and quicker improvement to grip strength compared to the open group. By 12-weeks follow-up, both groups were at a stronger grip strength than their preoperative status. Overall symptom severity: At 2-weeks follow-up, the endoscopic group had greater improvement to symptom severity compared to the open group, however, there was no significant difference between the two groups at 4-weeks follow-up. Grip strength regain: At 2-weeks follow-up, the grip strength of the endoscopic group was 97% of the preoperative status, compared to the 74% of the preoperative status in the open group. Overall symptom severity: By 3-months follow-up, 93% of participants had reduction in pain.

Overall symptom severity: The endoscopic group had significant improvement to symptom severity compared to the open group at 2-weeks follow-up. Symptom severity continued to improve at a similar rate

Long-term participant outcomes (>12 weeks follow-up)

Time taken to return to work/activities of daily living

No data provided.

No data provided.

Pillar pain was observed in one participant in both groups.

Overall symptom severity: By the time of final follow-up, all participants had relief from pain. 77% of the ECTR group had reduction in symptoms compared to the 67% in the OCTR group. Grip strength regain: Both groups showed significant improvement to grip strength by the time of final follow-up. Overall symptom severity: By 24 weeks follow-up, there was no significant difference between the two groups however, they both had improvement to symptom severity. Grip strength regain: By the time of final

OCTR mean time: 20 days ECTR mean time: 16 days

OCTR: Reflex sympathetic dystrophy developed in two participants.

No data provided as patients underwent on their other hand 1-minth after their first CTR procedure.

No serious complications were reported.

Complications

(continued)

162

E. MacDonald and P. M. Rea

Table 7.2 (continued)

Author (s)

Oh et al. (2017)

Saw et al. (2003)

Sennwald and Benedetti (1995)

Short-term participant outcomes (
12 weeks follow up)

Long-term participant outcomes (>12 weeks follow-up)

in both groups after 2-weeks follow-up. Grip strength regain: The ECTR group showed greater and quicker improvement to grip strength and by 12-weeks follow-up, the average grip strength was stronger than the average preoperative value. In contrast, No data provided.

follow-up, the open group remained under their preoperative value.

Overall symptom severity: Both groups improved symptom severity at a similar rate. Grip strength regain: The endoscopic group had regained grip strength at a quicker rate than the open group. By 12-weeks follow-up, both groups had returned to a grip strength which was similar to their preoperative status. Overall symptom severity: By 1-month follow-up, 88% of the participants in the ECTR group had complete reduction to symptom severity, compared to the 27% in the OCTR group. Grip strength regain: The endoscopic group had significantly quicker improvement to grip strength compared to the open group. Both groups seen a significant improvement to grip strength by 12-weeks follow-up.

Overall symptom severity: Both groups had significant improvement to symptom severity by the time of follow-up at 24-weeks. No data provided.

No data provided.

Time taken to return to work/activities of daily living

Complications

No data provided.

No participants required revision surgery and there were no serious complications reported.

OCTR mean time: 26 days ECTR mean time: 18 days

No serious and permanent neurovascular complications were reported. OCTR: One participant developed a wound infection and one participant had persistent symptoms. ECTR: One participant developed a wound infection.

OCTR mean time: 42 days ECTR mean time: 24 days

No serious nerve complications were observed in each intervention group. OCTR: One participant experienced exacerbated scar pain. One participant developed reflex sympathetic dystrophy.

(continued)

7

A Systematic Review of Randomised Control Trials Evaluating the Efficacy. . .

163

Table 7.2 (continued)

Author (s) Tian, Zhao, and Wang (2007)

Trumble et al. (2001)

Wong et al. (2003)

Short-term participant outcomes (
12 weeks follow up) No data provided.

Overall symptom severity: The endoscopic group had significantly lower CTS severity score. Grip strength regain: Compared to the OCTR group, the endoscopic group regained grip strength at a significantly quicker rate. Grip strength regain: By 3-months follow-up, grip strength was regained to the preoperative value. Those who were in the OCTR group had marginally stronger grip strength than those in the ECTR group.

Long-term participant outcomes (>12 weeks follow-up) Overall symptom severity: There was no significant difference between the two groups by the time of final follow-up, however, the endoscopic group did improve at a quicker rate than the open group. Grip strength regain: There was no significant difference between the two groups in the longterm follow-up, although they both had improvement from their preoperative status. Overall symptom severity: By the time of final follow-up, three was no significant difference between the two intervention groups. Grip strength regain: At the time of final follow-up, both groups had regained 1 kg grip strength from their preoperative status. Overall symptom severity: After 12-months follow-up – Within the OCTR group: 63% had complete reduction of symptoms, 27% had some reduction in symptoms and 10% had no change to their symptoms. Within the ECTR group: 57% had complete reduction of symptoms, 33% some reduction in symptoms, and 10% had no change to their symptoms.

Time taken to return to work/activities of daily living OCTR mean time: 28 days ECTR mean time: 12 days

Complications Scar tenderness was reported in both groups, but this improved by 3-months follow-up. Otherwise, there were no serious complications reported in each intervention group.

OCTR mean time: 38 days ECTR mean time: 18 days

No neurovascular damage was reported in either intervention group. OCTR: One participant required revision surgery and one participant developed reflex sympathetic dystrophy.

No data provided.

No persistent symptoms of CTS were reported and there were no serious complications reported in either group.

Allocation sequence ? Allocation sequence was not adequately outlined.

+ Block randomisation was carried out by a computer-generated sequence.

+ Block randomisation was carried out by a computer-generated sequence.

+ Randomisation was carried out by drawing a slip of paper from a container.

Author (s) Agee et al. (1992)

Atroshi et al. (2006)

Atroshi et al. (2015)

Brown et al. (1993)

? Allocation concealment was not adequately outlined.

+ Opaque sealed envelopes were used to conceal the allocation sequence.

Allocation concealment The allocation concealment had to be changed as most participants who underwent ECTR in one hand, then refused to undergo OCTR in the other. + Opaque sealed envelopes were used to conceal the allocation sequence.

+ The intervention groups had no significant differences at baseline.

+ The intervention groups had no significant differences at baseline.

+ The intervention groups had no significant differences at baseline.

Intervention group characteristics + The intervention groups had no significant differences at baseline.

The surgeon assessed the outcome and was therefore, not blinded to which intervention each participant received.

+ The researcher assessing the outcomes was blinded.

+ The researcher assessing the outcomes was blinded.

Blinding of the outcome assessor ? It was unclear whether the outcome assessor was blinded or not.

This missing outcome data increases with follow-up time and is inconsistent. No reasons were given

+ They declared that any missing participant data was not significant to the end results.

+ They declared that any missing participant data was not significant to the end results.

Missing participant outcome data ? It was unclear whether the missing participant outcome data was significant to the overall results.

+ Patient data was reported for all pre-selected outcomes at the pre-selected timepoints. + Patient data was reported for all pre-selected outcomes at the pre-selected timepoints. Not all the pre-selected outcomes were reported for each timepoint mentioned in the study outline.

Selective reporting + Patient data was reported for all pre-selected outcomes at the pre-selected timepoints.

+ The authors stated that there were no conflicts of interest.

+ The authors stated that there were no conflicts of interest.

Conflict of interest and funding There is no information regarding the source of study funding, however, the authors to declare there is conflict of interest. + The authors stated that there were no conflicts of interest.

Table 7.3 Table showing the assessment of the risk of bias in each individual study in the following domains—allocation sequence, allocation concealment, intervention group characteristics, blinding of the outcome assessor, missing participant outcome data, selective reporting, and conflict of interest and funding

Appendix 3. Table of Individual Study Bias Assessment

164 E. MacDonald and P. M. Rea

? Allocation sequence was not adequately outlined.

+ Randomisation was carried out by a number-generated sequence.

+ Randomisation was carried out using the ‘sealed envelope’ technique.

+ Randomisation was carried out by a

Dumontier et al. (1995)

Ejiri et al. (2012)

Erdmann (1994)

Ferdinand and Maclean. (2002)

? Allocation concealment was not adequately outlined.

+ Allocation concealment was achieved using sealed envelopes.

? Allocation concealment was not adequately outlined.

? Allocation concealment was not adequately outlined.

+ The intervention groups had no significant

+ The intervention groups had no significant differences at baseline.

+ The intervention groups had no significant differences at baseline.

+ The intervention groups had no significant differences at baseline.

+ The researcher assessing the

? It was unclear whether the outcome assessor was blinded or not.

The researcher assessing the outcomes was not blinded.

? It was unclear whether the outcome assessor was blinded or not.

? It was unclear whether there was

+ All data from participants was estimated using an intention-to-treat analysis. The authors have declared that one participant where ECTR could not be performed was excluded from the study and their missing values were not significant to the results. ? It was unclear whether there was any missing participant data.

for the missing outcome data. + They declared that any missing participant data was not significant to the end results.

+ Patient data was reported for all pre-selected

+ Patient data was reported for all pre-selected outcomes at the pre-selected timepoints.

+ Patient data was reported for all pre-selected outcomes at the pre-selected timepoints.

+ Patient data was reported for all pre-selected outcomes at the pre-selected timepoints.

? It was unclear whether there were any conflicts of interest or whether there was commercial funding. + The authors stated that there were no conflicts of interest. (continued)

? It was unclear whether there were any conflicts of interest or whether there was commercial funding. ? It was unclear whether there were any conflicts of interest or whether there was commercial funding.

7 A Systematic Review of Randomised Control Trials Evaluating the Efficacy. . . 165

+ Randomisation was carried out using the coin flip technique.

+ Randomisation was carried out by assigning those with odd number medical record identifiers to one group and even numbered record identifiers to the other.

? Allocation sequence was not adequately outlined.

+ Randomisation was carried out by a computer-generated sequence.

Gurpinar et al. (2019)

Jacobsen and Rahme (1996)

Kang et al. (2013)

computer-generated sequence.

Allocation sequence

Gumustas et al. (2015)

Author (s)

Table 7.3 (continued)

? Allocation concealment was not adequately outlined.

Allocation concealment was not achieved as participants may know their medical identifier number and thus, may be able to predict which group they were assigned to. ? Allocation concealment was not adequately outlined.

? Allocation concealment was not adequately outlined.

Allocation concealment

+ The intervention groups had no significant differences at baseline as it was a

+ The intervention groups had no significant differences at baseline.

+ The intervention groups had no significant differences at baseline.

differences at baseline as it was a bilateral CTS study. + The intervention groups had no significant differences at baseline.

Intervention group characteristics

+ The researcher assessing the outcomes was blinded.

? It was unclear whether the outcome assessor was blinded or not.

? It was unclear whether the outcome assessor was blinded or not.

+ The researcher assessing the outcomes was blinded.

outcomes was blinded.

Blinding of the outcome assessor

? It was unclear whether the missing participant outcome data was significant to the results.

? It was unclear whether there was any missing participant data.

? It was unclear whether there was any missing participant data.

? It was unclear whether the missing participant outcome data was significant to the results.

any missing participant data.

Missing participant outcome data

+ Patient data was reported for all pre-selected outcomes at the

Not all pre-selected outcomes were reported for each pre-selected timepoint.

+ Patient data was reported for all pre-selected outcomes at the pre-selected timepoints. + Patient data was reported for all pre-selected outcomes at the pre-selected timepoints.

outcomes at the pre-selected timepoints.

Selective reporting

? It was unclear whether there were any conflicts of interest or whether there was commercial funding. + The authors stated that there were no conflicts of interest.

+ The authors stated that there were no conflicts of interest.

+ The authors stated that there were no conflicts of interest.

Conflict of interest and funding

166 E. MacDonald and P. M. Rea

+ Randomisation was carried out using the ‘sealed envelope’ technique.

? Allocation sequence was not adequately outlined.

? Allocation sequence was not adequately outlined.

+ Randomisation was carried out using the ‘sealed envelope’ technique.

+ Randomisation was carried out by a computer-generated sequence.

Larsen et al. (2013)

MacDermid et al. (2003)

Mackenzie et al. (2000)

Malhotra et al. (2007)

Michelotti et al. (2018)

Participants were informed of which intervention they received on each hand.

Allocation was not concealed as the authors declared that the participants expressed preferences of one technique over the other. + Allocation concealment was achieved by sealed envelopes.

? Allocation concealment was not adequately outlined.

+ Allocation concealment was achieved using opaque sealed envelopes.

+ The intervention groups had no significant differences at baseline.

+ The intervention groups had no significant differences at baseline.

? There was no information provided about the baseline participant characteristics.

+ The intervention groups had no significant differences at baseline.

bilateral CTS study. + The intervention groups had no significant differences at baseline.

? It was unclear whether the outcome assessor was blinded or not.

? It was unclear whether the outcome assessor was blinded or not.

? It was unclear whether the outcome assessor was blinded or not.

+ The researcher assessing the outcomes was blinded.

+ The researcher assessing the outcomes was blinded.

? It was unclear whether there was any missing participant data.

? It was unclear whether the missing participant outcome data was significant to the results.

? It was unclear whether the missing participant outcome data was significant to the results.

? It was unclear whether there was any missing participant data.

+ They declared that any missing participant data was not significant to the end results.

+ Patient data was reported for all pre-selected outcomes at the pre-selected timepoints. + Patient data was reported for all pre-selected outcomes at the

pre-selected timepoints. + Patient data was reported for all pre-selected outcomes at the pre-selected timepoints. + Patient data was reported for all pre-selected outcomes at the pre-selected timepoints. + Patient data was reported for all pre-selected outcomes at the pre-selected timepoints.

A Systematic Review of Randomised Control Trials Evaluating the Efficacy. . . (continued)

+ The authors stated that there were no conflicts of interest.

+ The authors stated that there were no conflicts of interest.

? It was unclear whether there were any conflicts of interest or whether there was commercial funding.

+ The authors stated that there were no conflicts of interest.

+ The authors stated that there were no conflicts of interest.

7 167

Allocation sequence

+ Randomisation was carried out by a computer-generated sequence.

+ Block randomisation was carried out.

+ Randomisation was carried out by drawing a slip of paper from a container.

?23 Allocation sequence was not adequately outlined.

Author (s)

Oh et al. (2017)

Saw et al. (2003)

Sennwald and Benedetti (1995)

Tian, Zhao, and Wang (2007)

Table 7.3 (continued)

? Allocation concealment was not adequately outlined.

There was not adequate concealment of the allocation sequence.

+ Allocation concealment was achieved by sealed envelopes.

? Allocation concealment was not adequately outlined.

Allocation concealment

? There was no information provided about the baseline participant characteristics.

+ The intervention groups had no significant differences at baseline.

+ The differences at baseline are compatible with chance.

+ The intervention groups had no significant differences at baseline.

Intervention group characteristics

? It was unclear whether the outcome assessor was blinded or not.

? It was unclear whether the outcome assessor was blinded or not.

+ The researcher assessing the outcomes was blinded.

+ The researcher assessing the outcomes was blinded.

Blinding of the outcome assessor

? It was unclear whether there was any missing participant data.

+ They declared that any missing participant data was not significant to the end results.

+ They declared that any missing participant data was not significant to the end results.

? It was unclear whether there was any missing participant data.

Missing participant outcome data

+ Patient data was reported for all pre-selected outcomes at the pre-selected timepoints.

+ Patient data was reported for all pre-selected outcomes at the pre-selected timepoints.

pre-selected timepoints. + Patient data was reported for all pre-selected outcomes at the pre-selected timepoints. + Patient data was reported for all pre-selected outcomes at the pre-selected timepoints.

Selective reporting

? It was unclear whether there were any conflicts of interest or whether there was commercial funding. ? It was unclear whether there were any conflicts of interest or whether there was commercial funding. ? It was unclear whether there were any conflicts of interest or whether there was commercial funding.

+ The authors stated that there were no conflicts of interest.

Conflict of interest and funding

168 E. MacDonald and P. M. Rea

Wong et al. (2003)

Trumble et al. (2001)

+ Randomisation was carried out by drawing a slip of paper from a container. + Randomisation was carried out by a number-generated sequence.

Drawing randomly from an envelope does not guarantee adequate allocation concealment. ? Allocation concealment was not adequately outlined. + The intervention groups had no significant differences at baseline. + The intervention groups had no significant differences at baseline as it was a bilateral CTS study. ? It was unclear whether the outcome assessor was blinded or not.

+ The researcher assessing the outcomes was blinded. ? It was unclear whether the missing participant outcome data was significant to the results. ? It was unclear whether there was any missing participant data. + Patient data was reported for all pre-selected outcomes at the pre-selected timepoints.

Not all pre-selected outcomes were reported for each timepoint. + The authors stated that study funding did not come from a commercial body.

+ The authors stated that study funding did not come from a commercial body.

7 A Systematic Review of Randomised Control Trials Evaluating the Efficacy. . . 169

170

Appendix 4. Characteristics of Excluded Studies • Studies where only variations of OCTR were investigated without comparing to ECTR: Brüser et al. 1999; Cresswell et al. 2008; Jugovac et al. 2002; Lorgelly et al. 2005; Suppaphol et al. 2012; Tarallo et al. 2014. • Studies comparing standard OCTR with Knifelight® surgery: Bhattacharya et al. 2004; Cellocco et al. 2005; and Heidarian et al. 2013 • Study comparing standard OCTR to ulnarL incision OCTR: Citron and Bendall 1997 • Study comparing transverse mini-incisions to standard OCTR: Faraj, Ahmed and Saeed 2011.

References Included References Agee J, McCarroll H, Tortosa R, Berry D, Szabo R, Peimer C (1992) Endoscopic release of the carpal tunnel: a randomized prospective multicentre study. J Hand Surg Am 17(6):987–995 Atroshi I, Larsson G, Ornstein E, Hofer M, Johnsson R, Ranstam J (2006) Outcomes of endoscopic surgery compared with open surgery for carpal tunnel syndrome among employed patients: randomised controlled trial. BMJ 332(7556):1473 Atroshi I, Hofer M, Larsson G, Ranstam J (2015) Extended follow-up of a randomized clinical trial of open vs endoscopic release surgery for carpal tunnel syndrome. JAMA 314(13):1399 Brown R, Gelberman R, Seiler J, Abrahamsson S, Weiland A, Urbaniak J, Schoenfeld D, Furcolo D (1993) Carpal tunnel release. A prospective, randomized assessment of open and endoscopic methods. J Bone Joint Surg 75(9):1265–1275 Dumontier C, Sokolow C, Leclercq C, Chauvin P (1995) Early results of conventional versus two-portal endoscopic carpal tunnel release. J Hand Surg 20 (5):658–662 Ejiri S, Kikuchi S, Maruya M, Sekiguchi Y, Kawakami R, Konno S (2012) Short-term results of endoscopic versus palmar incision open carpal tunnel release: a prospective randomised controlled trial. Fukushima Med Sci J 58(1):49–59 Erdmann M (1994) Endoscopic carpal tunnel decompression. J Hand Surg 19(1):5–13

E. MacDonald and P. M. Rea Ferdinand R, MacLean J (2002) Endoscopic versus open carpal tunnel release in bilateral carpal tunnel syndrome. J Bone Joint Surg 84-B(3):375–379 Gumustas S, Ekmekçi B, Tosun H, Orak M, Bekler H (2015) Similar effectiveness of the open versus endoscopic technique for carpal tunnel syndrome: a prospective randomized trial. Eur J Orthop Surg Traumatol 25(8):1253–1260 Gurpinar T, Polat B, Polat A, Carkç{ E, Kalyenci A, Oztürkmen Y (2019) Comparison of open and endoscopic carpal tunnel surgery regarding clinical outcomes, complication and return to daily life: a prospective comparative study. Pakistan J Med Sci 35(6) Jacobsen M, Rahme H (1996) A prospective, randomized study with an independent observer comparing open carpal tunnel release with endoscopic carpal tunnel release. J Hand Surg 21(2):202–204 Kang H, Koh I, Lee T, Choi Y (2013) Endoscopic carpal tunnel release is preferred over mini-open despite similar outcome: a randomized trial. Clin Orthop Relat Res 471(5):1548–1554 Larsen M, Sørensen A, Crone K, Weis T, Boeckstyns M (2013) Carpal tunnel release: a randomized comparison of three surgical methods. J Hand Surg (European Volume) 38(6):646–650 MacDermid J, Richards R, Roth J, Ross D, King G (2003) Endoscopic versus open carpal tunnel release: a randomized trial. J Hand Surg Am 28(3):475–480 Mackenzie D, Hainer R, Wheatley M (2000) Early recovery after endoscopic vs. short-incision open carpal tunnel release. Ann Plast Surg 44(6):601–604 Malhotra R, Kiran E, Dua A, Mallinath S, Bhan S (2007) Endoscopic versus open carpal tunnel release: a shortterm comparative study. Ind J Orthopaedics 41(1):57 Michelotti B, Vakharia K, Romanowsky D, Hauck R (2018) A prospective, randomized trial comparing open and endoscopic carpal tunnel release within the same patient. Hand 15(3):322–326 Oh W, Kang H, Koh I, Jang J, Choi Y (2017) Morphologic change of nerve and symptom relief are similar after mini-incision and endoscopic carpal tunnel release: a randomized trial. BMC Musculoskelet Disord 18(1) Saw N, Jones S, Shepstone L, Meyer M, Chapman P, Logan A (2003) Early outcome and cost-effectiveness of endoscopic versus open carpal tunnel release: a randomized prospective trial. J Hand Surg 28 (5):444–449 Sennwald G, Benedetti R (1995) The value of one-portal endoscopic carpal tunnel release: a prospective randomized study. Knee Surg Sports Traumatol Arthrosc 3(2):113–116 Tian Y, Zhao H, Wang T (2007) Prospective comparison of endoscopic and open surgical methods for carpal tunnel syndrome. Chin Med Sci J 22(2):104–107 Trumble T, Gilbert M, McCallister W (2001) Endoscopic versus open surgical treatment of carpal tunnel syndrome. Neurosurg Clin N Am 12(2):255–266 Wong K, Hung L, Ho P, Wong J (2003) Carpal tunnel release. J Bone Joint Surg 85-B(6):863–868

7

A Systematic Review of Randomised Control Trials Evaluating the Efficacy. . .

Excluded References Bhattacharya R, Birdsall P, Finn P, Stothard J (2004) A randomized controlled trial of Knifelight and open carpal tunnel release. J Hand Surg 29(2):113–115 Brüser P, Richter M, Larkin G, Lefering R (1999) The operative treatment of carpal tunnel syndrome and its relevance to endoscopic release. Eur J Plast Surg 22 (2–3):80–84 Cellocco P, Rossi C, Bizzarri F, Patrizio L, Costanzo G (2005) Mini-open blind procedure versus limited open technique for carpal tunnel release: a 30-month followup study. J Hand Surg Am 30(3):493–499 Cresswell T, Heras-Palou C, Bradley M, Chamberlain S, Hartley R, Dias J, Burke F (2008) Long-term outcome after carpal tunnel decompression–a prospective randomised study of the Indiana tome and a standard limited palmar incision. J Hand Surg (European Volume) 33(3):332–336 Citron N, Bendall S (1997) Local symptoms after open carpal tunnel release. J Hand Surg 22(3):317–321 Faraj A, Ahmed M, Saeed O (2011) A comparative study of the surgical management of carpal tunnel syndrome by mini-transverse wrist incisions versus traditional longitudinal technique. Eur J Orthop Surg Traumatol 22(3):221–225 Heidarian A, Abbasi H, Hasanzadeh Hoseinabadi M, Hajialibeyg A, Kalantar Motamedi S, Seifirad S (2013) Comparison of Knifelight surgery versus conventional open surgery in the treatment of carpal tunnel syndrome. Iran Red Crescent Med J 15(5):385–388 Jugovac I, Burgiæ N, Miæoviæ V, Radoloviæ-Prenc L, Uraviæ M, Goluboviæ V, Stanèiæ M (2002) Carpal tunnel release by limited palmar incision vs traditional open technique: randomized controlled trial. Croat Med J 43(1):33–36 Lorgelly P, Dias J, Bradley M, Burke F (2005) Carpal tunnel syndrome, the search for a cost-effective surgical intervention: a randomised controlled trial. Ann R Coll Surg Engl 87(1):36–40 Suppaphol S, Worathanarat P, Kawinwongkovit V, Pittayawutwinit P (2012) The comparison between limited open carpal tunnel release using direct vision and tunnelling technique and standard open carpal tunnel release: a randomized controlled trial study. J Med Assoc Thail 95(4):532–536 Tarallo M, Fino P, Sorvillo V, Parisi P, Scuderi N (2014) Comparative analysis between minimal access versus traditional accesses in carpal tunnel syndrome: a perspective randomised study. J Plast Reconstr Aesthet Surg 67(2):237–243

Additional References Burton C, Chesterton L, Davenport G (2014) Diagnosing and managing carpal tunnel syndrome in primary care. Br J Gen Pract 64(622):262–263

171

Chow J (1989) Endoscopic release of the carpal ligament: a new technique for carpal tunnel syndrome. Arthroscopy 5(1):19–24 De Kesel R, Donceel P, De Smet L (2008) Factors influencing return to work after surgical treatment for carpal tunnel syndrome. Occup Med 58(3):187–190 Gerritsen A, Uitdehaag B, van Geldere D, Scholten R, de Vet H, Bouter L (2001) Systematic review of randomized clinical trials of surgical treatment for carpal tunnel syndrome. Br J Surg 88(10):1285–1295 Hermiz S, Kalliainen L (2017) Evidence-based medicine. Plast Reconstr Surg 140(1):120e–129e Higgins JPT, Savović J, Page MJ, Elbers RG, Sterne JAC. Chapter 8: assessing risk of bias in a randomized trial. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane handbook for systematic reviews of interventions version 6.1 (updated 2020). Cochrane, 2020. Available from www.training.cochrane.org/handbook Ibrahim I, Khan W, Goddard N, Smitham P (2012) Carpal tunnel syndrome: a review of the recent literature. Open Orthop J 6(1):69–76 Li Y, Luo W, Wu G, Cui S, Zhang Z, Gu X (2020) Open versus endoscopic carpal tunnel release: a systematic review and meta-analysis of randomized controlled trials. BMC Musculoskelet Disord 21(1) Lane J, Craig R, Rees J, Gardiner M, Green J, PrietoAlhambra D, Furniss D (2021) Serious postoperative complications and reoperation after carpal tunnel decompression surgery in England: a nationwide cohort analysis. Lancet Rheumatol 3(1):e49–e57 Moher D, Liberati A, Tetzlaff J, Altman D (2009) Preferred reporting items for systematic reviews and metaanalyses: the PRISMA statement. PLoS Med 6(7): e1000097 Oertel J, Schroeder H, Gaab M (2006) Dual-portal endoscopic release of the transverse ligament in carpal tunnel syndrome. Neurosurgery 59(2), pp. 333–340. Accessed Sep 23, 2020, from https://academic.oup. com/neurosurgery/article/59/2/333/2559286 Palmer D, Paulson J, Lane-Larsen C, Peulen V, Olson J (1993) Endoscopic carpal tunnel release: a comparison of two techniques with open release. Arthroscopy 9 (5):498–508 Rodner C, Katarincic J (2008) Open carpal tunnel release. Tech Orthop 23(3):199–207 Sanati K, Mansouri M, Macdonald D, Ghafghazi S, Macdonald E, Yadegarfar G (2011) Surgical techniques and return to work following carpal tunnel release: a systematic review and meta-analysis. J Occup Rehabil 21(4):474–481 Sayegh E, Strauch R (2014) Open versus endoscopic carpal tunnel release: a meta-analysis of randomized controlled trials. Clin Orthop Relat Res 473(3), pp. 1120–1132. Accessed Sep 23, 2020, from https:// www.ncbi.nlm.nih.gov/pmc/articles/PMC4317413/ Scholten R, Mink van der Molen A, Uitdehaag B, Bouter L, de Vet H (2007) Surgical treatment options

172 for carpal tunnel syndrome. Cochrane Database Syst Rev 3 Sevy J, Varacallo M (2020) Carpal tunnel syndrome. [online] Ncbi.nlm.nih.gov. Accessed Oct 10, 2020, from https://www.ncbi.nlm.nih.gov/books/ NBK448179/ Thoma A, Veltri K, Haines T, Duku E (2004) A metaanalysis of randomized controlled trials comparing endoscopic and open carpal tunnel decompression. Plast Reconstr Surg, pp. 1137–1146 Vasiliadis H, Nikolakopoulou A, Shrier I, Lunn M, Brassington R, Scholten R, Salanti G (2015) Endoscopic and open release similarly safe for the treatment of carpal tunnel syndrome. A systematic review and meta-analysis. PLoS One 10(12):e0143683 Venouziou A, Kerasnoudis A (2020) Endoscopic carpal tunnel release. [online] springer link. Accessed Dec 23, 2020, from http://link-springer-com-443.webvpn.

E. MacDonald and P. M. Rea fjmu.edu.cn/chapter/10.1007%2F978-3-030-37289-7_ 6#citeas Violante F, Farioli A, Graziosi F, Marinelli F, Curti S, Armstrong T, Mattioli S, Bonfiglioli R (2016) Carpal tunnel syndrome and manual work: the OCTOPUS cohort, results of a ten-year longitudinal study. Scand J Work Environ Health 42(4):280–290 Werner R, Andary M (2002) Carpal tunnel syndrome: pathophysiology and clinical neurophysiology. Clin Neurophysiol 113(9):1373–1381 Wipperman J, Goerl K (2016) Carpal tunnel syndrome: diagnosis and management. American Family Physician, [online] 94(12), pp. 993–999. Accessed Sep 25, 2020, from https://www.aafp.org/afp/2016/1215/ p993.html Zamborsky R, Kokavec M, Simko L, Bohac M (2017) Carpal tunnel syndrome: symptoms, causes and treatment options. Literature Reviev Ortopedia Traumatologia Rehabilitacja 19(1):1–8

8

Exploring Visualisation for Embryology Education: A Twenty-First-Century Perspective Eiman M. Abdel Meguid, Jane C. Holland, Iain D. Keenan, and Priti Mishall

Abstract

Embryology and congenital malformations play a key role in multiple medical specialties including obstetrics and paediatrics. The process of learning clinical embryology involves two basic principles; firstly, understanding time-sensitive morphological changes that happen in the developing embryo and, secondly, appreciating the clinical implications of congenital conditions when development varies from the norm. Visualising the sequence of dynamic events in embryonic development is likely to be challenging for students, as these processes occur not only in three dimensions but also in the fourth dimensions of time.

E. M. Abdel Meguid (*) Centre for Biomedical Sciences Education, School of Medicine, Dentistry and Biomedical Sciences, Queen’s University Belfast, Belfast, UK e-mail: [email protected] J. C. Holland Department of Anatomy, RCSI University of Medicine and Health Sciences, Dublin, Ireland e-mail: [email protected] I. D. Keenan School of Medical Education, Newcastle University, Newcastle upon Tyne, UK e-mail: [email protected] P. Mishall Department of Anatomy and Structural Biology & Department of Ophthalmology and Visual Sciences, Albert Einstein College of Medicine, Bronx, NY, USA e-mail: [email protected]

Consequently, features identified at any one timepoint can subsequently undergo morphological transitions into distinct structures or may degenerate and disappear. When studying embryology, learners face significant challenges in understanding complex, multiple and simultaneous events which are likely to increase student cognitive load. Moreover, the embryology content is very nonlinear. This nonlinear content presentation makes embryology teaching challenging for educators. Embryology is typically taught in large groups, via didactic lecture presentations that incorporate two-dimensional diagrams or foetal ultrasound images. This approach is limited by incomplete or insufficient visualisation and lack of interactivity. It is recommended that the focus of embryology teaching should instill an understanding of embryological processes and emphasise conceptualising the potential congenital conditions that can occur, linking pre-clinical and clinical disciplines together. A variety of teaching methods within case-based and problem-based curricula are commonly used to teach embryology. Additional and supplementary resources including animations and videos are also typically utilised to demonstrate complex embryological processes such as septation, rotation and folding. We propose that there is a need for embryology teaching in the twenty-first century to evolve. This is particularly required in terms of

# The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 P. M. Rea (ed.), Biomedical Visualisation, Advances in Experimental Medicine and Biology 1356, https://doi.org/10.1007/978-3-030-87779-8_8

173

174

E. M. Abdel Meguid et al.

appropriate visualisation resources and teaching methodologies which can ensure embryology learning is relevant to real-world scenarios. Here we explore embryology teaching resources and methodologies and review existing evidence-based studies on their implementation and impact on student learning. In doing so, we aim to inform and support the practice of embryology educators and the learning of their students. Keywords

Animation · Visualisation in embryology · Digital technology · 3D · Innovative teaching · Virtual reality

8.1

Introduction

A number of different modalities have been used to visualise embryology. These modalities range from simple static drawings to interactive dynamic 3D models. In the early nineteenth and twentieth centuries, the developmental stages of the embryo were demonstrated by drawings of human embryo (Hill 1948), wax models (His 1880), print photography (Hopwood 2007; Yamada et al. 2015), lab collection of comparative embryology of vertebrates (Richardson and Narraway 1999) and histology collection (Hamilton et al. 1972; Hill 2018). Furthermore, during the same era, embryologists worldwide collected specimens of real human embryos in different stages of development. Towards the end of the twentieth century, technological advances in biomedical visualisation such as magnetic resonance imaging (MRI) (Smith et al. 1999) and episcopic fluorescence image capture (EFIC) (Rosenthal et al. 2004) have enabled the digitisation of the Kyoto and Carnegie collection of the real human embryos, which are now available online through the Open Microscopy Environments (Allan et al. 2012; Cho et al. 2012). In the twenty-first century, embryology education research has focused on demonstrating the dynamic stages of embryonic development, which have been powerfully

illustrated by animations (Habbal and Harris 1995; Yamada et al. 2015), virtual reality (VR) (Alfalah et al. 2019), 3D printing (3DP) (Plunkett et al. 2019), computer graphics (Yamada et al. 2006, 2010) and the interactive 3D atlas (de Bakker et al. 2016). Furthermore, advanced technologies including Anatomage (Anatomage, Inc., Santa Clara, California) and Sectra (Sectra AB, Linkoping, Sweden) visualisation tables (VTs) can convert static images to short movies layered with user. Additionally, prenatal ultrasound technology is a boon to twenty-first century as it allows for in vivo embryo visualisation (Fakoya et al. 2017). Although the subject of embryology evolved in terms of research, its fate as an independent subject within medical school curricula has always been challenging. Embryology is now typically woven within multiple modules in system-based curricula (McBride and Drake 2018). The challenge is multi-fold and involves learners, educators and curriculum administrators. Learners can experience challenges with respect to cognitive load, as the subject of embryology demands understanding of the changes in spatial 3D orientation and terminology. A feature identified as X at one point can become structure Y while changing names and/or completely degenerating. For example, the mesoderm-derived structure nucleus pulposus in adults is an embryonic remnant of the degenerated notochord, which demonstrates changes in both structures in terminology during development. The endodermal extra-embryonic yolk sac degenerates and is replaced by the vitelline duct, which in turn also degenerates, while the associated embryonic endoderm persists and differentiates into the gut tube. The manner in which structures alter their morphology during development can be complex. For example, the inferior vena cava forms from multiple embryonic veins. If such transitions and transformations of structures are not explicitly explained by educators in terms of their relevance to realworld adult manifestations, both normal and abnormal, the course content is likely to become confusing, which may in turn make learning less engaging and relatable to the adult human body.

8

Exploring Visualisation for Embryology Education: A Twenty-First-Century Perspective

Traditionally, educators teach embryology content in lecture theatres to large groups of students, but this large group format lacks interactivity and active learner participation. Other pedagogical methodologies such as case-based (Moraes and Pereira 2010; Scott et al. 2013), problem-based (Cowan et al. 2010) or teambased (Prange-Kiel et al. 2016) learning are implemented by various institutions but often on a limited basis only. Student self-directed embryology learning is usually supported by asynchronous animations (Rao 2012; Upson-Taboas et al. 2019), YouTube videos (Raikos and Waidyasekara 2014) and screencasts (Evans 2011). Recently, social media has been used to harness the power of community-based learning. Furthermore, there are curricular demands to accommodate embryology within the limited time-frame of reduced pre-clinical contact hours. These changes have dramatically impacted embryology teaching in medical schools, forcing the educators to rethink on how and how much embryology should be taught. Here we highlight the history of visualisation in embryology and the theoretical and practical basis of embryology pedagogy. Furthermore, we explore current embryology teaching resources including animations, YouTube videos, VR, and 3D printing; discuss the learning outcomes of evidenced-based teaching methodologies; and conclude with a case study on integrating interactive 3D embryology learning resources in the medical school curricula.

8.2

8.2.1

History of Visualisation in Embryology and Challenges in the Twenty-First Century In the Nineteenth Century

Embryological research began in the decades around the turn of the nineteenth century. Wilhelm His (1831–1904), a Swiss embryologist, produced a series of both historic drawings of human embryos and well-known wax models of the developing human, chicken and fish (His

175

1880). Later, these drawings have been reproduced in historic embryology textbooks. His study of human embryology led to the creation of the valuable collection of medical models and specimens that established the basic principles for Franklin P. Mall’s collection of human embryos at the Carnegie Institution of Washington (Hopwood 2007). Franz Keibel (1861–1929), a German embryologist, created the crucial printed photographs of comparative evolutionary embryology and provided information for the study of human embryonic variation. His embryological plates were used as a starting point for embryologists who created the staging systems (Hopwood 2007). Hubrecht (1853–1915), a Dutch embryologist, founded a laboratory with a collection of comparative embryology of vertebrates. This collection is present in the Embryological Collection at the Museum für Naturkunde (Richardson and Narraway 1999).

8.2.2

In the Twentieth Century

In the United States, the Minot’s Harvard collection (Minot 1905) has been later on integrated into Franklin Mall’s Carnegie collection that was dated from the beginning of 1900s. This Carnegie collection made the ground of a huge number of essential studies of human development in the twentieth century. This human embryo collection is located at the National Museum of Health and Medicine (NMHM) and currently establishes part of the Developmental Anatomy Collections. From this collection, the Streeter’s Horizons (Streeter 1948) was set up that was thereafter reformed as Carnegie stages (O’Rahilly and Meyer 1979). These stages cover human development from fertilisation to the end of week 8. Week 8 is considered the end of the embryonic phase, when most of the parts of the human body have started to develop (O’Rahilly and Meyer 1979). In Europe, between 1950s and the 1970s, multiple fundamental collections were set up. In Germany, Blechschmidt (Blechschmidt 1961) followed by Hinrichsen (Hinrichsen 1990)

176

created two amazing histological collections focusing on embryology. In Spain, Orts Llorca followed by Puerta and Doménech Mateu set up three embryonic collections (Arechaga et al. 2009). In the United Kingdom (UK), between 1950s and 1960s in Cambridge, the Boyd collection of histological specimens focused on placentation (Hamilton et al. 1972). Later, in 1999, the Human Developmental Biology Resource (HDBR) and Human Developmental Studies Network (HUDSEN) collection commenced (Kerwin et al. 2010). In Japan, Nishimura et al. (1968) developed the Kyoto Collection that became one of the biggest human embryo collections for both abnormal and normal embryos around the globe, with over 44,000 human normal and abnormal specimens. The Kyoto collection is special for many reasons. First, the material is derived only from the Japanese population at the prenatal developmental stage, and the embryos were obtained after induced abortion (Dilation and Curettage 1961) of healthy females due to social as well as economic factors. Second, the material is from abnormal and normal embryos. Third, the material is reliable and consisted of histologically sectioned as well as whole embryos. The Kyoto collection has been electronically released in the Kyoto Human Embryo Visualisation Project and the Human Embryo Atlas and also in Japanese language (Atlas of Embryonic Development) (Yamada et al. 2006, 2010). The Kyoto and Carnegie embryo collections are currently the two main historic human collections (Hill 2018). The valuable uniqueness of all these twentieth century collections made the researchers obliged to plan trips to the collection site for further in situ studies. However, their increasing availability and popularity of these resources during the 1990s enabled major changes to the research field that initiated the evolution of digital references of the embryonic collections (Hill 2018). From the beginning of 1997, the Internet provided an opportunity for digital embryonic collections for the purposes of education and research (Hill 2018). Such novel

E. M. Abdel Meguid et al.

online resources have allowed a wide re-investigation of the Kyoto collection. Researchers and educators are currently using this embryo collection with other newer imaging and genetic analytical procedures in multiple digital projects. These embryo images are fundamental tools for understanding human embryonic development (Hill 2018). In the 1990s, magnetic resonance imaging (MRI) had been used to scan embryos (Smith et al. 1999). A few years later, episcopic fluorescence image capture (EFIC) emerged (Rosenthal et al. 2004). Recently, these approaches have been developed and used to scan the Kyoto embryos with the help of these two facilities, the MRI and the EFIC (Dhanantwari et al. 2009; Yamada et al. 2010). These electronic slide scans are available via the Open Microscopy Environments online (Allan et al. 2012; Cho et al. 2012).

8.2.3

In the Twenty-First Century

In 2009, advances in 3D imaging and computer graphics supported development of the 3D Atlas of Human Embryology project. The project produced more than 15,000 manually annotated sections and a duplicate series of fully reconstructed human embryos covering the phase of organogenesis, between Carnegie stage 7 (15–17 days old embryo) and 23 (56–60 days). The alignment of sections to create the 3D reconstructions was performed with Amira software (version 5.3–5.6, Thermo Fisher Scientific). Later, smooth-surfaced models were created by using Blender software (2.92, Blender Foundation (de Bakker et al. 2016). These models suitable for use in 3D-PDFs, applications or games viewed in Adobe reader (version X or higher) help develop an independent view on spatial relations. This labour-intensive project was sponsored by the Academic Medical Center (AMC) in Amsterdam and involved students and embryologists of the Department of Anatomy, Embryology and Physiology (de Bakker et al. 2016).

8

Exploring Visualisation for Embryology Education: A Twenty-First-Century Perspective

8.2.3.1

Challenges in Embryology Teaching in the Twenty-First Century Embryology is taught at the pre-clinical years of the medical course and in other programmes. For students, embryology may appear to be far from their physical reality. It is a subject to which minimum attention is devoted to by most medical educators (Hamilton and Carachi 2014); and yet it is a subject that not only possesses great scientific value but also includes important practical information (Alfalah et al. 2019). The rapid development of the field of embryology increases the difficulty in discovering the level of detail required for the medical course (Carlson 2002). Embryology education faces a number of challenges. Firstly, understanding embryology requires 3D visualisation of multiple events that occur rapidly and simultaneously within short periods of time. Secondly, over the last decade, the transition from content-based curricula to competence-based curricula has left embryology educators looking for a checklist of well-defined competencies that students need to learn during the pre-clinical years (Bergsmann et al. 2015; Holland and Pawlikowska 2019). Thirdly, problems such as non-expert educators, outmoded syllabus content and the reduction of the curricular time on basic biomedical sciences are likely to have made embryology education an un-inviting and disinteresting experience for students (Scott et al. 2013). Despite ongoing technological evolution, embryology is still typically taught using traditional methods that do not particularly serve the purpose of enhancing visualisation (McBride and Drake 2018). To overcome these challenges, we recommend that embryology educators should utilise the guiding principles of research-informed pedagogy underpinned by relevant educational theory. Through the use of such scholarly approaches, educators can effectively develop both engaging and informative course content, including the adoption of twenty-first-century interactive resources to support learner-centred teaching methods.

8.3 8.3.1

177

Learning Theories Cognitive Load Theory

Arguably, e-learning resources such as animations and videos may be more useful than standard texts in assisting students to understand what is essentially four-dimensional, and resources such as animations and videos have been reported to be effective for learning embryology (Yamada et al. 2006; Moraes and Pereira 2010; Upson-Taboas et al. 2019) and are the very embodiment of the multiple representation principle. This proposes that instructional media benefit from combining visual animation with verbal information, whether provided as an audible voice-over or written text labels, published as a video (Mayer and Moreno 2002, 2003; Ruiz et al. 2009). This principle is based upon an assumption that learners receive and process information through separate auditory-verbal and visual-pictorial channels and then combine this information to form long-term memories (Mayer 2010, 2014; Dong and Goh 2015) (Fig. 8.1). Videos and animations are useful resources to teach students how embryonic structures alter, both spatially and over time, and visualise embryological concepts such as rotation, folding and septation (Sweller 1988; Mayer and Moreno 2002; Marsh et al. 2008; Schleich et al. 2009; Upson-Taboas et al. 2019). Principles of cognitive load theory may assist in design decisions, for example, reducing distracting elements that may increase extraneous load, so that the final resource keeps students’ attention and focus in short, but high-value segments (Dong and Goh 2015). Videos do not necessarily need high-quality production values or digital imaging (high-fidelity); again, these highly detailed resources may in fact increase extraneous load, and low-quality or low-fidelity resources may be better for novice learners as they grasp the basics or core concepts (Mayer and Moreno 2003; Chen et al. 2015). Likewise, simple verbal explanations may reinforce learning more meaningfully than overly detailed or complex explanations, as students

178

E. M. Abdel Meguid et al.

Fig. 8.1 Dual-channel assumption of multimedia learning (Mayer 2014)

simultaneously watch new content on a video or animation (Ruiz et al. 2009). An additional difficulty occurs when attention is required to be focused on different regions of the visual field simultaneously, requiring students to constantly switch their visual focus. This is called the coherence effect, and the repeated shift in focus reduces the students’ ability to efficiently process the new information (Mayer and Moreno 2003). Thus, learners could perceive the information but might not be able to efficiently integrate it into their understanding. Similarly, animations are not always considered superior to the static pictures, as in some cases, it may impede learning by increasing extraneous cognitive load or decreasing the germane cognitive load, whereas static pictures offer time to review information as long as needed (Ruiz et al. 2009). Additionally, care must be taken not to assume that all individuals within this ‘net generation’ have the ability to use e-learning resources efficiently for study and learning (Helsper and Eynon 2010; Holland and Pawlikowska 2019). After all, the ‘book generations’ did not enter print-based libraries with innate knowledge of the Dewey Decimal System or index cards. The sheer paradox of choice within such a variety and volume of online resources, videos and podcasts potentially risks overloading the students with excessive options, such that ‘non-essential multimedia act to distract learners and actually decrease learning’ (Cook 2007; Gikas and Grant 2013). The educator plays a meaningful role in

assisting students to transition to self-regulated learning, reducing the potential for filter failure as students first foray into a potentially endless sea of social media sites or online video options (Shirky 2008; Flynn et al. 2015).

8.4 8.4.1

Current Resources in Embryology Teaching Videos and YouTube

While many videos and animations may be available via university websites or produced to a high standard by commercial publishers, and while some students may use the ‘intelligently filter’ resources provided to them from these educational sources, others will just type into Google or YouTube (Jaffar 2012; Henderson et al. 2015; Barry et al. 2016b; Holland and Pawlikowska 2019). Search engines and social media sites such as YouTubeTM (https://www.youtube. com/) remain popular, for both lay audiences and undergraduates alike, as a source of mostly free information. However, it lacks a peer-review process, so the accuracy of the information provided is not assessed (Jaffar 2012; Raikos and Waidyasekara 2014; Barry et al. 2016a; Holland and Pawlikowska 2019). YouTube in particular is a preferred source for many students when seeking videos to further study anatomy or embryology (Henderson et al. 2015; O’Carroll et al. 2015; Barry et al. 2016a;

8

Exploring Visualisation for Embryology Education: A Twenty-First-Century Perspective

Holland and Pawlikowska 2019; Curran et al. 2020). Videos on this platform cater to a wide audience, with different prior knowledge and educational needs, from children seeking to understand their own conditions to those undertaking advance postgraduate clinical studies (Bezner et al. 2014; Holland and Pawlikowska 2019). With so many resources and platforms seeking to gain students time and attention, and the more elements being added to modern curricula, educators should consider the merits in keeping videos concise, focused and closely aligned with the curriculum and course, so as not to overload students with excessive course work and resources (Evans 2011; Prober and Khan 2013; Dong and Goh 2015). Production values also vary, although as previously described, highfidelity or high-quality resources may be counter-productive, and low-quality animation or hand-drawn screen casts may demonstrate complex concepts clearly and concisely, if welldesigned (Cook 2007). What can be concerning, however, is the variability of the accuracy of the information included in some of these online videos, regardless of their production quality. Bezner et al. (2014) reviewed 40 videos covering aspects of common congenital conditions in paediatric surgery and found that 56.3% of videos ‘had accuracy scores of 3 or higher’ (Table 8.1). These were the top 40 videos displayed on YouTube.com using the default ranking algorithm of the platform, which incorporates view counts and viewer ratings, and so presumably the accuracy of any additional videos covering this content would rank and score lower (Bezner et al. 2014). Matthews et al. have presented a preliminary report on an ongoing study, examining the quality of cardiac embryology resources on YouTube, and have reported on some initial copyright and consent concerns. Ethical concerns have been voiced by previous authors about considerations such as evidence of patient consent (Jaffar 2012; Barry et al. 2016a; Hennessy et al. 2020). Even in circumstances where consent is clearly obtained, and documented, the depiction of graphical operative or cadaveric images without age restriction is still questionable, to say the least.

179

In summary, while YouTube hosts many videos of value to medical students studying embryology, there also exist many videos with irrelevant, or inaccurate, content. However, some critical judgement is required on the part of the students, and their teachers, when selecting appropriate videos for study and learning.

8.4.2

Animations

Animation is defined as, ‘A series of images designed so that each image appears as an alteration of the previous one, and in which the sequence of images is determined by either the designer or the user’ (Betrancourt 2005). Animations display concepts visually. These concepts might be difficult to explain just with words or static images that expect students to infer change on their own (Lowe 2004). For example, when embryological processes demonstrate changes to shape and size of organs or tissue shift or merging to adjoining structures (Clark and Paivio 1991; Mayer 2002; Plunkett et al. 2019), the use of animations allow to show the sequential changes by creating high-quality mental models that have the potential to selectively use the information and focus learner attention to make the new information stick (Lowe 2004). Hence, animations illustrate dynamic changes that occurred during the embryological processes in a unique way (Habbal and Harris 1995; Yamada et al. 2006; Schleich et al. 2009; Moraes and Pereira 2010). The factors that reportedly make the animations unique and impactful primarily concern learner interactivity, chunking the content in smaller segments and permitting learner to control the pace of the animation (Ruiz et al. 2009).

8.4.3

Virtual Reality

VR applications are considered one of the evolving methods that has the potential to transform the education process (Falah et al. 2015; Alfalah et al. 2019). VR allows the user to select the stage of embryonic development from a series

180

E. M. Abdel Meguid et al.

Table 8.1 The use of YouTube in embryology teaching Author Azer (2012)

Screened 235

Raikos and Waidyasekara (2014)

55,525

Bezner et al. (2014)

Content Surface anatomy Human heart anatomy Paediatric surgery

Reviewed as relevant for UG 57 294

Reviewed first 40 videos only

of sections and gain access to details on embryo anatomy and development according to gestational age in an interactive manner. VR is a unique tool that permits manipulation of the model by rotating, zooming in or out on specific features to enhance 3D understanding of the structures under observation (Alfalah et al. 2019). These facilities are likely to enhance and facilitate interaction between the student and the embryology under observation, by providing more realistic models that closely resemble real tissue and techniques, in an efficient, time-saving manner (Berman et al. 2008; Ungi et al. 2011; Alfalah et al. 2019). Several studies have suggested that the use of interactive 3D computer-generated models in an interactive VR environment has the potential to enhance the educational process, by improving visualisation, increasing flexibility of the learning process, reducing time and effort needed and making learning more enjoyable (Alfalah et al. 2019). This is especially vital during the teaching of embryology, as this subject requires a high level of visualisation of the spatial arrangements and 3D relationships between different components.

8.4.4

Virtual Dissection Tables

One further technological tool currently used for anatomy education is the virtual dissection tables (VTs). For example, the Anatomage and Sectra VTs can generate 3D digital anatomical images to support learning (Baratz et al. 2019; Keenan and

Finding ‘15 (27%) of the videos provided useful information on surface anatomy’ ‘Only 25.9% of the elaborated videos succeed to attain the pass mark score in our scoring system’ ‘Human heart anatomy videos available on YouTube conveyed our anatomical criteria poorly’ ‘Only 20.0% of the videos . . . directed toward medical professionals’ ‘Only 37.5% of the videos provided owner contact information’ ‘56.3% of videos had accuracy scores of 3 or higher’

Ben Awadh 2019; Ben Awadh et al. 2020). For anatomy learning, students use touch-screen interfaces to interact with and dissect various anatomical models. More recently, embryology content has been added to these tables, specifically scans of human embryos at Carnegie stages 13–23 or 28–56 days. Embryology teaching has evolved during the last decade to incorporate more visualisation such as asynchronous videos or animations (UpsonTaboas et al. 2019) and 3D printing (Plunkett et al. 2019). However, further research is required in order to identify the impacts on student learning when using these resources.

8.5

Summary of Evidence-Based Studies on Using Visualisation in Embryology Teaching

A review of the embryology education literature revealed a scarcity of evidence-based studies that demonstrate impacts of embryology resources on short-term and long-term student knowledge retention. In a study that involved educators, students and laypeople, participants were asked to watch six web-based animation series on cardiovascular embryology (Upson-Taboas et al. 2019). To investigate the impact of these animations in short-term knowledge retention, the participants took a pre-test before watching the animations and a post-test after watching the animations. The study has found that the number of post-test score was significantly higher after

8

Exploring Visualisation for Embryology Education: A Twenty-First-Century Perspective

watching the animation than the matched pre-test scores before watching the animation (P < 0.001). Another study, with findings that support the use of animation for long-term knowledge retention, assessed students by comparing classes who had used the animation module with classes who had not used the animation module. The study reported that students who used the animation module had their average score significantly higher than the control group (P < 0.05) (Marsh et al. 2008). A similar report compared formative assessments of first-year medical students who used an e-learning application along with faculty facilitation to previous batch of students who learned embryology via didactic lectures only. It was found that students who used the e-learning module showed improvement in mean scores from first (m ¼ 3.7) to third assessments (m ¼ 5.88). Moreover, the class that used the e-learning application reported a significantly higher average score than the class that used didactic lectures only (4.89 vs 4.32, p < 0.05) (Sagar and Viveka 2016.). A number of description-based studies have reported on the value of highly visual resources that have been used to teach embryology. Watt et al. (1996) received positive student feedback after using computer morphing techniques as a supplementary tool for embryology teaching (Watt et al. 1996). Yamada et al. (2006) developed a self-learning embryology program that used photographs, MRM images, computer graphics and histology sections of staged human embryos in the Kyoto collection to create lectures and formative quizzes. Students reported very favourable and positive experiences of using these resources (Yamada et al. 2006). Evans (2011) used screencast videos to create five mini-lectures and a formative quiz to augment embryology learning. Interestingly, the slide presentation of the screen recording used custom animation slides. These slides were designed particularly to increase the visual stimulation, and the narration was synchronised with slide animations. The study reported that students viewed the screencast favourably in terms of usefulness to their learning which also reflected

181

positively on their examination scores as compared to the previous years (Evans 2011). In another study, Plunkett et al. (2019) randomised medical students in four groups: A, B, C and D. Each group was given different resources for integrating embryology into gross anatomy. All groups received embryology images on a pamphlet. Group A did not receive any additional resources. Group B used 3D virtual models, Group C used 3D Print (3DP) models and Group D was given an instructor-led tutorial of embryology on an adult cadaver. The immediate impact of resource on retention of embryology knowledge tested by a pre- and post-test reported Group A and Group C showed an improvement in post-quiz scores by 8% and 12%, respectively. Students later explored all four learning resources and provided responses to a questionnaire. Results indicated that 3DP models were easiest to use and had the highest educational value (Plunkett et al. 2019). Due to the limitations of the findings of the studies described above, it is necessary to design follow-up longitudinal investigations of student performances, in order to understand the impact of various resources used to teach embryology. Despite this, we have been able to utilise the current available pedagogic research, educational theory and technological resources to develop effective embryology learning approaches in our practice.

8.6

8.6.1

Case Study: Integrating 3D Embryology Learning Resources Within a Medical School Curriculum Educational Context

8.6.1.1 Pedagogical Basis The curricular integration of clinical embryology at Newcastle University (NU) is based on underlying principles and hypotheses, which propose that an understanding of embryology and foetal development are crucially important elements within the education of undergraduate medical students. While it was previously identified that medical students consider embryology to be the

182

most challenging topic to learn when compared to other anatomical sub-disciplines (Ben Awadh et al. 2020), it has also been shown that medical students appreciate and value the importance and clinical relevance of embryology (Scott et al. 2013; Hamilton and Carachi 2014; Zaletel et al. 2016). In terms of broader medical training, gross anatomical knowledge is a fundamental area of understanding (Bãckers et al. 2010). It is vital therefore that medical students gain an appreciation of the embryological principles, developmental processes and three-dimensional morphology that support and underpin an understanding of the arrangement, structure and function of gross anatomical features in the adult. Furthermore, the inclusion of clinically relevant principles and scenarios is vital for contextualising undergraduate medical student understanding of course content and consequently for supporting the education of future clinicians (Sugand et al. 2010). Knowledge of embryology and developmental biology are important in this respect, through providing a basis for understanding, diagnosing and treating patients with congenital abnormalities, which arise within specific anatomical viscera and structures. Within the NU medical curriculum, developmental defects that are present in the embryonic heart, kidneys, gastrointestinal tract, respiratory system and craniofacial structures are considered and addressed through the specific inclusion of curricular learning outcomes. Moreover, in the context of specialised clinical cases and core conditions, embryology provides a basis for understanding the development of the reproductive system, which can underpin diagnosis and treatment of infertility and subfertility with respect to patients who are experiencing difficulties in conception. Within relevant course units of study, the delivery of content concerning early embryological processes and the development of the placenta are additionally intended to provide a deeper understanding of human pregnancy and to emphasise links between embryonic and foetal development and placental function and physiology, to support an appreciation of effective antenatal care and screening.

E. M. Abdel Meguid et al.

8.6.1.2 Pre-pandemic Curriculum Embryology education in the undergraduate pre-clinical Essentials of Medical Practice (EOMP) Bachelor of Medicine and Surgery (MBBS) curriculum (Year 1 and 2) at NU is embedded within specific case-based units of study. The current post-2017 EOMP curriculum consists of 25 consecutive units or ‘cases’, with the length of delivery of individual cases varying between periods of 1 and 4 weeks. The current and previous iterations of the undergraduate medical curriculum at NU have been described in detail in our previous work (Backhouse et al. 2017; Keenan and Ben Awadh 2019; Ben Awadh et al. 2020; Keenan and Powell 2020). Following a 3-week foundation block at the beginning of Year 1, which introduces novice students to fundamental clinical concepts and provides essential grounding in the biological sciences, each subsequent unit concerns a specific clinical case. For example, the early Year 1 cases focus on hypertension, cardiovascular disease, chronic kidney disease, anaemia and bowel cancer. Content is delivered in an integrated format, with teaching of the relevant basic life sciences, clinical and communication skills, clinical reasoning and early clinical experience delivered alongside knowledge-based learning outcomes within each case. Embryology learning outcomes are typically assessed in integrated summative closed-book single-best answer examinations. Prior to the onset of the global Covid-19 pandemic during academic year 2019–2020, core embryology content at NU was primarily delivered via didactic whole cohort lectures, which were supported by supplementary reading and online self-directed learning. Formative assessment of embryology was exclusively delivered in person via lecture polling software. A series of ten in-person embryology lectures were delivered across the 2-year duration of the EOMP portion of the course. Following an overview of embryology in the introductory foundation period, assessed learning outcomes relating to the clinically relevant embryology of the cardiovascular, renal, respiratory, gastrointestinal, and nervous systems are delivered, in addition to

8

Exploring Visualisation for Embryology Education: A Twenty-First-Century Perspective

content that considers embryology of the head and neck and craniofacial developmental abnormalities. A combined lecture and seminar series, which supplies further details of the key processes in early embryonic development, is included within the first case of Year 2. This specific case considers a series of patient narratives relating to conception, reproduction, subfertility and antenatal and neonatal care. In academic year 2019–2020, the embryology-related content of this case comprised two in-person small-group seminars on embryological developmental abnormalities and foetal and neonatal development. Five in-person lectures provided a detailed overview of early embryonic development, beginning with fertilisation and progressing through pre-implantation development, implantation, gastrulation, neurulation and embryonic folding.

8.6.1.3 Post-Pandemic Curriculum Since the onset of the Covid-19 pandemic in the United Kingdom March 2020, the EOMP curriculum has primarily been delivered in a blended format of remote synchronous and asynchronous teaching. This strategy of online learning has been provided via the bespoke medical virtual learning environment (VLE), which was originally designed, and is specifically tailored to support the post-2017 NU MBBS programme. Provision of in-person teaching has been delivered where possible within governmental and institutional guidelines and has primarily focused on essential practical training, including the performance of clinical examinations and procedures, and on laboratory-based gross anatomy. As such, it has been necessary for embryology teaching to be delivered exclusively via asynchronous means for the duration of the pandemic. Somewhat fortunately however, it has been possible to integrate embryology learning resources that had initially been planned and developed to be supplementary material to support face-to-face teaching within the pre-Covid19 curriculum. Since the onset of the pandemic, these resources have been further developed to fulfil requirements for core embryology learning.

8.6.2

8.6.2.1

183

Pre-Covid-19 Innovations for Embryology Learning

Social Media and Creative Art-Based Approaches In both current and pre-pandemic scenarios, it has been necessary to develop engaging, multimodal approaches that can effectively present human developmental processes as dynamic and threedimensional, with a view to supporting student understanding of clinically relevant embryology. The usage and effectiveness of social media (SoMe) in anatomical and medical education have been considered and outlined in detail in the previous work (Bahner et al. 2012; Cheston et al. 2013; Hennessy et al. 2016; Whyte and Hennessy 2017; Keenan et al. 2018). The fundamental pedagogic value of SoMe has been described in terms of the underlying theoretical principles of social learning (Bandura and Walters 1977; Vygotsky 1978; Bandura 2001), while the elements of collaboration, communication and active learning supported by SoMe have been identified as educational advantages (Ajjan and Hartshorne 2008; Bonzo and Parchoma 2010). However, despite the widespread use of SoMe in medical education, the specific application of SoMe within embryology education has not been comprehensively addressed in the literature. Similarly, art-based learning resources can support key elements of learning and cognition including observation, visualisation and spatial reasoning. In a comparable vein to SoMe, the educational literature with respect to the development and evaluation of art-based approaches in anatomy education is extensive (Nayak and Kodimajalu 2010; Naug et al. 2011; Tyler and Likova 2012; Lyon et al. 2013; Bell and Evans 2014; Keenan et al. 2017a, b; Pickering 2017; Greene 2018; Reid et al. 2019; Shapiro et al. 2019). While the three-dimensional and dynamic nature of embryonic development lends itself to delivery via corresponding sensory modalities (Lodge et al. 2016), descriptions of creative learning techniques in embryology education are notably lacking.

184

Based on the pedagogic principles of SoMe and art-based learning, we have proposed that combining these two educationally valuable approaches at NU could engage and support medical students in their learning of embryology while simultaneously providing additional opportunities for formative assessment. The contiguous series of early embryology lectures embedded within a single week of both the current NU EOMP curriculum and the previous pre-2017 curriculum have provided an opportunity to engage students for a specific and limited time-period in both creative pursuits and educational SoMe usage. A synergy of art-based approaches and SoMe were delivered in the form of encouraging student active participation in the competitive creation of innovative learning approaches to embryology. Once produced, students were encouraged to share their creations on Twitter using the hashtag #embryologyweek, with prizes awarded to students for the most effective approaches, as judged by educators. For accessibility and inclusion, students were also given the option to send their work directly to educators, for their entries to be considered and posted anonymously or to be shared only within the cohort. Entries included both static images and dynamic videos incorporating the use of stationery, foodstuffs and other household items, in addition to more conventional drawings, modelling and paper quilling to demonstrate key processes in embryonic development. Additionally, the polling feature of Twitter was utilised to provide functionality supporting the widespread delivery of single-best answer multiple-choice questions, created by both educators for their classes and by students for their peers. This was with a view to motivating and potentially enhancing student performance through formative assessments (Carrillo-de-laPena et al. 2009; Evans et al. 2014). Again, incentives were offered to those students who successfully engaged with the activity and in doing so created the most effective polls. While formal research was not conducted to identify the specific and comprehensive nature of the value of these approaches for student learning, educator observations, the extent of student engagement

E. M. Abdel Meguid et al.

with the hashtag and findings from formal course feedback evaluations indicated that those students who participated in these optional activities had been particularly stimulated to learn embryology. Due to the dynamic and advancing nature of SoMe, and the changing perceptions of successive generations, it is apparent from personal experience and wider educator perspectives (Border et al. 2019) that current student groups utilise of SoMe in a different and more cautious manner to previous cohorts. Moreover, it is likely that factors which can influence educator attrition in technology usage are becoming increasing apparent with respect to SoMe, including the phenomena of rapid replacement and changes in popularity of certain platforms by new and updated versions (Shelton 2016). While educator observations of decreases in EOMP SoMe engagement since 2016 would suggest that modern students prefer to independently engage in social messaging applications to facilitate communication and collaborative learning, the impact of Covid-19 on educational SoMe usage remains to be seen. However, it appears that the role of educators in facilitating SoMe-based learning has been significantly diminished in the time since notable anatomical and medical education studies were published (Cheston et al. 2013; Hennessy et al. 2016; Whyte and Hennessy 2017; Keenan et al. 2018), which is highlighted by recent attempts being limited to unilateral educator creation and SoMe dissemination of art-based embryology images (Fig. 8.2).

8.6.2.2

Development of a Prototype Digital Embryology Resource We have previously described the theoretical underpinnings and research-driven implementation of 3D digital visualisation technologies for gross anatomy education (Keenan and Ben Awadh 2019; Keenan and Powell 2020). Having done so, we propose that such principles can also be applied to embryology, with the dynamic and temporal elements of developmental processes additionally lending themselves to delivery via digital means. Consequently, in parallel to the integration of art-based SoMe approaches and the delivery of digital gross anatomy, we aimed

8

Exploring Visualisation for Embryology Education: A Twenty-First-Century Perspective

185

Fig. 8.2 Art-based demonstration of embryonic folding using modelling clay and Scotch egg. Posted on social media with #embryologyweek hashtag 27.09.2019

to produce digital embryology learning resources. This was with a view to delivering teaching with dynamic and three-dimensional models which could accurately capture and present the morphological and anatomical features of real human embryos. We proposed that such resources would benefit medical students through enhancing their embryology learning, when compared to existing imitative, static and two-dimensional images. The Human Developmental Biology Resource (HDBR) is a research tissue bank of donor embryological specimens ranging from 3 weeks to 20 weeks of development. Ethical approval is granted for the use of these human embryos for educational purposes. The HDBR is organised via two institutions, the Newcastle University Biosciences Institute and the Institute of Child Health at University College London (Casey and Evans 2011). A valuable resource for both researchers and educators, the HDBR Atlas (http://hdbratlas.org) is hosted by the HDBR group. The atlas consists of 3D-rendered embryo movies, with digitally painted anatomical domains, histological embryo sections and gene expression data (Kerwin et al. 2010; Gerrelli et al. 2015) (Fig. 8.3). Collaborating with, and utilising embryos provided by HDBR, we designed a student partnership project, based on key principles of student-partner development (Healey et al. 2014; Backhouse et al. 2017). The project aimed to produce a prototype embryology educational

resource which could display interactive, dynamic and three-dimensional digital models of the development of the gastrointestinal tract (Fig. 8.4) (Keenan et al. 2017c). Optical projection tomography (OPT) enabled cross-sectional embryological images to be generated from real human embryos at Carnegie stages 13–18, without the need to produce physical sections. For each cross section, the anatomical boundaries of the developing gut tube were identified, and MAPaint software (Edinburgh Mouse Atlas Project https://www.emouseatlas.org/emap/home. html) was used to define embryonic gastrointestinal structures by highlighting or ‘painting’ in distinct colours. Amira software (Thermo Scientific, version 5.3) was then used to generate short 3D video captures of embryos with painted domains rotating in 3600. The embryo models were then labelled using Adobe Photoshop (Photoshop CC 2015.5) with a view to embedding the annotated video captures and images into initially lecture-based and subsequently selfdirected, learning resources.

8.6.3

8.6.3.1

Approaches to Asynchronous Embryology Education During Covid-19

Integrated Embryology VLE Tutorial Previous work has identified the value of integrated embryology tutorials for enhancing

186

E. M. Abdel Meguid et al.

Fig. 8.3 HDBR Atlas ‘Organ Systems’ visual menu (http://hdbratlas.org/organ-systems.html) (top panels) and HDBR Atlas ‘Development of the Heart’ visual menu

(http://hdbratlas.org/organ-systems/cardiovascular-sys tem/heart.html) (lower panels), demonstrating a selection of resources available via the HDBR platform

student learning, through the delivery of interactive animations of developmental processes in two dimensions, in three dimensions and in cross sections (Marsh et al. 2008). We have attempted to recreate the value of this approach in our own resources, through developing an embryology VLE tutorial. Students follow our online embryology activities throughout the first 2 years of their integrated MBBS degree programme, accessing the resource at appropriate junctures, which are dictated by the sequence of learning within the course and each relevant individual case-based unit of study. Tutorial chapters initially introduce students to recommended external resources, to the descriptions of course learning outcomes, to the context and relationship to their current unit of study and to the clinical

relevance of the tutorial content. Learning activities include short (approximately 5–15 minutes) pre-recorded lecture captures (with picture in picture speaker view and accessible lecture slides). The first tutorial chapter, which students complete in the early weeks of the course, serves to guide them through some of the common terms, structures and processes in embryology. It is important for students to understand these basic terms and general principles before studying the embryology of specific anatomical regions and systems. The second chapter concerns heart development, which is relevant to learning outcomes relating to embryology of the heart and congenital abnormalities that are covered within in the ‘cardiovascular disease’ unit of study, including ventricular and

8

Exploring Visualisation for Embryology Education: A Twenty-First-Century Perspective

187

Fig. 8.4 HDBR sagittal sections of OPT embryos with painted anatomical domains of the gastrointestinal system (left panels), digitally painted anatomical domains of the gastrointestinal system mapped onto 360 rotating video captures of OPT embryos (middle panels) and labelled image of the gastrointestinal tract (right panels) (top,

Carnegie Stage 13; middle, Carnegie Stage 14; lower, Carnegie Stage 15). Consecutive videos and images at advancing Carnegie stages can demonstrate dynamic process of growth, rotation, herniation and retraction of the gut during embryonic development

atrial septal defects, and tetralogy of Fallot. It is recommended that students spend additional time on the first two chapters of the tutorial, in order to provide them with a strong basis for their future embryology learning. It is emphasised to students at this point, from the outset, that embryology can be a challenging topic due to the extensive terminology and the demands of understanding the dynamic and three-dimensional nature of development. Later Year 1 tutorial chapters explore the clinically relevant embryology of the kidneys and abnormalities including renal ectopia, agenesis and abnormal fusion; development of the gastrointestinal tract in which gastrointestinal differentiation, growth, rotation, herniation and retraction are considered, and stenosis, atresia, polyhydramnios, omphalocoele, congenital umbilical hernias, volvulus and malrotation are addressed. Year 2 chapters present activities to

support detailed understanding of early embryology and the development of the reproductive system, which deliver learning outcomes embedded in the unit of study concerning conception, fertility and development. The final chapter of the tutorial relates to the development of pharyngeal and craniofacial structures during the unit of study that covers motor neurone disease, head and neck anatomy and neuroanatomy. In addition to embedded formative single-best answer questions, each tutorial chapter is summarised by the integration of formative spotter questions and embryology crosswords.

8.6.3.2 HDBR Atlas Further to our prototype resource for gastrointestinal development, we have produced detailed guides and activities to support usage of HDBR resources and to supplement the threedimensional study of gastrointestinal and

188

E. M. Abdel Meguid et al.

cardiovascular embryology through integrating HDBR movies and labelled images (Fig. 8.5). HDBR activities have been embedded within tutorial chapters to summarise and supplement student understanding of three-dimensional morphological concepts in embryology once knowledge-based learning outcomes have been addressed by other means. When students first use the HDBR resources to study embryology within the cardiovascular unit, detailed instructions for using the atlas are provided alongside specific learning activities. When studying the heart, students are guided through usage of the HDBR Atlas to identify how the morphology of the heart alters during embryonic development, including how the process of septation and folding of the heart tube give rise to the progressive differentiation of chambers and outflow tract as development proceeds. Development of the paired aortic arch arteries can also be observed over time as these vessels are gradually remodelled into adult arteries. Labelled images are provided to support identification of named structures on digitally painted 3D models (Fig. 8.5).

8.6.3.3

Three-Dimensional Atlas of Human Embryology Having previously described our work on crosssectional anatomy using Sectra (Sectra AB, Linkoping, Sweden) resources (Keenan and Ben Awadh 2019; Ben Awadh et al. 2020), we have explored the capabilities of external resources that are available via the Sectra Educational Portal (Sectra AB, Linkoping, Sweden). Specifically, embryology resources provided by the University of Amsterdam 3D Atlas of Human Embryology (de Bakker et al. 2012, 2016) are accessible via Sectra, and we have begun to integrate them, in similar fashion to the HDBR resources, into asynchronous delivery of embryology through the use of interactive pdf documents. Within each document, from Carnegie stages CS7-CS23, students can identify the relative size of the embryo at the stage concerned, before activating three-dimensional functionality in order to interact with a digital embryo model. Using a mouse or touchscreen, students can zoom, rotate the embryo in 360 and select specific labelled anatomical structures. Features can be added, removed and faded to provide alternative views. These resources provide interactivity

Fig. 8.5 HDBR heart video captures of isolated digitally painted 360 rotating digital hearts and digitally painted hearts mapped onto 360 rotating video captures of OPT embryos (left panels). Labelled images of digitally painted

HDBR heart models created for integration into online asynchronous cardiovascular embryology tutorial (right panels)

8

Exploring Visualisation for Embryology Education: A Twenty-First-Century Perspective

and three-dimensional visualisation of embryonic structures and systems from gastrulation (15–17 days) to the beginning of the foetal stage (56–60 days). While we have primarily utilised these models thus far to support learning of the gastrointestinal and cardiovascular system, it is also possible to visualise and isolate the integumentary, skeletal and muscular systems; respiratory; urogenital; endocrine; lymphoid system; and nervous system and sense organs. To support student learning within postpandemic blended medical curricula, we will further develop and evaluate our asynchronous, interactive embryology resources included in VLE tutorials, HDBR and 3D Atlas resources at NU. Our experiences and research findings will have implications for educators aiming to enhance embryology education and for students seeking to improve their knowledge of human development and congenital abnormalities.

8.7

Conclusion

Embryology resources have evolved from handdrawn illustrations and photographic images to 3DP and VR. We propose that the development of embryology resources to support visualisation can initiate a dialogue with the students of embryology, with the aim of enhancing teaching and learning practices. In the future, it is crucial to evaluate the resources used for teaching embryology to strengthen and support evidence-based teaching practices. To improve student performance in embryology learning, we recommend that more time and effort should be devoted to the implementation of visualisation-based digital technology through videos and animations. Simultaneously, didactic large-group lectures should be transformed into learner-centred experiences with increased interaction and active student participation. Technology-enhanced learning strategies and modalities which support an understanding of dynamic and threedimensional embryological processes have the potential to develop deeper connections with learners. For twenty-first-century students, visual learning can contribute to increased interest,

189

interactivity, engagement and understanding of the educational content. Acknowledgements The author (IDK) would like to thank postgraduate and undergraduate student partners Abdullah Ben Awadh, Zahira Solim and Sasha Quigg for their contributions to embryology resource development and evaluation and to acknowledge Susan Lindsay, Janet Kerwin and the HDBR for their resource development and project collaborations. Website Resources for Visualisation in Embryology HDBR Atlas (http://hdbratlas.org) Online Embryology Animation Resources https://www.3dembryoatlas.com/copy-of-the-3d-atlasproject https://www.ehd.org/virtual-human-embryo/about. php?stage¼1 Sadler, T.W: Langman’s Medical Embryology with Simbryo CD-ROM http://embryo.soad.umich.edu Expand E-book - Kyoto Collection (first edn)

References Ajjan H, Hartshorne R (2008) Investigating faculty decisions to adopt web 2.0 technologies: theory and empirical tests. Internet High Educ 11(2):71–80. https://doi.org/10.1016/j.iheduc.2008.05.002 Alfalah S, Falah JF, Muhaidat N, Hudaib A, Koshebye D, AlHourani S (2019) Virtual reality learning environment in embryology education. World Academy of Science, Engineering and Technology. Int J Educ Pedagogical Sci 13:12 Allan C, Burel JM, Moore J, Blackburn C, Linkert M, Loynton S, Macdonald D, Moore WJ, Neves C, Patterson A, Porter M, Tarkowska A, Loranger B, Avondo J, Lagerstedt I, Lianas L, Leo S, Hands K, Hay RT, Patwardhan A, Best C, Kleywegt GJ, Zanetti G, Swedlow JR (2012) OMERO: flexible, model-driven data management for experimental biology. Nat Methods 9(3):245–253. https://doi.org/10. 1038/nmeth.1896 Arechaga J, Jimenez-Collado J, Ruano-Gil D (2009) A glance at Spanish embryology and teratology during the XX century through the academic life of Francisco orts-Llorca (1905-1993). Int J Dev Biol 53 (8–10):1165–1177. https://doi.org/10.1387/ijdb. 072492ja Azer SA (2012) Can “YouTube” help students in learning surface anatomy? Surg Radiol Anat 34:465–468. https://doi.org/10.1007/s00276-012-0935-x Bãckers A, Jerg-Bretzke L, Lamp C, Brinkmann A, Traue HC, Bockers TM (2010) The gross anatomy course: an analysis of its importance. Anat Sci Educ v3(n1):3–11. https://doi.org/10.1002/ase.124

190 Backhouse M, Fitzpatrick M, Hutchinson J, Thandi CS, Keenan ID (2017) Improvements in anatomy knowledge when utilizing a novel cyclical “observe-reflectdraw-edit-repeat” learning process. Anat Sci Educ 10 (1):7–22. https://doi.org/10.1002/ase.1616 Bahner DP, Adkins E, Patel N, Donley C, Nagel R, Kman NE (2012) How we use social media to supplement a novel curriculum in medical education. Med Teach 34 (6):439–444. https://doi.org/10.3109/0142159X.2012. 668245 Bandura A (2001) Social cognitive theory: an agentic perspective. Annu Rev Psychol 52(1):1–26 Bandura A, Walters RH (1977) Social learning theory Baratz G, Wilson-Delfosse A, Singelyn BM, Allan KC, Rieth GE, Ratnaparkhi R, Jenks BP, Carlton CE, Freeman BK, Wish-Baratz S (2019) Evaluating the Anatomage table compared to cadaveric dissection as a learning modality for gross anatomy. Med Sci Educ 29:499–506 Barry DS, Marzouk F, Chulak-Oglu K, Bennett D, Tierney P, O'Keeffe GW (2016b) Anatomy education for the YouTube generation. Anat Sci Educ 9(1):90–96 Barry DS, Tierney P, O'Keeffe GW (2016a) The need for ethical and pedagogical frameworks for developing online media in anatomy education. Anat Sci Educ 9 (5):498–499. https://doi.org/10.1002/ase.1628 Bell LT, Evans DJ (2014) Art, anatomy, and medicine: is there a place for art in medical education? Anat Sci Educ 7(5):370–378. https://doi.org/10.1002/ase.1435 Ben Awadh A, Clark J, Clowry G, Keenan ID (2020) Multimodal three-dimensional visualization enhances novice learner interpretation of basic cross-sectional anatomy. Anat Sci Educ. https://doi.org/10.1002/ase.2045 Bergsmann E, Schultes MT, Winter P, Schober B, Spiel C (2015) Evaluation of competence-based teaching in higher education: from theory to practice. Eval Program Plann 52:1–9. https://doi.org/10.1016/j. evalprogplan.2015.03.001 Berman NB, Fall LH, Maloney CG, Levine DA (2008) Computer-assisted instruction in clinical education: a roadmap to increasing CAI implementation. Adv Health Sci Educ 13(3):373–383. https://doi.org/10. 1007/s10459-006-9041-3 Betrancourt M (2005) The animation and interactivity principles in multimedia learning. In: Mayer R (ed) The Cambridge handbook of multimedia learning. Cambridge University Press, New York, NY, pp 287–296 Bezner SK, Hodgman EI, Diesen DL, Clayton JT, Minkes RK, Langer JC, Chen LE (2014) Pediatric surgery on YouTube™: is the truth out there? J Pediatr Surg 49 (4):586–589. https://doi.org/10.1016/j.jpedsurg.2013. 08.004 Blechschmidt E (1961) The stages of human development before birth; an introduction to human embryology. Saunders, Philadelphia Bonzo J, Parchoma G (2010) The paradox of social media and higher education institutions. In: Proceedings of the 7th international conference on networked learning, pp 45–56

E. M. Abdel Meguid et al. Border S, Hennessy C, Pickering J (2019) The rapidly changing landscape of student social media use in anatomy education. Anat Sci Educ 12(5):577–579. https://doi.org/10.1002/ase.1875 Carlson BM (2002) Embryology in the medical curriculum. Anat Rec 269(2):89–98. https://doi.org/10.1002/ ar.10075 Carrillo-de-la-Pena MT, Bailles E, Caseras X, Martinez A, Ortet G, Perez J (2009) Formative assessment and academic achievement in pre-graduate students of health sciences. Adv Health Sci Educ 14(1):61–67. https://doi.org/10.1007/s10459-007-9086-y Casey G, Evans T (2011) Designing for learning: online social networks as a classroom environment. Int Rev Res Open Dist Learn 12(7):1–26 Chen R, Grierson LE, Norman GR (2015) Evaluating the impact of high-and low-fidelity instruction in the development of auscultation skills. Med Educ 49 (3):276–285 Cheston CC, Flickinger TE, Chisolm MS (2013) Social media use in medical education: a systematic review. Acad Med 88(6):893–901. https://doi.org/10.1097/ ACM.0b013e31828ffc23 Cho BH, Cao-Berg I, Bakal JA, Murphy RF (2012) OMERO.Searcher: content-based image search for microscope images. Nat Methods 9(7):633–634. https://doi.org/10.1038/nmeth.2086 Clark JM, Paivio A (1991) Dual coding theory and education. Educ Psychol Rev 3(3):149–210. https://doi.org/ 10.1007/BF01320076 Cook DA (2007) Web-based learning: pros, cons and controversies. Clin Med 7(1):37–42 Cowan M, Arain NN, Assale TSA, Assi AH, Albar RA, Ganguly PK (2010) Student-centered integrated anatomy resource sessions at Alfaisal university. Anat Sci Educ 3(5):272–275. https://doi.org/10. 1002/ase.176 Curran V, Simmons K, Matthews L, Fleet L, Gustafson DL, Fairbridge NA, Xu X (2020) YouTube as an educational resource in medical education: a scoping review. Med SciEduc 30(4):1775–1782. https://doi. org/10.1007/s40670-020-01016-w de Bakker BS, de Jong KH, Hagoort J, de Bree K, Besselink CT, de Kanter FE, Veldhuis T, Bais B, Schildmeijer R, Ruijter JM, Oostra RJ, Christoffels VM, Moorman AF (2016) An interactive threedimensional digital atlas and quantitative database of human development. Science 354(6315). https://doi. org/10.1126/science.aag0053 de Bakker BS, de Jong KH, Hagoort J, Oostra RJ, Moorman AF (2012) Towards a 3-dimensional atlas of the developing human embryo: the Amsterdam experience. Reprod Toxicol 34(2):225–236. https:// doi.org/10.1016/j.reprotox.2012.05.087 Dhanantwari P, Lee E, Krishnan A, Samtani R, Yamada S, Anderson S, Lockett E, Donofrio M, Shiota K, Leatherbury L, Lo CW (2009) Human cardiac development in the first trimester. Circulation 120(4):343–351. https://doi.org/10.1161/CIRCULATIONAHA.108. 796698

8

Exploring Visualisation for Embryology Education: A Twenty-First-Century Perspective

Dong C, Goh PS (2015) Twelve tips for the effective use of videos in medical education. Med Teach 37(2):140–145. https://doi.org/10.3109/0142159x.2014.943709 Evans DJ (2011) Using embryology screencasts: a useful addition to the student learning experience? Anat Sci Educ 4(2):57–63. https://doi.org/10.1002/ase.209 Evans DJ, Zeun P, Stanier RA (2014) Motivating student learning using a formative assessment journey. J Anat 224(3):296–303. https://doi.org/10.1111/joa.12117 Fakoya FA, Emmanouil-Nikoloussi E, Sharma D, Moxham BJ (2017) A core syllabus for the teaching of embryology and teratology to medical students. Clin Anat 30(2):159–167. https://doi.org/10.1002/ca.22802 Falah J, Charissis V, Khan S, Chan W, Alfalah SFM, Harrison DK (2015) Development and evaluation of virtual reality medical training system for anatomy education. In: Intelligent Systems in Science and Information 2014. Springer International Publishing, Cham, pp 369–383 Flynn L, Jalali A, Moreau KA (2015) Learning theory and its application to the use of social media in medical education. Postgrad Med J: postgradmedj-2015-133358 Gerrelli D, Lisgo S, Copp AJ, Lindsay S (2015) Enabling research with human embryonic and fetal tissue resources. Development 142(18):3073–3076. https:// doi.org/10.1242/dev.122820 Gikas J, Grant M (2013) Mobile computing devices in higher education: student perspectives on learning with cellphones, Smartphones & Social Media. Internet High Educ 19:18–26. https://doi.org/10.1016/j. iheduc.2013.06.002 Greene SJ (2018) The use and effectiveness of interactive progressive drawing in anatomy education. Anat Sci Educ 11(5):445–460. https://doi.org/10.1002/ase.1784 Habbal OA, Harris PF (1995) Teaching of human anatomy: a role for computer animation. J Audiov Media Med 18(2): 69 –73 . h ttp s:/ /do i.o rg/1 0.3 10 9/ 17453059509022997 Hamilton WJ, Boyd JD, Mossman HW (1972) Prenatal development of form and function Hamilton, Boyd and Mossman's human embryology, 4th edn. Heffer, Cambridge Hamilton J, Carachi R (2014) Clinical embryology: is there still a place in medical schools today? Scott Med J 59(4):188–192. https://doi.org/10.1177/ 0036933014550953 Healey M, Flint A, Harrington K (2014) Engagement through partnership: students as partners in learning and teaching in higher education. Higher Education Academy. Accessed Jan 6, 2016, from https://www. heacademy.ac.uk/engagement-through-partnershipstudents-partners-learning-and-teaching-highereducation Helsper EJ, Eynon R (2010) Digital natives: where is the evidence? Br Educ Res J 36(3):503–520. https://doi. org/10.1080/01411920902989227 Henderson M, Selwyn N, Aston R (2015) What works and why? Student perceptions of ‘useful’ digital technology in university teaching and learning. Stud High Educ 1:13. https://doi.org/10.1080/03075079.2015. 1007946

191

Hennessy CM, Kirkpatrick E, Smith CF, Border S (2016) Social media and anatomy education: using twitter to enhance the student learning experience in anatomy. Anat Sci Educ 9(6):505–515. https://doi.org/10.1002/ ase.1610 Hennessy CM, Royer DF, Meyer AJ, Smith CF (2020) Social media guidelines for anatomists. Anat Sci Educ 13(4):527–539. https://doi.org/10.1002/ase.1948 Hill JP (1948) A biographical sketch of his career and an appreciation of his work, together with a bibliography of his published writings. J Anat 82(Pt 1–2):i4–i8 Hill MA (2018) Developing the digital Kyoto collection in education and research. Anat Rec (Hoboken) 301 (6):998–1003. https://doi.org/10.1002/ar.23783 Hinrichsen K (1990) Humanembryologie. SpringerVerlag, Berlin His W (1880) Anatomie menschlicher Embryonen. Verlag von F.C.W. Vogel, Leipzig Holland JC, Pawlikowska T (2019) Learning cardiac embryology–which resources do students use, and why? Med Sci Educ 20:1051–1060. https://doi.org/ 10.1007/s40670-019-00803-4 Hopwood N (2007) A history of normal plates, tables and stages in vertebrate embryology. Int J Dev Biol 51 (1):1–26. https://doi.org/10.1387/ijdb.062189nh Jaffar AA (2012) YouTube: an emerging tool in anatomy education. Anat Sci Educ 5(3):158–164. https://doi. org/10.1002/ase.1268 Keenan ID, Backhouse M, Fitzpatrick M, Hutchinson J, Thandi CS (2017a) Improvements in anatomy knowledge when using a novel cyclical artistic learning process. In: Anatomical society summer meeting at Brighton and Sussex Medical School, vol 2. Journal of Anatomy, pp 354–372. https://doi.org/10.1111/joa. 12558 Keenan ID, Ben Awadh A (2019) Integrating 3D visualisation Technologies in Undergraduate Anatomy Education. Adv Exp Med Biol 1120:39–53. https://doi. org/10.1007/978-3-030-06070-1_4 Keenan ID, Hutchinson J, Bell K (2017b) Twelve tips for implementing artistic learning approaches in anatomy education. MedEdPublish 6:2. https://doi.org/10. 15694/mep.2017.000106 Keenan I, Kerwin J, Lindsay S (2017c) Enhancing student learning of human embryology with aprototype e-learning resource. In: Anatomical society Winter meeting, vol 2016, pp 445–468 Keenan ID, Powell M (2020) Interdimensional travel: visualisation of 3D-2D transitions in anatomy learning. Adv Exp Med Biol 1235:103–116. https://doi.org/10. 1007/978-3-030-37639-0_6 Keenan ID, Slater JD, Matthan J (2018) Social media: insights for medical education from instructor perceptions and usage. MedEdPublish:7 Kerwin J, Yang Y, Merchan P, Sarma S, Thompson J, Wang X, Sandoval J, Puelles L, Baldock R, Lindsay S (2010) The HUDSEN atlas: a three-dimensional (3D) spatial framework for studying gene expression in the developing human brain. J Anat 217 (4):289–299. https://doi.org/10.1111/j.1469-7580. 2010.01290.x

192 Lodge JM, Hansen L, Cottrell D (2016) Modality preference and learning style theories: rethinking the role of sensory modality in learning. Learning: Research and Practice 2(1):4–17. https://doi.org/10.1080/23735082. 2015.1083115 Lowe R (2004) Interrogation of a dynamic visualization during learning. Learn Instr Course Lect 14:257–274. https://doi.org/10.1016/j.learninstruc.2004.06.003 Lyon P, Letschka P, Ainsworth T, Haq I (2013) An exploratory study of the potential learning benefits for medical students in collaborative drawing: creativity, reflection and 'critical looking'. BMC Med Educ 13:86. https://doi.org/10.1186/1472-6920-13-86 Marsh KR, Giffin BF, Lowrie DJ (2008) Medical student retention of embryonic development: impact of the dimensions added by multimedia tutorials. Anat Sci Educ 1(6):252–257. https://doi.org/10.1002/ase.56 Mayer RE (2002) Cognitive theory and the Design of Multimedia Instruction: an example of the two-way street between cognition and instruction. New Dir Teach Learn 2002(89):55–71. https://doi.org/10.1002/ tl.47 Mayer RE (2010) Applying the science of learning to medical education. Med Educ 44(6):543–549. https:// doi.org/10.1111/j.1365-2923.2010.03624.x Mayer RE (2014) Cognitive theory of multimedia learning. In: Mayer RE (ed) The Cambridge handbook of multimedia learning, 2nd edn. Cambridge University Press Mayer RE, Moreno R (2002) Animation as an aid to multimedia learning. Educ Psychol Rev 14(1):87–99 Mayer RE, Moreno R (2003) Nine ways to reduce cognitive load in multimedia learning. Educ Psychol 38 (1):43–52. https://doi.org/10.1207/s15326985ep3801_ 6 McBride JM, Drake RL (2018) National survey on anatomical sciences in medical education. Anat Sci Educ 11(1):7–14. https://doi.org/10.1002/ase.1760 Minot CS (1905) The Harvard embryological collection. J Med Res 13(5):499–522 491 Moraes SG, Pereira LA (2010) A multimedia approach for teaching human embryology: development and evaluation of a methodology. Ann Anat 192(6):388–395. https://doi.org/10.1016/j.aanat.2010.05.005 Naug HL, Colson NJ, Donner DG (2011) Promoting metacognition in first year anatomy laboratories using plasticine modeling and drawing activities: a pilot study of the "blank page" technique. Anat Sci Educ 4 (4):231–234. https://doi.org/10.1002/ase.228 Nayak SB, Kodimajalu S (2010) Progressive drawing: a novel "lid-opener" and "monotony-breaker". Anat Sci Educ 3(6):326–329. https://doi.org/10.1002/ase.172 O’Carroll AM, Westby EP, Dooley J, Gordon KE (2015) Information-seeking behaviors of medical students: a cross-sectional web-based survey. JMIR Med Educ 1 (1):e4. https://doi.org/10.2196/mededu.4267 O’Rahilly R, Meyer DB (1979) The timing and sequence of events in the development of the human vertebral column during the embryonic period proper. Anat Embryol 157(2):167–176. https://doi.org/10.1007/ BF00305157

E. M. Abdel Meguid et al. Pickering JD (2017) Measuring learning gain: comparing anatomy drawing screencasts and paper-based resources. Anat Sci Educ 10(4):307–316. https://doi. org/10.1002/ase.1666 Plunkett C, Dueñas A, Stratford J, Leppek N, Lee LM (2019) Embryos in gross anatomy laboratory? The educational impact of 3D printed embryo model integration in medical basic sciences education. FASEB J 33(S1):17.11–17.11. https://doi.org/10.1096/fasebj. 2019.33.1_supplement.17.1 Prange-Kiel J, Champine JG, Winkler AJ, Twickler DM (2016) Embryology, anatomy, and radiology of cervical cysts and cleft lip/palate: a team-based learning module for medical students. MedEdPORTAL 12: 10484. https://doi.org/10.15766/mep_2374-8265. 10484 Prober CG, Khan S (2013) Medical education reimagined: a call to action. Acad Med 88(10):1407–1410. https:// doi.org/10.1097/ACM.0b013e3182a368bd Raikos A, Waidyasekara P (2014) How useful is YouTube in learning heart anatomy? Anat Sci Educ 7(1):12–18. https://doi.org/10.1002/ase.1361 Rao MP (2012) Animations in medical education: you can do it! Med J Patil Univ 52011-02912-001:18–22. https://doi.org/10.4103/0975-2870.97502 Reid S, Shapiro L, Louw G (2019) How haptics and drawing enhance the learning of anatomy. Anat Sci Educ 12(2):164–172. https://doi.org/10.1002/ase.1807 Richardson MK, Narraway J (1999) A treasure house of comparative embryology. Int J Dev Biol 43 (7):591–602 Rosenthal J, Mangal V, Walker D, Bennett M, Mohun TJ, Lo CW (2004) Rapid high resolution three dimensional reconstruction of embryos with episcopic fluorescence image capture. Birth Defects Res C Embryo Today 72 (3):213–223. https://doi.org/10.1002/bdrc.20023 Ruiz JG, Cook DA, Levinson AJ (2009) Computer animations in medical education: a critical literature review. Med Educ 43(9):838–846. https://doi.org/10. 1111/j.1365-2923.2009.03429.x Sagar TV, Viveka S (2016) Assisted e-learning computer program as a teachinglearning resource on human embryology. Indian J Appl Res 5:540–544 Schleich J-M, Dillenseger J-L, Houyel L, Almange C, Anderson RH (2009) A new dynamic 3D virtual methodology for teaching the mechanics of atrial septation as seen in the human heart. Anat Sci Educ 2(2):69–77. https://doi.org/10.1002/ase.74 Scott KM, Charles AR, Holland AJA (2013) Clinical embryology teaching: is it relevant anymore? ANZ J Surg 83(10):709–712. https://doi.org/10.1111/ans. 12213 Shapiro L, Bell K, Dhas K, Branson T, Louw G, Keenan ID (2019) Focused multisensory anatomy observation and drawing for enhancing social learning and threedimensional spatial understanding. Anat Sci Educ. https://doi.org/10.1002/ase.1929 Shelton C (2016) Giving up technology and social media: why university lecturers stop using technology in teaching. Technology, Pedagogy and Education:1–19. https://doi.org/10.1080/1475939X.2016.1217269

8

Exploring Visualisation for Embryology Education: A Twenty-First-Century Perspective

Shirky C (2008) Web 2.0 Expo NY: it’s not information overload, it’s filter failure. Video, September Smith BR, Huff DS, Johnson GA (1999) Magnetic resonance imaging of embryos: an internet resource for the study of embryonic development. Comput Med Imaging Graph 23(1):33–40. https://doi.org/10.1016/s08956111(98)00061-5 Streeter GL (1948) Developmental horizons in human embryos. Contrib Embryol Wash 32:133–203 Sugand K, Abrahams P, Khurana A (2010) The anatomy of anatomy: a review for its modernization. Anat Sci Educ 3(2):83–93. https://doi.org/10.1002/ase.139 Sweller J (1988) Cognitive load during problem solving: effects on learning. Cogn Sci 12:257–285 Tyler CW, Likova LT (2012) The role of the visual arts in the enhancing the learning process. Front Hum Neurosci 6:8. https://doi.org/10.3389/fnhum.2012. 00008 Ungi T, Yeo CT, Paweena U, McGraw RC, Fichtinger G (2011) Augmented reality needle guidance improves facet joint injection training. Int Soc Optics Photo SPIE Med Image:79642E–79642E. https://doi.org/10.1117/ 12.877830 Upson-Taboas CF, Montoya R, O'Loughlin VD (2019) Impact of cardiovascular embryology animations on short-term learning. Adv Physiol Educ 43(1):55–65. https://doi.org/10.1152/advan.00121.2018 Vygotsky L (1978) Mind in society: the development of higher psychological processes Harvard University press. Massachusetts, Cambridge

193

Watt ME, McDonald SW, Watt A (1996) Computer morphing of scanning electron micrographs: an adjunct to embryology teaching. Surg Radiol Anat: SRA 18 (4):329–333. https://doi.org/10.1007/BF01627612 Whyte W, Hennessy C (2017) Social media use within medical education: a systematic review to develop a pilot questionnaire on how social media can be best used at BSMS. MedEdPublish 6:2. https://doi.org/10. 15694/mep.2017.000083 Yamada S, Hill M, Takakuwa T (2015) Human Embryology. In InTech. https://doi.org/10.5772/61453 Yamada S, Samtani RR, Lee ES, Lockett E, Uwabe C, Shiota K, Anderson SA, Lo CW (2010) Developmental atlas of the early first trimester human embryo. Dev Dyn 239(6):1585–1595. https://doi.org/10.1002/dvdy.22316 Yamada S, Uwabe C, Nakatsu-Komatsu T, Minekura Y, Iwakura M, Motoki T, Nishimiya K, Iiyama M, Kakusho K, Minoh M, Mizuta S, Matsuda T, Matsuda Y, Haishi T, Kose K, Fujii S, Shiota K (2006) Graphic and movie illustrations of human prenatal development and their application to embryological education based on the human embryo specimens in the Kyoto collection. Dev Dyn 235(2):468–477. https://doi.org/10.1002/dvdy.20647 Zaletel I, Maric G, Gazibara T, Rakocevic J, Labudovic Borovic M, Puskas N, Bajcetic M (2016) Relevance and attitudes toward histology and embryology course through the eyes of freshmen and senior medical students: experience from Serbia. Ann Anat 208:217– 221. https://doi.org/10.1016/j.aanat.2016.07.002

9

How Artificial Intelligence and Machine Learning Is Assisting Us to Extract Meaning from Data on Bone Mechanics? Saeed Mouloodi, Hadi Rahmanpanah, Colin Burvill, Colin Martin, Soheil Gohari, and Helen M. S. Davies

Abstract

Dramatic advancements in interdisciplinary research with the fourth paradigm of science, especially the implementation of computer science, nourish the potential for artificial intelligence (AI), machine learning (ML), and artificial neural network (ANN) algorithms to be applied to studies concerning mechanics of bones. Despite recent enormous advancement in techniques, gaining deep knowledge to find correlations between bone shape, material, mechanical, and physical responses as well as properties is a daunting task. This is due to both complexity of the material itself and the convoluted shapes that this complex material

The original version of the chapter has been revised. The missing author ‘Colin Burvill’ has been added to the author list. The correction to this chapter can be found at https://doi.org/10.1007/978-3-030-87779-8_15 S. Mouloodi (*) · H. Rahmanpanah · C. Burvill · S. Gohari Department of Mechanical Engineering, The University of Melbourne, Melbourne, Australia e-mail: [email protected]; saeed. [email protected]; [email protected] C. Martin Sperero Pty. Ltd., Melbourne, Australia

forms. Moreover, many uncertainties and ambiguities exist concerning the use of traditional computational techniques that hinders gaining a full comprehension of this advanced biological material. This book chapter offers a review of literature on the use of AI, ML, and ANN in the study of bone mechanics research. A main question as to why to implement AI and ML in the mechanics of bones is fully addressed and explained. This chapter also introduces AI and ML and elaborates on the main features of ML algorithms such as learning paradigms, subtypes, main ideas with examples, performance metrics, training algorithms, and training datasets. As a frequently employed ML algorithm in bone mechanics, feedforward ANNs are discussed to make their taxonomy and working principles more readily comprehensible to researchers. A summary as well as detailed review of papers that employed ANNs to learn from collected data on bone mechanics are presented. Reviewing literature on the use of these data-driven tools is essential since their wider application has the potential to: improve clinical assessments enabling realtime simulations; avoid and/or minimize injuries; and, encourage early detection of such injuries in the first place.

H. M. S. Davies Department of Veterinary Biosciences, The University of Melbourne, Melbourne, Australia # The Author(s), under exclusive license to Springer Nature Switzerland AG 2022, corrected publication 2023 P. M. Rea (ed.), Biomedical Visualisation, Advances in Experimental Medicine and Biology 1356, https://doi.org/10.1007/978-3-030-87779-8_9

195

196

S. Mouloodi et al.

Keywords

Latest trend · Bone mechanics · Artificial intelligence (AI) · Machine learning (ML) · Artificial neural network (ANN) · Fourth paradigm of science

9.1

An Introduction to the Book Chapter

Dramatic advancements in interdisciplinary research, especially the implementation of computer science, nourish the potential for artificial intelligence and machine learning algorithms to be applied to biomedical engineering and the studies concerning mechanics of living tissues. Artificial Intelligence is the title given to a rapidly expanding area of computer analysis of data sets or observations. A very successful application of AI is the process of regression tasks and pattern recognition. More recently artificial intelligence is being applied to observations of objects and situations that that are proving too complex for even the most advanced traditional analysis techniques. One area of application involves biological structures. Machine learning (ML) is a subset of artificial intelligence which involves a process of repetitive learning from a large set of observations. Machine learning uses algorithms and develops models from learning through data and observation. Deep learning algorithms such as feedforward artificial neural networks (ANNs) and multilayer artificial neural networks (ANN), are a sub-category of machine learning (Fig. 9.1). They are commonly employed to analyse complex models encompassing massive amounts of data (models with larger datasets). In contrast finite element analysis (FEA) and Computer aided design (CAD) are both limited by the requirement to include material properties into the calculations that underlie their models. Hence their analysis of data is limited by previous knowledge and assumptions concerning the structures that are being modelled. The ability of deep learning algorithms to develop models independent of any assumptions provides a way to

progress understanding of complex materials and structures as long as sufficient good quality data that is relevant to the structure being modelled can be employed to train and test the ANN. Being at the forefront of artificial intelligence, deep learning algorithms (such as feedforward artificial neural networks) are providing considerable impetus for advancing knowledge in the field of bone mechanics and to address many of the largely open forward and inverse problems associated with bone adaptation or modelling. These investigations gain advantage from the spectacular advances in imaging techniques, computers, and enhanced precision as well as the increased power of mechanical testing and our improved realization of previously ignored features associated with testing methods (Currey 2009). The recent advances in computational modelling of bones (Zadpoor 2013) stems from such breakthroughs in these fundamental features. Advances in science and engineering have produced sufficient design knowledge to produce the supersonic Concorde passenger aircraft in just 70 years since man’s first powered flight. While some of the design knowledge needed trial and error, this process has developed for mechanical systems to the level where unknowns are reduced to very small areas with low risk. In contrast biological structures have evolved over millennia by natural selection i.e. random trial and error resulting in physical shapes and material properties far more complex than their mechanical equivalents. As a direct consequence, understanding the properties of biological materials is proving extremely challenging. For example, bones might be considered one of the simpler biological components but even defining their mechanical properties under loading, is proving intractable with existing engineering approaches. This book chapter aims to review the employment of artificial intelligence and machine learning algorithms in the study of bone mechanics. Research on bone adaptation has progressed since the time when Wolff observed and stated that bones grow according to the forces exposed to them representing an inherent relationship between bone loading and their shape and morphology (Wolff 1892, 2012). Functions of the

9

How Artificial Intelligence and Machine Learning Is Assisting Us to. . .

197

Fig. 9.1 Relationships between artificial intelligence, machine learning, and deep learning

Machine learning (ML)

bone and the types of loads it is exposed to dictate its shape, because the shape of a bone is intimately correlated to its function. Investigations into long bone adaptations to loading are intriguing because long bones (such as the humerus, radius, ulna, femur, tibia, and metacarpals) possess a relatively simple shape, and furthermore, what they are required to do under loads is reasonably straightforward to understand (Currey 2006, 2014). A deeper understanding of extremely well-engineered, if not optimised, structures such as long bones offers generalised opportunities for researchers. Long bones have engineering features such as being thick-walled hollow tubes, expanded at the ends, and being built in a way to have a minimized mass while performing their particular function (Currey 2010). Bones are highly complex biological materials that possess exceptional mechanical properties, and therefore are characterized as complex engineering structures (Mouloodi et al. 2020b) and are also categorized as nanocomposite and anisotropic solids (Keaveny et al. 2001; Rahmanpanah et al. 2020a). Bones are only one component of the biological make-up of mammals and on face value a simple load-bearing component. While bones appear to have a simple function in biological structures, which is primarily to carry compressive loads, they exhibit complex shapes and are composed of complex materials. This living biological material possesses complex hierarchical structure that entails exceptional mechanical properties, such as specific strength and

Deep learning (DL)

Artificial intelligence (AI)

stiffness as well as high fracture toughness (Barkaoui et al. 2016). Such an extraordinarily hierarchical essence of bones’ structures and mechanical properties makes bone mechanics a current outstanding research topic. The mechanical properties and responses of these advanced engineering structures have been shown, for example, to be density-dependent (Carter and Hayes 1977; Rice et al. 1988), site- and lengthdependent (Fradinho et al. 2015; Nobakhti et al. 2017), force- and loading-rate-dependent (Kulin et al. 2011; Mouloodi et al. 2021a; Rostedt et al. 1998), time-dependent (viscoelasticity) (Manda et al. 2017; Mouloodi et al. 2020a; Xie et al. 2017), strain-rate-dependent (exhibiting both viscoelastic and viscoplastic behavior) (Johnson et al. 2010), size- and scale-dependent (Frame et al. 2017; Mirzaali et al. 2016), shape-dependent (Davies 2001; Mouloodi et al. 2019a), age-dependent (Kulin et al. 2011; Mouloodi et al. 2020b; Patton et al. 2019), sex-dependent (Patton et al. 2019). It would be difficult to find another material that possesses such multivariable and exceptional mechanical properties (Mouloodi 2020). In contrast the material properties of metals used in many high load applications possess simple, mostly linear, characteristics in their normal operating range. Such an extraordinarily hierarchical essence of its structures and mechanical properties makes bone a difficult material to fully comprehend. It hinders the development of universally accepted theoretical or empirical constitutive models incorporating all the mentionedabove influential parameters. It prevents

198

researchers from: (1) arriving at a consensus in implementing an optimal constitutive model in computational and numerical-based simulation packages (such as FEA and computational fluid dynamics); or (2) encourages their adoption of simple constitutive models with inclusion of many simplifying assumptions in these packages. This ignores the multi-factorial nature of bone. Nonetheless, such remarkable properties and tackling these challenges in the realm of engineering makes bones intriguing materials for researchers and engineers to investigate further, and if a true comprehension of these sophisticated materials is achieved using such novel techniques and innovative tools, our understanding of less complex engineering structures (such as composites, nanomaterials, and alloys) will be enhanced (Mouloodi 2020). This greatly assists with the establishment of a general framework that enables importing and inclusion of all the parameters or variables influencing responses and behavior of such intricate engineering materials and that eventually concludes with integrating or replacing the conventional constitutive models. Employing artificial intelligence techniques using different machine learning algorithms such as artificial neural networks is a key to establishing this general framework and to tackling the previously mentioned challenges in mechanical and biomedical engineering disciplines. It is an irony that machine learning which depends on a process of repetitive learning from a large set of observations is proving to be an important approach to understanding the properties of biological systems, which themselves have been created by repetitive selection. This book chapter introduces machine learning and artificial neural network algorithms to then summarize their successful applications to bone mechanics research. Section 9.2 offers an introduction to artificial intelligence and machine learning and it reinforces the importance of their employment in this challenging field of research. Section 9.3 provides an explanation of ML algorithms and provides the reader with taxonomy and several types of ML algorithms. Important features of ML algorithms are also explored

S. Mouloodi et al.

in that section. Feedforward artificial neural networks that are widely-employed in bone mechanics research are further explained in Sect. 9.4. A detailed elaboration of research that employed artificial neural networks in the study of bone mechanics is presented in Sect. 9.5. Finally, Sect. 9.6 summarizes the importance of this field of study and the implementation of these data-driven tools and provides some insights for future research.

9.2

The Applications of Artificial Intelligence and Machine Learning to Bone Mechanics Research

Table 9.1 summarizes the application of ML algorithms in the study of bone mechanics research. Feedforward ANNs are frequently employed in this field of science, and hence a detailed explanation of machine learning and its main features (Sect. 9.3) as well as feedforward ANNs (Sect. 9.4) are included later. To further elaborate on the details of some of the main papers reviewed in this book chapter, Sect. 9.5 is presented to make the readers more familiar with the types and in-depth details of work that has been performed in bone mechanics research.

9.2.1

What Are Artificial Intelligence and Machine Learning?

Artificial intelligence (AI) is a broad concept and can be considered as a predefined set of rules introduced to a computer to simulate some intellectual human behavior to then perform a task, for example, playing chess or finding a two-dimensional path for a moving object to chase another object. Artificial intelligence, also referred to as cognitive simulation or information processing psychology (Simon 1983), emerged as a computer science discipline in the mid-1950s, and since then AI has offered solutions to many practical engineering problems that otherwise required human intelligence (Pham and Pham 1999). Hence, AI is regarded as human

9

How Artificial Intelligence and Machine Learning Is Assisting Us to. . .

199

Table 9.1 Examples of research that employed machine learning algorithms in the study of bone mechanics Authors (Year) Mouloodi et al. (2021a) Rahmanpanah et al. (2020a)

Type of machine learning employed Feedforward ANN Two in-series feedforward ANNs

Mouloodi et al. (2020a)

Feedforward ANN

Mouloodi et al. (2020b)

Feedforward ANN

Mohanty et al. (2019) Dattatrey et al. (2019) Vukicevic et al. (2018) Tiwari and Kumar (2018) Taylor et al. (2017) Garijo et al. (2017)

Feedforward ANN Feedforward ANN Feedforward ANN Feedforward ANN Feedforward ANN Feedforward ANN

Barkaoui et al. (2016)

Feedforward ANN

Khaterchi et al. (2015) Ardestani et al. (2014a) Garijo et al. (2014)

Feedforward ANN

Barkaoui et al. (2014) Hambli and Hattab (2013)

Feedforward ANN

Zadpoor et al. (2013) Campoli et al. (2012) Favre et al. (2012) Hambli (2011a)

Feedforward ANN

Feedforward ANN Feedforward ANN and support vector machine

Feedforward ANN

Feedforward ANN Feedforward ANN Feedforward ANN

Main outcome Prediction of stiffness of a long bone (equine MC3) from the bones’ geometrical parameters and ex-vivo experimental measurements Prediction of cyclic load-displacement curves of MC3 bones: a complex biological structure possessing exotic mechanical properties) Prediction of total displacement of bones under cyclic loading (replacing constitutive equations and finite element analysis by datadriven methods using ANNs to enable rapid predictions that are useful for clinical applications) Prediction of cyclic mechanical loading of a long bone from experimental measurements (regression analysis) Classification of age of bones from experimental measurements (categorizing approach using pattern recognition algorithms) Prediction of micro-architectural parameters of human cortical bones: Pore diameter, pore density, and porosity Prediction of bones mineral apposition rate or mineralising surface from cyclic loading parameters (i.e. cycles, frequency, and strain) Prediction of bone fracture toughness: R-curve slope, toughness threshold, and stress intensity factor from clinical measures Prediction of bones mineral apposition rate from cyclic loading parameters (i.e. cycles, frequency, and strain) Prediction of femoral strains and fracture loads to fulfil clinical purposes in real time from clinically obtainable measurements Prediction of the forces that a subject-specific tibia is supporting from CT images of the patient (and hence knowing their bone geometry and density distribution) Prediction of Young’s modulus in the three scale levels (mineralized collagen microfibrils, mineralized collagen fibrils, mineralized collagen fibres) to study the elastic properties of different bone tissue levels Prediction of macroscopic properties (Young’s modulus and Poisson’s ratio) of bone fibril structure Prediction of the medial condyle knee contact force Prediction of proximal femur loads from bone geometry and bone density distribution. ANN revealed more accurate results than support vector machine Prediction of elastic properties of human cortical bone from the development of a novel multiscale hierarchical approach Prediction of crack density and crack length at a specific trabecular bone site to then develop a rapid multiscale approach combining ANN with FEA Prediction of trabecular bone loading from the bone density distribution Prediction of loading parameters (magnitude and angle) of the proximal femur Prediction of knee adduction moment during walking Prediction of the failure and fatigue damage of trabecular bone (continued)

200

S. Mouloodi et al.

Table 9.1 (continued) Authors (Year) Hambli (2010, 2011b)

Type of machine learning employed Feedforward ANN

Main outcome Prediction of density, damage, Young’s modulus, and stimulus (main variables used by clinicians to assess bone quality) combines with FEA to simulate the bone remodelling process

intelligence exhibited by machines (Jones et al. 2018). Machine learning (that is also referred to as data-driven tools) is a fascinating interdisciplinary science in which machines, algorithms, and expert systems are endowed with an approximation of human intelligence in order to learn from existing experience and massive real-world examples, decipher meanings, languages, images, and patterns as humans do, and perceive data and then anticipate their future trends. Hence, machine learning involves algorithms that learn models from data. Deep learning, such as multilayer artificial neural networks (ANN), are a sub-category of machine learning (Fig. 9.1). They are commonly employed to analyse complex models encompassing massive amounts of data (models with larger datasets). The definitions and boundaries as well as correlations of AI, ML, and deep learning (such as feedforward ANNs) have evolved over the years (Fig. 9.1). There is still ongoing development and expansion in the field of AI and the associated algorithms (Abiodun et al. 2018b; Hong et al. 2020). The increasing data storage and speed of digital computing are provided analysis and design solutions not available even to projects such as NASA’s Lunar landing in 1969. From the 1980s, instead of closed form equations we can now use finite elements to analyse both the external fluid flows as well as internal loading of complex shapes. However, FEA methods require the input of material properties or assumptions concerning them. The latest analytical digital analysis technique that is now impacting our understanding of the physical world is the development of artificial intelligence. This is because AI does not necessarily require previous knowledge of material properties. These methods are reliant on both the collection and storage of large data sets as well as the speed of computing both of which have benefitted from increased

computer processing speeds and micro electromechanical systems (MEMS) technology for data collection. Fuzzy logic, knowledge-based systems, inductive learning, genetic algorithms, and neural networks were identified in early 2000 as the most applicable AI tools to investigate engineering problems (Pham and Pham 1999). Artificial neural networks have substantially contributed to the biomechanical studies and investigations concerning bone mechanics (that are discussed later in this book chapter). We describe the evolution and growing application of ML in the following sections including specific examples such as ML and ANN algorithms which are particularly applicable to the analysis of bone mechanics.

9.2.2

Richness and Abundance of Data as Well as Powerful Computational Tools Motivate the Application of ML in Bone Mechanics

The fourth paradigm of science is referred to as the applications of different ML techniques in science and research. The two main reasons are: (1) machine learning algorithms exponentially accelerate computational discoveries in material science and (2) the number of applications of ML and deep learning algorithms (such as ANNs) is growing incredibly fast (Hong et al. 2020). The affordability and availability of huge data storage as well as current immensely powerful computers have enabled the widespread application of AI, ML, and ANN in science and engineering. Appealing objectives of ML and ANN algorithms and their applications to engineering research problems involve the acquisition of knowledge through experimental measurements, observations, theoretical, and numerical

9

How Artificial Intelligence and Machine Learning Is Assisting Us to. . .

simulations to then make predictions, classifications, propose new data-driven tools, take rational decisions and enhance performance of a variety of tasks, and perform certain averaging over realizations (Mitchell et al. 2013; Rahmanpanah et al. 2020b). These methodologies concerning the employment of ML and ANN are also referred to as data-driven tools (data-driven models, data-driven techniques, etc.) where physical modelling and constitutive equations are replaced by learning through examples/observations/data. To accomplish this objective several methodologies and approaches using AI, ML, and ANN have been devised and proposed (Bock et al. 2019; Ma et al. 2021; Oeser and Freitag 2009). The emergence of AI with the rapid development of ML algorithms offers new possibilities in the advancement of bone mechanic’s research. These algorithms that establish data-driven models or tools strive to enhance traditional and/or current statistical techniques (commonly adopted in bone mechanics research such as linear regression) to establish the mapping of nonlinear relationships among multiple variables: mechanical properties, physical properties, structures, morphology, geometrical parameters, data from CT/MRI/radiograph images, among others. Bone mechanics research involves complex biological structures where theory on the underlying relationships among influential parameters is scarce or complex to prescribe, while data is becoming plentiful or is easier to collect. This represents an area in which AI, ML, and ANN can provide impetus for research development in bone mechanics (Mouloodi 2020). These data-driven ML algorithms essentially require data on which the algorithm can be trained (to complete the learning process) and then computation through that specific ML algorithm demonstrates its potent ability to elucidate knowledge underlying the data and to investigate the meanings involved in the studied domain. Furthermore, extensive mechanical experiments have been performed on several bones in recent years with the high precision offered by the current experimental setups and techniques. This massive and comprehensive data collection has occurred

201

in humans and animals bones using techniques such as: strain gauges placed on different bones of the body both in-vivo (mostly for animals) and ex-vivo, computed tomography (CT performed in both macro and micro level) and magnetic resonance imaging (MRI) as well as radiographs, finite element analysis (FEA), mechanical testing like load-to-yield and load-to-failure experiments (compression, bending, tension, torsion, and impact loading) both in-vivo and in-vitro to investigate the mechanical performance and properties of bones. This has enormously grown the volume of datasets that are readily available to be imported into these data-driven techniques to provide a foundation for subsequent analysis, decisions, reasoning, and comprehension of underpinning phenomena. This assists in further elaboration, exploring new ideas, and testing new hypotheses. The availability of this huge dataset on bone mechanics has expedited the adoption of ML algorithms (with predominant use of feedforward artificial neural networks). This offers the employment of novel methodologies for interpretation and prediction of these highly nonlinear datasets, which is promising for studies involving regression analysis, classification, and pattern recognition problems in bone mechanics research. These methods can improve the understanding of the basic functional properties of bones and the relevance of specific anatomy and anatomical relationships to their function.

9.2.3

Main Areas of Bone Mechanics Where Machine Learning Is Worth-Employing

Table 9.1 lists some of the main features that are addressed by using ML in bone mechanics research. More specifically, the authors have found great promise in the use of AI, ML, and ANN algorithms in bone mechanics research to engage with collected and collated data to investigate/test/accomplish the following: 1. Investigate the useful application of these data-driven models in the quite complex

202

2.

3.

4.

5.

6.

S. Mouloodi et al.

area of bone mechanics which represents an interdisciplinary area. Address and delineate the limitations of traditional statistical tools and computational techniques. There are specific problems in bone mechanics research where employing deep neural networks is a necessity. For instance, when there are limited and/or fuzzy and/or noisy data or when there is partial/total lack of appropriate information concerning material properties or when there is no consensus regarding an optimal constitutive model to define material properties or when implementing a proper constitutive model with inclusion of information/data from several disciplines (biology, engineering, physics, etc.) is lacking in conventional simulation techniques. The development of multiscale approaches to couple among different scales in bones is needed in the field of bone mechanics. An exceptional ability of ANNs is their ability to lean from data/observation/examples (from experiments and/or results of computational tools), to generalize and adapt to changing situations. Hence, intelligent models in which FEA and ANNs are coupled to offer novel computational advantages show great promise in bone mechanics (Barkaoui et al. 2014; Hambli and Hattab 2013; Mouloodi et al. 2021a). For complex models where numerical analysis is time intensive or unfeasible, ANNs offer very useful techniques. There are many clinical applications in bone mechanics where a considerable reduction in the engineering computational effort is required. Real time knowledge of the effect of positioning of an implant to stabilize a complex bone fragment from intraoperation CT images is an example of this. Moreover, ANNs provide an environment where extracted information from macroscale FEA, experiments, and medical images that belong to different areas of research can be integrated to establish a universal framework

7.

8.

9.

10.

(Hambli 2010; Mouloodi et al. 2020a) and link mesoscopic and macroscopic scales to simulate the bone remodeling process. Advances made in the adoption of ANNs are providing contributions in the area of orthopaedic research that is focused on bone health improvement (Dattatrey et al. 2019). This enhances our comprehension of bone modelling and remodelling activities (Tiwari and Kumar 2018). To enhance and assist in development of subject-specific computer models in clinical practices where patient-centered treatments are essential (Garijo et al. 2017; Taylor et al. 2017). Statistical techniques/models have been widely used (in almost all literature concerning biomechanics) to obtain correlations between different variables governing the structure and mechanical properties of bones (Sohail et al. 2019). These variables have been collected and documented from experimental, theoretical, and numerical simulations over the years. These statistical models have evolved through the emergence of artificial intelligence and the advent of several machine learning algorithms (including ANNs) that offer exceptional data handling opportunities and better interpretation of collected/ documented data for enhancing predictions and clinical administration. Machine learning with the use of deep learning algorithms (such as ANNs) provides a potent mathematical environment where nonlinear mapping among the space of inputs and outputs is established, and the algorithm strives to adjust its parameters through learning from data (referred to as data-driven techniques). The flexibility of these datadriven techniques where inputs/outputs can be decided by the researcher is a major advantage of these techniques for challenging engineering problems concerning bone mechanics to find reliable and prompt (realtime) solutions to both forward problems and inverse problems (Arbabi et al. 2016b;

9

How Artificial Intelligence and Machine Learning Is Assisting Us to. . .

Rahmanpanah et al. 2020a; Zadpoor 2013). For example, reliable prediction of the history of bone loading from its density distribution (Zadpoor et al. 2013), bone geometry and density distribution (Garijo et al. 2014), deformation and strains (Mouloodi et al. 2020b). 11. A common bottleneck in the currently employed, and traditional, biomechanical modelling approaches is their stability, generalization, and computational speed. This because the adoption of these approaches involves a solution to a large system of equations that encode the complex mathematical representations that are involved in these biomechanical models. An improvement to studies in this field is to apply AI, ML, and ANN algorithms to train surrogate models, predict in-real time, and generalize results to a wider population (Giarmatzis et al. 2020b).

9.3

Machine Learning Algorithms

We have learned so far that machine learning in biomechanical modelling and research/clinical applications involving bone mechanics accelerates the procedure of research, expedites or enhances the quality of simulation techniques, produces efficient computational surrogate models, processes huge collected datasets, aids pre-existing computational models by data-driven components, can completely replace an existing technique with a simplified model, and expedites the comprehension of materials through establishing principles from observations. Such reasons facilitate the establishment of models that enable efficient and reliable predictions to advance design and performance analysis of bone materials (as enumerated in the last section). These are an ever-growing list yet are concluded from the current trend of research in the employment of machine learning. “Why should machines learn?” is a question that Herbert A. Simon asked and nicely addressed in his research (Simon 1983). Some of the reasons and

203

explanations to this inquiry were: the tediousness of human learning, lack of the ability to copy human learning, and the agonizingly slow rates of human learning and its inefficiency. Concerns associated with the use of ML and ANN algorithms traditionally focused on the requirement of comprehensive and extensive data for their training, testing and validation, as well as the necessities of calculating and optimizing millions of parameters (Lipton et al. 2015; Mouloodi 2020). These concerns have been alleviated thanks to: (1) the availability, affordability, and ease of access to enormous data storage; (2) extensive experimental measurements performed and recorded in recent years (with high precision offered by the current experimental setups and techniques) that enormously grow the volume of datasets; and (3) considerable advancement in parallel computing systems and the development of large computational capacity offered by rapidly growing powerful computers, among others. These have expedited the development of new algorithms and optimization techniques and have accelerated the use of diverse methodologies for interpretation and prediction of highly nonlinear datasets in classification, pattern recognition, and regression problems (Kim 2010). Similar to AI, there are numerous ML algorithms. Each excels in a specific area to accomplish a particular task in finding appropriate nonlinear mappings in the dataset, thus providing the most precise correlation among variables (input and output data) or discovering feature attributes in unlabelled datasets (applications where there is no specific output dataset: problems involving unsupervised learning, for example).

9.3.1

Types of Machine Learning Based on Learning Paradigm

Table 9.2 summarizes three dominant learning paradigms of ML algorithms including: the commonly adopted subtypes in science and engineering associated with each category; the ideas underpinning their application; and some examples of ML and ANN algorithms. Deep

204

S. Mouloodi et al.

Table 9.2 Taxonomy and examples of machine learning algorithms. Feedforward ANNs to perform regression tasks (using supervised learning) are the most employed ML algorithm in bone mechanics research Machine learning algorithms Type based on learning paradigm Subtype Supervised Regression learning

Unsupervised learning

Classification

Mathematically constructing a relationship between input dataset(s) and outputs to then classify/categorize/label discrete input dataset(s), i.e. to approximate a mapping function from input (s) dataset to discrete outputs that are classes/ labels/categories

Clustering

Input datasets are trained by a neural network to then group/partition/segregate the data by similarity into a certain number of clusters/groups/ categories, i.e. the neural network comes up with its own classifications according to patterns of similarity in the data Reducing the number of input variables or features in order to simplify the dataset and enable a subsequent model to enhance its classification or regression prediction

Dimensionality reduction

Reinforcement learning

Main idea Mathematically constructing a relationship between input and output datasets to predict the latter from the former, i.e. to define output(s) as a mathematical function of input(s) and then produce the regression performance value

Data is usually not given and yet is generated by the interaction with the environment to then sequentially make decisions

learning approaches such as a convolutional neural network (CNN) is a pioneering technique in ML that offers applications that are quite useful to medical image analysis and processing, speech recognition, feature and text understanding, and speech recognition. The employment of both supervised and unsupervised approaches has been shown to provide very promising results in pattern recognitions, classification, and multivariable regression analysis. The literature demonstrates that feedforward artificial neural networks for performing regression tasks (that follow a supervised learning paradigm) are the most employed ML algorithm in bone mechanics research (Table 9.1). This will be further

Examples Linear regression and nonlinear regression Support vector regression (SVR) Artificial neural networks (ANN) Support vector machines (SVM) k-nearest neighbors (KNN) Artificial neural networks (ANN) Decision trees/random forests/linear discriminant analysis (LDA) k-means Self-organized maps (SOM) Artificial neural networks (ANN) Gaussian mixture models Principle component analysis (PCA) Partial least squares (PLS) Multidimensional scaling (MDS) Kernel principal component analysis Epsilon-greedy Sarsa Q-learning

elaborated in the next sections along with examples of undertaken research. Supervised ANNs (also referred to as hypothesis verifying) generate a relationship between the input and output datasets to verify the prescribed hypotheses (Table 9.2). This allows correlation analysis (to perform either regression or classification tasks) on multiple variables that are involved in the bone mechanics problems. In this ML algorithm each of the network inputs is already being provided with a correct answer, target, class, or output. Then, supervised learning sets the parameters of an ANN from the training dataset. The trained and tested ANNs can then be generalized to make reliable predictions on a

9

How Artificial Intelligence and Machine Learning Is Assisting Us to. . .

wider population (samples or individuals that were not involved in the ANNs training session). These are the main concepts/flowcharts that researchers investigating bones mechanics have been following using this learning type of ANN algorithm.

9.3.2

Main Steps Involved in Machine Learning

Machine learning is primarily divided into four main steps: 1. Data collection. The availability of a proper dataset for training, validation, and testing is an essential, yet intimidating, element when employing ML algorithms. Data for this purpose is achieved via several sources, for example, through observations, online data sources, results from the published literature, results of ex-vivo and in-vivo experiments, 3D CAD models, CT/MRI/ or 2D radiographic or ultrasound images, simulations such as finite element analysis (FEA) and computational fluid dynamics (CFD), etc. The idea that data matters more than algorithms for complex problems was further popularized by Halevy et al. in a paper entitled “the unreasonable effectiveness of data” (Halevy et al. 2009). This implies that we may want to reconsider the trade-off between spending time and money on the development of algorithms versus spending it on corpus development (Géron 2019). The data can provide a way to develop the algorithms without requiring any assumptions or input of mechanical characteristics derived from ex vivo experiments for example. This reinforces the significance of collecting and collating more representative data in bone mechanics research that represents a field of study where the employment of AI, ML, and ANN still remains in its infancy (Mouloodi et al. 2021b). 2. Data representation. Pre-processing (cleanup) and initial analysis/exploration/examining of the training dataset are required to ensure that

205

there are not any obvious flaws or issues in the dataset. This makes the data appropriate for the ML algorithm which will thereby expedite and improve its learning during the training session of the ML. This step assists in discovering and identifying missing or unrealistic values (e.g., NaN’s and infinite values, unexpected negative/positive values, corrupt values), and discrepancies/ambiguities that have occurred due to the engineering units of some variables). Documentation of such data cleanup and pre-processing once this step is performed is critical to ensure reproducibility, which is often overlooked in ML investigations (Wang et al. 2020a). The quality and/or extent of the data will determine the effectiveness of the models derived from it. 3. Algorithm selection. Regarding the nature of the investigated problem and the main ideas to be addressed there are different types of algorithms to be adopted (Table 9.2). For example, supervised learning such as regression and pattern recognition or unsupervised learning such as clustering and associations. Choosing a suitable algorithm for a given classification or prediction problem is still more an art than a science (Suzuki 2011). However, the size of the training dataset will elucidate the available choices of ML models, where statistical approaches such as simple linear regression, support vector machines, k-nearest neighbour, and decision trees are more suitable for a small dataset and deep neural networks (such as ANNs) are better choices for a training dataset in the order of thousands of data points or more (Wang et al. 2020b). 4. Model optimization. This is performed to select the appropriate degree for the fitted mathematical function using regularization and cross-validation techniques to minimize the likelihood of overfitting and to reduce any unnecessary complexity of the model (Rajan 2005; Sha et al. 2020). Despite their remarkable development and enormous success, AI and ML are in their infancy and still undergoing continuous change (Bock et al. 2019).

206

9.3.3

S. Mouloodi et al.

Performance Metrics

Performance function (loss function or cost function) is an essential function in machine learning with which the performance of the training, validation, and testing datasets is evaluated during the ML training. At any point of training, the performance function essentially measures the difference between the estimation made by the ML model and the actual target value, and hence the effort is to minimize this measure to reduce the difference between ML prediction(s) values and the target counterparts. The process of adapting and tuning the weights and the biases of the neural network as well as minimizing the error uses a training algorithm (Sect. 9.3.4) which is referred to as optimising the performance function. Mean squared error (MSE) is an indication of the training, testing, and validation error of the ML, and is the principal performance function (loss function) typically employed in the reviewed literature. The optimisation in this context is to minimize the error (performance function such as MSE). An example of the performance plots or loss curves that are plotted versus epochs throughout the network training process is depicted in Fig. 9.2. An epoch is a single pass of the dataset through the ANN. There are many advantages associated with the use of MSE as the performance function in ML related to bone mechanics research. For example, the considerable ease of implementation without the need for tuning any variables/parameters, low computational complexity, and superior mathematical properties enables its adaption with many training algorithms. Moreover, its frequency of use across many publications allows a real comparison of results among different pieces of research. There are other performance functions as well: mean absolute error (MAE), root mean squared pffiffiffiffiffiffiffiffiffiffi error (RMSE ¼ MSEÞ, and normalized MAE/RMSE (Phellan et al. 2021). MSE or MAE is typically adopted for regression analysis when using a feedforward artificial neural network as the ML architecture (Géron 2019). While MSE is typically chosen as the

performance function (loss function) during the ML training, MAE is instead employed when there are lots of outliers in the training dataset. While feedforward ANNs in bone mechanics research were predominantly employed for regression tasks (Table 9.1), they can also be implemented to perform classification tasks (Mouloodi et al. 2020b). Cross entropy is the loss function (performance function) adopted for typical classification tasks (binary, multilabel binary, and multiclass classification) using feedforward ANNs. In such cases in which the model defines a distribution, cross entropy between the training data and the model’s prediction is used.

9.3.4

Training Algorithm

The implementation of a training algorithm (learning algorithm) is the essence of a training network that establishes an optimization model and attempts to optimize/adjust design variables (weights and biases) to obtain optimal values for the loss function of the network (for example, MSE as identified above). After every training cycle (referred to as an epoch), the variables involved in the neural network (weights and biases) are computed and then iteratively updated until the performance function (loss function) reaches its global optimum (its minimum as for the MSE/RMSE/MAE). Two main categories of algorithms adopted as the learning algorithms (training algorithms) of ANNs are: Gradient descent (GD) based algorithms and nonlinear optimization-based algorithms. GD based algorithms employ first-order derivatives of the loss function (performance function) with respect to the design variables of the network (weights and biases) while nonlinear optimization-based algorithms employ second order derivatives. Examples of GD based algorithms include standard GD backpropagation, GD with adaptive learning rate backpropagation, GD with additional momentum backpropagation, GD with momentum and adaptive learning rate backpropagation, and resilient backpropagation. Examples of nonlinear optimization-based

How Artificial Intelligence and Machine Learning Is Assisting Us to. . .

Fig. 9.2 An example of performance analysis (loss curve plot) of an artificial neural network during training. This shows the ANN model performance (loss that is assigned as MSE) evaluated on the training, validation, and testing datasets at each epoch throughout the training process. In this example ANN was adopted to predict surface strains measured on a long bone during ex-vivo experiments

Mean Squared Error (MSE)

9

10 6

207

Best Validation Performance is 4614.82 at epoch 186 Train Validation Test Best

10 4

10 2 0

20

40

60

80

100

120

140

160

180

Number of Epochs (192 Epochs) algorithms include conjugate gradient algorithm and Levenberg–Marquardt backpropagation algorithms. Three commonly adopted learning algorithms to train an ANN are: Bayesian regularization backpropagation, Levenberg–Marquardt backpropagation, and scaled conjugate backpropagation algorithms. These training (learning) algorithms outperform simple GD algorithms, are easy to implement, and less time- and memory-intensive than more complex techniques (Burden and Winkler 2008; Dong et al. 2020; Mouloodi et al. 2020b; Ranganathan 2004). Moreover, they have proven successful in the estimation, prediction, and classification problems in biomechanical and mechanical engineering. The literature demonstrates that feedforward neural networks trained with backpropagation algorithms are predominantly employed in the ML investigations in bone mechanics, and biomedical and mechanical engineering practices. For example in the studies involving bone mechanics research, Levenberg– Marquardt backpropagation (Campoli et al. 2012; Hambli 2011a; Hambli and Hattab 2013; Taylor et al. 2017; Zadpoor et al. 2013), Adam stochastic optimization (Dattatrey et al. 2019), Bayesian regularization backpropagation (Garijo et al. 2014; Hambli 2011b; Mouloodi et al. 2020a, 2021a; Rahmanpanah et al. 2020a; Vukicevic et al. 2018), and scaled conjugate

backpropagation (Mouloodi et al. 2020b) were employed. Neural networks trained with a backpropagation algorithm are trained by gradient descent methods in which the error (performance function) propagates in a backward manner through the network to adjust the weights and bias (design variables of the network) according to the employed training algorithm. For instance, Bayesian regularization backpropagation, which is well-suited for complex problems or noisy data points (Mouloodi et al. 2020a), employs Levenberg–Marquardt optimization in batch or incremental styles, thereby minimizing MSE between the ANN prediction (output) and the target output (experiments).

9.3.5

Training, Validation, and Testing Datasets

As previously emphasized data collection is an essential step prior to performing a machine learning technique. Data for this purpose is achieved via several sources, for example, through observations, online data sources, results from the published literature, results of ex-vivo and in-vivo experiments, 3D CAD models, CT/MRI/radiograph images, simulations such as finite element analysis (FEA) and computational fluid dynamics (CFD), etc.

208

An essential step prior to initiation of training a machine learning algorithm is to split data into training, validation, and testing datasets. The availability of a proper dataset to achieve this aim (data split into training, validation, and testing) is a critical, yet intimidating, element when employing ML algorithms. A training dataset is essentially a set of examples employed for training the algorithm, and that is why ML algorithms are referred to as data-driven techniques where traditional physical modelling is replaced by learning from data/examples/observations. The training algorithm (that was explained in Sect. 9.3.4) uses the training dataset to learn from existing experience and massive real-world examples (to adjust weights and biases), decipher meanings, languages, images, and patterns as humans do, and perceive data to then anticipate their future trends. Hence, in a nutshell when the collected data is split into training, validation, and testing datasets, the training dataset is used by the training algorithm to optimize the performance function (loss function) to make the prediction/ classification accurate. This is also referred to as fitting the model. The validation dataset is a portion of the entire collected data (held back from the training data) that is used to assess the reliability of the ML model when tuning the models’ hyperparameters. Therefore, a validation dataset is not used in the training, but instead is used to offer an unbiased evaluation of the network performance when comparing among different models (for example, to choose the number of hidden neurons in an ANN) to then select a final model. Hyperparameters in a neural network such as the number of hidden layers, number of neurons in each hidden layer, number of epochs for training the network, activation functions, and learning rate need to be tuned. A validation dataset is employed for tuning hyperparameters, however default values or empirical tuning is also adopted (Phellan et al. 2021). A proper selection of this percentage depends on each application and there is no specific rule to perform this task. However, the amount of data available on which the neural networks are being trained, the architecture (topology) of the selected network, and the

S. Mouloodi et al.

number of inputs and outputs can influence this choice. The testing dataset is another portion of the entire collected data (held back from the training and validation data) that is used to assess the performance of a fully specified ML model (the final fitted model on the training dataset). The error on the test dataset provides an unbiased measure of the generalization error of the trained network. This evaluates the performance of the already trained network on unseen samples (new datasets that ANNs have not seen before). This guarantees a correct and unbiased generalization of the already trained neural networks which have avoided overfitting to the training datasets and it provides a robust verification of the networks post-training. The ability of an ANN to generalize, i.e. to properly respond to previously unseen input data, is critically important. As explained this is evaluated by the measures obtained from the testing dataset. An indication of overfitting is when the accuracy of a testing session is considerably worse than a training dataset. This is when the training is stopped to avoid overfitting of the ML model to the training dataset. A typical visualization curve that is commonly plotted in ML studies (to assess and demonstrate a good performance of the network that avoided overfitting) is the performance plots or loss curves that are plotted versus epochs throughout the network training process. An example of this plot is depicted in Fig. 9.2. Figure 9.2 displays the performance of the neural network in terms of MSE for a total of 192 epochs, in which the log of MSE is plotted versus the number of epochs. Epochs are the number of times the algorithm sees the entire dataset. In this example there is a similar trend in the error reduction for both training and testing as well as validation datasets and a small difference in both errors was observed. This indicates a well-trained ANN which is well-suited for generalization to predict real-world data that the ANN has not seen before. For a convincing performance, the ultimate value of MSE should be small, the error of training and testing datasets should have similar characteristics, and there should be no significant overfitting. Figure 9.2

9

How Artificial Intelligence and Machine Learning Is Assisting Us to. . .

demonstrates that the best validation performance in this example occurs at epoch 186. This indicates the iteration at which the validation performance reached a minimum. The training in the current ML network continued for six more iterations before the training stopped at epoch 192. Generally speaking, if the test curve increases significantly before the validation curve does, then it is possible that some overfitting might occur. Hence, in a well-trained network the performance plot should reveal similar characteristics and trends for the training and validation datasets. Hence, the training is stopped when either the error between ANN output and target is small enough or the learning iteration reaches its higher level without a considerable decrease in the error (default value is set by the user, for example 1000 iterations or epochs). This is when it is stated that the errors are minimal, and that convergence occurs. The corresponding weights and bias are then stored, which makes the trained ML model prepared for production to be fed with a separate and fully new dataset. A percentage of the entire dataset is assigned to each set by the user, but evidently a majority of the percentage is set to the training dataset and the remaining retained for validation and testing. Different practices have been adopted to perform the data split, for example: (Campoli et al. 2012) assigned 80% to the training dataset, 15% to validation, and 5% to testing, (Zadpoor et al. 2013) assigned 90% to the training dataset, 5% to validation, and 5% to testing, (Taylor et al. 2017) assigned 80% to the training dataset, 10% to validation, and 10% to testing. Alternatively, for research where Adam optimizer or Bayesian regularization backpropagation was employed as the training algorithm 90% and 10% of the datasets (Garijo et al. 2017; Mouloodi et al. 2020b), 80% and 20% of the datasets (Mouloodi et al. 2020a, 2021a), and 70% and 30% of the datasets (Dattatrey et al. 2019) were used for the training and testing sets, respectively. The employment of Bayesian regularization backpropagation as the training algorithm of an ANN eliminates the need to present the validation dataset, because the regularization technique which is the main purpose of having the

209

validation dataset is already built into this training algorithm. This training algorithm reduces or eliminates the need for the lengthy process involved in cross-validation, is difficult to overtrain and overfit, is advantageous to use where data are scarce and expensive to acquire, and removes the validation effort that might be demanding for large datasets (Burden and Winkler 2008).

9.4

Artificial Neural Networks

Artificial neural networks (ANNs) were first introduced in 1943 by Warren McCulloch (a neurophysiologist) and Walter Pitts (a mathematician) (McCulloch and Pitts 1943) in their paper entitled “A logical calculus of the ideas immanent in nervous activity”. In that study a simplified computational model was proposed that explained how biological neurons in animals’ brain might perform collaboratively to undertake complex real computations. Artificial neural networks, being one of the most commonly employed machine learning algorithms, are massively parallel computing systems, comprise a parallel processing architecture (Fig. 9.3), and derive their power through this parallel distributed structure. Figure 9.3 displays a schematic diagram of a feedforward backpropagation neural network, also referred to as multilayer feedforward artificial neural network or multilayer perceptron (MLP) (Mouloodi et al. 2021a). This implies that the information only moves in one direction (forward) from the input layers (and their associated nodes) through the hidden layers and finally to the output layer. In contrast to a recurrent neural network, there are no feedback cycles or loops in the network structure. Several techniques and tools exist to establish these networks, for example, MATLAB toolbox, Scikit learn, Keras, Tensorflow, and PyTorch. Artificial neural networks are believed to be inspired by the structure of biological neural networks in the human brain (Hertz 2018; McCulloch and Pitts 1943). Figure 9.3 demonstrates that an ANN consists of a large number of processing elements (known as nodes

210

S. Mouloodi et al.

Output 1

Input 2

Output 2

Input 3

. . .

. . .

. . .

. . .

. . .

. . .

Input 1

m

. . .

. . .

Input i

Output 3

. . .

Output n Input Layer

1

2

Output Layer

Hidden Layers Information flow: Feedforward Errors back propagated to the neural network to adjust weights and bias, using gradient descent based algorithms or nonlinear optimization based algorithms What happens in a single neuron?

Input 1 × w1 Weights

Input 2 × w2 Input 3 × w2 . . .

Ʃ b

Output of the neuron is introduced as an input for the subsequent neurons

Bias

Ʃ(wi × Input i )+b

y = f (Ʃ(wi × Input i )+b) f Activation function

Input i × wi Sigmoid (logistic or log-sigmoid) activation function

hyperbolic transfer functions activation function

Fig. 9.3 A schematic drawing of the architecture and the working principle of a feedforward artificial neural network trained with a backpropagation algorithm is shown herein. There are several activation functions (also referred

to as transfer functions) that are adopted in neural network algorithms. Only sigmoid (logistic or log-sigmoid) and hyperbolic (Tanh) activation functions are depicted in this figure

or neurons) that are linked together to perform information transmission and processing. Neurons in each layer are connected to all surrounding neurons in the adjacent layer(s), and each connection is weighted with numerical values. The number of neurons in the input layer

and the number of neurons in the output layer equal the number of input and output variables, respectively. This highly interconnected pattern of links along with the existence of a massive number of neurons enables neural networks to dive into high-dimensional data to learn their

9

How Artificial Intelligence and Machine Learning Is Assisting Us to. . .

intricate patterns (Jones et al. 2018). The main idea is to mimic the way by which humans learn and decipher meaning through examples and observations available to them. This structure allows the ANN to perform specific tasks such as regression analysis, data classification, optimization, pattern recognition, among others, which are analogous to the tasks accomplished by humans, for example for speech recognition, facial recognition, and object recognition. ANNs offer researchers very powerful computational techniques allowing the comprehension of complicated nonlinear relationships that are involved in datasets (Abiodun et al. 2018a; Olden and Jackson 2002). The structure of artificial neurons and the working principles involved in a neuron are shown in Fig. 9.3. Input data has an associated weight (wi), also known as the synaptic weight, which mathematically represents a degree of significance for that neuron. The input signals of the neurons are multiplied by their synaptic weight, and the summation of this result, which is added to the bias (b), forms the input information of the neuron (Fig. 9.3). The number of hidden layer (s) for a feedforward ANN for either regression or classification tasks depends on the problem, however it is typically assigned as 1 to 5. The number of neurons (nodes) per hidden layer is also dependent upon the problem but is typically set as 10 to 100. Each neuron has an activation function ( f ). Weights (wi) are assigned to the signals (xi) by the neurons. The weighted signals are then summed up and biased. The outcome is eventually introduced to the activation function. The activation function (also referred to as transformation function) is used to transform the output into a desired range (usually 0–1 with Sigmoid function), to introduce nonlinearity to the data where needed, and finally to achieve the desired ANN output. Sigmoid (logistic) activation function, hyperbolic (Tanh) activation function, and rectified linear unit (ReLU) activation function are some of the commonly used activation functions for artificial neurons. Sigmoid (logistic or log-sigmoid) and hyperbolic (Tanh) activation functions are depicted in Fig. 9.3.

9.5

211

Applications of Artificial Neural Networks to Bone Mechanics

An overview of research undertaken in the study of bone mechanics was presented in Table 9.1. This section elaborates more on some of these papers to make the readers more familiar with the types of works that have been performed in the area of bone mechanics research. Extensive experimental recordings are available in the literature which can be imported into ANN algorithms for computer simulation analysis to predict responses of bones (Currey 2009). Each ANN input data point represents recorded in-vivo or ex-vivo measurements, such as loading, load exposure time, individual age, displacement, strains, stiffness, modulus of elasticity (Young’s modulus), density distribution, shape and geometrical parameters, kinematic parameters, etc. Several types of ANNs will be employed to find the optimum ANN in gaining desirable outcomes (i.e. completing a solution to either an inverse problem or a forward problem). Furthermore, different architecture types and learning methods, feedforward or recurrent ANNs (based on ANNs connection pattern), single layer or multilayer (based on the number of hidden layer), and fixed or adaptive (based on the nature of weights adjustment during ANNs training), will be examined to present a reliable and efficient model for bone tissue adaptation. Sensitivity and error analysis will also be performed using different training algorithms for neural networks, such as Levenberg-Marquardt backpropagation, Bayesian regularization backpropagation, and scaled conjugate backpropagation (Mandal 2017). A backpropagation neural network (BPNN) is a wellsuited network for predicting mechanical responses of bones and tissues. BPNN is an artificial neural network that employs a supervised learning method (Deng et al. 2008), possesses feedforward architecture, accommodates complex and nonlinear data relationships, and hence is well-suited for regression and pattern recognition analysis and also practical application (Mandal 2017).

212

The connection weights approach, which uses raw input-hidden and hidden-output connection weights in the neural network (Mandal et al. 2009; Olden et al. 2004), can be employed to quantify the relative importance of ANNs input variables. The connection weights approach calculates the product of the raw input-hidden and hidden-output connection weights between the neuron of input and the neuron of output and then sums these products across all hidden layers. This approach defines the influential input variables (e.g. strains, load, displacement, shape) that have considerable influence on the prediction of the ANN (output). For example, in a recent study it was observed that strains measured experimentally on the several surfaces of equine MC3 long bones significantly affect the load predicted by ANNs, exhibiting the profound effects of measurements taken of bone midshaft strains on the estimation of loads (Mouloodi et al. 2020b). Remaining parameters, though important, revealed a relatively moderate effect on load prediction. Supervised and unsupervised ANNs offer considerable benefits to clinical biomechanics in general. For instance, they can be employed to classify gait and running data, classify different movement tasks (such as classifying the relationship between pain and vertebral motion (Dickey et al. 2002)), and analysing muscle activities through classification of EMG data. Because mechanical properties and responses of bones and tissues are multivariable and depend to a great extent on multiple factors, in order to model the elusive relationship among loading, strains, kinematic and dynamic parameters, EMG and so forth, employing ANNs are attractive and provide encouraging outcomes in biomedical engineering. Furthermore, future finite element studies will benefit from employing hybrid modelling, also referred to as intelligent finite element method, in which neural networks algorithms are integrated within a finite element framework (facilitating bypassing the FEA calculation process). This assists in time-sensitive clinical applications where using traditional FEA requires long computing times to obtain a final simulation result (Liang et al. 2018; Zadpoor

S. Mouloodi et al.

2013). In such scenarios, neural networks are incorporated in an FEA as an alternative to the constitutive model of a bone. ANNs are then trained using raw in-vivo experimental measurements representing the mechanical response of the bone to applied loads. The trained network is then used in the FEA to predict different types of stress and strain (Javadi et al. 2003). Supervised and unsupervised ANNs using radiographs of lower limb long bones belonging to children (femur, tibia, and fibula) were employed to investigate lower limb fractures and to classify healing time (Malek et al. 2016, 2018). Age of the children, displacement, angulation, type, and contact area of the fracture were used as the ANNs input variables to classify fracture healing time and to evaluate accuracies exhibited by two different ANN models: self-organizing maps (SOM) and multilayer perceptron (MLP) neural networks. Feedforward ANNs were adopted to estimate musculoskeletal loading of the proximal femur that have resulted in the bone density distribution measured via CT imaging (Campoli et al. 2012). 1581 different combinations of loading parameters (including various loading magnitude and load angles) via solving a forward tissue adaptation model were established in that study to form the ANNs training dataset. The angle and the magnitude of contact force constructed two ANNs outputs. Although cartilage biomechanics is not the concern of this book chapter, ANNs have demonstrated their successful application to this research area too (Arbabi et al. 2016a, b). ANNs were also employed as a nonlinear system identification approach to solve an inverse problem, i.e. to predict loading of trabecular bone from its density distribution (Zadpoor et al. 2013). The use of ANNs was demonstrated in that study enabling the mapping of the nonlinear relationship from the space of bone density distribution (serving as the ANNs input parameters) to the space of applied loads (ANNs outputs). This is called a backward mapping (finding a solution to an inverse problem) for which there are no straightforward approaches using traditional tools such as FEA, and moreover, differential equations representing such inverse models are

9

How Artificial Intelligence and Machine Learning Is Assisting Us to. . .

typically not known (Zadpoor 2013; Zadpoor et al. 2013). The development of investigations on other bones, and not merely the femur, has also been recommended in the literature. Garijo et al. also offered a general methodology using machine learning techniques (ANN and support vector machines) and a linear statistical technique (linear regression) to make predictions on the likely loads supported by the proximal femur (Garijo et al. 2014). Different loading conditions with a wide variation were simulated using bone remodelling problems through FEA to obtain corresponding bone density distribution, thereby input data were formed to establish linear regression, ANN, and support vector machines. Such methodologies elucidate patient-specific musculoskeletal loads and assist in finding effective remedies for musculoskeletal treatments in a timely manner, because it is critically important to determine physiological loading that a bone has been exposed to from continuous changes that have occurred in the bone morphology (geometry, shape, and density distribution) (Christen et al. 2012). Knee adduction moment was predicted using a simple method where a multilayer ANN was employed from some measurements including force plate recordings, and the extracted results demonstrated a significant correlation with those achieved from inverse dynamics (Favre et al. 2012). ANNs were similarly employed to predict knee loading (Brisson et al. 2021; Liu et al. 2009). ANNs have been further employed in other areas of bone mechanics to elucidate mechanical responses exhibited by bones substantiating our knowledge of this advanced well-engineered biological structure. Several artificial intelligent expert systems and linear regression were fused to assess resistance of cortical bone to fracture (quantification of bone fracture toughness) where R-curve slope, toughness threshold, and stress intensity factor were estimated from patients age and crack length as the input variables of the algorithms, while stress intensity factor that is a critical factor related to crack, shaped the ANN output (Vukicevic et al. 2018). These input parameters are frequently available during clinical examinations, and thus,

213

such studies are of profound significance to tackle the challenges of establishing cost-effective and competent management tools for the associated bone fracture risks. Inspired by the importance of bone age, relevant studies have been undertaken in the application of machine learning algorithms such as ANNs and convolutional neural networks (CNNs) to complete significant assessments of bone age from different perspectives (Dallora et al. 2019a, b; Liu et al. 2019). In order to offer potential insights into the enhancement of bone repair analyses, an ANN was developed using measurements such as apparent modulus of elasticity, bone volume fraction, and bone ash density as ANN input variables to make reliable and quick prediction of apparent damage at specific bone sites (ANN outputs were the failure and fatigue damage of trabecular bone) (Hambli 2011a). That study, incorporating ANN expert systems and FEA, also revealed a superior ability of the ANN in terms of considerable reduction in computational time versus FEA, which accentuated the development of a rapid computational technique using ANNs to simulate fatigue analysis of bones (rapid prediction of damage accumulation in a bone). Similar approaches were carried out by Hambli and colleagues to further investigate fracture and fatigue analysis of bone tissues using FEA and neural networks as well as a combination of both which represents hybrid multiscale modelling (Barkaoui et al. 2016; Hambli 2014; Hambli et al. 2016; Hambli and Hattab 2013). In-vivo measurements of applied forces on different parts of humans and animal skeleton are extremely challenging, and sometimes, impossible. For example, knee contact force that is highly affected by gait patterns is not always straightforward to quantify through in-vivo measurements, and moreover, widely employed computational techniques to calculate this force such as inverse dynamics analysis have their own drawbacks (Ardestani et al. 2014a). In that research a feedforward ANN was proposed and was trained on pre-rehabilitation gait patterns including ground reaction forces and marker trajectories (since knee contact force is significantly affected by gait patterns and knee joint

214

alignment (Fregly et al. 2009; Shelburne et al. 2008)), and then the ANN predicted knee contact force related to rehabilitation gait pattern. Another noteworthy feature of that study, which is often overlooked in literature concerning applications of ANN, is assessing: (1) the generalization ability of the already trained and tested ANN to demonstrate its feasibility to be employed to a wider population, and (2) the validation of ANNs predictions against a reliable source such as experimental data. Similarly, machine learning using feedforward ANNs were employed to address clinical problems and to investigate gait biomechanics with inclusion of influential parameters from limbs, for example, to predict lower limb joint angles and moments (Mundt et al. 2020), to predict appropriate kinematic waveforms to reduce the contact force in knee joint implant (Ardestani et al. 2014b), and to predict knee contact forces (Giarmatzis et al. 2020a) and joint forces (Giarmatzis et al. 2020b). The potential applications in bone engineering enable monitoring and prediction of solutions associated with clinical practice. A multilayer feedforward ANN was applied to undertake multidimensional data analysis associated with physical and mechanical properties of trabecular bone. That study emphasized the requirement to develop stand-alone ANN models that integrate a variety of mechanobiological parameters and their correlations with clinical parameters, which eventually accelerate clinical diagnosis and hence increases the chances of prevention of bone disorders (Khovanova et al. 2015). Age of the patients was predicted in that study from the mechanobiological parameters of bones: porosity, gender, morphology, level of interconnectivity, and compressive strength. A surrogate model using ANN was developed to predict femoral strains and fracture loads to fulfil clinical purposes in real time from clinically obtainable measurements including body weight, femoral neck bone mass, and femoral neck length (Taylor et al. 2017). To detect essential input variables that make a significant contribution to the strain prediction (ANN output), prior to their injection to the ANNs, a multivariate linear regression was

S. Mouloodi et al.

adopted in that study. Due to the fact that the ANNs were trained from the previously developed population based FEA (requiring 300 CPU seconds per femur, 150,000 in total for 500 femurs, accounting for the bones image segmentation, mesh generation, and definition of appropriate load and boundary conditions), the computational cost involved was considered as a drawback. However, it was also emphasized by the authors that the preparation of these population-based data is performed only once, and then the already trained and tested ANN is able to be fed with input variables of a new patient in the clinic to produce satisfactory results in real time (enabling prediction of strains and fracture loads on a new patient in 0.002 s). It has also been shown that feedforward backpropagation ANNs, trained with the results of finite difference time domain simulations, are successful at estimating micro-architectural parameters of human cortical bones (pore diameter, pore density, and porosity) (Mohanty et al. 2019). The main focus of this book chapter is to summarize research undertaken on bone mechanics using machine learning algorithms. However, it might be useful to mention some examples of research where several machine learning algorithms were employed to perform bone age assessment (Dallora et al. 2019a; Gerges et al. 2020). This is an essential topic for clinical applications to, for example, evaluate the biological maturity of children. Such deep learning algorithms are making this field of research an ever-emerging topic and strive to address several problems such as prediction, segmentation, and classification (Nadeem et al. 2020). Artificial neural networks (Liu et al. 2008; Tang et al. 2019), convolutional neural networks (Iglovikov et al. 2018; Liu et al. 2019; Pan et al. 2020; Ren et al. 2018), support vector machines (Cunha et al. 2014; Harmsen et al. 2012), fuzzy neural networks (Lin et al. 2012), decision trees and random forest (Urschler et al. 2015), and K-nearest neighbours are among the machine learning algorithms that were adopted to assess age of bones.

9

How Artificial Intelligence and Machine Learning Is Assisting Us to. . .

9.6

Perspectives, Conclusions, and Future Directions

As we have repeatedly stressed, bones are highly complex biological materials, possess exotic mechanical properties, are characterized as complex engineering structures and nanocomposite as well as anisotropic solids, possess complex hierarchical structure that entails exceptional mechanical properties, and are surrounded by an irregular geometry. A proper comprehension to understand the relationships among mechanical responses/ properties/behaviour of the whole entity, material properties, shape and geometrical parameters and responses to internal and external changing situations (such as loading, modelling, remodelling, temperature, etc.) is an extreme challenge. This is almost always the case in biology and in the investigations concerning bone mechanics. This stems from the multivariable and exceptional mechanical properties that influence this advanced biological material. To advance such understanding, collaboration of clinicians, veterinarians, biomechanical engineers and bone biology researchers is required. Even though researchers from different disciplines concur with the wide range of parameters affecting bone mechanics responses under load, importing these parameters/variables into a universal equation to form a widely-accepted model is a challenge. Artificial intelligence using several types of machine learning algorithm that follow datadriven methodologies allows researchers from different areas of expertise to investigate the same problem and to then import all the different potential influential parameters into the AL, ML, and ANN models for further investigations. Efficient management of the large amounts of data analysis in a range of very specialised areas has been, and continues to be, a main restriction to this area, yet ML is exhibiting a promising trend to assist researchers in this complex area. The literature on the application of AI, ML, and ANN algorithms to investigate bone mechanics research reviewed in this book chapter has revealed that feedforward artificial neural networks (Fig. 9.3) are the dominant ML

215

algorithm adopted in this field (Table 9.1). Performing multivariate regression tasks (that follow a supervised learning paradigm) are the main problems that bone mechanics research have addressed. Regression analysis using ANNs, which allows correlation analysis among many influential parameters, provides a very powerful tool to investigate multivariable problems (to conduct multivariate regression). This is always the case in bone mechanics research that has been addressed at the beginning of this section and throughout this book chapter. It was demonstrated that multivariate regression using feedforward ANNs was adopted in bone mechanics research to predict mechanical loading parameters (Ardestani et al. 2014a; Campoli et al. 2012; Mouloodi et al. 2020b; Zadpoor et al. 2013), failure and fracture features (Hambli 2011a; Vukicevic et al. 2018), micro-architectural parameters (Mohanty et al. 2019), strains and deformation/displacement (Mouloodi et al. 2020a; Taylor et al. 2017), and mechanical properties (Barkaoui et al. 2014, 2016; Hambli 2010, 2011b; Mouloodi et al. 2021a; Rahmanpanah et al. 2020a). Detailed explanations were offered in Table 9.1 and Sect. 9.4. Mechanical testing on bones (Sharir et al. 2008) to collect information to then be analysed using simple statistical techniques, developing mathematical models that strive to provide universally approved methods, and performing finite element analysis are widely employed to study bone mechanics. However, introducing elaborate mathematical expressions to quantify the mechanical features of these exotic materials is not straightforward and it does involve focusing on one aspect of bone’s features. It is noteworthy to employ ML and ANN using superfluous data collected over the years to advance bone mechanics research. Much remains to be done at the macro, micro, and nanolevels to obtain a full comprehension of these complex structures. These techniques can eventually allow us to determine when and how to intervene before breakdown occurs. This substantially assists in better comprehension of the mechanism of injury in human and animal bones, joints and other

216

biological tissues and structures, and increases the understanding of the underpinning biomechanics, leading to more effective prevention strategies. This is a promising field for clinicians to discover influential parameters and conduct real-time multivariate regressions tasks that are required in their clinical investigations. Fruitful outcomes exhibited from the employment of ML and neural networks will also assist in data collection strategies and will elucidate which data (specific measurements) and of what type (physiological, anatomical, environmental) is worth-collecting for further analysis. Moreover, the employment of deep learning algorithms has demonstrated that it provides impetus to advance traditional computational techniques either to replace conventional techniques or to offer real-time applications that substantially reduces computational time in biomechanical engineering. Computer-aided-design (CAD) and finite element analysis (FEA) are indispensable tools in attempts to model and analyse bones and to replicate experimental data (Mouloodi et al. 2019b). FEA provides highly accurate results and is a potent tool to analyse structures that possess irregular geometry and quite complex boundary conditions. These scenarios exist in the structural and mechanical analysis of bones, and hence FEA is predominantly employed in biomedical engineering and studies concerning bone mechanics. However, FEA’s main disadvantage is its reliance on mesh statistics and quality, and for complicated problems (such as investigations involving anatomical structures and bone mechanics) the establishment of hundreds of thousands of, if not millions of, elements is required (Dong et al. 2020; Mouloodi et al. 2020a). This is associated with time-intensive computational mechanical simulations, thereby minimizing practical applications where real-time response is essential, for example in clinical applications (Phellan et al. 2021). One of the strong motivations to follow this path with the potential replacement of FEA with ML methodologies is the exponential computational time-saving without compromising robustness and performance of simulations. This disadvantage of FEA further worsens their

S. Mouloodi et al.

usefulness in structural optimization problems. It has also been shown that an essential part of an FEA should be dedicated to convergence and error analysis prior to publishing the results (Mouloodi et al. 2019b). A limitation to most of the previous studies concerning the application of FEA, is the lack of evidence in terms of quantifying dimensional error of reconstructed geometries, performing mesh quality assessment, determining the error of FEA, or assigning linear tetrahedron meshes to the models, which would all potentially lead to misleading stress and strain results. As an example, a surrogate of FEA using artificial intelligence, machine learning approaches can be used to develop a deep learning model to promptly and accurately estimate stress and strain distribution in bone tissues to develop a rapid multiscale approach. Such a machine learning model can be designed and trained to take its input information from FEA to then directly output the stress and strain distributions (bypassing the FEA calculation process). The bones’ shape can be generated from a statistical shape model, then FEA can be performed to obtain stress and strain distribution of each shape sample. Therefore, the ML takes shape as an input and it results in the stress and strain distributions. A similar model was implemented to address aorta wall stress and strain distributions (Liang et al. 2018). Moreover, the complexities that are involved in biological systems (in plants and animals/ humans soft and hard tissues (San Ha and Lu 2020)) provide inspiration for mechanical and biomedical engineering sciences to embrace this complexity for the development of bioinspired materials and structures that exhibit a significant improvement over conventional structures in energy absorption capacity (San Ha and Lu 2020), crashworthiness under different loading conditions (Nian et al. 2019; Wang et al. 2020c), auxetics behavior in structures and implants (Bodaghi et al. 2020; Ghavidelnia et al. 2021), among others. Data-driven methodologies elucidate such complexities by learning from examples and will offer prompt prediction on new datasets never seen by the network.

9

How Artificial Intelligence and Machine Learning Is Assisting Us to. . .

This book chapter has demonstrated how artificial intelligence and machine learning is assisting us to extract meaning from data collected on bone mechanics research. Bone mechanics research represents a field of study where the employment of AI, ML, and ANN still remains in its infancy. There are huge data already available in the literature, and indeed there is a requirement to collect and collate more representative data, to be analysed using not only feedforward ANNs but also other powerful ML algorithms (Table 9.2) to make more discoveries in this challenging field of biology. In a nutshell, it is hoped that these novel data-driven methodologies be implemented to: (1) address the limitations of traditional statistical tools and computational techniques, (2) assist in problems where some data features are missing or a proper constitutive model is lacking, (3) develop multiscale approaches to couple among different scales in bones, (4) tackle challenges exhibited by time-intensive computational simulations widely adopted in biomechanics, (5) establish universal frameworks, (6) link mesoscopic and macroscopic scales to simulate the bone remodelling process, (7) design strategies and offer recommendations to clinicians regarding bone health improvement and avoidance of injury in the first place, (8) develop subject-specific computer models to enhance clinical practices where patient-centred treatments are essential, (9) tackle challenging forward and inverse bone mechanics problems where traditional mathematical models and numerical simulations fail to offer a solution; and more significantly, to combine all these different areas to learn from ever-growing data to make generalizations to new specimens, samples, and cases.

References Abiodun OI, Jantan A, Omolara AE, Dada KV, Mohamed NA, Arshad H (2018a) State-of-the-art in artificial neural network applications: a survey. Heliyon 4: e00938 Abiodun OI, Jantan A, Omolara AE, Dada KV, Mohamed NA, Arshad HJH (2018b) State-of-the-art in artificial neural network applications: a survey. 4:e00938

217

Arbabi V, Pouran B, Campoli G, Weinans H, Zadpoor AA (2016a) Determination of the mechanical and physical properties of cartilage by coupling poroelastic-based finite element models of indentation with artificial neural networks. J Biomech 49:631–637 Arbabi V, Pouran B, Weinans H, Zadpoor AA (2016b) Combined inverse-forward artificial neural networks for fast and accurate estimation of the diffusion coefficients of cartilage based on multi-physics models. J Biomech 49:2799–2805 Ardestani MM, Chen Z, Wang L, Lian Q, Liu Y, He J, Li D, Jin Z (2014a) Feed forward artificial neural network to predict contact force at medial knee joint: application to gait modification. Neurocomputing 139: 114–129 Ardestani MM, Chen Z, Wang L, Lian Q, Liu Y, He J, Li D, Jin Z (2014b) A neural network approach for determining gait modifications to reduce the contact force in knee joint implant. Med Eng Phys 36:1253– 1265 Barkaoui A, Chamekh A, Merzouki T, Hambli R, Mkaddem A (2014) Multiscale approach including microfibril scale to assess elastic constants of cortical bone based on neural network computation and homogenization method. Int J Numer Meth Biomed Eng 30:318–338 Barkaoui A, Tlili B, Vercher-Martínez A, Hambli R (2016) A multiscale modelling of bone ultrastructure elastic proprieties using finite elements simulation and neural network method. Comput Methods Prog Biomed 134:69–78 Bock FE, Aydin RC, Cyron CJ, Huber N, Kalidindi SR, Klusemann B (2019) A review of the application of machine learning and data mining approaches in continuum materials mechanics. Front Mater 6:110 Bodaghi M, Serjouei A, Zolfagharian A, Fotouhi M, Rahman H, Durand DJIJoMS (2020) Reversible energy absorbing meta-sandwiches by FDM 4D printing. Int J Mech Sci 173:105451 Brisson NM, Gatti AA, Damm P, Duda GN, Maly MR (2021) Association of machine learning based predictions of medial knee contact force with cartilage loss over 2.5 years in knee osteoarthritis. Arthritis Rheumatol 73:1638–1645 Burden F, Winkler D (2008) Bayesian regularization of neural networks. Artif Neural Netw 23–42 Campoli G, Weinans H, Zadpoor AA (2012) Computational load estimation of the femur. J Mech Behav Biomed Mater 10:108–119 Carter DR, Hayes WC (1977) The compressive behavior of bone as a two-phase porous structure. JBJS 59:954– 962 Christen P, van Rietbergen B, Lambers FM, Müller R, Ito K (2012) Bone morphology allows estimation of loading history in a murine model of bone adaptation. Biomech Model Mechanobiol 11:483–492 Cunha P, Moura DC, López MAG, Guerra C, Pinto D, Ramos I (2014) Impact of ensemble learning in the assessment of skeletal maturity. J Med Syst 38:1–10

218 Currey JD (2006) Bones: structure and mechanics. Princeton University Press Currey J (2009) Measurement of the mechanical properties of bone: a recent history. Clin Orthopaed Relat Res 467:1948–1954 Currey J (2010) Mechanical properties and adaptations of some less familiar bony tissues. J Mech Behav Biomed Mater 3:357–372 Currey JD (2014) The mechanical adaptations of bones. Princeton University Press Dallora AL, Anderberg P, Kvist O, Mendes E, Diaz Ruiz S, Sanmartin Berglund J (2019a) Bone age assessment with various machine learning techniques: a systematic literature review and meta-analysis. PLoS One 14:e0220242 Dallora AL, Berglund JS, Brogren M, Kvist O, Ruiz SD, Dübbel A, Anderberg P (2019b) Age assessment of youth and young adults using magnetic resonance imaging of the knee: a deep learning approach. JMIR Med Inform 7:e16291 Dattatrey M, Tiwari AK, Ghoshal B, Singh J (2019) Predicting bone modeling parameters in response to mechanical loading. IEEE Access 7:122561–122572 Davies H (2001) The relationship between surface strain and measurements of bone quality, quantity and shape. Equine Vet J 33:16–20 Deng W-J, Chen W-C, Pei W (2008) Back-propagation neural network based importance–performance analysis for determining critical service attributes. Expert Syst Appl 34:1115–1125 Dickey JP, Pierrynowski MR, Bednar DA, Yang SX (2002) Relationship between pain and vertebral motion in chronic low-back pain subjects. Clin Biomech 17: 345–352 Dong J, Qin Q-H, Xiao Y (2020) Nelder–Mead optimization of elastic metamaterials via machine-learningaided surrogate modeling. Int J Appl Mech 12: 2050011 Favre J, Hayoz M, Erhart-Hledik JC, Andriacchi TP (2012) A neural network model to predict knee adduction moment during walking based on ground reaction force and anthropometric measurements. J Biomech 45:692–698 Fradinho MJ, Vale AC, Bernardes N, Caldeira RM, Vaz MF, Ferreira-Dias G (2015) Biomechanical properties of the equine third metacarpal bone: in vivo quantitative ultrasonography versus ex vivo compression and bending techniques. J Equine Vet Sci 35:198–205 Frame J, Wheel M, Riches P (2017) A numerical investigation and experimental verification of size effects in loaded bovine cortical bone. Int J Numer Meth Biomed Eng Fregly BJ, D’Lima DD, Colwell CW Jr (2009) Effective gait patterns for offloading the medial compartment of the knee. J Orthop Res 27:1016–1021 Garijo N, Martínez J, García-Aznar J, Pérez M (2014) Computational evaluation of different numerical tools for the prediction of proximal femur loads from bone

S. Mouloodi et al. morphology. Comput Methods Appl Mech Eng 268: 437–450 Garijo N, Verdonschot N, Engelborghs K, García-Aznar J, Pérez M (2017) Subject-specific musculoskeletal loading of the tibia: computational load estimation. J Mech Behav Biomed Mater 65:334–343 Gerges M, Eng H, Chhina H, Cooper A (2020) Modernization of bone age assessment: comparing the accuracy and reliability of an artificial intelligence algorithm and shorthand bone age to Greulich and Pyle. Skelet Radiol 49:1449–1457 Géron A (2019) Hands-on machine learning with Scikitlearn, Keras, and TensorFlow: concepts, tools, and techniques to build intelligent systems. O’Reilly Media Ghavidelnia N, Bodaghi M, Hedayati RJM (2021) Femur auxetic meta-implants with tuned micromotion distribution. Materials 14:114 Giarmatzis G, Zacharaki EI, Moustakas K (2020a) Neural network based prediction of knee contact forces for different gait speeds. In: 2020 IEEE international conference on bioinformatics and biomedicine (BIBM), pp 2590–2595 Giarmatzis G, Zacharaki EI, Moustakas K (2020b) Realtime prediction of joint forces by motion capture and machine learning. Sensors 20:6933 Halevy A, Norvig P, Pereira FJIIS (2009) The unreasonable effectiveness of data. IEEE Intell Syst 24:8–12 Hambli R (2010) Application of neural networks and finite element computation for multiscale simulation of bone remodeling. J Biomech Eng 132 Hambli R (2011a) Apparent damage accumulation in cancellous bone using neural networks. J Mech Behav Biomed Mater 4:868–878 Hambli R (2011b) Numerical procedure for multiscale bone adaptation prediction based on neural networks and finite element simulation. Finite Elem Anal Des 47:835–842 Hambli R (2014) 3D finite element simulation of human proximal femoral fracture under quasi-static load. Adv Biomech Appl 1:1 Hambli R, Hattab N (2013) Application of neural network and finite element method for multiscale prediction of bone fatigue crack growth in cancellous bone. In: Multiscale computer modeling in biomechanics and biomedical engineering. Springer, pp 3–30 Hambli R, Frikha S, Toumi H, Tavares JMR (2016) Finite element prediction of fatigue damage growth in cancellous bone. Comput Methods Biomech Biomed Engin 19:563–570 Harmsen M, Fischer B, Schramm H, Seidl T, Deserno TM (2012) Support vector machine classification based on correlation prototypes applied to bone age assessment. IEEE J Biomed Health Inform 17:190–197 Hertz JA (2018) Introduction to the theory of neural computation. CRC Press Hong Y, Hou B, Jiang H, Zhang J (2020) Machine learning and artificial neural network accelerated computational discoveries in materials science. Wiley Interdisc Rev: Comput Molecular Sci 10

9

How Artificial Intelligence and Machine Learning Is Assisting Us to. . .

Iglovikov VI, Rakhlin A, Kalinin AA, Shvets AA (2018) Paediatric bone age assessment using deep convolutional neural networks. In: Deep learning in medical image analysis and multimodal learning for clinical decision support. Springer, pp 300–308 Javadi A, Tan T, Zhang M (2003) Neural network for constitutive modelling in finite element analysis. Comput Assist Mech Eng Sci 10:523–530 Johnson T, Socrate S, Boyce M (2010) A viscoelastic, viscoplastic model of cortical bone valid at low and high strain rates. Acta Biomater 6:4073–4080 Jones L, Golan D, Hanna S, Ramachandran M (2018) Artificial intelligence, machine learning and the evolution of healthcare: a bright future or cause for concern? Bone Joint Res 7:223–225 Keaveny TM, Morgan EF, Niebur GL, Yeh OC (2001) Biomechanics of trabecular bone. Annu Rev Biomed Eng 307–333 Khaterchi H, Chamekh A, BelHadjSalah H (2015) Artificial neural network analysis for modeling fibril structure in bone. Int J Precis Eng Manuf 16:581–587 Khovanova NA, Shaikhina T, Mallick KK (2015) Neural networks for analysis of trabecular bone in osteoarthritis. Bioinspired Biomimetic Nanobiomater 4:90–100 Kim T-h 2010. Pattern recognition using artificial neural network: a review. In: International conference on information security and assurance. Springer, pp 138–148 Kulin RM, Jiang F, Vecchio KS (2011) Effects of age and loading rate on equine cortical bone failure. J Mech Behav Biomed Mater 4:57–75 Liang L, Liu M, Martin C, Sun W (2018) A deep learning approach to estimate stress distribution: a fast and accurate surrogate of finite-element analysis. J R Soc Interface 15:20170844 Lin H-H, Shu S-G, Lin Y-H, Yu S-S (2012) Bone age cluster assessment and feature clustering analysis based on phalangeal image rough segmentation. Pattern Recogn 45:322–332 Lipton ZC, Berkowitz J, Elkan C (2015) A critical review of recurrent neural networks for sequence learning. arXiv preprint arXiv:1506.00019 Liu J, Qi J, Liu Z, Ning Q, Luo X (2008) Automatic bone age assessment based on intelligent algorithms and comparison with TW3 method. Comput Med Imaging Graph 32:678–684 Liu Y, Shih S-M, Tian S-L, Zhong Y-J, Li L (2009) Lower extremity joint torque predicted by using artificial neural network during vertical jump. J Biomech 42:906– 911 Liu Y, Zhang C, Cheng J, Chen X, Wang ZJ (2019) A multi-scale data fusion framework for bone age assessment with convolutional neural networks. Comput Biol Med 108:161–173 Ma J, Dong S, Chen G, Peng P, Qian L (2021) A datadriven normal contact force model based on artificial neural network for complex contacting surfaces. Mech Syst Signal Process 156:107612

219

Malek S, Gunalan R, Kedija S, Lau C, Mosleh MA, Milow P, Amber H, Saw A (2016) A primary study on application of artificial neural network in classification of pediatric fracture healing time of the lower limb. In: International conference on practical applications of computational biology & bioinformatics. Springer, pp 23–30 Malek S, Gunalan R, Kedija S, Lau C, Mosleh MA, Milow P, Lee S, Saw A (2018) Random forest and self organizing maps application for analysis of pediatric fracture healing time of the lower limb. Neurocomputing 272:55–62 Manda K, Wallace RJ, Xie S, Levrero-Florencio F, Pankaj P (2017) Nonlinear viscoelastic characterization of bovine trabecular bone. Biomech Model Mechanobiol 16:173–189 Mandal P (2017) Artificial neural network prediction of buckling load of thin cylindrical shells under axial compression. Eng Struct 152:843–855 Mandal S, Sivaprasad P, Venugopal S, Murthy K (2009) Artificial neural network modeling to evaluate and predict the deformation behavior of stainless steel type AISI 304L during hot torsion. Appl Soft Comput 9:237–244 McCulloch WS, Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys 5:115–133 Mirzaali MJ, Schwiedrzik JJ, Thaiwichai S, Best JP, Michler J, Zysset PK, Wolfram U (2016) Mechanical properties of cortical bone and their relationships with age, gender, composition and microindentation properties in the elderly. Bone 93:196–211 Mitchell R, Michalski J, Carbonell T (2013) An artificial intelligence approach. Springer, New York Mohanty K, Yousefian O, Karbalaeisadegh Y, Ulrich M, Grimal Q, Muller M (2019) Artificial neural network to estimate micro-architectural properties of cortical bone using ultrasonic attenuation: a 2-D numerical study. Comput Biol Med 114:103457 Mouloodi S (2020) 3D modelling, finite element analysis, and artificial neural network analysis of a complex nonlinear structure: a study on a long bone. PhD Thesis, The University of Melbourne Mouloodi S, Rahmanpanah H, Burvill C, Davies H (2019a) Converging-diverging shape configuration of the diaphysis of equine third metacarpal bone through computer-aided design. Compar Exer Physiol 15:349– 358 Mouloodi S, Rahmanpanah H, Burvill C, Davies HM (2019b) Accuracy quantification of the reverse engineering and high-order finite element analysis of equine MC3 forelimb. J Equine Vet Sci 78:94–106 Mouloodi S, Rahmanpanah H, Burvill C, Davies HM (2020a) Prediction of displacement in the equine third metacarpal bone using a neural network prediction algorithm. Biocybernet Biomed Eng 40:849–863 Mouloodi S, Rahmanpanah H, Burvill C, Davies HM (2020b) Prediction of load in a long bone using an

220 artificial neural network prediction algorithm. J Mech Behav Biomed Mater 102:103527 Mouloodi S, Rahmanpanah H, Burvill C, Gohari S, Davies HMS (2021a) Experimental, regression learner, numerical, and artificial neural network analyses on a complex composite structure subjected to compression loading. Mech Adv Mater Struct 1–17 Mouloodi S, Rahmanpanah H, Martin C, Burvill C, Davies HM (2021b) The use of deep learning algorithms to predict mechanical strain from linear acceleration and angular rates of motion recorded from a horse hoof during exercise. Int J Mech Sci (Under Review) Mundt M, Thomsen W, Witter T, Koeppe A, David S, Bamer F, Potthast W, Markert B (2020) Prediction of lower limb joint angles and moments during gait using artificial neural networks. Med Biol Eng Comput 58: 211–225 Nadeem MW, Goh HG, Ali A, Hussain M, Khan MA (2020) Bone age assessment empowered with deep learning: a survey, open research challenges and future directions. Diagnostics 10:781 Nian Y, Wan S, Li X, Su Q, Li M (2019) How does bio-inspired graded honeycomb filler affect energy absorption characteristics? Thin-Walled Struct 144: 106269 Nobakhti S, Katsamenis OL, Zaarour N, Limbert G, Thurner PJ (2017) Elastic modulus varies along the bovine femur. J Mech Behav Biomed Mater 71:279– 285 Oeser M, Freitag S (2009) Modeling of materials with fading memory using neural networks. Int J Numer Meth Eng 78:843–862 Olden JD, Jackson DA (2002) Illuminating the “black box”: a randomization approach for understanding variable contributions in artificial neural networks. Ecol Model 154:135–150 Olden JD, Joy MK, Death RG (2004) An accurate comparison of methods for quantifying variable importance in artificial neural networks using simulated data. Ecol Model 178:389–397 Pan X, Zhao Y, Chen H, Wei D, Zhao C, Wei Z (2020) Fully automated bone age assessment on large-scale hand X-ray dataset. Int J Biomed Imaging 2020: 8460493 Patton DM, Bigelow EM, Schlecht SH, Kohn DH, Bredbenner TL, Jepsen KJ (2019) The relationship between whole bone stiffness and strength is age and sex dependent. J Biomech 83:125–133 Pham D, Pham P (1999) Artificial intelligence in engineering. Int J Mach Tools Manuf 39:937–949 Phellan R, Hachem B, Clin J, Mac-Thiong JM, Duong L (2021) Real-time biomechanics using the finite element method and machine learning: review and perspective. Med Phys 48:7–18 Rahmanpanah H, Mouloodi S, Burvill C, Gohari S, Davies HM (2020a) Prediction of load-displacement curve in a complex structure using artificial neural networks: a study on a long bone. Int J Eng Sci 154:103319

S. Mouloodi et al. Rahmanpanah H, Mouloodi S, Burvill C, Gohari S, Davies HMJIJoES (2020b) Prediction of load-displacement curve in a complex structure using artificial neural networks: A study on a long bone 154:103319 Rajan K (2005) Materials informatics. Mater Today 8:38– 45 Ranganathan A (2004) The levenberg-marquardt algorithm. Tutoral LM Algorithm 11:101–110 Ren X, Li T, Yang X, Wang S, Ahmad S, Xiang L, Stone SR, Li L, Zhan Y, Shen D (2018) Regression convolutional neural network for automated pediatric bone age assessment from hand radiograph. IEEE J Biomed Health Inform 23:2030–2038 Rice JC, Cowin SC, Bowman JA (1988) On the dependence of the elasticity and strength of cancellous bone on apparent density. J Biomech 21:155–168 Rostedt M, Ekström L, Broman H, Hansson T (1998) Axial stiffness of human lumbar motion segments, force dependence. J Biomech 31:503–509 San Ha N, Lu G (2020) A review of recent research on bio-inspired structures and materials for energy absorption applications. Compos Part B 181:107496 Sha W, Guo Y, Yuan Q, Tang S, Zhang X, Lu S, Guo X, Cao Y-C, Cheng S (2020) Artificial intelligence to power the future of materials science and engineering. Adv Intell Syst 2:1900143 Sharir A, Barak MM, Shahar R (2008) Whole bone mechanics and mechanical testing. Vet J 177:8–17 Shelburne KB, Torry MR, Steadman JR, Pandy MG (2008) Effects of foot orthoses and valgus bracing on the knee adduction moment and medial joint load during gait. Clin Biomech 23:814–821 Simon HA (1983) Why should machines learn?, Machine learning. Elsevier, pp 25–37 Sohail A, Younas M, Bhatti Y, Li Z, Tunç S, Abid M (2019) Analysis of trabecular bone mechanics using machine learning. Evol Bioinforma 15: 1176934318825084 Suzuki K (2011) Artificial neural networks: methodological advances and biomedical applications. BoD–Books on Demand Tang FH, Chan JL, Chan BK (2019) Accurate age determination for adolescents using magnetic resonance imaging of the hand and wrist with an artificial neural network-based approach. J Digit Imaging 32:283–289 Taylor M, Perilli E, Martelli S (2017) Development of a surrogate model based on patient weight, bone mass and geometry to predict femoral neck strains and fracture loads. J Biomech 55:121–127 Tiwari AK, Kumar N (2018) Establishing the relationship between loading parameters and bone adaptation. Med Eng Phys 56:16–26 Urschler M, Grassegger S, Štern D (2015) What automated age estimation of hand and wrist MRI data tells us about skeletal maturation in male adolescents. Ann Hum Biol 42:358–367 Vukicevic AM, Jovicic GR, Jovicic MN, Milicevic VL, Filipovic ND (2018) Assessment of cortical bone fracture resistance curves by fusing artificial neural

9

How Artificial Intelligence and Machine Learning Is Assisting Us to. . .

networks and linear regression. Comput Methods Biomech Biomed Engin 21:169–176 Wang AY-T, Murdock RJ, Kauwe SK, Oliynyk AO, Gurlo A, Brgoch J, Persson KA, Sparks TD (2020a) Machine learning for materials scientists: an introductory guide toward best practices. Chem Mater 32: 4954–4965 Wang AY-T, Murdock RJ, Kauwe SK, Oliynyk AO, Gurlo A, Brgoch J, Persson KA, Sparks TDJCoM (2020b) Machine Learning for materials scientists: an introductory guide towards best practices Wang Z, Zhang J, Li Z, Shi C (2020c) On the crashworthiness of bio-inspired hexagonal prismatic tubes under axial compression. Int J Mech Sci 186:105893

221

Wolff J (1892) Das gesetz der transformation der knochen. A Hirshwald 1:1–152 Wolff J (2012) The law of bone remodelling. Springer Science & Business Media Xie S, Manda K, Wallace RJ, Levrero-Florencio F, Simpson AHRW, Pankaj P (2017) Time dependent behaviour of trabecular bone at multiple load levels. Ann Biomed Eng 45:1219–1226 Zadpoor AA (2013) Open forward and inverse problems in theoretical modeling of bone tissue adaptation. J Mech Behav Biomed Mater 27:249–261 Zadpoor AA, Campoli G, Weinans H (2013) Neural network prediction of load from the morphology of trabecular bone. Appl Math Model 37:5260–5276

Visual Communication and Creative Processes Within the Primary Care Consultation

10

Holly Quinton

Abstract

Discussion of the value of image, metaphor and creative principles to good consulting skill and patient education within the Primary Care setting is important in enhancing improved patient–physician interactions. A broad-based view of the techniques used in undergraduate and postgraduate teaching within Medical Education in the UK and US are canvassed to establish the best practices and efficacy of using drawings and images as communication tools between physicians and patients. A descriptive analysis of the author’s use of image and metaphor is analysed to assess how such convey medical information and help in the improvement of consultation and patient understanding. Keywords

Communication · Drawing · Colour · General practice · Visual communication Manual drawing is a communication method that occurs within clinical settings. It is carried out by a health professional for a patient, parent or patient’s carer. More widely, drawing and visual communication are part of professional and educational interactions with colleagues, trainees or

H. Quinton (*) Glasgow School of Art Visual CommunicationIllustration, Glasgow, Scotland

students. Diagrams or drawings are made and offered in conjunction with spoken or written explanations (sometimes with annotations added to the drawing) and deal with everything from anatomy, health conditions and planned treatments. Person centred health care is characterised by being tailored very specifically to the patient’s health condition and body, and incorporates personalised information needs based on the patient’s level of comprehension and emotional state, which evidence suggests, is important (Lyon and Turland 2019; Berkman et al. 2011; Coleman 2014; Lam 2004). As doctors we do not formally learn to use drawing in our communication with colleagues or patients and yet in our training; communication and interpersonal skills are honed. We are trained to gather information in order to facilitate accurate diagnosis, counsel and give therapeutic instructions—whilst establishing caring relationships with patients (Lyon and Turland 2019). Patient surveys show that patients want better communication with their doctors (Duffy et al. 2004; Webster 1989). We pay attention to our clothes, our rooms, the semiotics of the signage in hospital and waiting rooms; all of which have been visually considered. Medicine is often pondered upon, as being more of an art or a science and it is unarguably both. Harnessing the potential for using image within the General Practice (GP) consultation would be in keeping with the explosion of smartphone apps following the 2015 release of ‘ResearchKit’, Apple’s

# The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 P. M. Rea (ed.), Biomedical Visualisation, Advances in Experimental Medicine and Biology 1356, https://doi.org/10.1007/978-3-030-87779-8_10

223

224

open-source platform to build smartphone apps for medical research of Asthma, Type 2 Diabetes, Cardiovascular and Parkinson’s disease (Dorsey et al. 2017). One-fifth of smartphone owners have health apps (Fox and Duggan 2012) and they cover a wide range of chronic disease management. For patients with asthma; the British Lung Foundation app provides regular reminders to participants about the importance of maintenance inhaler therapy and provides a symptom and medication efficacy rating diary (British Lung Foundation 2020). Ear Nose and Throat symptoms are also being included in innovative visual app technology (Rak et al. 2019). Medical apps enhance widespread participation by removing geographic barriers to participation and enable more objective data collection (Dorsey et al. 2017), and increase the ability for virtual reality research and intervention. (Overkamp 2019) and ever improving 3D printing (Meglioli et al. 2020; Meskó 2019). All primary care surgeries encourage online booking of appointments, ordering of prescriptions online. Virtual consultations were written into the NHS England Long Term Plan NHS to ensure that every patient in England will have access to online and video consultation (Alderwick and Dixon 2019) ‘high quality primary care’ from 2021. Simply put; our world is a heavily visual and technological one. Lectures in medical school use images, cartoons, animations, and demonstration models. As General Practitioners, we point to the anatomical posters in our rooms, use surgical drawings for minor surgery consent or the ‘clock test’ (Fig. 10.1) in dementia/cognition screening (Shulman 2000). The drawings are often schematic and rarely a traditional completed artwork. As juniors in hospital, we mark the lung fields with little crosses or musical notes (Fig. 10.2) to communicate the sound of infection, wheeze or fibrosis in the hand-held notes (Scott et al. 2013). In GP, some of us use electronic IT management, system drawing capabilities to document pressure sores, bruises or skin changes in child protection documentation, dermatology or bedridden patients (Broadbent et al. 2018).

H. Quinton

Fig. 10.1 Clock drawing as part of cognition screening

Evidence based medicine requires primary research and drawing in the primary care consultation is currently quite scant in research projects, but features much more prevalently in other specialties, and is slowly being given more attention. Doctor C Kearns as accredited in Kearns (2019), is a clinical teaching fellow at the College of Medicine in Edinburgh who, in collaboration with his team, claim to be the first to document the prevalence of drawing by surgeons while Wright (2019) assesses the value of drawing in research. In the study, Kearns (2019) interviewed 100 surgeons, (92 of whom reported the positive value of drawing in surgical practice and research) and found that of 244 patients, 99% valued the educational value of a drawing in moderate or complex surgery consent explanation. Research evidence highlights the neurological and cognitive benefits of using image in undergraduate and postgraduate pedagogy alongside the neuroscience and cognitive processing of learning, (Lyon and Turland 2019; Duffy et al. 2004; Finn et al. 2011; Ainsworth et al. 2011; Backhouse et al. 2016) through observational drawing (Chamberlain et al. 2014) especially clay work and retention of concomitant anatomical lecture principles. When asked to

10

Visual Communication and Creative Processes Within the Primary Care Consultation

Fig. 10.2 Image to show the presence of wheeze and drawing to indicate clear, normal sounding lung fields

draw, often in medical humanities sessions, it has been noted that there is usually initial recoil by the student, and a claim that they are not creative and cannot draw (Keenan et al. 2017). Keenan et al. (2017) encourage colleagues and students to see drawing as a form of learning and expression rather than a pursuit of creating artwork in undergraduate education. The authors have written about the use of technology enhanced learning (TEL) and the importance of implementing artistic learning approaches in order to present information memorably (Keenan et al. 2017). Keenan et al. observe that, good and thorough observation/visualisation are ‘crucial for the learning of anatomy and are also relevant for clinical observation and diagnosis’. This pedagogical advice is meant for teachers of the medical curricula, but the process of imparting knowledge from lecturer to student is not too dissimilar to the dynamic between the patient and the GP in the consultation. It is appropriate for similar methods to be implemented in improving the patients’ experience and education in

225

attempts to improve the quality of care. Just as a student receives pharmacodynamic and anatomical information in a seminar, so does the patient in the consultation room. All medical professionals are striving for patient-centred and individualised care with a heavy emphasis of shared care decision models in modern medicine (Elwyn et al. 2010). The drawing created in the General Practice consultation is often the summation of the topic discussed and tailor-made for that individual patient or maybe even drawn by the patient themselves. There is little data of the prevalence of manual clinical drawing as a practice, but anecdotally and from Dr. Phillipa Lyon’s referenced writing (Lyon and Turland 2016, 2019) drawing occurs, often quite informally, between patient and professional from the GP surgery to dementia wards. Often, patients ask for the image created during their consultation as an aide memoire, a foundation for further research or for discussion with a partner. Patient information leaflets are widely available, and it is proposed that sometimes adding to these or creating patient specific imagery can be more efficacious in being more patientcentred. A literature search revealed some exciting and innovative approaches being used generally in medical education and other specialities—particularly within the discipline of anatomy teaching, academic medicine and also in surgical practice. Professor Gabrielle Finn Professor of Medical Education at the University of Manchester and events director of the Association for the study of Medical Education (ASME), fosters a network of artists, science educators and health professionals; all interested in alternative andragogy—by painting anatomical structures directly onto the body. Her work can be found at paintmeeanatomy.com. She has contributed to and initiated research, considering the impact of visual and colour psychology on learning, retention of medical knowledge. Interestingly no specific link was found between colourful imagery and retention of medical knowledge in the short or long term in a medical student cohort study (Finn et al. 2011). Her pioneering use of

226

fluorescent paint to show layering of the human anatomy has proven visually striking and memorable, specifically when describing layers of human tissue (Finn 2018). Professor Paul M. Rea, Professor of Digital and Anatomical Education at University of Glasgow created the world’s first Medical Visualisation and Human Anatomy MSc—combining anatomy and digital technologies (Rea 2021). MSc student Dr. Angela Douglas along with Dr. Craig Daly created an animation describing hypertension, using clay models and animation (Daly and Douglas 2020), which simply and visually describes the difficult concepts in audio and video for patient information and innovative public engagement (Rea 2021). These projects create accurate information for patients which can consolidate information given during consultation in the primary care setting. Dr. Mike Todorovic in Australia uses the social media platforms Twitter and YouTube creating sound-bite videos, intended primarily for the medical education community but accessible enough for the general public (Todorovic 2021). Familiarisation with the innovative patient and health information sites like this can be incorporated into practice by GPs. Should there be fear of directing to non-NHS validated sources then viewing the information and adopting suggestions into the consultation itself could be considered. Medical Education resources give ideas for more creative and effective ways of personally improving physician practice and provide ideas of communicating visually. They are a useful adjunct to verbal consultation. Professor Alice Roberts Professor of Public Engagement in Science at the University of Birmingham uploads anatomical videos which are lyrical and visually appealing. She uses her illustrative skills to impart complex human biological process including embryological development and hand anatomy. These recordings simplify, and in an accessible narrative structure convey the wonder and complexity of biology without cognitive overload and feel accessible to anybody—and that is the point, public engagement through storytelling and entertainment (Roberts 2021).

H. Quinton

Christmas Lectures, started by Michael Faraday in 1825 are the United Kingdom’s flagship series an annual festival of science promotion through performance science often with the use of theatrics and comedy. An example of this would be in 2020 when geologist Chris Jackson (Jackson 2020) showed how the planet’s oldest rocks and fossils tell a story of radical climate change, whilst physicist and oceanographer Helen Czerski (Czerski 2020) showed how shifting ocean water creates an engine that distributes heat and nutrients around the planet. Imparting some difficult theoretical concepts. This is public engagement at its very finest. An inspiring example of how imagery can convey complex information. The potential for innovative visual explaining to make a consultation in General Practice more innovative and efficacious is feasible. The outcome would be a patient who understands the facts, has tailored information and treatment and to have benefitted from the experience—biologically psychologically and socially-irrespective of their cultural or socioeconomic background. A flagship course at Brown University in the United States entitled, ‘The Physician As Illustrator’ is an undergraduate course run by Professor Francois Luke who recognises the power of art and that visuals can speak louder than words. The course is advertised with the following: We are not all artists, but we often use pictures instead of thousands of words—whether to explain medical concepts to students or procedures to patients. Cultural, language and educational barriers may hinder verbal communication, and the use of simplified diagrams can enhance patients’ understanding of their medical condition (Brown Alpert Medical School).

The course aims and objectives include the need to understand the place of visual arts in improving communication across language and socio-educational barriers and to better communicate complex medical ideas (Brown Alpert Medical School 2021). The objectives are best linked to the simplification of medical processes for patients to ensure they can better acquaint themselves with an understanding of the illness and treatment process. Visual aids communicate and

10

Visual Communication and Creative Processes Within the Primary Care Consultation

deconstruct complex medical information to enhance the interactions between physicians and their patients. Hence, the course at Brown University is designed to equip physicians with the capacity of using visual aids to communicate with patients. Dr. Phillipa Lyon is a senior lecturer at the Brighton School of Art and she has researched drawing by clinicians for patients focusing on the doctor patient relationship with a meticulous exploration of the role of creating image within the medical consultation itself although not specifically within a primary care setting. In her work (Lyon and Turland 2016, 2019) she references Roland Barthes, author of ‘Rhetoric of the Image’ (Barthes 1977; Haustein 2014) a flagship work of semiology—the messaging of image making. This theory looks at the, ‘linguistic’, ‘symbolism’ and ‘literal’ components of image. This analysis is deeply theoretical and extensive, so deep discussion is beyond the scope here, but Barthes’ theories are pertinent in how as a clinician, ‘anchors’ and ‘relays’ medical concepts (Haustein 2014; Lyon and Turland 2016). When considering the relationship between verbal or written communication and image Barthes refers to the idea of ‘anchorage’ as the most frequent function of linguistic messaging (Barthes 1977). It is an academic description of the use of verbal explanation or text with an image to reduce ambiguity of one modality of communication (verbal alone or text alone). Barthes gives the example of a small collection of fruits in an image, with accompanying text reading ‘as if from your own garden’. On its own, the picture could look like a failed harvest, Barthes explains but the accompanying text banishes the idea of a scarcity and re-orientates the reader to the idea of a smaller but home grown nutritious collection of fruits (Barthes 1977). This anchorage makes sure the context is clear. This idea, extrapolated out to the General Practice consultation could be—a hand drawn circle on the page, which may not initially seem to represent anything much; but if the clinician verbalises that, ‘the circle is a red blood cell, it could balance on the end of a pin head’ or, ‘this is a life sized drawing of a cervix’, the drawing context is made clear and ‘anchored’

227

to ensure the achievement of a descriptive meeting point between patient and clinician. Another example of this may be a drawing a retina; the clinician verbally describing the circle to represent the retina, relative in size to the anatomy and adding a metaphor, i.e., a rugby ball, then suggesting the patient visualises making a hole in the ball, looking through the hole and adding illumination, to see inside to the back wall (the drawn circle), where the medical problem lies. The function of relay is less common (Barthes 1977), and is seen in comic strips and cartoons. The image and text are complimentary. This style of communication is more likely to be pre-populated, for example a patient information leaflet or a medical cartoon. To illustrate the idea of anchorage and relay, an image was created of an electrical plug to describe the anatomy and processes occurring in carpal tunnel syndrome based on the model described by Barthes. Using a familiar object to describe anatomy encompasses the concept of making the transfer of information accessible and specific (Fig. 10.3). A description of the tendons of the hand running into a channel and held down by retinaculum is made. The space is described as small, not allowing for swelling or

Fig. 10.3 Image comparing left hand palmar sensory distribution with a plug metaphor

228

H. Quinton

fluid and, if these are present—tingling or pain can be felt in a specific distribution in the hand. The verbally created imagery is enhanced using pictures-ensuring new terms are explained in a clear and simple way.

10.1

Ethics

Ethical consideration is paramount in all our interactions with our patients. This is not a reason to avoid engaging in the creative activity but there is a requirement to consider the value of the image-once it has been completed. Often, a scribble on paper is simply discarded into the waste as it describes a principle or generic concept and other times it’s highly personal in the description of an unusual identifiable body part (following surgery, congenital or traumatic disfigurement) or even the presence of a tattoo. The Data Protection Act 1998 (Gov.Uk 2018) is the current legislation in the UK that dictates how individuals and organisations handle, store and process personal information. The 8 Caldicott principles apply to images as part of a record (GOV.UK 2020). This can sometimes limit publication due to the probability of directly referencing the patient. The Institute of Medical Illustrators (IMI) National Guideline ‘Confidentiality and Consent. A Guide to Good Practice’ was updated in June 2020. It provides mandatory practice guidance for medical illustration and other healthcare professionals making, using and storing visual recordings of patients’ data, rights to confidentiality, advice regarding obtaining appropriate consent and complying with relevant legislation. In summary, if an image is created in the GP consultation, it then requires confidentiality protection (Cull and Gilson 1986; Cull 1988) and therefore approaching drawing in the same way we do patient photographs and patient data is safest (Kearns et al. 2019). When I was a junior-doctor I kept a visual diary which contained quotes, sketches and documentation often following a period of reflection or analysis of a problem I’d encountered, for

Fig. 10.4 Drawing summarising the phenomena of forgetting tablet names

example, a difficult conversation with an unhappy family, or a sad and traumatic revelation by a patient. These drawings were more prolific during my job in Accident and Emergency and Elderly care. This was a catharsis for me—a debriefing and a summation. For many years I didn’t show anybody these despite most of them not containing patient identifying information. I felt the drawings held sanctity where I, along with the patient was present in the image. I was reminded of the privilege bestowed upon medical professionals with the public, during some of the most desperate or intimate parts of their lives. It is only recently I sought to edit these images and somehow the authenticity leaves them less powerful in their altered state. A summative image not based on a specific patient, but of multiple patient exposures can often for physician catharsis and professional development. These images illustrate concepts rather than individuals to highlight possible health system improvement. Figure 10.4 communicates lack of patient unawareness of medication names. Initially, the image is amusing—showing vague

10

Visual Communication and Creative Processes Within the Primary Care Consultation

information gathering, on deeper reflection, it becomes emotionally touching given the clear vulnerability of the patient in the situation. There is also a learning point for health improvement—possibly evaluation of the data gathering system from patients on admission to hospital. Many patients do not know what their medication is for (Linsky and Simon 2012) and half of chronic disease medication is not taken (Jones 2003), patients often forget the names and doses (Tranchard et al. 2019). Hence, the use of drawings is important in understanding the perception of patients and facilitating their understanding of their health status (Broadbent et al. 2018). The health professional is able to understand how patients perceive their illnesses in a process that facilitates improved care (Broadbent et al. 2018). All of this is information is present in one image. It was something I drew in a minute but captured months of patient exposure and interaction. I see it as a form of reflection but also to identify my education needs and pointers towards health improvement. Figure 10.5 is a drawing summarising patients with chronic lung disease claiming they stopped smoking last week, or last night, in reply to the social questions we as doctors are trained to ask to gain insight into lifestyle factors in disease processes. All medical staff have these experiences, they are extremely valuable. We need to encourage their fruition from lived experience or thought to initiatives in designing a new medication system or improvements with a department. Graphic novels and more recently graphic medicine cartoons are becoming popular as an exploration between image and the discourse of Medicine. The website graphicmedicine.org is an example—where the blending of image and text attempts to engage medics and the public with the complex challenges of the medical world. This fits in with health promotion and public engagement principles. It is also a form of learning and entertainment. Cartoons are not, in themselves very different from formal illustrations or surgical pictures but can perhaps describe movement or consequence of health choices unfolding on a page.

229

Fig. 10.5 A drawing showing the need for exploratory questioning

10.2

Combining Visual Communication and the Medical Consultation

Before Medicine I attended the Glasgow School of Art and gained a BA in Visual Communication. I was taught to consider what it was I wanted to say and then to work out how best to communicate it visually, thus avoiding a fixed artistic style as a blanket approach to my drawing, and fostering an authenticity in the images I made. This meant thinking about which art material to use to capture the spirit of a subject I was drawing—using appropriately descriptive markings. This Multimodality communication is a semiotic theory from the 1980s, by Gunther Kress who examined how speech, gesturing, gaze and the

230

H. Quinton

elements of the material context blend together and contribute to the production of meaning, referencing literature, especially Children’s books (Kress 1980). Philipa Lyon and Martha Turland Turland reference the lack of descriptors of drawing, which somewhat impedes a research framework. They propose a way of acknowledging the presence of multimodality within the consultation (Lyon and Turland 2016, 2019). More broadly in the referenced papers, they explain that semiotic theory, applied to the consultation can help to give names to the elements of the medical consultation This leads on to the structures of the medical consultation itself.

10.3

Medical Consultation Models

Medical consultation models are the structures taught in medical school in the training of General Practitioners that make it easy to understand the medical consultation process and how best to personalise it. These models have, of course changed over the years and much research has been done to examine the best way to utilise the usual 10- or 12-min consultation allotted to patient and doctors (Coleman 2014). In 2020 Simon Stevens, Chief Executive of the National Health Service of England (NHSE) launched the ‘Long Term Plan’—a 10 year projection drawn up by frontline staff, patients groups, and national experts to be ambitious but realistic; with directives regarding the future role of remote consultations and patient triage (NHS England 2019). This directive has encouraged a shift towards initial telephone triage and then face to face examination, according to medical need. This process of offering remote consultation escalated during the COVID-19 Pandemic as advised by Public Health England according to The Royal College of General Practitioners (RCGP) who report around half of consultations in general practice are being delivered face to face (in May 2021). At the peak of the pandemic, the ways in which patients accessed their GP reversed from around 70% face to face and 30% by phone,

video or online pre-pandemic to around 30% face to face and 70% remote (RCGP 2021). The structure of the consultation will also need to adapt with or without visual communication elements. The consultation structures currently remain firmly in place, but research papers learning about the differences in consultation skill requirements and the efficacy and suitability of telephone consultations have predated the 2019 Sars COVID-19 Pandemic (Jiwa et al. 2002; McKinstry et al. 2017) and E-Learning modules are being currently being published to help with the difficulties of Practicing during a global pandemic (gponline.com 2020). Dr. Michael Balint introduced the idea of the ‘doctor as a drug’ describing the interaction with the physician being therapeutic (Balint 1955). This writing highlighted the importance of listening to our patients and the belief that attentive listening can make the patient feel better. Identifying a patient’s, ‘Ideas Concerns and Expectations’ can help to formulate the structure of the doctor patient interaction (Coleman 2014). The idea is to form rapport, for the doctor to elucidate the reason for visit, the performance of a medical examination, to form an opinion, formulate a plan and end the consultation. The Calgary Cambridge structure framework is now most commonly used and forms the triaxial Royal College of General Practitioners consultant exit exam standard against which candidates are compared (RCGP 2021). It was created by Silverman, Kurtz and Draper in 1998 (Kurtz et al. 2003). Biological, psychological and social factors are combined into a model of consultation. It is the only model that actively encourages the use of written or diagrammatic information to help clarify explanations. Validated sources of further reading are available from Patient.co.uk or NHS Inform (NHS 2021a, b, c, d). It’s important to get the style right as Silverman Kurtz and Draper 2008 find as if the doctor fails to elucidate the reason for visiting or explain well, the patient partnership process is dysfunctional (Lam 2004). When a consultation is less paternalistic and patient centred satisfaction is increased (Faulkner 2015).

10

Visual Communication and Creative Processes Within the Primary Care Consultation

10.4

231

Examples of Illustration and Visual Communication in my GP Consultations

These examples are not intended to be wholly didactic nor a panacea for every patient or every encounter within the territory of illness but examples of everyday encounters—which can be adapted to the skills and personality of the physician or patient.

10.5

Congestive Heart Failure

Classic presentation is with cough, shortness of breath, ankles oedema, reduced exercise tolerance and the patient often presents once function is affected, unless signs are picked up at an annual review with the GP or Practice Nurse. Field discusses the demographic, age and comorbidity of patients; notes the average age is 76, and that patients can also have hypertension and sometimes cognitive impairment (Field et al. 2006; Cline et al. 1999). Many respondents expressed uncertainty about what having heart failure meant, about how it differed from other kinds of heart disease and about whether they would recover (Field et al. 2006).

Field et al. (2006) and Cline et al. (1999); both summarise that, “Many patients do not understand what heart failure means, and also, we have shown how even when patients work hard to take their medicines as directed, they do not necessarily understand what different drugs are intended for” and that non compliance is common as are shortcomings in the knowledge of medications despite clinicians verbally explaining (Cline et al. 1999).

Rogers (2000) stated in the summary of the referenced study that, ‘Effective and better ways of communicating with patients with chronic heart failure need to be tested’. I noticed that many of my patients express concern with the terminology of cardiac failure as the semantic is fatalistic and does not adequately describe the presence of a scale of severity, nor the ability to

Fig. 10.6 Working drawing to help describe the mechanisms of heart failure and associated symptoms

significantly move up and down the severity scale. Anecdotally, patients told me they didn’t understand how breathlessness nor oedema were related to the heart. I realised a mismatch of intention. Clinicians want to impart the knowledge of the pathophysiology, order tests and quantify the illness and then medicate and promote heath education (Cheung et al. 2016), but the patient wants to feel well and usually wants to understand ‘why’. The correlation between breathlessness and oedema do not seem linked to many patients and the drawing above show how I use analogy and explanations to convey this in Fig. 10.6. I describe fluid running ‘backwards’ and filling the legs—making fluid boots and lungs, as a rough descriptor and then go into more detail. More research is needed here but patients often thank me for using simple starting language alongside a picture made during the consult to show movement of blood flow.

232

10.6

H. Quinton

Ear Nose and Throat Conditions

Ear Nose and Throat problems lend themselves well to diagram or analogy because the structures are so memorable in shape. The cochlea looks like a snail, the stapes bone is shaped like a horse-riding stirrup and the ear canal is like a tunnel leading to the tympanic membrane or ear ‘drum’ and can be described as ‘like tough clingfilm’. The function of hearing and balance lend themselves well to descriptors of fluid and closure, caves and pressure chambers. There are some great posters showing the anatomy of the ear but what I found most helpful is having a poster or photocopy of anatomically correct medical illustrations and then create my own ink drawing to show effusion in otitis media or post virally congestion. The drawings I make most are for the symptoms of otalgia, ‘fullness’ or ‘dizziness’, relating to vestibular disease (viral or bacterial otitis media) and Eustachian tube dysfunction. Lots of patients seem to re-consult multiple times (DG 2017) when effusion post virally can last for up to 3 months (Walker et al. 2017). As discussed earlier, the colour chosen does not seem to matter for retention of theoretical knowledge but mark making can covey something heavy or light, reflective or dull and block colouring a space implies lack of air and if a dense material is used, this is better conveyed than a chalk or biro markings (Fig. 10.7). Words used in the description of the middle ear, like ‘tunnel’ referring to external auditory canal, leading to a ‘cling film door’ can help anchor and hook knowledge onto familiar imagery, referred to as a ‘cognitive scaffold’ (Ausubel et al. 1978; Hart 1937). Improving comprehension, problem solving and higher patient engagement (Hart 1937). I often find linking the proposed medication to a particular part of the metaphor or drawing results in patients verbally communicating they understand why it is important to take a medicine regularly or when an antibiotic might not help. Doctor patient decision making as a partnership requires explanation of the reasoning for prescribing or withholding

Fig. 10.7 Mixed media to describe different parts of the ear and effusion in the middle ear

treatment unless the patient explicitly asks for a paternalistic approach.

10.7

Analogy and Metaphor

A metaphor is a direct comparison of two totally different things whereas an analogy is comparing two things with a set of another two things. There may be times when there is a need for the patient to come up with these themselves, and when they do, they are often extremely valuable and a relief from medical terminology (Casarett et al. 2010). Analogy is used to demonstrate how two things are similar while metaphor is used to get your point across in a more emphatic manner. Metaphor when describing pain is well documented (Butler and Moseley 2018). In one study it was found that physicians use metaphors in two-thirds of serious illnesses consultations; patients reported that doctors using metaphor better understood their conditions (Casarett et al. 2010). The consultation is often described as a performing art (Woolliscroft and Phillips 2003). Some patients feel it is not enough a description-vehicle on its own. The essayist Susan Sontag, an American writer and philosopher, discussed the widespread use of metaphor in relation to certain diseases in her writings. She

10

Visual Communication and Creative Processes Within the Primary Care Consultation

wrote whilst being treated for breast cancer and her critique was particularly focused on the narratives surrounding HIV and TB; and she argues that miscommunication can be inherent in metaphor and needs to be carefully explored. My point is that illness is not a metaphor, and that the most truthful way of regarding illness—and the healthiest way of being ill—is one most purified of, most resistant to, metaphoric thinking (Sontag 2005).

This interpretation views the use of a tool to covey information akin to depriving a person of the true meaning of something. Like many things in life, one rule does not fit all. If a patient initiates with the use of a verbal metaphor and feels comfortable discussing in these terms then checking understanding and empowering with the correct terminology and contextualising, then it should not be discouraged. Using a minimising metaphor could patronise or confuse, so care must be made when physician initiate metaphor is used. Care also needs to be considered of the language used within the metaphor or analogy with an ‘unknown bias’ considered. Author Arthur Frank who published The Wounded Story-Teller in 1995 fosters the positive use of metaphor with those who suffer from some type of illness or disability and analysing stories within a framework of narrative theory as a way of making sense of the underlying need of a person (Frank 1997). In summary, the implicit comparison of one thing to another is one of the most common literary techniques, because effective communication relies heavily on drawing comparisons and parallels to portray the clarity of an idea. Health care practitioners employ metaphors and analogy for two main reasons: first, to foster clarity by transferring meaning effectively and economically, and secondly, to highlight caution. Some patient examples follow—offered as opening consultation presentation of their illness characteristics. Sometimes the hyperbolic descriptions match the patient’s anxiety—of body language or story telling passion. “My tongue feels weird after antibiotics; like licking a battery!” (oral thrush) “You know when you put your hand in a bag of rice? That nice sort of tingly feeling? Well I get that

233

in my head—but it’s not nice” (antidepressant discontinuation syndrome SSRI) “When I bend my knee, it sounds like walking on gravel” (crepitus) “There’s a humming bird living inside my head” (Tinnitus) “I feel dizzy and then there’s a feeling like my heart has stopped, it falls out of my body and I pass out”

In my practice I see value in the way a patient conveys ideas this way during the initial discussion of a health discussion as it is an authentic description of the illness symptom or good health experience. Other clinicians use metaphor and (Butler and Moseley 2018) have collated a helpful collection of them. If I sense or a patient communicates they do not understand, I sometimes use a metaphor. I like to follow this up with added information about the illness with a leaflet or contextualising with image or anatomy model. This is not a new concept and is referenced as being a vehicle for improving understanding (Sutherland 2001).

10.8

Patient Drawings and Pain

Drawing by patients as a communication of their experience and perspective along with analysis of the images made have revealed correlations between patient beliefs expressed in drawing and health outcomes following MI and in chronic obstructive pulmonary disease (COPD) (Broadbent et al. 2018; Kaptein et al. 2017). It was noted by Broadbent that those who drew larger areas of damage within a heart diagram following heart attack (myocardial infarction— MI) were more pessimistic-took longer to return to work and believed that recovery would take longer (Broadbent et al. 2018). Opiate and opioid prescribing is an ongoing ‘hot topic’, and prescribing has more than doubled between 1998 to 2018 NHS England have referred to this as an ‘opioid epidemic’ (NHS West 2021). National Institute of Clinical Excellence (NICE) guidance suggests improving conversation about opiates and opioid with patients experiencing pain. These conversations are often extremely challenging not only due to deeply held

234

beliefs. I have found using imagery and metaphor to describe pain helpful. These can then be analysed with the patient and conversations about the psychology and non-pharmaceutical treatments can applied to the personalised image. I often use the comparison of pain in migraine in contrast to pain during the burning process of the skin, to differentiate between the consequences of pain. Increasing the vocabulary of pain and describing sensations can be helpful in promoting massage or exercise as a management strategy. Mobile phones are used to relay information between the patient and doctor and encouraged, particularly in seizure management plans, where unusual seizure activity is encouraged to be ‘caught’ on camera’ to show the clinician (Epilepsy Foundation 2021). Anecdotally I’ve seen videos of neurological symptoms, absence seizures, apnoeic pauses and unusual movements in children. Patients sometimes want to show before and after photographs, particularly of skin conditions before or after a treatment, or to show a period of stability whilst consulting with relapse symptoms, for example eczema or acne. Patients say they find it difficult to find the words to describe the problem at times or want to accurately communicate. There is often a feeling of urgency; a desire to succinctly explain and this forms an important consultation skill of agenda setting, and rapport building. Patients also bring their own drawings. The following drawings, (with patient permission) show personal representation of the pain experienced in cluster headache (Figs. 10.8 and 10.9) and end stage osteoarthritis with spondylolisthesis (Fig. 10.10). The feeling of dragging and vice like pain can be seen (Fig. 10.11). Poulsen et al. analysed drawings of patients with osteoarthritis and discovered a link between patient drawings of hip pain at the greater trochanter, groin, thigh buttock, but not so much in the knees, or lower leg. Interestingly the patient in the drawing also required a bilateral hip replacement.

H. Quinton

Fig. 10.8 Showing cluster headache and migraine headache—with patient permission

Fig. 10.9 Showing cluster headache and migraine headache—with patient permission

10.9

Diabetes

Poor health literacy has substantial health consequences: people with low health literacy report worse physical and mental health, which is supported by a higher prevalence of a number of serious health conditions, including diabetes (Berkman et al. 2011). The RCGP created a health literacy report in 2014. The data is worrying. Forty-three per cent of the English adult working-age population cannot fully understand and use health information containing only text. When numerical information is included in health information, this proportion increases to 61%.

10

Visual Communication and Creative Processes Within the Primary Care Consultation

235

Fig. 10.10 Patient drawings with permission describing dragging pain and vice-like pain Fig. 10.12 Diabetes cartoon

literacy skills. At times I draw cartoons for some of my diabetic patients to help with pharmacological understanding and to encourage medicine compliance (Fig. 10.12) or a version of it as an alternative to the written information as written health literature and doctors’ spoken communication are often not pitched at a level that is inclusive of people with low health literacy. I used an idea of text messaging as a familiar analogy and set out the cartoons—using metaphor in a humorous and accessible way. Fig. 10.11 Patient drawings with permission describing dragging pain and vice-like pain

Rowlands goes on in the 2014 report to suggest that all GPs recognise the issues caused by low health literacy and learn to develop consultation techniques to improve the clarity of their communication with patients with low health literacy and support patients to develop their health

10.10 Gynaecology Being the only female GP in my last two workplaces means that a lot of the Women’s health has migrated into my consultation room. A common observation is the need for description of the menstrual cycle and the role of hormonal messaging—particularly when talking about contraception containing progesterone. Menstrual art,

236

sometimes called ‘Menstrala’, a term coined by the artist Vanessa Tiegs (Tiegs 2021) has history back to the 1970s (Røstvik 2019). Of course, shock culture always grabs attention, but there are subtle ways of discussing difficult topics especially considering that globally there remains a stigma of menstruation. Recent campaigns are pushing for free sanitary products in school (Red Box Project 2021). An exhibition of menstrual art was held in Nepal in 2018 which contained a pile of money to show how much was spent on sanitary products each year by the average woman. and can be a powerful means to confront and subvert stigma around menstruation. Linked with this is The Real Period Project who’s mantra is, We want the menstrual cycle to be recognised as a vital body system that is about more than just reproduction, and for everyone to feel safe to talk about and experience menstruation whoever and wherever they are (The Real Period Project 2021).

In the last few years, more women present to me knowing the length of their cycle through the use of Apps on smart phones and it’s quite common for women to use their phone to remind themselves and inform me of the day of their last menstrual period (The Real Period Project 2021). These applications replace the paper diaries that women used to bring in and I often find the description of normal vaginal secretions and or regular bleeding to be exactly recorded in the menstrual cycle mostly due to their userfriendly interfaces. Although I still frequently find myself resorting to pen and paper—to try to make sense with the chronology of symptoms. Figure 10.13 shows the simple drawing I use to describe why a patient should attend on day three or four of the menstrual cycle and at day 21, shows the hypothalamic–pituitary–thyroid axis (HPT axis for short, a.k.a. thyroid homeostasis or thyrotrophic feedback control) as neuroendocrine control system simplified. The pre-populated image can then be drawn on to describe the process to show the dynamic hormonal messaging. When talking about dysfunctional uterine bleeding particularly with patients who are taking progesterone contraceptives, I am often surprised

H. Quinton

Fig. 10.13 Gynaecology explainer. Day 3–4 and Day 21 hormone menstrual cycle aide memoir for practice improvement

at how often I remind patients that this is a common side effect in the first 6 months of about 1/3 of women using these preparations. Much more work needs to be done empowering women with the knowledge that they need to manage their fertility; whilst educating on the disruption that a contraceptive itself can cause. Spending more time talking about the variations of normal and making sure the fundamentals of biology are understood is vital. Reinforcing the fact that a uterus is a muscle, whilst drawing the build-up of blood with a red inky pen, creates a richer description, which, is more authentic than looking at a black and white drawing on some of the traditional patient information leaflets. Documenting with appropriate visuals can only help improve an appreciation of the internal unseen workings of the female body to explain the visible workings. I picked up some paint colour swatches from a home improvement store and they ranged from bright red to brown and white yellow and green. I sometimes, lay these out on my desk and talk about the variations of normal vaginal secretion colour throughout the monthly cycle of vaginal discharge. Not everyone knows how to describe events, and not all female patients have intellectual ability, cognitive ability or English language vocabulary (if English is not the first language) to accurate describe what is happening to them. Spending time using these techniques frequently leads to reassurance without investigation and

10

Visual Communication and Creative Processes Within the Primary Care Consultation

sometimes need for referral is identified when a ‘red flag’ warning symptom is identified. If a communication technique is found to be particularly helpful in data gathering—an audit can be done and added to computer, IT system templates for other clinicians to use.

10.11 Urology Older men are more likely to have difficulties with micturition and prostate symptoms due to enlargement which is often benign but bothersome (Urology Care 2019). Patients usually come in complaining of frequency at night, stream change or difficulty initiating urination— perhaps with some post micturition urinary dribble. Sexual dysfunction often accompanies this chronic condition. The key to the initial investigations are fundamentally rooted in the initial data gathering. A questionnaire validated by the Royal College of Urology known as the LUTS questionnaire (lower urinary tract symptoms) is a point based scale, to ascertain the severity of prostatism now called urinary outflow symptoms (Sahai et al. 2014). This is a validated tool and one that correlates to evidence-based medicine advice (Häkkinen et al. 2007) for prescribing, referring or ordering further tests. When the prostate enlargement occurs, there is the likelihood of an associated chronic illness or cognitive impairment. There have been times when formulaically repeating the same explanation for a cognitively intact 50-year-old, did not suit another patient with aphasia following a stroke. Creativity was required for information gathering and has resulted in the use of the tap in my clinic room as a vehicle to answer the questions. The patient altered the flow of the tap water stream to answer the questionnaire as each category was read to him to answer questions about the quality of his urine stream and control, along with the strength of urinary flow. This approach retained the patient’s autonomy to be able to express himself and due to the activity being novel and messy, there was an easing of the initial awkward atmosphere when he struggled to engage. Some charts are visual, the pain

237

scale and Bristol Stool Chart, where stool types are categorised according to their shape and consistency (WebMD 2021). The paediatric version compares stool to the shape of common chocolate bars which children love to fill in, in macabre delight (WebMD 2021). Novel ideas and amusing anecdotes and quips are often remembered by patients, but it is also important to check when a metaphor is used that the common and classical names are verbalised. This seems to be more pertinent when discussing subjects relating to genitals, breast tissue, sexual identity and mental health concerns. When a patient uses slang, checking and correlating with the correct anatomical term should be implemented to ensure both patient and doctor are discussing the same body structure. The mantra ‘never presume’ has served me well, having been planted is an idea when I was a junior doctor.

10.12 Colour Despite widespread belief, the evidence-base doesn’t reveal much, if any, scientific evidence that colour affects mood, emotions or psychological well-being in any systematic manner (Lenham 2013). An extensive review of colour literature carried out in 2004 by Tofle et al. (2004) concluded that there is insufficient evidence to assert that specific colours can evoke a certain mood, nor is there a link that can be made between specific colours and health outcomes (Tofle et al. 2004) but there are systems of contrasting colours that aim to improve navigation or human function (Gerritsen 1975; Gibson et al. 2004). Within the context of this writing there is no intention of being didactic about the use of colour but a hope to convey the need to be thoughtful when using line and colour to describe illnesses to patients in the knowledge that the material and the lines or shapes drawn need to be contextualised and/or sensitive to the subject matter. There is a need to also acknowledge that colour is important within a person’s culture which is outside the scope of this discussion, but it is worth

238

mentioning that in India, the colour red is documented to be linked to purity and birth (Morton 2004) whereas in Germany, red is traditionally associated with fear, warning and jealousy (De Botoli and Maroto 2001) and in South Africa—red is associated with mourning (Morton 2004). Nursing homes and outpatient clinics, catering to patients with dementia, often use yellow or red door handles and toilet seats so there is a higher visual contrast—shown to aid navigation (Lenham 2013; Gibson et al. 2004) colour preference with advancing age correlates with decreasing preference for blue and increasing choice of red or green (Dittmar 2001). My elderly care placement taught me that avoiding blue floor coverings that look like water help patients who can mistake it for water. Brightly coloured plates are used so that traditional food is more visible— thus improving dietary intake in those who are cognitively and or visually impaired (Lenham 2013; Reading 2015). Drawings by adults with aphasia have been researched and found to be clinically relevant as they may help specify location and severity of brain damage in identifying thought and can be a new form of expression, aiding social communication and integration (Lyon 1995). I learned from community palliative care nurses that the stress and anxiety, (particularly for observing relatives) in patients who have head and neck cancer—which invariably bleed within the last few minutes of life can be visually reduced simply by choosing the colour of the towel that is used to surround the bleeding neck of the patient. Red or green towels soak up the blood, which is less striking and stark than a white towel (Gagnon et al. 1998; McGrath and Holewa 2006; Lenham 2013). This is the level of consideration I hope we can use when making a drawing for patients. This is a minimising tactic and could be used for overly anxious patients in other contexts. I have also used the opposite of this idea with patients who didn’t seem to grasp the severity of an illness and we’re not keen to attend the Accident and Emergency department of hospital with clear signs of stroke or heart attack. I have used a simple line drawing and

H. Quinton

then bold marker to indicate my concern of a bleed or a thrombus in combination with verbal warning. The tone of voice, the words used, and a drawing powerfully communicate why the attendance is medically required.

10.13 Inclusive Visual Communication in Dementia, Autistic Spectrum Disorder and Learning Disability A report published by the Alzheimer’s Society found that in 2013 there were approximately 815,000 people living with dementia in the UK If current trends continue, this number is expected to increase to 1,143,000 by 2025 (Alzeimers.org. uk 2018). GPs are the first point of contact for referral into memory clinic in most areas of the UK (RCGP 2010; Iliffe et al. 2009). Assessment of memory is done, using image heavy validated screening tests (Shulman 2000). Patients are requested to copy a drawing of interlocking pentagons (Fig. 10.14), the clock test mentioned earlier and writing a sentence. Guidance is heavily focused on allowing the patient as much autonomy as they are capable of and to be patient and allow time for word finding (NICE 2018).

Fig. 10.14 Intersecting pentagon (IP) drawing component of the Mini-Mental State Examination (MMSE). This part of the MMSE is a test of understanding, execution and co-ordination

10

• • • • • •

Visual Communication and Creative Processes Within the Primary Care Consultation

the 10-point cognitive screener (10-CS) the 6-item cognitive impairment test (6CIT) the 6-item screener the Memory Impairment Screen (MIS) the Mini-Cog Test Your Memory (TYM)

Cue cards are useful for people living with advanced dementia, aphasia, anomia (difficulty in finding words), and other related conditions affecting communication. They • Activities of daily living (bathing, dressing, eating, toileting) • Emotions and feelings (angry, frustrated, depressed, sad) • Events, visitors, outings (anticipation, expectation, leisure) • Memory and cognition (confusion, recognition, anxiety, worry) • Empathy (support, legitimisation, respect, partnership) Discussion of an innovative approach obtaining information for the Urology screening tool was mentioned earlier. Being aware of the aids that can help a patient should be considered to encourage autonomy and equitable treatment. In Autistic spectrum disorder, often patients have also been (or need to be) provided with cards containing and image, colour or single word to help provide structure and routine (National Autistic Society 2021). Familiarisation with these aids is vital to encourage independence and build confidence with the patient. The National Autistic society say card use can improve understanding; help to avoid frustration and anxiety and ease a person into interact with others using a familiar and comfortable communication style (National Autistic Society 2021). The Easy Read system is intended to help communication with learning difficulties Easy read is an accessible format of providing information designed for people with a learning disability. The easy read format is easy to understand because it uses simple, jargon free language, shorter sentences and supporting images (GOV.UK 2021).

239

The Mencap website advises we should consider what it is like to not being able to read, nor be able to tell someone else about a worry or concern. Not being able to find the words you wanted to say and words coming out jumbled up or incorrectly. They hope we will approach those not being able to understand, the words, phrases or expressions used by others (Mencap 2021). Simple words and image help to achieve the legal and ethical requirements in providing care for our patients (NHS 2021a, b, c, d).

10.14 Art Therapy and Use of Allegory Art therapy and the use of allegory is a form of psychotherapy that uses art media as its primary mode of expression and communication. Within this context, art is not used as a diagnostic tool but as a medium to address emotional issues which may be confusing and distressing. Art therapy is suitable for all ages and can help those with emotional, behavioural or mental health problems (Keenan et al. 2017). Of real note, patients are not expected to be able to draw or have any experience of expression this way. Art therapy often focuses on the colour and drawing quality with children in emotional disturbance. Children’s drawings are analysed for attaining certain characteristics as part of development screens. The initial scrubbing with a pencil, then circles leading on, as the child develops to a circle with stick arms and legs is fascinating (Farokhi and Hashemi 2011). Regression can be seen in children’s drawings when there is turbulence within the family or the child’s experience (Lingren 1971). I have not always had the option of referring my patients to an art therapist as it has not always been a service that has been commissioned in the local area by the funding body, but I have lost count of the times patients and I talk about coping with chronic pain, grief, difficult feelings or loneliness by discussing art or craft. Part of the NHS future plans (Long Term Plan 2020) is the role of

240

social prescribing, including community based support and promoting exercise and participation of holistic activities. Promoting film and books and can be helpful if patients really don’t know where to start. There’s a film entitled ‘Amelie’ directed by Jean-Pierre Jeunet (2001), the protagonist Amelie (played by Audrey Tatou) sets out to help change the lives of those around her, encouraging others to expand their vision of possibility. In one scene she meets a visually impaired man in the street and offers her arm to guide him to his destination—her ulterior motive to give him the present of a rich visual life she enjoys but he lacks. She sees it as an act of goodness, of benevolence and during his guided walk she verbally relays in detail, the rich street life playing out around them. The pub figure head of a Horse has lost an ear and the florist has ‘crinkly eyes’ she tells him. Ham is 79 Francs, Spare-Ribs 45 Francs and at the Butcher’s shop a baby is watching a Dog drooling over roasting Chickens. This scene of this film is helpful in describing one of the ways we can help our patients. If we are looking at the bio-psycho-social aspect of a person, then enrichment of the visual sense is of huge value. Encouraging a new hobby is fine as a broad concept but I can recall tens of patients who have asked me for help with this. “I’m no good at anything”, “I don’t like sport”, “I don’t know where to start”. To stimulate curiosity, I inquire about areas of a person’s interests and hobbies during consultations, and sometimes point them towards a film or a book. On other occasions I have constructed a list of seemingly random but carefully selected topics, given as a list of starting points for research. I call these ‘Creative Lists’. These topics are often unusual or quirky references or places intended to spark interest and to encourage inquisitiveness in those who expressed feeling lost or depressed or wanting to improve knowledge and skill. They may include a recipe for making flat bread (BBC Goodfood 2021), or direct to beautiful furniture design like that of Robert Thompson (also known as ‘Mouse Man’. He is a sculptor and furniture maker in North Yorkshire, known for the carving of mice

H. Quinton

on furniture in overt or hidden detailing (Thompson 2021). Some patients have brought pictures of their own projects after conversations of social prescribing. To date, I have seen sculptures, redecorated dolls houses and reported engagement in fitness and mindfulness. In Primary Care sometimes there is not a medical need, but a focus more on the psychosocial, and it is here that enrichment of a visual and cultural life can be therapeutic, to make life more tolerable or exciting. It is truly patient-centred and can start with a broad recommendation from the clinician and should be open enough to allow the patient to take ownership of it and yet, different enough to challenge and excite. “I don’t know where to start, I’m lost, I need to move on, I want to get the old me back, but I don’t know how’, I have to live with this pain but I can’t think of anything else”. These are the cues which are communicating a need to help with guidance. In follow up, if it’s truly helped or valued by the patient, you won’t need to mention it. It will spontaneously be bought back into the consultation. There may be a request for more, or that may well have been enough as an input. The direction of people to cultural activity is only an enrichment of life and fits into any Cognitive Behavioural Therapy (CBT) or counselling and well-being programme. There is the opportunity for anaphorous health messaging within context which makes it relatable and individualised. I’ve often reflected on the origins of these ideas and techniques and see clear links with the teaching approach at The Glasgow School Of Art. The body of work I would go on to produce from a book, poem or a creative brief for an illustration. As a child I was lucky to be read stories and partake in treasures hunts; following slips of paper (encouraging me to read and have fun) to lead me around the house to find a new book as the prize. The painting, drawing or photograph on the wall of the clinical office has often triggered a discussion and helped to create rapport in the form of small talk. The selection of the image,

10

Visual Communication and Creative Processes Within the Primary Care Consultation

painting, poster or leaflet could then also be considered as a waiting room or consulting room exhibition requiring curation. This space has great potential as does the waiting room (Gignon et al. 2012; Ward and Hawthorne 1994).

10.15 Summary Images are integral to verbal communication according to the semiologists discussed in this chapter. Drawing or creating can help to consolidate, convey a whole message, or just a fundamental part of it between a doctor and the patient or amongst colleagues. Research in chronic disease shows new techniques are needed to improve health outcomes. The image can be stopped mid-draw if the function was to convey a small bit of information, a finalised image or cartoon and has the potential to have an ethical value of its own according to confidentiality principles. A wide range of medical educators use imagery as a core method for teaching medical students; with new course development. Some clinicians are researching the value of the same pedagogic principles within the clinical setting. Literature using creative metaphorical language already exists to creatively discuss pain and difficult concepts with patients. Within the GP curricula the use of image is encouraged within current consultation models and drawing is just visual-communication, so it can be honed. There is not a requirement of having a formal knowledge of drawing, and the clinician should not be concerned about the perceived lack of skill or ability to create a masterpiece. The hope is to encourage and use continuing personal development time to look at existing resources, gain ideas and reflect on areas of clinically difficulty. This may include describing biological principles, designing ways to improve medication concordance or to promote a rich and varied healthy lifestyle management. Once identified, looking for resources and innovative ways of visually conveying information to the patient to improve understanding, rapport and health outcomes, is the intended message of this chapter.

241

References Ainsworth S, Prain V, Tytler R (2011) Drawing to learn in science. Science 333:1096–1097. https://doi.org/10. 1126/science.1204153 Alderwick H, Dixon J (2019) The NHS long term plan. BMJ 364:l84. https://doi.org/10.1136/bmj.l84 Alzeimers.org.uk (2018) Accessible via https://www. alzheimers.org.uk/. Accessed 29 June 2021 Ausubel, D Novak JD, Hanesian H (1978) Educational psychology: a cognitive view. Holt, Rinehart And Winston, New York Backhouse M, Fitzpatrick M, Hutchinson J et al (2016) Improvements in anatomy knowledge when utilizing a novel cyclical “observe-reflect-draw-edit-repeat” learning process. Anat Sci Educ 10:7–22 Balint M (1955) The doctor, his patient, and the illness. Lancet 265:683–688. https://doi.org/10.1016/s01406736(55)91061-8 Barthes R (1977) Image, music, text: essays. Fontana, London BBC GoodFood (2021) Recipe for toasted cumin flatbreads. Accessible via https://www.bbcgoodfood. com/recipes/toasted-cumin-flatbreads. Accessed 29 June 2021 Berkman ND, Sheridan SL, Donahue KE et al (2011) Low health literacy and health outcomes: an updated systematic review. Ann Intern Med 155:97. https://doi. org/10.7326/0003-4819-155-2-201107190-00005 British Lung Foundation (2020) Smart peak flow: asthma tracker & meter app. In: Available via British Lung Foundation. https://www.blf.org.uk/technology-forlung-health/smart-peak-flow-asthma-diary-0. Accessed 30 June 2021 Broadbent E, Schoones JW, Tiemensma J, Kaptein AA (2018) A systematic review of patients’ drawing of illness: implications for research using the common sense model. Health Psychol Rev 13:406–426. https://doi.org/10.1080/17437199.2018.1558088 Brown Alpert Medical School (2021) BIOL 6527 – The Physician as Medical Illustrator. Med.brown.edu. http://med.brown.edu/pedisurg/IllustrationClass/ MedicalIllustrationDescription.htm. Accessed 28 June 2021 Butler DS, Moseley GL (2018) Explain pain. Noigroup, Adelaide Casarett D, Pickard A, Fishman JM et al (2010) Can metaphors and analogies improve communication with seriously ill patients? J Palliative Med 13:255–260. https://doi.org/10.1089/jpm.2009.0221 Chamberlain R, McManus IC, Brunswick N et al (2014) Drawing on the right side of the brain: a voxel-based morphometry analysis of observational drawing. NeuroImage 96:167–173. https://doi.org/10.1016/j. neuroimage.2014.03.062 Cheung M, Saini B, Smith L (2016) Using drawings to explore patient’s perceptions of their illness: a scoping review. J Multidiscip Healthc 9:631–646

242 Cline CMJ, Björck-Linné AK, Israelsson BYA et al (1999) Non-compliance and knowledge of prescribed medication in elderly patients with heart failure. Eur J Heart Fail 1:145–149. https://doi.org/10.1016/s1388-9842 (99)00014-8 Coleman D (2014) Skills for communicating with patients. Silverman Jonathan et al Radcliffe Publishing. 26:10. https://doi.org/10.7748/nop2014.04.26.4.10.s12 Cull P (1988) Aspects of confidentiality, copyright, and accreditation. J Audiovisual Media Med 11:8–10. https://doi.org/10.3109/17453058809063647 Cull PG, Gilson CC (1986) Confidentiality of illustrative clinical records-a code of practice, guidance notes and recommendations. J Audiov Media Med 9:124–130. https://doi.org/10.3109/17453058609156050 Czerski H (2020) The Royal Institute: Christmas Lectures 2020. Available via https://www.rigb.org/christmaslectures/2020-planet-earth-a-users-guide/meet-helenczerski. Accessed 29 June 2021 Daly D, Douglas A (2020) What is hypertension. Available via https://www.gla.ac.uk/schools/lifesciences/ news/headline_744311_en.html. Accessed 29 June 2021 De Botoli M, Maroto J (2001) Colors across cultures: translating colors in interactive marketing communications. University of Paisley DG (2017) Unilateral otitis media with effusion in adults: a challenging disease. Otolaryngol Open Access J 2. https://doi.org/10.23880/ooaj-16000146 Dittmar M (2001) Changing colour preferences with ageing: a comparative study on younger and older native Germans aged 19–90 years. Gerontology 47:219–226. https://doi.org/10.1159/000052802 Dorsey ER, Yvonne, Chan Y-F, McConnell MV et al (2017) The use of smartphones for health research. Acad Med 92:157–160. https://doi.org/10.1097/ACM. 0000000000001205 Duffy FD, Gordon GH, Whelan G et al (2004) Assessing competence in communication and interpersonal skills: the Kalamazoo II report. Acad Med 79:495–507. https://doi.org/10.1097/00001888-200406000-00002 Elwyn G, Laitner S, Coulter A et al (2010) Implementing shared decision making in the NHS. BMJ 341:c5146. https://doi.org/10.1136/bmj.c5146 Epilepsy Foundation (2021) Understanding epilepsy and seizures: a management tool. Accessible via https:// epilepsyfoundation.org.au/understanding-epilepsy/epi lepsy-and-seizure-management-tools/recordingseizures-and-seizure-diaries/. Accessed 29 June 2021 Farokhi M, Hashemi M (2011) The analysis of children’s drawings: social, emotional, physical, and psychological aspects. Procedia Soc Behav Sci 30:2219–2224 Faulkner B (2015) Five steps to a successful consultation. In Pract 37:146–149. https://doi.org/10.1136/inp.h989 Field K, Ziebland S, McPherson A, Lehman R (2006) “Can I come off the tablets now?” A qualitative analysis of heart failure patients’ understanding of their medication. Fam Pract 23:624–630. https://doi.org/10. 1093/fampra/cml036

H. Quinton Finn G (2018) Current perspectives on the role of body painting in medical education. Adv Med Educ Pract 9:701–706. https://doi.org/10.2147/amep.s142212 Finn GM, White PM, Abdelbagi I (2011) The impact of color and role on retention of knowledge: a bodypainting study within undergraduate medicine. Anat Sci Educ 4:311–317. https://doi.org/10.1002/ase.253 Fox S, Duggan M (2012) Mobile health 2: half of smartphone owners use their devices to get health information and one-fifth of smartphone owners have health apps. Pew Research Center Web site. November 8, 2012. Available via http://emr-matrix.org/wpcontent/uploads/2012/12/PIP_MobileHealth2012.pdf. Accessed 14 Mar 2016 Frank AW (1997) The wounded storyteller: body, illness, and ethics. University of Chicago Press, Chicago Gagnon B, Mancini I, Pereira J, Bruera E (1998) Palliative management of bleeding events in advanced cancer patients. J Palliat Care 14:50–54. https://doi.org/10. 1177/08258597980140040 Gerritsen F (1975) Theory and practice of color. A color theory based on laws of perception: color. Studio Vista, London Gibson MC, MacLean J, Borrie M et al (2004) Orientation behaviors in residents relocated to a redesigned dementia care unit. Am J Alzheimers Dis Other Dement 19:45–49. https://doi.org/10.1177/ 153331750401900110 Gignon M, Idris H, Manaouil C et al (2012) The waiting room: vector for health education? The general practitioner’s point of view. BMC Res Notes 5. https://doi.org/10.1186/1756-0500-5-511 GOV.UK (2018) The data protection act 2018. Accessible via https://www.gov.uk/data-protection. Accessed 29 June 2021 Gov.UK (2020) The Caldicott principles. Available via http://www.gov.uk/government/publications/thecaldicott-principles. Accessed 30 June 2021 GOV.UK (2021) Accessible communication formats. [online] GOV.UK. Available at: https://www.gov.uk/ government/publications/inclusive-communication/ accessible-communication-formats. Accessed 29 June 2021 Häkkinen JT, Hakama M, Huhtala H et al (2007) Impact of LUTS using bother index in DAN-PSS-1 questionnaire. Eur Urol 51:473–478 Hart FT (1937) Scaffolding in education. Music Educators J 23:22. https://doi.org/10.2307/3384718 Haustein K (2014) How to read Barthes’ image – music – text. Fr Stud 68:126–126. https://doi.org/10.1093/fs/ knt289 Iliffe S, Robinson L, Brayne C et al (2009) Primary care and dementia: 1. Diagnosis, screening and disclosure. Int J Geriatr Psychiatry 24:895–901. https://doi.org/10. 1002/gps.2204 Jackson C (2020) The Royal Institute, Christmas lectures. Available via https://www.rigb.org/christmas-lectures/ 2020-planet-earth-a-users-guide/meet-chris-jackson. Accessed 29 June 2021

10

Visual Communication and Creative Processes Within the Primary Care Consultation

Jiwa M, Mathers N, Campbell M (2002) The effect of GP telephone triage on numbers seeking same-day appointments. Br J Gen Pract 358:390-1 Jones G (2003) Prescribing and taking medicines. BMJ 327:819–819. https://doi.org/10.1136/bmj.327.7419. 819 Kaptein A, Tiemensma J, Broadbent E, Asijee G, Voorhaar M (2017) COPD depicted & dash patients drawing their lungs. Int J Chronic Obstructive Pulmonary Dis 12:3231–3236 Kearns C (2019) Is drawing a valuable skill in surgical practice? 100 surgeons weigh in. J Vis Commun Med 42(1):4–14 Kearns C, Kearns N, Paisley A (2019) The art of consent: visual materials help adult patients make informed choices about surgical care. J Vis Commun Med 43 (2):76–83 Keenan I, Hutchinson J, Bell K (2017) Twelve tips for implementing artistic learning approaches in anatomy education. MedEdPublish 6(2) Kress G (1980) The development of the concept of “sentence” in children’s writing. Austral Rev Appl Ling 3:63–75. https://doi.org/10.1075/aral.3.2.06kre Kurtz S, Silverman J, Benson J, Draper J (2003) Marrying content and process in clinical method teaching. Acad Med 78(8):802–809 Lam CL (2004) The new consultation: developing doctorpatient communication. In: Pendleton D, Schofield T, Tate P, Havelock P (eds) Oxford University Press, 143 p, 19.95, 2003. ISBN 0-19-263288. Fam Pract, 21(1), pp 117 Lenham J (2013) Colour, contrast and comfort: interior design in dementia. Nurs Residential Care 15 (9):616–618 Lingren RH (1971) An attempted replication of emotional indicators in human drawings by shy and aggressive children. Psychol Rep 29(1):35–38 Linsky A, Simon SR (2012) Patients’ perceptions of their “most” and “least” important medications: a retrospective cohort study. BMC Res Notes 5. https://doi.org/ 10.1186/1756-0500-5-619 Lyon J (1995) Drawing: its value as a communication aid for adults with aphasia. Aphasiology 9(1):33–50 Lyon P, Turland M (2016) Manual drawing in clinical communication: understanding the role of clinical mark-making. In: Visual methodologies, 5, 1, pp 39–44, 6 p Lyon PM, Turland M (2019) Visualising the body: health professionals’ perceptions of their clinical drawing practices. Med Humanit 46:454–463. https://doi.org/ 10.1136/medhum-2019-011675 McGrath P, Holewa H (2006) Missed opportunities: nursing insights on end-of-life care for haematology patients. Int J Nurs Pract 12(5):295–301. https://doi. org/10.1111/j.1440-172X.2006.00585.x McKinstry B, Campbell J, Salisbury C (2017) Telephone first consultations in primary care. BMJ 358:j4345 https://doi.org/10.1136/bmj.j4345

243

Meglioli M, Naveau A, Macaluso GM et al (2020) 3D printed bone models in oral and cranio-maxillofacial surgery: a systematic review. 3D Print Med 6(1):30. https://doi.org/10.1186/s41205-020-00082-5. Erratum in: 3D Print Med. 2020 Dec 2;6(1):36. PMID: 33079298; PMCID: PMC7574578 Mencap (2021) Learning disability – autism – down’s syndrome. [online] Mencap. Available at: https:// www.mencap.org.uk. Accessed 28 June 2021 Meskó B (2019) The real era of the art of medicine begins with artificial intelligence. J Med Internet Res 21(11): e16295 Morton J (2004) Global color clues and taboos. Colorcom ISBN: 0970912552 National Autistic Society (2021) Visual supports. [online] Autism.org.uk. Available at: https://www.autism.org. uk/advice-and-guidance/topics/communication/com munication-tools/visual-supports. Accessed 29 June 2021 NHS (2021a) Health A to Z. [online] nhs.uk. Available at: https://www.nhs.uk/conditions/. Accessed 28 June 2021 NHS (2021b) NHS inform. Available via https://www. nhsinform.scot/symptoms-and-self-help/a-to-z. Accessed 29 June 2021 NHS (2021c) Integrating care: next steps to building strong and effective integrated care systems across England. [ebook], pp 1–39. Available at: https:// www.england.nhs.uk/wp-content/uploads/2021/01/ integrating-care-next-steps-to-building-strong-andeffective-integrated-care-systems.pdf. Accessed 29 June 2021 NHS (2021d) Long term plan: digitising healthcare. Available at: https://www.england.nhs.uk/wp-content/ uploads/2020/01/online-consultations-implementa tion-toolkit-v1.1-updated.pdf. Accessed 29 June 2021 NHS England (2019) Using online consultations in primary care toolkit, 1st edn. Accessible at https://www. england.nhs.uk/wp-content/uploads/2020/01/onlineconsultations-implementation-toolkit-v1.1-updated. pdf. Accessed 29 June 2021 NHS West (2021) NHS England and NHS improvement south West “Opioid prescribing for chronic pain”. [online] England.nhs.uk. Available at: https://www. england.nhs.uk/south/info-professional/safe-use-ofcontrolled-drugs/opioids/. Accessed 29 June 2021 NICE (2018) Dementia: assessment, management and support for people living with dementia and their carers. [ebook] NICE, pp 1–43. Available at: https:// www.nice.org.uk/guidance/ng97/resources/ dementia-assessment-management-and-support-forpeople-living-with-dementia-and-their-carerspdf-1837760199109. Accessed 29 June 2021 Overkamp F (2019) Apps and more; potentials of digitalization in breast medicine. Breast Care (Basel) 14 (3):136–139. https://doi.org/10.1159/000501020 Rak K, Völker J, Taeger J et al (2019) Medizinische Apps in der HNO-Heilkunde. Laryngo-Rhino-Otologie 98: S253–S289. https://doi.org/10.1055/a-0740-4866

244 RCGP (2021) General practice COVID-19 recovery: the future role of remote consultations & patient ‘triage’. [online] Rcgp.org.uk. Available at: https://www.rcgp. org.uk/policy/general-practice-covid-19-recoveryconsultations-patient-triage.aspx. Accessed 29 June 2021 RCGP Clinical Innovation and Research Centre (2010) Dementia: diagnosis and early intervention in primary care. RCGP, London. Available via http://www. wamhinpc.org.uk/sites/default/files/dementia-rcgpdiagnosis-and-early-intervention.pdf. Accessed 29 June 2021 Rea P (2021) University of Glasgow – Schools – School of Life Sciences – Our Staff – Professor Paul M Rea. [online] Gla.ac.uk. Available at: https://www.gla.ac. uk/schools/lifesciences/staff/paulrea/#biography, researchinterests,publications,articles. Accessed 29 June 2021 Reading H (2015) Brighter plates make for better mealtimes. Nursing Older People 27(8):7. https://doi. org/10.7748/nop.27.8.7.s6 Red Box Project (2021). Available at http://redboxproject. org/. Accessed 29 June 2021 Roberts A (2021). https://www.birmingham.ac.uk/staff/ profiles/biosciences/roberts-alice.aspx. Accessed 29 June 2021 Rogers AE (2000) Knowledge and communication difficulties for patients with chronic heart failure: qualitative study. BMJ 321:605–607. https://doi.org/10. 1136/bmj.321.7261.605 Røstvik CM (2019) Blood works: Judy Chicago and menstrual art since 1970. Oxford Art J 42:335–353. https:// doi.org/10.1093/oxartj/kcz021 Sahai A, Dowson C, Cortes E et al (2014) Validation of the bladder control self-assessment questionnaire (B-SAQ) in men. BJU Int 113:783–788. https://doi.org/10.1111/ bju.12521 Scott G, Presswood EJ, Makubate B et al (2013) Lung sounds: how doctors draw crackles and wheeze. Postgrad Med J 89:693–697 Shulman KI (2000) Clock-drawing: is it the ideal cognitive screening test? Int J Geriatric Psychiat 15:548–561. https://doi.org/10.1002/1099-1166(200006) 15:6 ‘Project Window’ > ‘New’, rename the project and choose the desired save location on your computer. Then click ‘Select’. Once the model is imported into an Autodesk Maya 3D scene, a studio setup similar to a photography studio with a lighting rig will need to be created (see Fig. 11.20). At a minimum, a ‘floor’ and ‘walls’ should be added so the lights can be contained within the area where your 3D model is placed. From the menu at the top of the Maya interface choose ‘Create’, ‘Polygon Primitives’ and ‘Plane’. Once the plane is inserted into the 3D scene, use the ‘Transform’, ‘Move’ and ‘Rotate’ tools from the left-hand toolbar to position it into the desired location. Do this as many times as required to create a ‘photography studio’. Reposition and resize the 3D head model to fit the studio space. Default ‘Perspective’ and

‘Orthographic’ virtual cameras are added when a new scene is created, and the user can move between them by selecting them from the ‘Panels’ tab at the top of the viewport. Without lights in the scene, it will not be possible to visualise the 3D models when images are rendered. A standard 3-point light set up will illuminate the 3D models evenly and replicate real-world environments. To add lights, navigate to ‘Create’ in the menu at the top of the Autodesk Maya interface and choose ‘Lights’ then ‘Directional Light’. Once a light is inserted into the 3D scene, use the ‘Transform’, ‘Move’ and ‘Rotate’ tools from the left-hand toolbar to position it into the desired location. Do this as many times as required to create a 3-Point light setup. The exposure, diffusion and temperature of the virtual lights can be adjusted accordingly in the ‘Attribute Editor’ panel, found to the right of the Autodesk Maya interface. A warm soft pink light and a cooler soft blue light are recommended to be placed at either side of the 3D head, with a

260

M. Roughley and C. Y. J. Liu

Fig. 11.17 A 3D model and its generated texture map, visible in Pixologic ZBrush

Fig. 11.18 A 3D model and its generated displacement map, visible in Pixologic ZBrush

11

Digital 2D, 2.5D and 3D Methods for Adding Photo-Realistic Textures to 3D. . .

Fig. 11.19 3D model imported into Autodesk Maya, and the ‘Outliner’ window

Fig. 11.20 A studio setup and 3-point lighting system in Autodesk Maya

261

262

M. Roughley and C. Y. J. Liu

Fig. 11.21 Applying an ‘aiStandardSurface’ shader to a 3D model in Autodesk Maya

brighter white light pointing towards the face for optimum illumination. Experimentation with positioning of the lights and their associated settings is recommended until a desirable setup is reached. The default material assigned to a 3D model when imported into Autodesk Maya is ‘Lambert’, which is “a material (shader) that represents matte surfaces, such as chalk, matte paint, and unpolished surfaces with no specular highlights” (Autodesk 2016). A different shader would need to be assigned for the surface of the 3D model to resemble human skin. In this example, with the 3D model eventually being rendered using the Arnold rendering engine, an ‘aiStandardSurface’ shader is used. This shader is capable of producing many types of materials (Autodesk Arnold n. d.-a) and a surface that resembles skin can be generated. With the 3D model selected, press and hold the right mouse button for the menu in Fig. 11.21 (left) to appear. Navigate to ‘Assign New Material’ and choose ‘aiStandardSurface’ shader. In

the ‘Attribute Editor’, accessed to the right of the Autodesk Maya interface, the material shader options will appear. Click the ‘Presets*’ button and choose ‘Skin’ and ‘Replace’. This shader is now applied to the 3D model and can be further customised by altering material settings and adding texture and displacement maps, either within the ‘Attribute Editor’ or via the ‘Hypershade’ window (Autodesk 2014a). With the 3D model selected, in the ‘Attribute Editor’, add the ‘diffuse’ colour texture map that was exported from Pixologic ZBrush to the ‘Subsurface Colour’ node. Click on the checkerboard icon, then choose ‘File’ in the ‘2D Textures’ menu. Click the folder icon next to ‘Image Name’ then navigate to the colour texture map on your computer and click ‘Open’. The colour texture map is now applied to the 3D model, as demonstrated in Fig. 11.22. If the texture map does not align correctly when applied and viewed in the Autodesk Maya viewport, the map may need to be transformed. Depending on the version of Pixologic ZBrush used to digitally paint the 3D

11

Digital 2D, 2.5D and 3D Methods for Adding Photo-Realistic Textures to 3D. . .

263

Fig. 11.22 Applying texture maps and adjusting shader settings in Autodesk Maya

model, the exported colour texture map might first need to be flipped horizontally to be in the correct orientation for use in Autodesk Maya. For additional customisation of the skin’s material properties, the ‘Subsurface’, ‘Specular’ and ‘Sheen’ properties can be adjusted, however, the settings added earlier by the ‘Skin’ preset should suffice. If the 3D model was resized, the ‘Radius’ in the ‘Subsurface’ properties may need to be reduced to avoid the skin layer appearing too washed out. A grayscale specular map can be added to the ‘Specular’ > ‘Colour’ node for more advanced control of where highlights appear on the skin surface. The connections between the material nodes and shaders can be visualised in the ‘Hypershade’ window. The ‘Hypershade’ window is accessed from the ‘Windows’ > ‘Rendering Editors’ menu from the top of the Autodesk Maya interface. With the shader selected in the ‘Hypershade’, under ‘Shading Group Options’ click on the icon to the right of ‘Displacement mat.’ and the additional nodes will be added to the node tree in

the viewport (visible at the bottom of Fig. 11.23). Click on the ‘image’ node and add the displacement map in the same way that the colour texture map was added to the ‘Subsurface Colour’ node. In this node, select ‘Alpha is Luminance’ under ‘Colour Balance’. To adjust the displacement settings select the 3D model in the ‘Outliner’ and under the ‘Shape’ tab in the ‘Attribute Editor’. Scroll down to ‘Subdivision’, select ‘catclark’ under the ‘Type’ with a recommended four iterations. This figure controls the subdivision of a surface and a higher number increases the displacement quality. However, a high value will also increase render time (Autodesk Arnold n.d.-b). Under ‘Displacement Attributes’ the ‘Height’ of the displacement map can be altered until the desired final appearance of skin layer is reached (Fig. 11.24). Multiple test renders might be required. Adjustments to the virtual lights in the scene may be required if the model is too exposed or the scene is too dark. To add realistic human eyes to the scene, eyes can be 3D modelled in Autodesk Maya or

264

M. Roughley and C. Y. J. Liu

Fig. 11.23 A node tree for a skin shader, highlighting the workflow for adding a displacement map to the 3D model in Autodesk Maya

downloaded and modified under Creative Commons licenses to suit your project needs. These are available from online repositories such as TurboSquid and Sketchfab. For this workflow, 3D eye models freely available from the Autodesk website have been used (https:// www.autodesk.in/campaigns/arnold/asset-3deye). Figure 11.25 shows the setup of these eye models in the ‘Hypershade’ window after importing the models, resizing them to fit the 3D head, and connecting texture and displacement maps. The 3D eye models exist in two parts; the ‘eyeball’ and the ‘cornea’, with preloaded material settings for each. The colour texture map for the eyeball can be altered in Adobe Photoshop CC if the pre-existing iris colour is incorrect for the facial depiction. At this stage, additional objects such as garments and jewellery can be imported or 3D modelled, and digital textures added to them following the aforementioned workflow. To create hairs, the XGen tool within Autodesk Maya

should be used. XGen is a geometry instancer, able to create and style hair, fur, feathers, grass and forests (Autodesk 2014b). Select the 3D skin model in the ‘Outliner’ that hairs should populate from. To create hair in specific areas on the 3D model select the faces of the 3D model in the desired locations. Once a specific area of the 3D model is selected, click on ‘Create a new XGen description’ under the ‘XGen’ menu at the top of the Autodesk Maya interface. In the popup window, name the new description and click ‘Create a Collection’. To generate hair, select ‘Splines’ as a primitive, and to be generate ‘Randomly across the surface’. The controls should be ‘Placing and shaping guides’ (see Fig. 11.26). Once ‘Create’ is selected, the new collection will appear in the ‘Outliner’ as a folder in green. Hair is generated by placing guides on the surface of the selected area. Hair will ‘grow’ according to the direction of the guides. Under the XGen tab, select the ‘Add or Move guides’ (the icon with a ‘+’), click on the surface of the model and adjust

11

Digital 2D, 2.5D and 3D Methods for Adding Photo-Realistic Textures to 3D. . .

265

Fig. 11.24 Altering displacement map settings in the ‘Attribute Editor’ in Autodesk Maya

Fig. 11.25 The node tree and shader settings for 3D eye models available from the Autodesk website. Viewed in the ‘Hypershade’ window in Autodesk Maya

266

M. Roughley and C. Y. J. Liu

Fig. 11.26 Creating XGen Descriptions for hair modelling in Autodesk Maya

the guide using the ‘Transform’ and ‘Rotate’ tools from the left-hand toolbar. The guide can be moved by holding ‘Ctrl’ on the keyboard when ‘Add or Move guides’ is selected. The ‘Move’ tool from the left-hand toolbar is ineffective to adjust the position of the guide. The shape of the guide can be manipulated using ‘Sculpt guides’. Create a new description for different types of hair e.g. one for eyebrows, one for a beard, one for head hair etc. (see Fig. 11.27). The hairs can be visualised in the viewport by using the ‘Update the Preview’ (icon with an eye highlighted in Fig. 11.28). The ‘Generator Attributes’ tab in the XGen window (Fig. 11.27) can adjust the density and randomness of the hairs across the surface of the 3D model. The shape, length and width of the generated hairs can be adjusted under ‘Primitive Attributes’ (Fig. 11.27). To create realistic hair, ‘Modifiers’ such as ‘Noise’ and ‘Clumping’ can be added and adjusted, as shown in Fig. 11.28. Expressions can also be used to create more randomness in the structure, as explained in detail by JesusFC (https://jesusfc.net/tutorials/introduction-toxgen).

A new hair shader can be assigned to the hair systems in the ‘Hypershade’. The Arnold ‘aiStandardHair’ shader allows users to adjust the level of melanin, redness and randomness of the hair colours (Fig. 11.29). This is an intuitive system in adjusting the colour of the hair. Throughout this process, test renders of the 3D scene can be performed at lower resolutions, which reduces render time. Select ‘Render’ from the top menu bar in the Autodesk Maya interface, then ‘Test Resolution’ and choose one of the reduced percentage settings. Then navigate back up to ‘Arnold’ in the menu bar and click ‘Render’ to render the scene in a new ‘Arnold RenderView’ window (Fig. 11.30). If the render doesn’t automatically start, click the ‘Play’ button. When satisfied with the appearance of the 3D model, hairs and shader settings, a final image of the facial depiction can be rendered at 100% resolution. To choose an image format, change the render settings in ‘Render’ > ‘Render Settings’ from the top menu bar in the Autodesk Maya interface. Specific cameras, image formats, images sizes and image resolutions can be chosen

11

Digital 2D, 2.5D and 3D Methods for Adding Photo-Realistic Textures to 3D. . .

267

Fig. 11.27 Adding and editing XGen hair guides in Autodesk Maya, using different hair description for different hair types

here. For more advanced image rendering, the CPU and GPU settings can be changed in the advanced options. Please refer to this guide provided by Arnold for more information (https://docs.arnoldrenderer.com/display/ A5ARP/Getting+Started+With+Arnold+GPU). Select ‘Render’ from the top menu bar in the Autodesk Maya interface and select ‘Render Current Frame’. Any images rendered will be saved to the ‘Images’ folder within your Autodesk Maya project on your computer. The 3D model is often positioned to face the camera in the Frankfurt Horizontal Plane prior to rendering. If you were animating the 3D model, for example, getting it to move on a turntable, the models can be translated and rotated then keyframed in the scene by using the tools in the ‘Animation’ menu and the timeline at the bottom of the Autodesk Maya interface. Alternatively, the camera can be animated to move around the 3D model. In ‘Render Settings’ choose start and end frame numbers for the sequence length, then

choose ‘Render Image Sequence’ in the ‘Render’ menu to render all layers. For detailed animation and sequence rendering steps consult guides on the Autodesk Knowledge Network (https:// knowledge.autodesk.com/support/maya/learnexplore.html).

11.2.3

2.5D Digital Composite Method

3D modelling of eyes, hair and clothing, plus skin material and virtual lighting choices that require advanced CGI skills can be time consuming to generate in VFX software. Like the 3D digital painting method, the hybrid 2.5D method adds skin textures directly on the skin layer of the 3D facial reconstruction model. However, more complex textures such as hair and eyes, are composited on a 2D image of the painted 3D model. Figure 11.31 visually demonstrates the 2.5D workflow. The image shows a digital facial

268

M. Roughley and C. Y. J. Liu

Fig. 11.28 Visualising the XGen hair in the viewport with ‘Update the XGen preview’. The addition of modifiers to generate realistic hair using Arnold in Autodesk Maya

depiction of a 1200-year-old bog body from Bernuthsfeld, East Frisia (Wilkinson and Roughley 2019) for which a front-facing 2D image of the face was requested. The 3D facial reconstruction was created in Geomagic Freeform and the skin layer of the 3D model painted in Pixologic ZBrush using the aforementioned 3D digital painting techniques. Eyes, hair and clothing were added to a 2D, front-facing image of the facial depiction in Adobe Photoshop CC using the techniques described in the 2D digital composite method. When first approached by the archaeological team, the choice of eye colour, hair colour and style, and clothing materials were yet to be confirmed as scientific analyses of the human remains, which included preserved hair and fabrics were incomplete. The facial reconstruction and skin painting was undertaken and the depiction process was then paused until information regarding eyes, hair and clothing were made available from the scientific researchers. In

addition to this flexible approach to working and an increased speed in texturing the face model, compositing these features in 2D provided opportunities to produce multiple iterations of the depiction for consideration. For example, generating different combinations of clothing material and hairstyle, until a scientific and artistic consensus was reached.

11.2.3.1 Workflow Like the digital 3D painting method, the skin layer of the 3D facial reconstruction is exported from the software that it was created in and imported into Pixologic ZBrush. The model will usually require a simple retopology, and the ‘ZRemesher’ tool can be used to create a mesh with organised topology ready for painting. If the mesh appears to have too few polygons, more can be added by using the ‘Divide’ button in the ‘Geometry’ palette. This increases the number of subdivisions that the 3D model has. It is recommended to also generate a UV map at this

11

Digital 2D, 2.5D and 3D Methods for Adding Photo-Realistic Textures to 3D. . .

269

Fig. 11.29 Arnold ‘aiStandardHair’ shader with options to adjust the level of melanin, redness and randomness, to generate realistic hair using Arnold in Autodesk Maya

stage in case the model is to be used in VFX software in the future. The previously described method for UV mapping can be implemented, however, it may be unnecessary for this type of project, therefore simply pressing the ‘UV Unwrap’ button in ‘UV Master’ will be sufficient. The 3D model will need to be at the lowest subdivision in the ‘Geometry’ palette for UV unwrapping to take place. This can be increased to the highest subdivision level again prior to start the next stage. If desired, skin textures such as creases and wrinkles can be sculpted as demonstrated in Fig. 11.14, however, the 2.5D method can continue without the need for sculpted skin textures. Digital painting of the skin layer commences by using brushes and alphas or by using

photographic donor images and the ‘Spotlight tool’. The workflow for digital painting (shown in Figs. 11.15 and 11.16) can be followed until the model is ready for export. Pixologic ZBrush also possesses a number of other tools that can be used prior to exporting the front-facing image, to enhance the photo-realism of the generated face image. These include adjustable material settings, built in virtual lights, and an integrated rendering engine. We have recommended the use of the ‘SkinShade4’ material when painting in Pixologic ZBrush due to its specularity adjustment options, and this material’s properties (and any other material’s properties) can be altered in the ‘Material’ dropdown menu at the top of the Pixologic ZBrush interface. The diffuse colour, transparency or

270

M. Roughley and C. Y. J. Liu

Fig. 11.30 ‘Arnold RenderView’ in Autodesk Maya under ‘Arnold’. Test resolution can be adjusted from the ‘Render’ menu

specularity settings could be changed to increase realism of the painted skin surface. The built-in lighting system allows the user to add, move and adjust the intensity and colour of

the virtual lights in the 3D scene. The settings of these lights can be altered in the ‘Light’ dropdown menu at the top of the Pixologic ZBrush interface until a desired appearance is reached.

Fig. 11.31 Example of 2.5 texturing method: 3D digital facial depiction of a 1200-year-old bog body known as “Bernie” created in Geomagic Freeform; exported to Pixologic ZBrush for digital painting; exported to Adobe

Photoshop for composite texturing of eyes, hair and clothing. Image courtesy of Face Lab, Liverpool John Moores University

11

Digital 2D, 2.5D and 3D Methods for Adding Photo-Realistic Textures to 3D. . .

271

Fig. 11.32 Exporting an image of the current scene from ZBrush as an image document

The scene can be rendered using the inbuilt BPR renderer. This is found under the ‘Render’ dropdown menu at the top of the Pixologic ZBrush interface and it has options to adjust appearance features like shadow intensity. While the BPR renderer may not appear as effective as the Arnold render in Autodesk Maya, it is very quick and may enhance the photo-realism of the painted skin. When the model is sufficiently painted, instead of exporting the 3D model as an. OBJ file with its associated maps, go to ‘Document’ in the menu bar at the top of the Pixologic ZBrush interface and choose ‘Export’ (Fig. 11.32). This will allow for the export of the current view as an image file to be opened in Adobe Photoshop CC. It is recommended that the 3D model be placed in the Frankfurt Horizontal Plane and exported a frontal facing image. When the image is opened in Adobe Photoshop CC, digital compositing methods can be used to add realistic eyes, hair, clothing and accoutrements to the 2D image of the 3D painted facial depiction (Fig. 11.33). Once complete, the

layer can be flattened and a final image can be exported for publication.

11.3

Discussion

Facial reconstruction and depiction visualizes a facial appearance of an individual from their skeletal remains. While the prediction of face shape follows anatomical standards and the accuracy of methods available have been established (Wilkinson et al. 2006; Lee et al. 2012), the texturing process is driven by the preference and skillset of the facial depiction practitioner, alongside access to required technologies and availability of supporting information relating to textural appearance. Technologies are also changing rapidly, and this requires a significant investment from the practitioner to learn or keep up to date with advancements. The three methods described in this chapter for adding textures to digital 3D facial reconstructions each have their own affordances, and although these methods are described in the context of adding digital textures

272

M. Roughley and C. Y. J. Liu

Fig. 11.33 Compositing of eyes, hair and clothing in Adobe Photoshop CC on top of an image exported from Pixologic ZBrush as an image document

to 3D models for facial depiction purposes, they are transferrable and can be utilised for anatomical and medical visualisation.

11.3.1

Comparing Methods for Adding Digital Textures to 3D Facial Reconstructions

Facial depiction practitioners can use various offthe-shelf image graphics software with image transformation and digital painting capabilities to add digital textures to facial depictions (Taylor 2000; Roughley and Wilkinson 2019). Adobe Photoshop CC contains suitable tools for adding digital textures to 2D images of 3D facial reconstructions. It facilitates compositing of numerous photographic images and digital painting to create realistic 2D digital facial depictions for display on screen, online or in print. In comparison to the other two techniques described in this chapter, the 2D digital composite method is the fastest—taking approximately 8–10 h to

complete. It is also relatively low cost and does not require advanced 3D painting and VFX skills to add textures to 2D images of 3D facial reconstructions. Care should be taken when applying donor textures to the facial depiction image during the composite texturing process. Application of identifiable textures might inadvertently affect recognition or be recognised as the donor. Good practice is to use less than 10% of one donor image for anonymization. For individual features, Shrimpton (2018) suggests the compositing of more than two donor images per feature, and if one face donor image is being used to composite multiple parts of the facial depiction, no more than half of the donor face image should be used. With facial features that include identifiable biometrics (iris patterns and lip prints) it is possible to anonymise them. If the shape of these features has been chosen because they best match the morphology of the 3D facial reconstruction, but further anonymisation is required, parts of the features can be blurred, additional

11

Digital 2D, 2.5D and 3D Methods for Adding Photo-Realistic Textures to 3D. . .

273

Fig. 11.34 Potential impact on feature shape from different photographic donor images via the 2D digital composite method in Adobe Photoshop CC

details digitally painted on new layers, and parts of the image can be swopped for parts of other images. For example, using a lip shape from one image and a lip print from another, or blurring the iris pattern. In addition to an awareness of the impact of applying identifiable donor textures to a depiction, caution when choosing and applying photographic donor textures should be taken to ensure that any ‘shape’ information transferred does not infer incorrect morphology. Errors can occur when attempting to texture an image of a 3D facial reconstruction with 2D images. In the 2D digital composite method, the contour, lighting and shape from the donor photographs can be difficult to match to the 3D shape of the facial reconstruction. This potentially creates opportunities for the suggestion of different morphology through the application of unsuitable textures. Figure 11.34 shows different photographic donor images of a nose applied to two iterations of the same 3D face model via the 2D composite method in Adobe Photoshop CC. Transformation of the donor images was limited to rotate, scale and opacity. It is demonstrated that the nose shape added by the image compositing process appears to be different to that of the 3D facial reconstruction and more advanced transformations are

required if this donor texture is to be used. If caution is not taken to transform and edit the donor images sufficiently, the shape of the face can be affected. Shadows present on the image of the 3D facial reconstruction in the 2D digital composite method are important in helping to define morphology. These shadows are created by virtual lights in the 3D software where the facial reconstruction was created and they can be harnessed by duplicating the layer that contains the imported 3D facial reconstruction image and moving it to the top of the layer stack. Then, in the ‘Blend Mode’ drop-down menu in the ‘Layers’ palette, change ‘Normal’ to ‘Multiply’. The image may appear oversaturated and require some minor image adjustments or a reduction in opacity to reach a desired finish but importantly, the morphology-defining shadows will remain. ZBrush is a digital sculpture software that combines 3D modelling, texturing and painting, and its resemblance to traditional sculpture makes ZBrush easier to learn and adapt to than other available 3D modelling software. Its application in anatomical and medical visualisation has been effectively demonstrated (see Erolin et al. 2017; Webster 2017; Christ et al. 2018; Erolin 2019; Jędrzejewski et al. 2020; Knight et al. 2019; Maniam et al. 2020; Šulek et al. 2020; Adams

274

M. Roughley and C. Y. J. Liu

Fig. 11.35 The same photographic donor image applied to two iterations of one 3D face model. On the left, the digital 2D composite method was used with little transformation of the donor image. On the right, the digital 3D painting method and ‘Spotlight’ tool was used with little transformation of the donor image

and Erolin 2021), however, there is limited documentation of its use for texturing 3D facial depictions with only a small number of examples currently available (see Charlier et al. 2019; Roughley and Wilkinson 2019; Wilkinson et al. 2019; Smith et al. 2020; Martínez-Labarga et al. 2021). The software facilitates digital painting and sculpting in three-dimensions and has real use in applying textures to 3D facial reconstructions. Unlike the 2D digital composite method, painting textures directly onto the surface of the 3D model, and compositing with the ‘Spotlight’ tool in Pixologic ZBrush, could reduce the likelihood of morphological errors occurring between the ‘clay-like’ model and the final textured face. This is because the paintbrushes and projected images follow the contours of the 3D skin layer. Figure 11.35 shows a photographic donor image applied to two iterations of the same 3D face model. On the left, the digital 2D composite method was follow using a screenshot of the 3D model in Adobe Photoshop CC. The transformations applied to the donor image were rotate, scale and opacity. A ‘Multiply’ blend mode was also applied to use shadows from the facial reconstruction image. On the right is the application of the same donor image via the digital 3D painting method using the ‘Spotlight’ tool in Pixologic ZBrush. The donor image was rotated and scaled, and the opacity reduced before being painted directly onto the surface of the 3D model. It is possible that ‘shape’ information from donor images can be transferred to the 3D

model in both methods, however, the risk of extreme morphological error is reduced in the 3D digital painting method. Working entirely in 3D enables the head model to be moved, viewed and textured from different angles (Fig. 11.36), unlike the 2D methods of facial depiction where only the frontal view is available. However, difficulties can arise when trying to apply 2D frontal-facing photographic donor images using the ‘Spotlight’ tool to the oblique surfaces of the 3D model. Unless there is access to donor facial images from multiple viewpoints—frontal, medial or lateral, it can be difficult to project 2D textures onto 3D surfaces that are not directly visible in a frontal view, and some image distortion may occur. Painting skin textures using virtual and brushes directly onto the surface of the 3D model instead of using the ‘Spotlight’ tool avoids these difficulties and potential morphological errors adopted from donor images. If the addition of textures using the ‘Spotlight’ tool is preferred, another option for texturing 3D models with photographic images is to create a library of anonymised ‘textures’ in Adobe Photoshop CC and apply these to the 3D model. These textures do not include and ‘shape’ from the donor image but are simply square images with softened edges, as shown being used in the 3D digital painting method in Fig. 11.15. They can easily be composited using the ‘Spotlight’ tool, with additional blending and edits undertaken using virtual brushes and image alphas. Using these textures ensures that any

11

Digital 2D, 2.5D and 3D Methods for Adding Photo-Realistic Textures to 3D. . .

275

Fig. 11.36 3D digital painting of skin textures directly onto the surface of the 3D model most likely reduces shape error compared to the 2D digital composite and 3D ‘Spotlight’ methods. 3D facial depiction of a Seventeenth

Century Scottish Solider unearthed at Durgam Cathedral. Image courtesy of Face Lab, Liverpool John Moores University

shape information in the final facial depiction is driven by the shape of the 3D facial reconstruction. The realism and appearance of the 3D painted skin in the 3D digital painting and rendering method depends on the CGI skills of the artist, and additional access to software and hardware needed for rendering. Human skin has several properties that include “variation in colour, surface roughness, and translucency over different parts of the body, between different individuals” (Nagano et al. 2015), which make it difficult to replicate. The material properties of skin shaders applied to a 3D model in Autodesk Maya, such as sub-surface scattering, can be altered to produce more realistic outcomes with good success. However, an understanding of skin structure and reflectance (Angelopoulou 1999), and the shader’s ability to “simulate the scattering of light beneath the surface of the skin” (Jensen et al. 2001) is required for the skin to “appear fleshy and organic” (Nagano et al. 2015). Virtual lighting of a 3D facial reconstruction model in a 3D scene can be controlled to generate a desired aesthetic. Shadows are cast on the 3D model from the virtual lights in the 3D scene, and these shadows help to define morphology. Using

the 3D painting and rendering method, the 3D depiction is unaffected by erroneous shadows that might be present from donor photographs applied during a composite texturing method. Tweaking the material shader and light settings in a 3D scene until a desired appearance is reached, will be a process of trial and error for individual practitioners. The skin material shader, sub-surface and specular settings are easily affected by small changes to light intensity and colour. If a light is moved within the 3D scene or its intensity is changed, you may need to spend time re-adjusting the skin shader settings and performing multiple test renders until the desired appearance is reached. This adds additional time and complexity to the 3D digital texturing and rendering process. Rendering time will also increase with the addition of 3D hair and clothing. In comparison to the other methods described in this chapter the 3D digital painting and rendering method is the most time-consuming and has the most complex workflow. A facial depiction textured using this method will take approximately 80+ hours. The 2.5D method requires some 3D modelling and painting skills, but no advanced VFX or rendering skills, or access to expensive rendering

276

M. Roughley and C. Y. J. Liu

software and hardware. Similar to the 3D digital painting method, lighting can be controlled within Pixologic ZBrush, and painting directly onto the surface of the 3D model reduces the inference of incorrect morphology. This method reduces time compared to the digital 3D painting and rendering method, taking approximately 10–20 h to complete a facial depiction. It could also increase accuracy when compared to the 2D composite method. Where possible, it is recommended that the digital 3D painting and rendering method is used to add realistic textures to 3D facial reconstructions, and that virtual lights are used to cast shadows. In the absence of advanced rendering and VFX skills, the 2.5D method presents a suitable compromise and a potentially more accurate methods of adding digital textures to 3D models than the digital 2D composite method alone.

11.3.2

Artistic Proficiency and Cognitive Biases

The digital workflows described in this chapter use Adobe Photoshop CC, Pixologic Zbrush and Autodesk Maya with default workspaces, but it should be noted that “no two artists work identically or have identical training” (Erickson et al. 2016). Even if the base 3D reconstruction model is the same, artistic variability will result in different aesthetics on the textural layer, especially in relation to information such as hair, colour and surface details. There are no known studies that note if an artists’ proficiency with digital texturing software can affect the morphological accuracy of digital facial depictions. Further studies are required to establish this with certainty. Some studies highlight that artistic proficiency can affect recognition of an individual from a facial depiction in forensic settings. Erickson et al. (2016) suggest that the professional and artistic experience of the practitioner could be a contributing factor to the performance of a facial depiction for identification purposes. However, Lampinen et al. (2015) found no correlation between resemblance to the target and the

experience or artistic training of the artist. Even archaeological facial depictions, where there is access to historic written descriptions of the individuals, portraits and DNA analysis to suggest textural appearance, may rely on stereotypes in order to create super-realism (Wilkinson 2020). The aforementioned 3D digital facial depiction of King Robert II (Fig. 11.5) is an example of a multi-year project that involved an interdisciplinary research team to make decisions about the applied textures and overall presentation of the facial reconstruction based on available historical evidence. The application of brown hair and eye colour was informed by analysis of portraits of confirmed descendants held in art galleries by a geneticist at the University of Glasgow (Scotland), challenging the pre-existing stereotype of a blue-eyed and red-haired Scotsman. The bascinet and crown were produced through a collaboration between a curator of European Arms and Armour at Kelvingrove Art Gallery & Museum (Scotland), and a 3D artist, who used historic depictions of King Robert II (statues, coinage) and armour held in European museums as references (Wilkinson et al. 2019). Care was taken to ensure that the choices made were appropriate, down to the leather and twine materials used to attach the aventail to the bascinet, and the type of mail (steel rings with a domed fastening). However, not every project will have access to similar data, records or collaborators. Forensic DNA phenotyping provides more comprehensive details relating to individual appearance; the phenotyping process is useful for characteristics that involve pigmentation such as eye, hair and skin colour (Kayser 2015), and the addition of this information to facial depictions can update or change portraits of people from the past. Figure 11.37 shows depictions of two Mesolithic individuals from Norway; cases where aDNA phenotyping suggested dark skin and blue eyes, contrary to popular theory for this ancient population. However, caution must be taken when relying on forensic DNA phenotyping as it presents a spectrum of possibilities. In contemporary art, Heather Dewey-Hagborg’s artwork ‘Stranger Visions’ (2012–13) received criticism due to its

11

Digital 2D, 2.5D and 3D Methods for Adding Photo-Realistic Textures to 3D. . .

277

Fig. 11.37 2.5D facial depictions of two Mesolithic individuals from Norway on display at Bergen City Museum. Image courtesy of Face Lab, Liverpool John Moores University

ancestry bias in relation to textures chosen and applied to 3D printed face models produced from DNA profiling. The follow up work ‘Radical Love: Chelsea Manning’ (Dewey-Hagborg 2017) drew attention to this problematic use of forensic DNA phenotyping through the generation of multiple (approx. 30) faces with different textural appearances from one DNA sample. Although the facial reconstruction process is rooted in scientific knowledge, subjective material is often added during the depiction process, especially when the facts prove insufficient for a realistic appearance. This subjectivity is conditioned by confirmation bias and any accepted knowledge/beliefs. The facial depiction practitioner should be aware of their own cognitive biases when applying textures to facial depictions, especially for forensic purposes whereby application of incorrect colour textures can affect recognition. One solution is to present facial depictions as solid colour models (see grayscale 3D printed replica of King Robert II in Fig. 11.5 as an example) to lessen subjective addition of textures that may be incorrect or shaped by cognitive biases. However, research has shown that the degree of realism in facial depictions influences their reception by the publics as valid faces (Lewis 1997; Johnson 2016), and that faces are generally more difficult to recognise without textural information (Bruce et al. 1991). Textural information is

therefore an important component of a facial depiction, however, the desire for hyper-realistic depictions of human remains, particularly for museum display, sometimes outweighs available evidence for justification of texture choices; potentially ignoring the fact that the chosen textures are some of many possible options (Wilkinson 2020).

11.4

Conclusion

The presentation of a facial depiction can be critical to public perception, academic dialogue and scientific progress. This chapter describes three methods of adding realistic textures to digital facial depictions produced using 3D software: a 2D photo-composite method, a 3D digital painting and rendering method, and a previously undescribed hybrid 2.5D method. Each method has its advantages, including reduction in time taken to digitally texture a 3D facial reconstruction model, or increased opportunities to present facial depictions in different formats. The affordances of each method are highlighted, and it is suggested that the digital 3D texturing and rendering process for adding textures to 3D facial reconstructions can reduce the inference of incorrect morphology compared to the 2D digital composite method. It is recognised that practitioner biases may impact

278

the choice of textures and final display of the facial depiction for public audiences.

References Adams EM, Erolin C (2021) The devil is in the details: developing a modern methodology for detailed medical illustrations. J Vis Commun Med 1–20. https://doi. org/10.1080/17453054.2021.1921566 Angelopoulou E (1999) The reflectance spectrum of human skin. Technical reports (CIS), p 584. https:// repository.upenn.edu/cgi/viewcontent.cgi? article¼1616&context¼cis_reports Autodesk (2014a) Open and navigate Hypershade [Online]. Available at: https://knowledge.autodesk. com/support/maya/learn-explore/caas/CloudHelp/ cloudhelp/2015/ENU/Maya/files/Asts-Open-and-navi gate-Hypershade-htm.html Autodesk (2014b) XGen overview [Online]. Available at: https://knowledge.autodesk.com/support/maya/learnexplore/caas/CloudHelp/cloudhelp/2015/ENU/Maya/ files/GUID-47644337-40F0-4766-BD3B4104F9F9B7E2-htm.html Autodesk (2016) Lambert [Online]. Available at: https:// knowledge.autodesk.com/support/maya/learn-explore/ caas/CloudHelp/cloudhelp/2016/ENU/Maya/files/ GUID-97B781A0-7F21-42D4-B0408EB967CB8563-htm.html Autodesk Arnold (n.d.-a) Standard surface [Online]. Available at: https://docs.arnoldrenderer.com/display/ A5AFMUG/Standard+Surface Autodesk Arnold (n.d.-b) Displacement [Online]. Available at: https://docs.arnoldrenderer.com/display/ A5AFMUG/Displacement Brinkmann R (2008) The art and science of digital compositing: techniques for visual effects, animation and motion graphics. Morgan Kaufmann Bruce V, Healey P, Burton M, Doyle T, Coombes A, Linney A (1991) Recognising facial surfaces. Perception 20(6):755–769. https://doi.org/10.1068/p200755 Buti L, Gruppioni G, Benazzi S (2017) Facial reconstruction of famous historical figures: between science and art. Studies. Forensic Biohist 75:191 Challoner A, Erolin C (2013) Creating pathology models from MRI data: a comparison of virtual 3D modelling and rapid prototyping techniques. J Vis Commun Med 36(1–2):11–19. https://doi.org/10.3109/17453054. 2013.790011 Charlier P, Froesch P, Benmoussa N, Morin S, Augias A, Ubelmann Y, Weil R, Morin S, Straub F, Deo S (2019) Computer-aided facial reconstruction of “Mary-Magdalene” relics following hair and skull analyses. Clin Med Insights 12:1179550618821933. https://doi.org/ 10.1177/1179550618821933 Christ R, Guevar J, Poyade M, Rea PM (2018) Proof of concept of a workflow methodology for the creation of basic canine head anatomy veterinary education tool

M. Roughley and C. Y. J. Liu using augmented reality. PLoS One 13(4):e0195866. https://doi.org/10.1371/journal.pone.0195866 Claes P, Vandermelen D, De Greef S, Williems G, Clement J, Suetens P (2010) Computerized craniofacial reconstruction: conceptual framework and review. Forensic Sci Int 201(1–3):138–114. https://doi.org/10. 1016/j.forsciint.2010.03.008 Clark JG (2020) Face-to-face with a medieval Abbot. Available at: https://www.stalbanscathedral.org/ Blogs/learn-with-us/face-to-face-with-a-medievalabbot Crossland Z (2009) Acts of estrangement. The postmortem making of self and other. Archaeol Dialog 16 (1):102–125. https://doi.org/10.1017/ S1380203809002827 Denham T (n.d.) What is UV mapping & unwrapping? Concept art empire. https://conceptartempire.com/uvmapping-unwrapping/. Accessed 01/06/2021 Dewey-Hagborg H (2012–2013) Stanger visions. https:// deweyhagborg.com/projects/stranger-visions. Accessed 01/06/2021 Dewey-Hagborg H (2017) Radical love: Chelsea Manning. https://deweyhagborg.com/projects/radical-love. Accessed 01/06/2021 Erickson WB, Lampinen JM, Frowd CD, Mahoney G (2016) When age-progressed images are unreliable: the roles of external features and age range. Sci Justice 57(2):136–143. https://doi.org/10.1016/j.scijus.2016. 11.006 Erolin C (2019) Interactive 3D digital models for anatomy and medical education. In: Rea P (ed) Biomedical visualisation. Advances in experimental medicine and biology, vol 1138. Springer, Cham. https://doi.org/10. 1007/978-3-030-14227-8_1 Erolin C, Jarron M, Csetenyi LJ (2017) Zoology 3D: creating a digital collection of specimens from the D0 Arcy Thompson zoology museum. Dig Appl Archaeol Cult Herit 7:51–55. https://doi.org/10.1016/j.daach. 2017.11.002 Fernandes CMS, de Sena Pereira FDA, da Silva JVL, da Costa Serra M (2013) Is characterizing the digital forensic facial reconstruction with hair necessary? A familiar assessors’ analysis. Forensic Sci Int 229 (1–3):164–1e1. https://doi.org/10.1016/j.forsciint. 2013.03.036 Frowd CD, Skelton F, Atherton C, Pitchford M, Hepton G, Holden L, McIntyre AH, Hancock PJ (2012) Recovering faces from memory: the distracting influence of external facial features. J Exp Psychol 18 (2):224. https://doi.org/10.1037/a0027393 Iscan MY, Steyn M (2013) The human skeleton in forensic medicine. Charles C Thomas Publisher Jędrzejewski Z, Loranger B, Clancy JA (2020) Virtual anatomy museum: facilitating public engagement through an interactive application. In: Rea P (ed) Biomedical visualisation. Springer, Cham, pp 1–18. https://doi.org/10.1007/978-3-030-43961-3_1 Jensen HW, Marschner SR, Levoy M, Hanrahan P (2001) A practical model for subsurface light transport. In:

11

Digital 2D, 2.5D and 3D Methods for Adding Photo-Realistic Textures to 3D. . .

Proceedings of the 28th annual conference on Computer graphics and interactive techniques, pp 511–518. https://doi.org/10.1145/383259.383319 Johnson H (2016) Craniofacial reconstruction and its socio-ethical implications. Mus Soc Iss 11(2):97–113. https://doi.org/10.1080/15596893.2016.1204598 Kähler K, Haber J, Seidel HP (2003) Reanimating the dead: reconstruction of expressive faces from skull data. ACM Trans Graphics (TOG) 22(3):554–561. https://doi.org/10.1145/882262.882307 Kayser M (2015) Forensic DNA phenotyping: predicting human appearance from crime scene material for investigative purposes. Forensic Sci Int Genet 18:33–48. https://doi.org/10.1016/j.fsigen.2015.02. 003 Knight O, Carter CR, Loranger B, Rea PM (2019) Recommended workflow methodology in the creation of an interactive application for patient’s diagnosed with pancreatic cancer. In: Rea P (ed) Biomedical visualisation. Springer, Cham, pp 85–103. https://doi. org/10.1007/978-3-030-24281-7_8 Lampinen JM, Erickson WB, Frowd C, Mahoney G (2015) Mighty Morphin’age progression: how artist, age range, and morphing influences the similarity of forensic age progressions to target individuals. Psychol Crime Law 21(10):952–967. https://doi.org/10.1080/ 1068316X.2015.1077245 Lee WJ, Wilkinson CM, Hwang HS (2012) An accuracy assessment of forensic computerized facial reconstruction employing cone-beam computed tomography from live subjects. J Forensic Sci 57(2):318–327. https://doi.org/10.1111/j.1556-4029.2011.01971.x Lewis MB (1997) Familiarity, target set and false positives in face recognition. Eur J Cogn Psychol 9(4):437–459. https://doi.org/10.1080/713752567 Mahoney G, Wilkinson C (2010) Computer generated facial depiction. In: Wilkinson CM, Rynn C (eds) Craniofacial identification. Cambridge University Press, Cambridge, pp 222–237 Maniam P, Schnell P, Dan L, Portelli R, Erolin C, Mountain R, Wilkinson T (2020) Exploration of temporal bone anatomy using mixed reality (HoloLens): development of a mixed reality anatomy teaching resource prototype. J Vis Commun Med 43(1):17–26. https://doi.org/10.1080/17453054.2019.1671813 Martínez-Labarga C, Carbone R, Ridolfi V, Parrino M, Vitali S, Bravi L, Rickards O, Falconi M (2021) Craniofacial reconstruction of Raphael Sanzio from Urbino: face and features of a “mortal god”. Dig Appl Archaeol Cult Herit e00190 Miranda GE, Wilkinson C, Roughley M, Beaini TL, Melani RFH (2018) Assessment of accuracy and recognition of three-dimensional computerized forensic craniofacial reconstruction. PLoS One 13(5): e0196770. https://doi.org/10.1371/journal.pone. 0196770 Nagano K, Fyffe G, Alexander O, Barbic J, Li H, Ghosh A, Debevec PE (2015) Skin microstructure deformation with displacement map convolution.

279

ACM Trans Graph 34(4):109–1. https://ict.usc.edu/ pubs/Skin%20Microstructure%20Deformation% 20with%20Displacement%20Map%20Convolution. pdf Osgood R (2018) Facing Magwitch: the Rat Island prisoners. Sanctuary 47:18–19. https://assets.publish ing.service.gov.uk/government/uploads/system/ uploads/attachment_data/file/915742/Sanctuary_ 2018_web_secured.pdf Parsons D, MacCallum K (2021) Current perspectives on augmented reality in medical education: applications, affordances and limitations. Adv Med Educ Pract 12:77. https://doi.org/10.2147/AMEP.S249891 Prag J, Neave RAH (1997) Making faces. British Museum Press, London Roughley M (2020) Pores, pimples and pathologies: 3D capture and detailing of the human skin for 3D medical visualisation and fabrication. In: Rea P (ed) Biomedical visualisation. Springer, Cham, pp 141–160. https://doi.org/10.1007/978-3-030-474836_8 Roughley M, Wilkinson C (2019) The affordances of 3D and 4D digital technologies for computerized facial depiction. In: Rea P (ed) Biomedical visualisation. Springer, Cham, pp 87–101. https://doi.org/10.1007/ 978-3-030-14227-8_7 Shrimpton SL (2018) Facial creation: using compositing to conceal identity. Doctoral thesis, Liverpool John Moores University Smith K (2018) Portraits, likenesses, composites? Facial difference in forensic art. In: Skinner P, Cock E (eds) Approaches to facial difference: past and present. Bloomsbury, London, pp 84–111. ISBN: 9781350028296 Smith K, Wilkinson C, Roughley M, Balachandran S, Swaney M, Garcia JR (2019) One of us? Navigating ‘rehumanisation’ questions in the depiction and display of two ancient Egyptians from the Johns Hopkins archaeological museum. International Association of Craniofacial Identification Conference, Baton Rouge. http://10.0.51.84/RG.2.2.36529.33122 Smith K, Roughley M, Harris S, Wilkinson C, Palmer E (2020) From ta-Kesh to ta-Kush: the affordances of digital, haptic visualisation for heritage accessibility. Dig Appl Archaeol Cult Herit 19:e00159. https://doi. org/10.1016/j.daach.2020.e00159 Stephan CN, Claes P (2016) Craniofacial identification: techniques of facial approximation and craniofacial superimposition. In: Handbook of forensic anthropology and archaeology. Routledge, pp 444–457 Šulek N, Poyade M, Ferguson E (2020) A methodology for visualising growth and development of the human temporal bone. In: Rea P (ed) Biomedical visualisation. Advances in experimental medicine and biology, vol 1262. Springer, Cham. https://doi.org/10.1007/978-3030-43961-3_8 Suppersberger Hamre S, Ersland GA, Daux V, Parson W, Wilkinson C (2017) Three individuals, three stories, three burials from medieval Trondheim, Norway.

280 PLoS One 12(7):e0180277. https://doi.org/10.1371/ journal.pone.0180277 Taylor KT (2000) Forensic art and illustration. CRC Press Vanezis M, Vanezis P (2000) Cranio-facial reconstruction in forensic identification—historical development and a review of current practice. Med Sci Law 40 (3):197–205. https://doi.org/10.1177/ 002580240004000303 Vermeulen L (2012) Manual forensic facial reconstruction. In: Wilkinson C, Rynn C (eds) Craniofacial identification. Cambridge University Press, pp 184–192 Webster NL (2017) High poly to low poly workflows for real-time rendering. J Vis Commun Med 40(1):40–47. https://doi.org/10.1080/17453054.2017.1313682 Wilkinson C (2005) Computerized forensic facial reconstruction. Forensic Sci Med Pathol 1(3):173–177. https://doi.org/10.1385/FSMP:1:3:173 Wilkinson C (2010) Facial reconstruction–anatomical art or artistic anatomy? J Anat 216(2):235–250. https:// doi.org/10.1111/j.1469-7580.2009.01182.x Wilkinson C (2018) Archaeological facial depiction for people from the past with facial differences. In: Skinner P, Cock E (eds) Approaching facial difference past and present. Bloomsbury, London

M. Roughley and C. Y. J. Liu Wilkinson C (2020) Cognitive bias and facial depiction from skeletal remains. Bioarchaeol Int 4(1):1–14. https://doi.org/10.5744/bi.2020.1001 Wilkinson C, Roughley M (2019) Facial depiction of “Bernie”, man from the bog. In: Bauerochse A, Hassmann H, Püschel K, Jopp-van Well E, Jahn W, Schultz M (eds) Bernie – the bog body from Bernuthsfeld: results of the interdisciplinary research and reconstruction of an early medieval find complex from East Frisia. VML Verlag Marie Leidorf, pp 273-277. Rahden/Westf 9783896468499 Wilkinson C, Rynn C, Peters H, Taister M, Kau CH, Richmond S (2006) A blind accuracy assessment of computer-modeled forensic facial reconstruction using computed tomography data from live subjects. Forensic Sci Med Pathol 2(3):179–187. https://doi.org/10. 1007/s12024-006-0007-9 Wilkinson C, Roughley M, Moffat R, Monckton D, MacGregor M (2019) In search of Robert Bruce, part I: craniofacial analysis of the skull excavated at Dunfermline in 1819. J Archaeol Sci Rep 24:556–564. https://doi.org/10.1016/j.jasrep.2019.02. 018

Teaching with Cadavers Outside of the Dissection Room Using Cadaveric Videos

12

Danya Stone, Catherine M. Hennessy, and Claire F. Smith

Abstract

The rise of Information and Communication Technologies and Computer Assisted Instruction have led to the adoption of digital visual learning aids to improve anatomy instruction. Creation of cadaveric video resources surged during 2020–2021 as they provided one option to continue teaching anatomy using cadaveric specimens in a time when all in-person practical teaching was prohibited to maintain safety during the Covid-19 pandemic. Cadaveric videos are relatively inexpensive to create and with the correct set up can be filmed independently by one anatomist. This makes cadaveric videos a feasible option for anatomists to create using their own specimens and tailored to their own curriculum. The use of cadaveric videos is not limited to instances where practical teaching is not an option and can provide an excellent supplementary exercise. Using cadaveric videos in conjunction with in-person dissection sessions could enhance student’s self-efficacy, promote autonomous learning and reduce the likelihood of students experiencing cognitive overload while learning in the dissection room environment. However, sharing resources that contain cadaveric material online should be D. Stone (*) · C. M. Hennessy · C. F. Smith Brighton and Sussex Medical School, Brighton, UK e-mail: [email protected]; [email protected]; [email protected]

approached with caution and anatomists should ensure they have a secure method of distributing cadaveric video content to the intended audience only. Keywords

Cadaveric video · Cognitive load · Covid-19 · Digital fluency · Self-efficacy

12.1

Introduction

Throughout anatomy’s history, anatomical illustrations have been created, based on observations made during human dissection, and used as a method of advancing understanding in anatomy (Calkins et al. 1999). Nowadays, the practise of anatomical illustration has been largely replaced by digital visualisations in the format of photos, videos, and digital illustrations (Losco et al. 2017). These digital visualisations have come to form a key resource in anatomy education. Hulme and Strkalj (2017) suggested two main drivers for adoption of digital visual learning aids in anatomy education: advancements in technology and reduced curricula time. Around the turn of the millennium, the rise of Information and Communication Technologies (ICT) and Computer Assisted Instruction (CAI) led to the adoption of digital visual learning aids to improve anatomy instruction. In particular, increases in

# The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 P. M. Rea (ed.), Biomedical Visualisation, Advances in Experimental Medicine and Biology 1356, https://doi.org/10.1007/978-3-030-87779-8_12

281

282

the quality of video recording (Hulme and Strkalj 2017), and the development of user-friendly editing software (i.e., iMovie™ video editing software (Apple, Inc., Cupertino, CA)) have made it easier for anatomists to create their own video recordings of cadaveric material to support their students. This surge in the availability of ICT means that cadaveric videos are no longer only available on computers in anatomy laboratories, but rather, are now supported for mobile or remote learning (Trelease 2008, 2016). This has led to the emergence of cadaveric video resources that can be used to learn anatomy anywhere and at any time. The impact that cadaveric videos have on learning gain, remains unclear, and implementation of these videos appears to have no significant effect on assessment scores (Choi-Lundberg et al. 2016; Greene 2020; Mahmud et al. 2011; Saxena et al. 2008; Topping 2014). However, perhaps it is not just the learning gain that is important to consider; a sense of enjoyment and satisfaction are important feelings to recognise as part of the student learning experience. The evidence demonstrates that students enjoy using cadaveric videos made by their instructors to learn anatomy and find them positive additions to their learning (Choi-Lundberg et al. 2016; DiLullo et al. 2006; Langfield et al. 2018; Mahmud et al. 2011; Ogunranti 1987; Saxena et al. 2008; Topping 2014). Cadaveric videos may be popular resources for students learning anatomy as they reduce cognitive load (Sweller 1988) and increase student’s self-efficacy during practical dissection teaching (Choi-Lundberg et al. 2016; Langfield et al. 2018; Saxena et al. 2008). Additionally, cadaveric videos provide an opportunity for students to acclimatise to learning using cadaveric material (Casado et al. 2012; Greene 2020). Moreover, creating bespoke cadaveric videos affords anatomists the opportunity to make content that closely aligns with their curriculum and may therefore help to compensate for the reduction in taught contact hours allocated to teaching anatomy (Drake et al. 2009, 2014). Despite the benefits and flexibility that sharing cadaveric videos offers to anatomy educators and

D. Stone et al.

students, care must be taken to ensure that ethical and professionalism standards are in place. Concerns have rightly been raised regarding the sharing of cadaveric images and videos on digital media platforms due to the issues around receiving explicit informed consent from anatomy donors (Hennessy and Smith 2020) and content containing cadaveric images reaching an unintended or inappropriate audience (Hennessy et al. 2020). Additionally, anatomy educators must ensure that secure storage systems are in place when sharing images or videos containing cadaveric content, and that only the intended student audience can access these resources (Anderson et al. 2021).

12.1.1

Transition During Covid-19

In March 2020, there was an emergency transition to online learning across the globe because of the coronavirus (Covid-19) pandemic (GOV.UK 2021; WHO 2020). In anatomy education, the immediacy of this move meant that new online resources needed to be implemented quickly to support students who were now suddenly learning anatomy remotely (Evans et al. 2020; Longhurst et al. 2020; Pather et al. 2020). With the cancellation of in-person, face-to-face teaching during the Covid-19 lockdowns, anatomy educators had to navigate teaching a subject without access to the best-established methods of doing so: cadaveric dissection and prosection sessions. Many anatomy departments invested in new technologies to attempt to maintain student exposure to cadaveric material while teaching online. Some invested in live streaming equipment to provide synchronous video content straight from the dissection room. Other anatomists chose to instead use pre-existing resources to provide cadaveric teaching to their students online, such as Acland’s Video Atlas of Human Anatomy (Acland 2013; Longhurst et al. 2020). Additionally, many anatomists opted to invest in video recording equipment to create and implement their own cadaveric videos (Longhurst et al. 2020). It is worth highlighting that students may

12

Teaching with Cadavers Outside of the Dissection Room Using Cadaveric Videos

have also chosen to source material independently given the availability of anatomy videos (including videos containing cadaveric material) online, on platforms such as YouTube™. Medical students have been found to use YouTube often to support their anatomy learning (Jaffar 2012; Barry et al. 2016). Instructor made cadaveric videos have the advantage of being a relatively inexpensive resource to create as they require only a small investment in equipment (a video recorder and a stand). Additionally, with the right set up, as shown in Fig. 12.1, these resources can be filmed

by one academic, thereby alleviating the time investment required by academic staff. In many universities, the decisions surrounding implementation of these new, bespoke, cadaveric videos were made in immediate response to the cancellation of practical sessions. However, cadaveric videos can be easily stored and accessed in the future, meaning they could potentially continue to be used to support students in their anatomy learning going forward. As a wealth of new cadaveric video content has been generated by anatomists across the globe during the Covid-19 lockdowns, it is important to evaluate the effectiveness of these videos and their potential place in anatomy teaching (Evans et al. 2020). This piece is written from the perspective of three anatomists with 33 years collective experience teaching anatomy. Views are shared based on the authors experiences of teaching anatomy to undergraduate medical students at Brighton and Sussex Medical School (BSMS). The authors refer to their experiences throughout but situate these in the existing literature.

12.1.2

Fig. 12.1 A set up to allow independent filming. This set up was used at Brighton and Sussex Medical School to achieve aerial views of specimens (Model: 18 Part Coloured Skull, developed by Adam, Rouilly Co., Sittingbourne, UK)

283

Anatomy at Brighton and Sussex Medical School

At BSMS, medical students study anatomy following a systems-based approach. Medical students are predominantly taught anatomy during year one and year two, with some sessions in year four. The anatomy provision at BSMS includes teaching using human cadaveric material in one of two formats: dissection sessions, where students dissect with the intention of revealing structures of interest; or prosection sessions, where students view structures on specimens that have already been dissected. Students are expected to use numerous resources to aid their learning in the dissection room, including the dissection notes, cadaveric specimens, osteological models/specimens, potted specimens, and three dimensional (3D) models. While in the dissection room, students are asked to leave all personal belongings outside. Most dissection sessions are attended by approximately

284

D. Stone et al.

100 students, facilitated by six instructors. Students are therefore expected to arrive to these sessions prepared to work somewhat autonomously. A full outline of the anatomy provision at BSMS has been described previously by Smith et al. (2018).

12.2

Cadaveric Videos

At BSMS, a complete series of cadaveric videos were recorded and edited between March and November 2020. A total of 63 videos were finalised with a total running time of 5 h and 27 min, covering all anatomical regions. Videos were filmed using a camera mounted to a tripod to generate an aerial view of the specimens, allowing the students to focus solely on the specimen and the hands of the instructor, as demonstrated in Fig. 12.2. The cadaveric videos were filmed in this manner to aid students with orientation of the specimen, as suggested in the literature (DiLullo et al. 2006; Langfield et al. 2018; Saxena et al. 2008). The cadaveric videos recorded at BSMS, also followed the advice of DiLullo et al. (2006) and used a wide camera view at the beginning of the video to allow

Fig. 12.2 Instructor made video of the skull. An example of the aerial view used during filming at Brighton and Sussex Medical School. (Model: 18 Part Coloured Skull, developed by Adam, Rouilly Co., Sittingbourne, UK)

students to orientate themselves, before then zooming in on the anatomical area being demonstrated. The cadaveric videos created by anatomists at BSMS were later edited using iMovie™ video editing software (Apple, Inc., Cupertino, CA). Other editing software described in the literature includes Camtasia™ video editing software (TechSmith Corp., Okemos, MI) (Greene 2020; Langfield et al. 2018) and Vegas Movie Studio HD Platinum Suite 11 (Sony Electronics, Tokyo, Japan) (Topping 2014). During the editing process, clips were stitched together to ensure seamless transitions between views of different specimens and all unnecessary information was removed. This allowed the authors to cap all videos at 10 min running length, following the suggestion from the literature (Choi-Lundberg et al. 2016; Greene 2020; Hulme and Strkalj 2017; Langfield et al. 2018; Saxena et al. 2008) to keep the running time as short as possible to maintain student engagement. At BSMS, videos were filmed using already dissected specimens (prosections) rather than recording the dissection process. In the literature, some implementations also opted to film prosected specimens (Langfield et al. 2018; Theoret et al. 2007; Topping 2014), while others chose to film videos of the dissection process (Choi-Lundberg et al. 2016; DiLullo et al. 2006; Greene 2020) or a combination of both (Saxena et al. 2008). Prior to the Covid-19 lockdowns, overview videos showing the cadaveric material covered during most cadaveric practical sessions existed at BSMS. This complement of cadaveric videos was built upon in response to the forced suspension of in-person cadaveric based teaching during Covid-19, to act as a temporary replacement while access to the laboratory was limited. In other reports, cadaveric videos were implemented as a supplemental resource for cadaveric practical sessions for students to access if or when they saw fit (Greene 2020); as a replacement for some cadaveric practical sessions (Theoret et al. 2007) or incorporated into dissection guides to be used alongside cadaveric dissection (Koop et al. 2020; Greene 2020; Nation et al. 2020).

12

Teaching with Cadavers Outside of the Dissection Room Using Cadaveric Videos

12.2.1

Student Opinion

Students view cadaveric videos as valuable resources to aid them in their anatomy learning (Choi-Lundberg et al. 2016; DiLullo et al. 2006; Langfield et al. 2018; Mahmud et al. 2011; Ogunranti 1987; Saxena et al. 2008; Topping 2014). In a study by Mahmud et al. (2011) almost half of the students surveyed stated that they considered anatomy videos containing cadaveric material as the best resource to learn anatomy. Students have also reported that they find cadaveric videos helpful as they aid revision and/or preparation for cadaveric practical sessions, deepen understanding and increase familiarity with visual features and landmarks (ChoiLundberg et al. 2016; Langfield et al. 2018; Topping 2014). Evaluation data from BSMS also suggests that the cadaveric videos were well received as a short-term replacement for cadaveric practical teaching during the Covid-19 lockdowns.

12.2.2

Learning Gain

The impact of cadaveric videos on examination scores as a measurement of learning gain appears to be less favourable. When measuring overall assessment scores between a control group and a group who were provided with access to cadaveric videos made by their instructors, most reports found no significant difference in assessment scores (Choi-Lundberg et al. 2016; Greene 2020; Koop et al. 2020; Mahmud et al. 2011; Saxena et al. 2008; Topping 2014) or a small decrease in assessment scores (Langfield et al. 2018). However, two factors were identified that highlight the impact of cadaveric videos on performance in anatomy assessments. First, when the number of videos watched, or the amount of time spent viewing by each student were compared with assessment scores, a positive correlation was found (Choi-Lundberg et al. 2016; Greene 2020; Langfield et al. 2018; Saxena et al. 2008). Second, when performance specifically in practical assessments rather than written assessments

285

was measured, cadaveric videos were again found to have a positive impact (Choi-Lundberg et al. 2016; Granger and Calleson 2007; Topping 2014).

12.2.3

Engagement

Measurements of student engagement with cadaveric videos created by their instructors include viewing frequency and the timepoint at which students viewed the videos. Most students (79%) watched at least one video (Saxena et al. 2008). However, the number of students viewing the cadaveric videos tended to decline throughout the year (Greene 2020; Langfield et al. 2018). At BSMS, the cadaveric videos were viewed by a large majority of students, perhaps reflecting the fact that they were acting as a temporary replacement for cadaveric dissection rather than as a supplemental resource. Eighty percent of students watched at least one of the cadaveric videos provided, with most of these students watching all five videos provided during their module. Literature suggests that the time points at which students are most likely to view cadaveric videos are before their practical session, as preparation (Langfield et al. 2018; Saxena et al. 2008), or during the week preceding an examination, as revision (Choi-Lundberg et al. 2016; Langfield et al. 2018). Although students were encouraged to use cadaveric videos as preparatory material, Choi-Lundberg et al. (2016) found that only 58% of students had watched the videos by the day of their dissection class. Like other reports, at BSMS, the videos associated with the first practical session received the most views (69% of students), compared to the final video which was only accessed by 54% of students. Most students accessed each video once or twice, however, some students accessed videos up to 15 times, as shown in Fig. 12.3.

12.3

Cognitive Load Theory

Cognitive load theory (Sweller 1988) presents one explanation for positive student opinions of

286

D. Stone et al.

Fig. 12.3 Average viewing frequency for a set of five cadaveric videos at Brighton and Sussex Medical School. Viewing frequency for each video was calculated using the statistics tracking feature on the virtual learning environment, Blackboard

cadaveric videos. Cognitive load is described as the capacity of the working memory to consciously perceive sensory stimuli and to construct schemas to be stored in long term memory (Sweller et al. 1998). Schemas can be defined as a framework that facilitates organisation of information. Cognitive load theory suggests that instructors should aim to reduce the cognitive load placed on the working memory by distracting elements of instruction (extraneous load) to allow students to focus their working memory on the development of schemas to aid learning (germane load). A high extraneous load will increase the cognitive load placed on the working memory, thereby decreasing the capacity of the working memory to develop schemas for learning (germane load) (Khalil et al. 2005; Sweller et al. 1998). Finally, the intrinsic cognitive load is influenced by the difficulty of the subject matter, rather than by the instructional technique (Chandler and Sweller 1991; Sweller and Chandler 1994). Instructors should, therefore, aim to reduce extraneous and intrinsic cognitive loads to increase germane load and prevent cognitive overload, by choosing the most suitable format of instruction (Khalil et al. 2005). Anatomy has been described as a subject with a high intrinsic cognitive load. Learning anatomy requires students to retain a large amount of new information and anatomical terminology (Greene 2020). This is compounded by the intrinsically 3D nature of anatomy, requiring students to develop their spatial awareness to understand the

relationships between structures (Vorstenbosch et al. 2013). Also, anatomy is often situated at the beginning of a medicine degree, which is already a time full of transition and potential stress for students. These factors all contribute to the high intrinsic cognitive load associated with learning anatomy. Learning that occurs in dissection room environments also places high extraneous cognitive loads on students (Andersen et al. 2016), as there are many distractions to learning. If we take a typical dissection session at BSMS as an example, students are processing a vast array of sensory information. Visually, students are focussing on the cadaveric and osteological specimens, the potted pathological specimens around the periphery of the room, the initially unfamiliar dissection instruments, the anatomical models, the dissection notes and perhaps also a 3D digital model (i.e., Complete Anatomy, 3D4Medical/Elsevier, Dublin, Republic of Ireland) provided on an iPad. Audibly the dissection room is filled with discussions between students working in their peer groups and with facilitators, creating a constant background noise. Additionally, students are confronted with the distinct smell of formaldehyde and experience the textures of various tissues through palpation and dissection. The amount of sensory information students must process in the dissection room is compounded by the expectation that students simultaneously use many different resources to aid their learning and hence must effectively select what to focus

12

Teaching with Cadavers Outside of the Dissection Room Using Cadaveric Videos

their attention on. Moreover, the emotionally confronting nature associated with working with human cadaveric material contributes to a further increase in extraneous cognitive load (Dinsmore et al. 2001; Fraser et al. 2012). Finally, anatomy dissection classes require students to master basic dissection skills (Flack and Nicholson 2018) to identify the relevant structures for that session. Without first mastering these skills, students often end up with imperfect views of the structures, thereby also contributing to the high extraneous load associated with teaching anatomy in the format of cadaveric dissection sessions (Greene 2020). Since Sweller (1988) first introduced the concept of cognitive load, several instructional techniques have been developed with the aim of reducing it (Mayer and Moreno 2003; Mousavi et al. 1995; Sweller et al. 1998). The introduction of cadaveric videos, supplementary to cadaveric practical classes, inadvertently encompasses many of the suggestions to reduce cognitive load: by minimising the split attention effect and utilising the modality effect and by reducing task complexity and fidelity. Before discussing strategies to reduce cognitive load and hence free up space in the working memory for germane load and prevent cognitive overload, it must first be noted that in the correct circumstances, cognitive load can be beneficial for learning. It is therefore only redundant cognitive load (that doesn’t have a positive impact on learning) that instructors should aim to reduce (Leppink and van den Heuvel 2015).

12.3.1

Split Attention and Modality Effects

The split attention effect states that requiring students to split their attention between different resources, results in a higher extraneous cognitive load (Sweller et al. 1998). During cadaveric dissection and prosection sessions, students are normally required to read the notes, hold the information in their working memory and then complete the dissection or identify structures on a prosection. Therefore, students must mentally

287

integrate the sources of information to derive meaning, resulting in their attention being split between the written text and the cadaveric specimen. In cadaveric videos, on the other hand, the two sources of information are integrated, thereby reducing cognitive load by alleviating the need to search for relations between the written notes and the specimen (Chandler and Sweller 1996; Sweller and Chandler 1994; Mayer and Moreno 2003). Students recognise this benefit of cadaveric video resources, with one student explaining the videos were good because the “explanation and visualisation are simultaneous” (Ogunranti 1987). Moreover, the variety of resources available during cadaveric practical sessions (cadaveric specimens, dissection notes, osteological specimens, 3D models, 3D digital atlases, pathological specimens) means that students must be confident in selecting the best resource to support their learning to pay attention to. Dual-coding theory (Paivio 1991) and Baddeley’s model of working memory (Baddeley 1992) suggest that working memory processes both visual and auditory information separately. The modality effect states that using these separate processing streams (visual and auditory) in conjunction, can increase the capacity of the working memory (Baddeley 1992; Mayer and Moreno 2003; Paivio 1991; Penney 1989; Sweller et al. 1998), thereby increasing the capacity for germane load and subsequent schema formation. When watching cadaveric video resources, students are processing visual information (the cadaveric specimen), and auditory information (the instructor’s narration). When completing the same task of identification of structures in the dissection room, students are processing both sources of information using the visual processing stream (the cadaveric specimen and the dissection notes). Utilising independent streams of working memory (visual and auditory) in cadaveric video resources is beneficial for learning anatomy (Collins et al. 2015) as it provides an alternative to dissection room tasks that increases the capacity of the working memory. It is interesting to consider the addition of subtitles to cadaveric videos and the resultant

288

effect on cognitive load. In the United Kingdom (UK), a requirement to provide digital content that is accessible (e.g., subtitling of audio content) came into force under the Public Sector Bodies (Websites and Mobile Applications) (No. 2) Accessibility Regulations (Public Sector Bodies 2018). This means that from September 2020, all video content produced by Higher Education Institutions in the UK should include the option of displaying subtitles. The effect of subtitles on cognitive load is debated in the literature. Subtitles offer an additional processing stream (visual verbal). When subtitles are included alongside cadaveric videos, the visual-verbal (subtitles) and auditory-verbal (narration) information being presented is identical, which could create unnecessary (extraneous) cognitive load, referred to in the literature as the redundancy principle (Sweller 2005). This is supported by Kalyuga and Sweller (2014) and Mayer and Fiorella (2014) who found that the addition of subtitles for adult native speakers resulted in poorer comprehension, presumably due to the increased extraneous cognitive load. Conversely, Kruger et al. (2013) found that the inclusion of subtitles decreased cognitive load and reduced frustration, which they attributed to the high intrinsic load of the subject material. Therefore, as anatomy is a subject associated with a high intrinsic load, the inclusion of subtitles may offer a form of support and effectively reduce the cognitive load and frustration experienced by students. Subtitles also appear to be particularly beneficial to students on reading instruction and language acquisition courses (D’ydewalle and Van De Poel 1999) and to support non-native speaking students (Lavaur and Bairstow 2011). At least initially, learning anatomy requires a relatively high amount of language acquisition as the roots for much of the terminology used comes from Greek and Latin-languages unfamiliar to most students. Encouraging students to read the subtitles may, therefore, be of particular benefit as it allows them to read the anatomical terminology while also listening to the narration, without introducing unnecessary (extraneous) cognitive load. However, once students feel comfortable with the anatomical vocabulary, subtitles

D. Stone et al.

may become redundant and increase the cognitive demands. Subtitles should therefore be provided in a format that gives students control over whether they are displayed (Yang 2014) and students should be made aware of the benefits of displaying subtitles while learning anatomy.

12.3.2

Task Complexity and Self-Efficacy

Gradually increasing task complexity is one method to reduce intrinsic cognitive load (Leppink and Duvivier 2016; Leppink and van den Heuvel 2015). This is essential for novice learners, as it allows them to develop schemas to aid them in their understanding through easier subtasks. This is in line with Vygotsky’s (1980) influential concept: the zone of proximal development, being the difference between learning that a student can achieve independently and content that requires assistance to master. By gradually increasing task complexity, students can build schemas to support them as the task becomes more difficult, ensuring that students are able to work autonomously as the task remains within their zone of proximal development. Cadaveric practical sessions are a high complexity task. Consequently, students often arrive to cadaveric practical sessions without the knowledge required to work autonomously or the confidence to independently apply what they have learned previously to a cadaveric specimen. Cadaveric videos offer a format of anatomy learning with significantly less task complexity than cadaveric prosection and dissection sessions and may therefore act as a ‘steppingstone’ for understanding. Gradually increasing task complexity, by starting with cadaveric videos and progressing to cadaveric practical teaching, enables students to build schemas for their understanding and achieve the desired level of knowledge through an easier subtask. This results in students being able to work more autonomously during cadaveric practical sessions, as the tasks remain within the students’ zone of proximal development.

12

Teaching with Cadavers Outside of the Dissection Room Using Cadaveric Videos

Cadaveric videos offer lower task complexity as they often use one ‘key’ view of a specimen, giving students an opportunity to identify key structures before appreciating the 3D relationships. This effectively reduces the intrinsic load of the content, by first encouraging schema formation for basic identification that can later be built upon to incorporate the relationships between structures. While there is currently no literature to confirm that cadaveric videos reduce the intrinsic cognitive load of anatomy in this manner, Andersen et al. (2016) found that using virtual reality before attending the anatomy laboratory improved outcomes by reducing cognitive load and Küçük et al. (2016) reported a reduction in cognitive load for neuroanatomy students following the implementation of augmented reality. This suggests that digital representations of anatomy that display key views of structures, are effective resources that can minimise intrinsic cognitive load as they reduce task complexity. Task complexity is also reduced by the short length of the videos, effectively dividing the content into shorter, moreachievable subsections (Greene 2020; Mayer and Moreno 2003) and by providing an opportunity for students to adapt to learning from a cadaveric specimen by watching how anatomy faculty orientate themselves on specimens and navigate through different structures. Due to the lower task complexity offered by the simplified views of specimens, the division of the content into shorter subsections and the introduction to learning anatomy using cadaveric specimens, cadaveric videos place a lower intrinsic cognitive load on the student than cadaveric practical sessions. However, it is important to continue to evaluate the proficiency of learners as the course progresses so that task complexity can be increased at a suitable pace (Leppink and Duvivier 2016; Leppink and van den Heuvel 2015). As a result of these factors, cadaveric videos used as preparation material for dissection classes have also been shown to improve self-efficacy for practical classes (Choi-Lundberg et al. 2016; Langfield et al. 2018; Saxena et al. 2008). Selfefficacy can be defined as an individual’s

289

confidence in their own ability to complete a task (Bandura et al. 1996) and is therefore important for students to be able to successfully work autonomously in the dissection room. Reducing task complexity allows students to feel more confident in their ability to work autonomously during practical sessions, by bringing the aims for the in-person dissection session within the zone of proximal development for the student (Vygotsky 1980). Students often express concerns about working autonomously during cadaveric practical sessions as they require confirmation from their teachers that they are identifying structures correctly. This sometimes prevents students from confidently working autonomously and encourages a reliance on the facilitators during practical sessions. Reassuring students that they will be provided with videos that have all the relevant structures they must be able to identify for their assessment, may also alleviate some of this reliance and further promote self-efficacy and autonomous working in the dissection room (Ogunranti 1987). The creation of bespoke cadaveric video resources further supports this as content can be tailored to the requirements of the course. By supplying cadaveric videos that are constructively aligned to the intended learning outcomes for the session (Biggs 1996), students can be reassured that everything they must know for an assessment can be covered using the videos outside of the laboratory, thereby promoting the use of in person time to explore the cadaveric specimen. The introduction of cadaveric videos, either prior to or following a cadaveric practical session, can lower the intrinsic cognitive load by offering a task with lower complexity and promoting selfefficacy. However, it cannot be assumed that because a resource is made available, students are going to interact with it. Often, virtual learning environments are littered with resources, making it difficult for students to locate resources and ensure they are receiving the resources most valuable to them. It is therefore essential to consider if and when students should be encouraged to interact with cadaveric video resources. It is imperative that material designed to be completed

290

D. Stone et al.

prior to a session is made clear and that students are reminded of this, as despite students being encouraged to watch cadaveric videos prior to attending their dissection session, the number of students accessing these resources often remains low (Choi-Lundberg et al. 2016; Langfield et al. 2018).

12.3.3

Task Fidelity and Affect

Another method to reduce cognitive load is to gradually increase task fidelity. Leppink and Duvivier (2016) describe task fidelity as the degree to which the task resembles real life. Applied to anatomy, fidelity therefore increases as we move from teaching using textbooks, to teaching using 3D models, teaching using digitized cadaveric resources and finally teaching directly using cadaveric specimens (Leppink and van den Heuvel 2015). Leppink and Duvivier (2016) suggest that to avoid cognitive overload, teaching should begin with lower fidelity tasks and progress to higher levels of fidelity as students’ progress. The cognitive overload associated with increasing the fidelity of the task can be partly attributed to the increasing emotional impact as fidelity increases. Although this is not an area of research that has so far received much attention, one current theory is that negative emotions can increase extraneous cognitive load and thereby reduce the capacity of the working memory (Fraser et al. 2012; Plass and Kalyuga 2019). This needs to be considered, particularly when discussing cognitive load in relation to cadaveric dissection; an activity that often elicits negative emotions which perhaps impacts the ability of students to process information (Fraser et al. 2012; Leppink and van den Heuvel 2015). Cadaveric videos are a comparatively low fidelity task compared to cadaveric practical sessions. Cadaveric videos may therefore provide an opportunity for students to start to process emotions surrounding working with human cadaveric material. Findings from self-report questionnaires (Arráez-Aybar et al. 2008; Goss et al. 2019) and interviews (Finkelstein and Mathers 1990) suggest that a significant number

of students experience emotions such as uncertainty, anxiety and revulsion when first introduced to the dissection room. Students have also reported experiencing physiological symptoms such as syncope, queasiness, palpitations, dizziness, and nausea (ArráezAybar et al. 2008). Due to the unique circumstances during 2020, students at BSMS watched a video containing cadaveric material before they entered the dissection room for the first time. It was interesting to find that when students were finally introduced to the dissection room environment, there was a lower incidence of syncope and anxiety than is normally observed. This finding was supported by Casado et al. (2012) who reported a reduction in anxiety experienced by students after they were shown a cadaveric video prior to first in-person exposure. Guiding students through any anxiety provoked by their experiences in the dissection room is essential and could be a good justification for encouraging students to view the cadaveric video resources prior to cadaveric practical teaching. Greene (2020) also hypothesised that the higher number of students watching the videos in year one compared to year two could be attributed to their role in supporting students entering the dissection room for the first time and adjusting to the environment. Conversely, Attardi et al. (2021) found that encouraging students to view relevant YouTube™ videos prior to human cadaveric dissection was not sufficient to cause a reduction in anxiety experienced by students. However, the videos used in this study (Attardi et al. 2021) obscured all views of cadaveric material to enable them to be shared via a public platform (YouTube™) meaning the effect of bespoke cadaveric video content on emotion experienced by students in the dissection room is not yet fully understood.

12.4

Sharing Cadaveric Images Online with Students

Over recent years, sharing cadaveric images and videos for educational purposes online has been an increasing trend (Rai et al. 2019). The rise of

12

Teaching with Cadavers Outside of the Dissection Room Using Cadaveric Videos

the use of social media within anatomy education (circa 2012–2017) (Chytas 2019) has led to a drastic and somewhat concerning increase in the number of cadaveric images and videos that are available publicly online (Hennessy and Smith 2020). However, being able to share cadaveric images online, including on social media, does have numerous benefits and has allowed many anatomy educators to continue to teach students using cadaveric material during the Covid-19 pandemic (Longhurst et al. 2020). At BSMS, without sharing the cadaveric videos with students and streaming live online dissection sessions, anatomy faculty would not have been able to engage with students directly on the specific teaching material being taught on the BSMS anatomy courses. Despite these benefits, sharing cadaveric videos online with students does bring ethical and professionalism challenges to anatomy education (Hennessy and Smith 2020). In the UK, informed consent must be received from anatomy donors before images of cadaveric material can be captured and shared for educational or research purposes according to the Human Tissue Authority (HTA 2021), the regulators for the use of human tissue in the UK. As previously argued by Hennessy et al. (2020) there is uncertainty around the level of informed consent received from donors regarding the sharing of cadaveric images on online platforms, even if the purpose is educational. Likewise, the production of 3D anatomical models from images of anatomy donors has been criticised for lacking explicit informed consent from donors (Jones 2019). Donors who have consented to images being taken are likely to expect such images to be used for educational and research purposes, since this is what is stated on the HTA website (HTA 2021), however donors may not anticipate that these images will be used for the production of 3D models. It is also unlikely donors would anticipate that images of their body would be shared on social media and other online platforms, even if for educational purposes; donors might not have received this information before donating their body (Farsides

291

and Smith 2020). Currently, anatomy educators who share cadaveric images online with students are assuming that anatomy donors who have consented to images being taken of their bodies, have therefore consented to these images being shared on online platforms also, since the purpose is for education. Due to the success and popularity of sharing cadaveric material on online platforms during the Covid-19 pandemic, it is likely that anatomy educators will continue to use online platforms and social media to share cadaveric content in the future. With this in mind, it is more important than ever that anatomy educators can be confident that explicit informed consent for this practice is received from anatomy donors. There has already been a drive for making consent forms more transparent around the capturing of images and the use of images for online educational purposes due to the rise of cadaveric images appearing on social media (Hennessy et al. 2020), and this drive appears more relevant and important than ever following the creation of cadaveric video content to be shared online during the Covid-19 pandemic. Receiving specific informed consent for sharing images on online platforms would allow anatomy educators to be confident that they are maintaining the ethical and professionalism standards expected of the profession, and this is one of the recommendations of the social media guidelines for anatomists published in 2020 (Hennessy et al. 2020). The widespread adoption of online streaming of anatomy teaching sessions during the Covid19 pandemic has led to the development of further guidance for anatomy educators, specifically for sharing cadaveric images for online streaming and/or pre-recorded videos (Anderson et al. 2021). These guidelines focus on educating the students on how they should use and handle cadaveric images to ensure that the dignity of the anatomy donors is maintained at all times. Ensuring informed consent is received from anatomy donors, is again, one of the three key principles of these guidelines. Setting a digital code of conduct is the second key principle of

292

these guidelines. Setting codes of conduct for how students should behave around anatomy donors in the anatomy laboratory and for how students should handle cadaveric images that are shared on lecture slides is common practice for anatomy educators. However, sharing cadaveric videos brings a unique set of risks due to the potential vast reach of the content if misused. Digital codes of conduct should highlight to students the importance of viewing the cadaveric material in an appropriately private place, away from family, friends, or other members of the public (non-medical students), for whom the content was not intended. Students must also be instructed that saving videos and capturing photos on their own devices and the onward sharing of the cadaveric content is forbidden, just as it would be if they took photos of the donors when in the anatomy laboratory. Usually, anatomy educators have much more control and can police the students from capturing images on their own devices when they are in the anatomy laboratory, but this control must be somewhat loosened when sharing content online and students must be trusted to abide by a set digital code of conduct. An element of trust will always be required regarding how students handle cadaveric content. However, to minimize the risk of students breaching the code of conduct, at BSMS, at the beginning of every session or video that is streamed to or shared with students online, a disclaimer is displayed including a reminder to students that capturing and onward sharing of the cadaveric material is strictly forbidden and considered a breach of professionalism standards. Maintaining professionalism and instilling probity in students is the third key principle of the online teaching guidelines by Anderson et al. (2021). Students should understand that anatomy donors should be treated with the same level of respect as living patients, and this includes cadaveric images shared with students online. Students should also understand that any breaches of professionalism standards when handling the cadaveric images that have been shared during online teaching sessions will result in a “fitness to practice” investigation.

D. Stone et al.

12.4.1

Opportunity to Develop Digital Professionalism and Fluency

Although there is an increased risk of students breaching professionalism standards by cadaveric images being shared with them on online platforms, there is an opportunity to develop medical student’s awareness of their responsibility to maintain professionalism standards when online, also known as digital professionalism (Ellaway et al. 2015). The importance of medical students developing digital professionalism was gaining momentum prior to the move to the Covid-19 induced surge in online teaching, due to the increasing use of social media and online platforms for communicating and sharing resources amongst doctors and medical students. However, implementing digital professionalism education sessions into medical curricula has been lacking (Gomes et al. 2017; Mostaghimi et al. 2017). The move towards more online teaching sessions within anatomy and medical education could fill the gap in developing digital professionalism amongst medical students. How students handle cadaveric images when online is transferrable to how students should handle patient images and confidential patient information. The respect for anatomy donors that is instilled in students from when students first enter the anatomy laboratory (which is thought to form part of the hidden curriculum on professionalism development (Kumar Ghosh and Kumar 2019)) must not be lost due to the move to online teaching sessions, therefore it is important that this respect and dignity that is afforded to anatomy donors also forms part of the education students receive when anatomy educators share cadaveric images with students during online teaching sessions. Digital fluency is an emerging skill that many educators are advocating for development in today’s student cohorts who learn largely in a digital age (Wang et al. 2012; White 2013). Digital fluency has been defined as “the ability to reformulate knowledge and produce information to express oneself creatively and appropriately in a digital environment” (Wang et al. 2012). For

12

Teaching with Cadavers Outside of the Dissection Room Using Cadaveric Videos

medical students learning in the digital age, having the ability to generate knowledge, source information and communication appropriately with colleagues on online platforms, are likely to be key aspects of the digital fluency and digital professionalism skills they need to develop to ensure that they can handle and manage patient information when they become doctors. Sharing cadaveric images and videos with students on online platforms has brought an unexpected added advantage to medical education in that it has forced medical students to learn appropriate behaviour and use of images from anatomy donors, which is transferrable to how students will behave and use patient information. Although digital fluency and digital professionalism should be incorporated throughout medical curricula it is apparent that anatomy offers a key opportunity for developing these skills early in the medical curricula.

12.4.2

Storage of Cadaveric Images

It is likely that anatomy departments will already have policies in place for capturing images of cadaveric material. At BSMS, permission must first be received by the Designated Individual (Head of Anatomy) of the Anatomy Licence or a Persons Designated (any member of the anatomy faculty) before any images of cadaveric material can be captured for educational or research purposes. Image capture is only permitted on donors who have consented for images to be taken. Once permission has been received, faculty must ensure that images are captured on the designated anatomy cameras (where possible) and stored on the closed, password-protected university storage systems. Where it is not practical for faculty or guest faculty to capture cadaveric images on the anatomy cameras, they are permitted to capture images on their own devices but are then required to read and sign an ‘After-care form’ which sets out specific guidance for how the images should be handled, used, and stored once captured. This puts the responsibility for the care of such images on the individual who has captured the images. Any images that are shared

293

for online teaching sessions or cadaveric videos must be shared using University protected systems (which require users to login), rather than being shared on public platforms such as YouTube™. At BSMS, during the Covid-19 pandemic, live-streamed sessions of cadaveric dissection were shared with students using the University of Brighton Microsoft Teams channel, which only medical students in specific cohorts had access to. The cadaveric videos made by BSMS faculty were stored on a university protected application (app), called the Anatomy Lab Interface, which was specially created by the university learning technologists so that only specific students had access to the app. Again, students were required to login to the app using their university accounts before they could access the cadaveric videos. As previously explained, at the beginning of each session, students were reminded that any capturing or storage of images of any cadaveric material shared with them by anatomy faculty during live-streamed sessions or pre-recorded videos was strictly prohibited. Also, at the beginning of each online session or pre-recorded video, a disclaimer was announced to students that consent for taking images had been received from all donors in these learning materials.

12.4.3

Existing Online Learning Resources Versus Bespoke Cadaveric Video Content

The authors recognise that many anatomy educators may be wary of creating online material that contains cadaveric images and sharing this material with students, however the authors advocate creating bespoke instructor-made content for their students. The advantage of instructor made content is that faculty can ensure the material being shared follows ethical professionalism guidelines and includes information on how students can handle the cadaveric images shared with them in a responsible and professional way. There are perhaps some advantages in letting students to sift through the vast amount of

294

D. Stone et al.

learning content that is available to students online, in that it allows students to practice their critical analysis skills which Schmitt et al. (2012) have suggested is an essential skill for current students since students use platforms like YouTube regularly to support their learning. Jaffar (2012) reported that 40% of students watched anatomy YouTube videos “always” or “frequently” to support their learning and Barry et al. (2016) supported these findings when finding that 50% of students accessed online anatomy videos at least on a weekly basis. Some instructors may decide to recommend certain online resources to students that have been well used and endorsed by anatomy educators such as the Acland anatomy video series (Acland 2013), however these resources are unlikely to cover the exact content or level of detail that is being taught by individual instructors. Furthermore, there is evidence that students prefer to use learning materials that have been designed for them by their own anatomy educators (Pickering and Bickerdike 2017) and Junco et al. (2013) highlighted that instructor engagement is vital for the success of online educational tools. Raikos and Waidyasekara (2014) also found that the accuracy of anatomy content on YouTube™ is not reliable and that can often be of low quality, which is a further reason for instructors creating bespoke content for students rather than relying on students to find accurate content to learning support their learning.

12.5

Conclusion

Expanding the teaching of anatomy using cadaveric material to environments outside of the dissection room is complicated due to concerns of content being circulated inappropriately. However, with the right storage and online distribution systems in place to ensure security of content, videos containing cadaveric material can form valuable teaching resources. Advancements in technology and limitations on in-person practical time have led to an influx of cadaveric videos made by anatomists. These videos have been

received well by students, potentially because they offer a format of learning anatomy using cadaveric material that places less cognitive load on students than cadaveric practical teaching. With the return to face-to-face practical teaching following the Covid-19 lockdowns, anatomists should consider the benefits of retaining these video resources alongside cadaveric practical sessions. Further research is needed to confirm whether, when used in conjunction with cadaveric practical sessions, cadaveric videos promote self-efficacy and autonomous working in the dissection room, and whether they reduce the initial emotional impact experienced by students learning from cadaveric material for the first time.

References Acland R (2013) Acland’s Video Atlas of Human Anatomy®. https://aclandanatomy.com. Accessed 1 Apr 2021 Andersen SW, Mikkelsen PT, Konge L et al (2016) Cognitive load in mastoidectomy skills training: virtual reality simulation and traditional dissection compared. J Surg Educ 73:45–50 Anderson SI, Hennessy CM, Matthan J et al (2021) Guidance for institutions/anatomists when utilising cadaveric material for online teaching sessions. In: Oral presentation at trans european pedagogic anatomical research group conference, 2021 Arráez-Aybar LA, Castaño-Collado G, Casado-Morales MI (2008) Dissection as a modulator of emotional attitudes and reactions of future health professionals. Med Educ 42:563–571 Attardi SM, Gould DJ, Pratt RL et al (2021) YouTubebased course orientation videos delivered prior to matriculation fail to alleviate medical student anxiety about anatomy. Anat Sci Ed. https://doi.org/10.1002/ ase.2107 Baddeley A (1992) Working memory. Science 255:556– 559 Bandura A, Barbaranelli C, Caprara GV et al (1996) Multifaceted impact of self-efficacy beliefs on academic functioning. Child Dev 67:1206–1222 Barry DS, Marzouk F, Chulak-Oglu K et al (2016) Anatomy education for the YouTube generation. Anat Sci Educ 9:90–96 Biggs J (1996) Enhancing teaching through constructive alignment. High Educ 32:347–364 Calkins CM, Franciosi JP, Kolesari GL (1999) Human anatomical science and illustration: the origin of two inseparable disciplines. Clin Anat 12:120–129 Casado MI, Castano G, Arraez-Aybar LA (2012) Audiovisual material as educational innovation strategy to

12

Teaching with Cadavers Outside of the Dissection Room Using Cadaveric Videos

reduce anxiety response in students of human anatomy. Adv Health Sci Educ Theory Pract 17:431–440 Chandler P, Sweller J (1991) Cognitive load theory and the format of instruction. Cogn Instr 8(4):293–332 Chandler P, Sweller J (1996) Cognitive load while learning to use a computer program. Appl Cogn Psychol 10:151–170 Choi-Lundberg DL, Cuellar WA, Williams AM (2016) Online dissection audio-visual resources for human anatomy: undergraduate medical students’ usage and learning outcomes. Anat Sci Educ 9:545–554 Chytas D (2019) Use of social media in anatomy education: a narrative review of the literature. Ann Anat 221: 165–172 Collins AM, Quinlan CS, Dolan RT et al (2015) Audiovisual preconditioning enhances the efficacy of an anatomical dissection course: a randomised study. J Plast Reconstr Aesthet Surg 68:1010–1015 Dilullo C, Coughlin P, D’angelo M et al (2006) Anatomy in a new curriculum: facilitating the learning of gross anatomy using web access streaming dissection videos. J Vis Commun Med 29:99–108 Dinsmore CE, Daugherty S, Zeitz HJ (2001) Student responses to the gross anatomy laboratory in a medical curriculum. Clin Anat 14:231–236 Drake RL, Mcbride JM, Lachman N et al (2009) Medical education in the anatomical sciences: the winds of change continue to blow. Anat Sci Educ 2:253–259 Drake RL, Mcbride JM, Pawlina W (2014) An update on the status of anatomical sciences education in United States medical schools. Anat Sci Educ 7:321–325 D’ydewalle G, Van De Poel M (1999) Incidental foreignlanguage acquisition by children watching subtitled television programs. J Psycholinguistic Res 28:227– 244 Ellaway RH, Coral J, Topps D et al (2015) Exploring digital professionalism. Med Teach 37(9):844–849 Evans DJ, Bay BH, Wilson TD et al (2020) Going virtual to support anatomy education: a STOPGAP in the midst of the Covid-19 pandemic. Anat Sci Educ 13 (3):279–283 Farsides T, Smith CF (2020) Consent in body donation. Eur J Anat 24:239–248 Finkelstein P, Mathers LH (1990) Post-traumatic stress among medical students in the anatomy dissection laboratory. Clin Anat 3:219–226 Flack NA, Nicholson HD (2018) What do medical students learn from dissection? Anat Sci Educ 11: 325–335 Fraser K, Ma I, Teteris E et al (2012) Emotion, cognitive load and learning outcomes during simulation training. Med Educ 46:1055–1062 Gomes AW, Butera G, Chretien KC et al (2017) The development and impact of a social media and professionalism course for medical students. Teach Learn Med 29(3):296–303 Goss AL, Viswanathan VB, Delisser HM (2019) Not just a specimen: a qualitative study of emotion, morality, and

295

professionalism in one medical school gross anatomy laboratory. Anat Sci Educ 12:349–359 GOV.UK (2021) Staying at home and away from others (social distancing) https://www.gov.uk/government/ publications/full-guidance-on-staying-at-home-andaway-from-others/full-guidance-on-staying-at-homeand-away-from-others. Accessed 28 Mar 2021 Granger NA, Calleson D (2007) The impact of alternating dissection on student performance in a medical anatomy course: are dissection videos an effective substitute for actual dissection? Clin Anat 20:315–321 Greene SJ (2020) The use of anatomical dissection videos in medical education. Anat Sci Educ 13:48–58 Hennessy CM, Smith CF (2020) Digital and social media in anatomy education. In: Rea PM (ed) Biomedical visualisation, 8th edn. Springer, pp 109–122 Hennessy CM, Royer DF, Meyer AJ et al (2020) Social media guidelines for anatomists. Anat Sci Educ 13: 527–539 HTA (2021) Human tissue authority. Donating your body. Human Tissue Authority, London. https://www.hta. gov.uk/guidance-public/body-donation. Accessed 23 June 2021 Hulme A, Strkalj G (2017) Videos in anatomy education: history, present usage and future prospects. J Morphol 35(4):1540–1546 Jaffar AA (2012) YouTube: an emerging tool in anatomy education. Anat Sci Educ 5:158–164 Jones DG (2019) Three-dimensional printing in anatomy education: assessing potential ethical dimensions. Anat Sci Educ 12(4):435–443 Junco R, Elavsky CM, Heiberger G (2013) Putting twitter to the test: assessing outcomes for student collaboration, engagement and success. Br J Educ Technol 44 (2):273–287 Kalyuga S, Sweller J (2014) 10 the redundancy principle in multimedia learning. In: The Cambridge handbook of multimedia learning, 2nd edn. Cambridge University Press, pp 247–262 Khalil MK, Paas F, Johnson TE et al (2005) Interactive and dynamic visualizations in teaching and learning of anatomy: a cognitive load perspective. Anat Rec 286B:8–14 Koop CF, Marschollek M, Schmiedl A et al (2020) Does an audiovisual dissection manual improve medical students’ learning in the gross anatomy dissection course? Anat Sci Educ. https://doi.org/10.1002/ase. 2012 Kruger JL, Hefer E, Matthew G (2013) Measuring the impact of subtitles on cognitive load: eye tracking and dynamic audiovisual texts. In: Proceedings of the 2013 conference on eye tracking, South Africa Küçük S, Kapakin S, Göktaş Y (2016) Learning anatomy via mobile augmented reality: effects on achievement and cognitive load. Anat Sci Educ 9:411–421 Kumar Ghosh S, Kumar A (2019) Building professionalism in human dissection room as a component of hidden curriculum delivery: a systematic review of good practice. Anat Sci Educ 12:210–221

296 Langfield T, Colthorpe K, Ainscough L (2018) Online instructional anatomy videos: student usage, selfefficacy, and performance in upper limb regional anatomy assessment. Anat Sci Educ 11:461–470 Lavaur JM, Bairstow D (2011) Languages on the screen: is film comprehension related to the viewers’ fluency level and to the language in the subtitles? Int J of Psychol 46:455–462 Leppink J, Duvivier R (2016) Twelve tips for medical curriculum design from a cognitive load theory perspective. Med Teach 38:669–674 Leppink J, Van Den Heuvel A (2015) The evolution of cognitive load theory and its application to medical education. Perspect Med Educ 4:119–127 Longhurst GJ, Stone DM, Dulohery K et al (2020) Strength, weakness, opportunity, threat (SWOT) analysis of the adaptations to anatomical education in the United Kingdom and Republic of Ireland in response to the Covid-19 pandemic. Anat Sci Educ 13:301–311 Losco CD, Grant WD, Armson A et al (2017) Effective methods of teaching and learning in anatomy as a basic science: a BEME systematic review: BEME guide no. 44. Med Teach 39:234–243 Mahmud W, Hyder O, Butt J et al (2011) Dissection videos do not improve anatomy examination scores. Anat Sci Educ 4:16–21 Mayer RE, Fiorella L (2014) 12 principles for reducing extraneous processing in multimedia learning: coherence, signaling, redundancy, spatial contiguity, and temporal contiguity principles. In: Mayer R (ed) The Cambridge handbook of multimedia learning, 279 edn. Cambridge University Press, pp 279–315 Mayer RE, Moreno R (2003) Nine ways to reduce cognitive load in multimedia learning. Educ Psychol 38:43– 52 Mostaghimi A, Olszewski AE, Bell SK et al (2017) Erosion of digital professionalism during medical students’ core clinical clerkships. JMIR Med Educ 3 (1):e6879 Mousavi SY, Low R, Sweller J (1995) Reducing cognitive load by mixing auditory and visual presentation modes. Educ Psychol 87:319 Nation H, Kaliski D, Ortiz A (2020) Narrated dissection videos and peer-mentoring to enhance anatomy performance of underrepresented minority students in physical therapy education. Anat Sci Educ 13:794–799 Ogunranti J (1987) Video technology in integrated anatomy education. J Educ Telev 13:63–67 Paivio A (1991) Dual coding theory: retrospect and current status. Can J Psychol 45:255 Pather N, Blyth P, Chapman JA et al (2020) Forced disruption of anatomy education in Australia and New Zealand: an acute response to the Covid-19 pandemic. Anat Sci Educ 13:284–300 Penney CG (1989) Modality effects and the structure of short-term verbal memory. Mem Cogn 17:398–422

D. Stone et al. Pickering JD, Bickerdike SR (2017) Medical student use of Facebook to support preparation for anatomy assessments. Anat Sci Educ 10(3):205–214 Plass JL, Kalyuga S (2019) Four ways of considering emotion in cognitive load theory. Educ Psychol Rev 31:339–359 Public Sector Bodies (2018) Public Sector Bodies (Websites and Mobile Applications) (No 2) Accessibility Regulations. http://www.legislation.gov.uk/uksi/ 2018/952/contents/made. Accessed 3 June 2021 Rai R, Shereen R, Protas M et al (2019) Social media and cadaveric dissection: a survey study. Clin Anat 32: 1033–1041 Raikos A, Waidyasekara P (2014) How useful is YouTube in learning heart anatomy? Anat Sci Educ 7(1):12–18 Saxena V, Natarajan P, O’sullivan PS et al (2008) Effect of the use of instructional anatomy videos on student performance. Anat Sci Educ 1:159–165 Schmitt TL, Sims-Giddens SS, Booth RG (2012) Social media use in nursing education. Online J Issues Nurs 17(3):1–15 Smith CF, Tollemache N, Covill D et al (2018) Take away body parts! An investigation into the use of 3D-printed anatomical models in undergraduate anatomy education. Anat Sci Educ 11:44–53 Sweller J (1988) Cognitive load during problem solving: effects on learning. Cogn Sci 12:257–285 Sweller J (2005) The redundancy principle in multimedia learning. In: Mayer R (ed) The Cambridge handbook of multimedia learning, pp 159–167 Sweller J, Chandler P (1994) Why some material is difficult to learn. Cognition Instruct 12:185–233 Sweller J, Van Merrienboer JJG, Pass FGWC (1998) Cognitive architecture and instructional design. Educ Psychol Rev 10:251–296 Theoret CL, Carmel É-N, Bernier S (2007) Why dissection videos should not replace cadaver prosections in the gross veterinary anatomy curriculum: results from a comparative study. J Vet Med Educ 34:151–156 Topping DB (2014) Gross anatomy videos: student satisfaction, usage, and effect on student performance in a condensed curriculum. Anat Sci Educ 7:273–279 Trelease RB (2008) Diffusion of innovations: smartphones and wireless anatomy learning resources. Anat Sci Educ 1:233–239 Trelease RB (2016) From chalkboard, slides, and paper to e-learning: how computing technologies have transformed anatomical sciences education. Anat Sci Educ 9:583–602 Vorstenbosch MA, Klaassen TP, Donders A et al (2013) Learning anatomy enhances spatial ability. Anat Sci Educ 6:257–262 Vygotsky LS (1980) Mind in society: the development of higher psychological processes. Harvard University Press

12

Teaching with Cadavers Outside of the Dissection Room Using Cadaveric Videos

Wang E, Myers MD, Sundaram D (2012) Digital natives and digital immigrants: towards a model of digital fluency. Bus Inf Syst Eng 5(6):409–419 White GK (2013) Digital fluency: skills necessary for learning in the digital age. https://research.acer.edu. au/digital_learning/6. Accessed 24 June 2021 WHO (2020) WHO Director-General’s opening remarks at the media briefing on COVID-19. https://www.who.

297

int/director-general/speeches/detail/who-director-gen eral-s-opening-remarks-at-the-media-briefing-oncovid-19%2D%2D-11-march-2020. Accessed 21 Mar 2021 Yang HY (2014) The effects of advance organizers and subtitles on EFL learners’ listening comprehension skills. CALICO J 31:345–373

A Novel Cadaveric Embalming Technique for Enhancing Visualisation of Human Anatomy

13

Brian Thompson, Emily Green, Kayleigh Scotcher, and Iain D. Keenan

Abstract

Within the discipline of anatomical education, the use of donated human cadavers in laboratory-based learning activities is often described as the ‘gold standard’ resource for supporting student understanding of anatomy. Due to both historical and educational factors, cadaveric dissection has traditionally been the approach against which other anatomy learning modalities and resources have been judged. To prepare human donors for teaching purposes, bodies must be embalmed with fixative agents to preserve the tissues. Embalmed cadavers can then be dissected by students or can be prosected or plastinated to produce teaching resources. Here, we describe the history of cadaveric preservation in anatomy education and review the practical strengths and limitations of current approaches for the embalming of human bodies. Furthermore, we investigate the pedagogic benefits of a

range of established modern embalming techniques. We describe relevant cadaveric attributes and their impacts on learning, including the importance of colour, texture, smell, and joint mobility. We also explore the emotional and humanistic elements of the use of human donors in anatomy education, and the relative impact of these factors when alternative types of embalming process are performed. Based on these underpinnings, we provide a technical description of our modern Newcastle-WhitWell embalming process. In doing so, we aim to inform anatomy educators and technical staff seeking to embalm human donors rapidly and safely and at reduced costs, while enhancing visual and haptic tissue characteristics. We propose that our technique has logistical and pedagogic implications, both for the development of embalming techniques and for student visualisation and learning. Keywords

B. Thompson School of Medical Education, Newcastle University, Newcastle upon Tyne, UK School of Medicine, University of Sunderland, Sunderland, UK e-mail: [email protected] E. Green · K. Scotcher · I. D. Keenan (*) School of Medical Education, Newcastle University, Newcastle upon Tyne, UK e-mail: [email protected]; kayleigh. [email protected]; [email protected]

Anatomy education · Human embalming · Cadaveric dissection · Visual learning · Haptic learning

13.1

History of Embalming

Ensuring the safe and effective preservation of human cadaveric material is crucial to the usage of donor specimens in anatomy education (Balta

# The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 P. M. Rea (ed.), Biomedical Visualisation, Advances in Experimental Medicine and Biology 1356, https://doi.org/10.1007/978-3-030-87779-8_13

299

300

et al. 2015a). Embalming is performed with a view to limiting the decomposition of tissues while maintaining the physical appearance of the cadaver (Brenner 2014). In this respect, modern anatomical embalming procedures are performed with a similar purpose to ancient methods of human body preservation. The earliest historical instances of embalming typically arose from religious or cultural beliefs however, rather than because of a need to preserve cadavers as anatomy education resources, or for anatomical examination and research. A notable example of classical embalming approaches concerns human mummification. While the most wellknown examples of mummification may be those occurring in ancient Egypt, this technique of human body preservation may have been first carried out by the South American Chinchorro culture in 6000 B.C.E. (Brenner 2014; Marquet et al. 2012). Egyptian mummification is likely to have been originally performed more recently, with archaeological explorations suggesting that such procedures began to emerge in Egypt in around 3000 B.C.E. (Brenner 2014; Abayomi and Odiri 2017). The usage of embalming for religious purposes in Egypt continued during the Roman Period between 27 B.C.E. and 476 A.D., and consequently spread across Europe. Subsequently, processes of treating and artificially preserving the deceased persisted into the Middle Ages. During this time, human embalming commonly involved evisceration, immersion of the body in alcohol and the use of preservative herbs (Brenner 2014). Cadaveric embalming and dissection for the purposes for anatomical education, as opposed to preservation for religious regions, began in earnest throughout Europe during the Renaissance period of the fifteenth and sixteenth centuries. Such advancements involved the development of various embalming techniques to preserve cadavers for gross anatomy teaching (Srivastava and Nagwani 2019). The importance of anatomy and human dissection during medical training was strongly advocated by practitioners (Ghosh 2015), and cadaveric dissection became the defining experience of medical education throughout the sixteenth and seventeenth

B. Thompson et al.

centuries (McLachlan and Patten 2006). In turn, the widespread popularity of teaching with human specimens drove the scientific development of improved embalming techniques. Renaissance anatomists including Leonardo Da Vinci attempted the embalming of human cadavers through injection of waxes or inks into hollow anatomical structures and cavities, such as the ventricles of the heart (Brenner 2014). However, embalming by arterial injection, a technique that is currently in widespread usage within modern preservation procedures, was not popularised until the discovery of the circulatory system by William Harvey in the early seventeenth century (Ribatti 2009). Consequently, an understanding of the vasculature enabled William Hunter to become the first anatomist to report successful use of arterial injection for embalming during the middle years of the eighteenth century (Ajileye et al. 2018). A methodological publication authored by Thomas Pole in the late eighteenth century, ‘The Anatomical Instructor’ (Pole 1790), described in detail several recommended contemporary techniques for producing human specimens for teaching students of anatomy. Such procedures included the drying of specimens, the injection of coloured wax or mercury, and the preservation of soft tissues and visceral organs using resins, balsams, oils, turpentine, spirits of wine, and varnishes (Mitchell et al. 2011). Despite improvements in the preservation of cadavers, attempts to apply antiseptics during embalming processes were either absent or severely limited, and gloves were seldom worn during the dissection procedure. The absence of even rudimentary health and safety considerations resulted in many deaths of students and anatomical educators due to infections acquired from the cadaveric specimens they had prepared and studied (Shoja et al. 2013). Limitations in adequate cadaveric preservation also partially contributed to the rising demand for human bodies for dissection throughout the eighteenth and nineteenth centuries. In turn, the need for human specimens for use in medical education led to practices of grave robbing, body snatching and as in the famous case of Burke and Hare in 1828, even murder (Ghosh 2015).

13

A Novel Cadaveric Embalming Technique for Enhancing Visualisation of Human Anatomy

The development of the modern embalming procedure is widely accredited to Dr. Thomas Holmes, who vastly improved cadaver preservation techniques during the 1861 American Civil War (Ajileye et al. 2018). The most effective preserving agents of the time were simultaneously developed for use in both funeral embalming and in anatomical education during the late nineteenth century. Embalming fluids primarily consisted of alcoholic solutions containing arsenic until the introduction of phenol for anatomical embalming in 1886 (Brenner 2014). Crucially, this period coincided with the discovery and increased availability of formaldehyde, which was originally identified in 1859 by Alexander Butlerov. Practical manufacture of this naturally occurring compound was developed and established soon after by Wilhelm van Hoffman in 1868 (Fox et al. 1985). The subsequent introduction of formaldehyde in 1893 as a fixative was an important step in the history of preservation and has formed the foundation of modern practices in the embalming of human cadavers for anatomical education. Thereafter, ongoing modification and improvement to formaldehydebased embalming solutions continued throughout the twentieth century (Balta et al. 2015a), which was later accompanied by the introduction of Thiel embalming (Thiel 1992, 2002). Although not strictly by definition embalming, and therefore beyond the scope of this review, a further notable development was the invention of cadaver plastination by Gunter von Hagens (Von Hagens et al. 1987; Fruhstorfer et al. 2011), a relatively expensive and time-consuming process of tissue preservation that is achieved through the replacement of fluids with specific types of polymer.

13.2

Modern Approaches to Cadaveric Preservation

A variety of modern anatomy learning resources, including art-based and technology-enhanced learning approaches, are now available to supplement the use of cadaveric specimens, for supporting comprehensive and detailed

301

anatomical understanding (Keenan et al. 2017; Backhouse et al. 2017; Keenan and Ben Awadh 2019). The availability of such alternative approaches, in addition to financial and logistical barriers to cadaveric provision and ethical considerations of body donation, constitute arguments opposing the usage of human specimens in anatomy education (Guttmann et al. 2004; Mclachlan 2004; Mclachlan et al. 2004). Nonetheless, the significant educational advantages of embedding cadaveric anatomy within curricula have been comprehensively described (Dinsmore et al. 1999; Dyer and Thorndike 2000; Korf et al. 2008; Sugand et al. 2010). Benefits include humanistic (Getachew 2014) and pedagogic factors (Lempp 2005; Winkelmann et al. 2007), in addition to the surgical and fine motor skills that can be acquired from performing anatomical dissection (Liu et al. 2015). Such advantages have resulted in descriptions of cadavers as the ‘gold standard’ resource in anatomy education (Chytas et al. 2020). Techniques for preserving the deceased have altered extensively since the early methods of embalming. This is likely due to the development of modern artificial approaches having been influenced by contemporary demands beyond the need to prolong the integrity of human tissue for extended periods of time. Current determinants of anatomical embalming are driven by requirements for enhancing the identification and examination of anatomical structures, and for increasing the lifelike realism of cadaveric specimens for the teaching of anatomy (Balta et al. 2015a). Simultaneously, modern anatomical embalmers have obligations and responsibilities in mitigating concerns regarding formaldehyde and formalin toxicity and resulting impacts on the health of anatomy students, educators, and technical staff (Elshaer and Mahmoud 2017). Such wider considerations have led to the ongoing development and improvement of embalming formulas. These modifications have been implemented with a view to supporting the effective preservation of human tissue while simultaneously considering important financial, logistical and safety concerns, and pedagogically

302

B. Thompson et al.

relevant factors such as colour retention and flexibility in human tissue (Balta et al. 2019; Crosado et al. 2020). Furthermore, different institutions have distinct traditions, teaching approaches and logistical requirements, resulting in varied approaches to the preservation of human material. Consequently, descriptions of specific and modified embalming fluids, comparisons between alternative techniques, and their respective benefits and impacts on anatomical education, are not uncommon in the literature (Balta et al. 2015b, 2019; Ajileye et al. 2018; Coleman and Kogan 1998; Crosado et al. 2020; Eisma et al. 2013; Hammer et al. 2012; Hunter et al. 2014; Kennel et al. 2018; Miyake et al. 2020; O’Sullivan and Mitchell 1993).

13.2.1

Phenol

Phenol was one of the first major preserving agents to be introduced for the purposes of cadaveric preservation and has been used consistently, and in varying quantities, within multiple distinct types of embalming mixture (Haizuka et al. 2018; Benet et al. 2014; Coleman and Kogan 1998; Tomalty et al. 2019; Noël 2020). In the late twentieth century, the embalming methods of 16 higher education institutions within the United Kingdom were explored (O’Sullivan and Mitchell 1993). While similarities were identified in the chemical composition of embalming fluids, there was typically a disparity in the proportions of each component used. However, the embalming solutions of all participating medical schools contained comparable proportions of phenol, due to the antifungal and disinfectant properties of the compound. Despite providing these benefits to the embalming process however, phenol does present toxicity and fire safety concerns, which must be taken into consideration when preparing cadavers for anatomical education (Dikshith 2010; Noël 2020).

13.2.2

Formaldehyde

Conventional embalming techniques commonly involve an embalming fluid with a component of

formalin, the aqueous form of formaldehyde, as a tissue fixative. Typically, formalin embalming solutions are injected through the internal carotid or femoral arteries, allowing widespread perfusion and fixation of human tissue. Formalinbased approaches are likely to have become established within higher education institutions due to the rapid and long-lasting preservation properties of the compound, in addition to widespread availability and affordability. Formalin is particularly beneficial in circumstances requiring cadavers to be dissected for the duration of an anatomy programme, or for long-term usage of prosections, when tissue must be preserved for periods from several months up to many years (Balta et al. 2015a, b, 2017, 2019). Formaldehyde or formalin-based embalming are described as hard-fix techniques because tissues and joints do not retain their original lifelike flexibility and mobility. Range of motion is often used as a measurement during the evaluation of embalming approaches, with reductions in motion commonly understood to be a technical limitation. Formaldehyde embalmed cadavers have demonstrated significantly lower levels of joint mobility when compared to alternative approaches, in addition to displaying alterations in colour (Balta et al. 2019). Furthermore, formaldehyde produces a strong odour within anatomical dissection laboratories, which can cause optical and nasal irritation, and may impact upon student concentration levels. Consequently, there are concerns with respect to the provision of a safe and effective learning environment when working with embalmed bodies, which in turn requires the incorporation of appropriate ventilation within anatomy facilities (Bhat et al. 2019; Fischer 1905; Elshaer and Mahmoud 2017).

13.2.3

Thiel

The thirty-year project to produce a soft-fix, low-odour embalming solution led by Walter Thiel (Thiel 1992, 2002) has been developed to improve upon the educational benefits of traditional formaldehyde-based techniques, while reducing staff and student exposure to toxic

13

A Novel Cadaveric Embalming Technique for Enhancing Visualisation of Human Anatomy

compounds. In the United Kingdom, this embalming method has been adopted at the University of Dundee (Eisma et al. 2013), where Thiel has also been used to enhance the characteristics of previously formalin-fixed prosected specimens (Hunter et al. 2014). In contrast to hard-fixation, Thiel embalming enables the lifelike visual appearance, texture, and colouration of tissues, and the flexibility and range of motion of cadaveric joints to be retained post-preservation (Balta et al. 2015b; Abayomi and Odiri 2017). Such characteristics can be beneficial for the study of musculoskeletal structures and when students require an understanding of functional anatomy. In soft-fixed specimens, the actions of specific muscles or muscle groups on certain joints can be directly observed. In contrast, the usage of hard-fixed cadavers requires some imagination on the part of students to be able to visualise muscle function. Furthermore, Thiel provides a more realistic alternative to synthetic or animal specimens, an important consideration with respect to the safety of future surgical patients. Specifically, the qualities of Thielembalmed tissue enable the provision of effective soft-fixed cadaveric resources for use in surgical training procedures including microvascular suturing and endoscopic surgery (Yiasemidou et al. 2017; Balta et al. 2015b; Veys et al. 2020). Despite the benefits that this embalming method can provide for anatomical education, the Thiel method is not widely used (Benkhadra et al. 2011; Balta et al. 2015a). This is likely due to the necessary prerequisites of storage space, equipment, and skilled staff, in addition to the time and costs involved in producing Thielembalmed cadavers. The Thiel technical protocol involves arterial intravascular perfusion of the femoral or brachial arteries, and via the venous system through the superior sagittal sinus or brachial vein, followed by submersion for 4–6 months. While this length of time may constitute a practical obstacle, this aspect of the process can be advantageous when it is extended to enable long term storage of submerged cadavers without the need for refrigeration (Eisma and Wilkinson 2014). However, limitations in the Thiel embalming process can

303

arise during the study of soft tissues, including the brain and abdominal organs. Soft-fixed specimens can degrade, while reductions in rigidity can negatively impact upon student identification of key anatomical structures when holding and examining isolated viscera (Balta et al. 2019). To counteract this, the use of Thiel embalming combined with intra-cerebral ventricular formalin injection has been developed to preserve brain tissue for surgical training (Miyake et al. 2020). Nonetheless, a study comparing Thiel and phenol-based soft embalmed cadavers has identified that the preservation of histological specimens was less effective in Thiel-embalmed tissues (Venne et al. 2020).

13.2.4

Alternative Fixatives

Further methods of preservation have been introduced for anatomical education, with the aim of enhancing the embalming process and achieving more desirable results when compared to traditional formaldehyde-based approaches. For example, the Crosado embalming technique used at the University of Otago in New Zealand contains the antifungal agent phenoxyethanol as the key ingredient (Crosado et al. 2020). Crosado embalmed tissues have been received positively by anatomists in comparison to conventional techniques, due to the odourless, durable, and visual characteristics produced (Crosado et al. 2020). While this approach can incur greater costs than formaldehyde and ethanol-based preservation techniques, Crosado embalming is relatively inexpensive when compared to Thiel embalming (Crosado et al. 2020; Hammer et al. 2012, 2015). An ethanol-glycerin fixation approach with post-fix thymol conservation has been developed as a low toxicity method for generating flexibility in embalmed muscles (Hammer et al. 2012). However, the widespread adoption of this approach is significantly limited by the relatively high costs involved, including the requirement of an explosion-proof facility. Genelyn embalming solution, which contains methanol and formaldehyde but no phenol, (Jaung et al. 2011), has been

304

B. Thompson et al.

shown to produce relatively reduced ranges of motion in embalmed cadavers when compared to Thiel and formalin (Balta et al. 2019), and specific limitations have been identified in the characteristics of Genelyn-fixed tissues when used in the development in 3D and 4D ultrasound approaches for epidural catheter insertion (Belavy et al. 2011). The Imperial College London softembalming technique was first reported as a novel technique for gynaecological surgical training (Barton et al. 2009), and has been favourably compared to Thiel, Genelyn and formalin methods in terms of ranges of joint motions and the dimensions of in embalmed viscera (Balta et al. 2019).

13.2.5

Fresh-Frozen Preservation

Although not an embalming approach per se, the fresh-frozen preparation of human tissue is a notable cadaveric preservation technique, which is achieved by the freezing of human donor bodies shortly after death (Doomernik et al. 2016). Fresh-frozen cadavers can then be thawed at room temperature (or below) before use (Doomernik et al. 2016; Jansen et al. 2020), a process which can take only 4–5 days and can be repeated up to three four times (Chai et al. 2019; Jansen et al. 2020). While the realistic tissues produced in fresh-frozen cadaveric preparations are often used for surgical training (Reed et al. 2009), notable limitations with respect to freezer space, time constraints, and the specific duration of each surgical course, can mean that such methods may not always be feasible. In an attempt to counteract such drawbacks, commercially available embalming agents including ESCO (Embalmer’s Supply Company, Ontario, Canada) have been used to extend the life of fresh tissues for surgical training (Messmer et al. 2010). Further considerations of the fresh-frozen approach and the use of human cadaveric specimens in general are particularly significant within the context of the current Covid-19 pandemic. The management of infection risks in the deceased (UK Health and Safety Executive 2018), potential increases in such risks in the

absence of embalming fluids (UK Government 2021), required donor consent for post-mortem Covid-19 testing (Human Tissue Authority 2021), and the combined impact of Covid-19 on body donation programmes (Human Tissue Authority 2021), may result in the reduced availability of human cadavers for anatomical and surgical education in the medium and longer term.

13.3

Learning and Teaching with Cadavers

Human specimens constitute a realistic threedimensional resource for the learning and teaching of anatomy. The layered arrangement of the human body from the superficial integumentary system through deeper fascial and muscular layers; the size, shape, position, and anatomical relations of visceral organs; the structural relationships of the musculoskeletal system; and the course and supply of neurovascular structures; can all be observed in three-dimensions within individual human donor bodies. Furthermore, many common and clinically relevant anatomical variations and notable pathologies can be observed in realistic detail, and such elements can be compared when multiple cadavers are examined. Because the visual appearance and texture of these features can be influenced by the type of embalming technique that is used, pedagogic implications can vary depending on the choice of preservation method. Therefore, we explore not only the factors that determine successful curricular integration of cadaveric anatomy, but also the specific elements of cognition and anatomy learning that may be produced from usage of distinct embalming methods.

13.3.1

Curricular Integration

In many contemporary higher education institutions, the use of human dissection or cadaveric prosection is viewed as a fundamental and essential element in the provision of anatomical education (Ghosh 2015, 2017; Sugand et al.

13

A Novel Cadaveric Embalming Technique for Enhancing Visualisation of Human Anatomy

2010; Flack and Nicholson 2018). Through the examination, dissection and observation of embalmed cadavers and prosected specimens, students are provided with opportunities for comprehensively visualising essential concepts and structures in human gross anatomy. Established traditional and cultural influences on the usage of human cadavers in medical schools are likely to be major drivers in modern anatomy educational practice. Differing ideological perspectives surrounding the pedagogic strengths and limitations of human specimens, and conflicting views regarding the respective value of the usage of cadaveric material for dissection versus prosection, have been debated comprehensively within the anatomy education literature (Ghosh 2017; Guttmann et al. 2004; Mclachlan et al. 2004; Sheikh et al. 2016; Winkelmann 2007; Wilson et al. 2018; Topp 2004; Granger 2004; Pawlina and Lachman 2004). Further debate has surrounded the integration of technologyenhanced learning approaches within anatomy curricula to supplement or even replace cadaveric anatomy, a conversation which has increased at pace during the last two decades (Moore et al. 2017; Sheikh et al. 2016; Miller 2016; Slon et al. 2014; Biasutto et al. 2006; Fasel et al. 2016; Aziz et al. 2002; Keenan and Ben Awadh 2019; Saltarelli et al. 2014). Moreover, the decisions of individual institutions when choosing and maintaining preferred embalming techniques may have pragmatic rather than scholarly underpinnings and may be determined by contextual factors relating to technical skills, programme curricula, and anatomy teaching approaches (Balta et al. 2015a). While such differences exist, cross-institutional transparency and the discussion and sharing of best practice with respect to embalming approaches is likely to provide valuable insights for anatomy educators and technicians. A survey-based study seeking the perspectives of anatomists in the United Kingdom and Ireland has identified educator insights with respect to the use of preserved cadavers in anatomy teaching (Balta et al. 2017). Respondents indicated that

305

soft-fixed embalming methods provided the most accurate representations of anatomical structures and proposed that realistic learning resources could enhance student retention of anatomy knowledge. Nonetheless, it is important to note that the subjective educator perceptions obtained in this study were not directly based upon data identifying the perspectives and performance of students pursuing anatomy programmes (Balta et al. 2017). Reported comparisons of softfixed and hard-fixed embalming techniques have been performed, typically through the observation of colour changes, rigidity of viscera, the ability to dissect certain structures, the range of motion of joints, the changes of dimensions of viscera and ability to retain the shape of anatomical structures (Eisma et al. 2011; Balta et al. 2015b, 2019; Kennel et al. 2018). While such studies have differentiated specific embalming techniques in terms of tissue fixation and appearance, direct comparisons of educational value are limited. For example, students who experienced Thiel-embalmed cadavers have been shown to perceive greater levels of confidence in the identification of anatomical structures in living individuals (Kennel et al. 2018). However, there were no significant differences in functional anatomy knowledge retention observed when students used soft-fix Thiel-embalmed specimens to learn anatomy, when compared a cohort that utilised hard-fix formalin embalmed cadavers (Kennel et al. 2018). This finding suggests that although there may be disparity in the characteristics and student perceptions of distinct embalming methods, the impact on student learning may be minimal. Consequently, there is broad scope for the implementation of larger scale research to determine the pedagogic value of specific embalming methods. In the absence of such studies, relationships between established pedagogic principles and the key physical characteristics of embalmed bodies can be identified and described, with a view to providing a basis for selecting effective preservation techniques when preparing human cadavers for anatomy education.

306

13.3.2

B. Thompson et al.

Visualisation, Sensation and Emotion

We have previously outlined our approach to the concept of visualisation within the context of anatomical education (Keenan and Ben Awadh 2019). Our understanding concerns the existence of visualisation as an important element of the anatomy learning process, which comprises experiences of cognitively creating mental images that represent three-dimensional anatomical structures. Furthermore, we concur with previous work that has defined visualisation as a process which can be effectively achieved via the development and application of student spatial and observational skills (Pandey and Zimitat 2007). Additionally, there is an increasing number of studies within the anatomical education literature that provide an evidence-base supporting the principle that the study of anatomy, and more specifically the use of cadaveric dissection, can improve the mental-rotation elements of medical student visuo-spatial abilities (Bogomolova et al. 2020; Vorstenbosch et al. 2013). In turn, a reciprocal relationship may exist, with higher levels of visuo-spatial abilities resulting in the enhancement of anatomy learning and assessment performance (Langlois et al. 2017; Lufler et al. 2012; Guillot et al. 2007; Sweeney et al. 2014). Despite the extent and nature of these findings, we have been unable to reproduce these observations from our own experiences and research, having failed to show any strong correlations between spatial abilities and anatomy learning performance within our own context, when considering both cadaveric and digital anatomy (Ben Awadh et al. 2021). Nonetheless, the development and application of student skills in visualisation and visual observation gained from the use of human cadaveric specimens may provide significant advantages supporting the use of these physical, threedimensional resources. We have identified in our earlier work (Keenan and Powell 2020; Ben Awadh et al. 2021; Keenan and Ben Awadh 2019), that the modality appropriateness model (Lodge et al. 2016) provides an

effective basis for understanding and integrating relevant learning approaches in anatomy education. In essence, this hypothesis supports an emphasis on the provision of three-dimensional and visual approaches to anatomy learning, due to the fundamental nature of anatomy as a visual and three-dimensional discipline. While we have previously considered the importance of visualisation and visual observational modalities in terms of educational technologies (Ben Awadh et al. 2021; Keenan and Ben Awadh 2019) and art-based learning approaches (Keenan and Powell 2020; Shapiro et al. 2019), such principles can be similarly and effectively applied to optimising the development of human cadaveric resources. Anecdotally, an important facet of the visualisation process in anatomy learning is likely to be the presence and variety of colours presented within anatomical images. Building on the modality appropriateness hypothesis (Lodge et al. 2016), it can be proposed that because the pigmentation of living human tissues exist within a range of visible hues, that the use of colour is relevant and necessary for learning with cadaveric anatomy education resources. Indeed, perceptions of the importance of colour in the anatomy learning process are supported by the popularity of anatomy education textbooks and resources, which often emphasise the presentation of anatomical images as colour figures (Logan and Hutchings 2011; Gartner and Hiatt 2012; Gosling et al. 2016), or provide specific anatomy colouring activities to support learning (Fehrenbach 2013; Hansen 2010; Mccann and Wise 2017; Mcconnell 2012). As described in detail elsewhere, there is substantial evidence supporting the importance of colour in learning, memory, and behaviour (Dzulkifli and Mustafar 2013). Additionally, several research studies have identified that the use of coloured text (Hall and Hanna 2004), coloured multimedia slide presentations (Farley and Grant 1976) and coloured objects (Vernon and LloydJones 2003; Lloyd-Jones and Nakabayashi 2009) can enhance identification, memory performance and knowledge retention when compared to

13

A Novel Cadaveric Embalming Technique for Enhancing Visualisation of Human Anatomy

equivalent monochrome controls. Furthermore, support for the significance of colour with respect to learning and student engagement has been presented in terms of synesthetic (Smilek et al. 2002) and environmental (Spence et al. 2006; Greene et al. 1983) experiences of colour. However, there is a notable scarcity of anatomical education studies that have investigated the role of colour in anatomy learning. Moreover, in one study that has considered this element, limited influences of colour on anatomical understanding, and no significant impacts of colour on the knowledge retention of undergraduate anatomy students, were identified (Finn et al. 2011). Nonetheless, this single study specifically considered and evaluated the use of colour in body painting activities, rather than cadaveric anatomy learning resources. Therefore, the direct relevance and application of these findings to the impact of colouration in human specimens, and to anatomy education more widely, should be considered with caution. Rather, a strong basis for the use of coloured educational resources (Dzulkifli and Mustafar 2013) supports effective application of this principle to a variety of physical and digital anatomy learning modalities, including human cadaveric material. Additionally, differences between the colour of anatomical features in cadaveric specimens, and the colour of the same structures presented within reference texts, may also impair the identification of anatomical structures in cadavers (Crosado et al. 2020; Kennel et al. 2018). It is recommended therefore, that tissue colouration should be carefully considered when determining optimal cadaveric embalming approaches for anatomy education. In addition to differences in the visual appearance of cadavers, the range of available embalming approaches can produce various outcomes with respect to the size, shape and texture of human tissue produced (Balta et al. 2019). Although it is likely that realistic representations of preserved human anatomy will support an appreciation of anatomical structures in living patients, there are likely to be cognitive learning processes that are influenced by the sensory experience of handling cadaveric material. When considering physical

307

differences in not only the appearance, but also the texture of tissues and the mobility of joints in soft-fixed and hard-fixed cadavers, it is possible that students may be subject to additional cognitive load (Khalil et al. 2005; Sweller 1988), when attempting to reconcile living anatomy with cadavers that do not faithfully represent living tissues. While one particular study has found that both digital and physical approaches to dissection have been shown to be equally valuable for learning, the key element in both scenarios was found to involve visual inputs (Hisley et al. 2008). Nonetheless, haptic observation is likely to be an important facet of anatomy learning with physical cadaveric specimens. Indeed, we have explored the importance of tactile sensation in visualisation and anatomy education, in work which has shown positive educator perceptions of haptic activities in spatial and holistic anatomical understanding (Shapiro et al. 2019), and which is based upon established principles of the crucial role of haptics and multimodal haptico-visual observation in learning (Reid et al. 2019; Klatzky and Lederman 2011; Loomis et al. 2013; Jones et al. 2006; Minogue and Jones 2006). Furthermore, we have recently identified the value of combining physical and digital resources for studying anatomy (Ben Awadh et al. 2021), suggesting that combining haptic and visual learning, as is the case in cadaveric dissection, may be advantageous. In addition to the sensations of sight and touch, there are established links between olfactory inputs, visual memory and spatial learning (Zelcer et al. 2006; Olofsson et al. 2020). When using Thiel-embalmed cadavers, students have perceived strong or even pungent odours, but in this case the sensation of smell did not appear to influence learning (Kennel et al. 2018). Indeed, any changes in the level and quality of haptic or olfactory learning due to differences in tissue textures and odours produced by distinct embalming techniques are yet to be identified, but there is certainly scope for future work in this area. Alongside accounts of learning and teaching, there have been detailed explorations of the impacts on donors, students, and educators with

308

B. Thompson et al.

respect to various logistical, financial, sociocultural, and ethical factors that have arisen from the usage of cadavers for anatomical science and anatomy education (Chen et al. 2018; Bati et al. 2013; Rasiah et al. 2014; Chiu et al. 2012; Richardson and Hurwitz 1995). With particular relevance to embalming approaches, several studies have explored and identified the emotional impacts of human cadaveric dissection on anatomy students (Bati et al. 2013; Ghosh 2017; Khan and Mirza 2013; Bockers et al. 2012; Abrams et al. 2020; Tseng and Lin 2016; Williams et al. 2014; Plaisant et al. 2011; Quince et al. 2011). Consequently, although there may be considerable benefits for anatomical learning to be gained from the production and usage of lifelike cadavers, it can be postulated that the soft-fix methods of embalming that produce more realistic cadaveric appearances may result in greater psychological distress for students than when hard-fix cadavers are used. Indeed, one particular study has shown that Thiel-embalmed cadavers can elicit negative emotional experiences in anatomy students due to the authentic appearance and texture of the specimens (Balta et al. 2015b). Approaches have also been introduced to further investigate and address the emotional impacts of cadaveric resources on anatomy students, including the implementation of induction courses in advance of the use of cadaveric specimens (Bockers et al. 2012), the provision of counselling and pastoral support (Khan and Mirza 2013), and the introduction of reflective writing activities to identify individual impacts and support student wellbeing (Abrams et al. 2020). It is therefore recommended that such measures are considered when introducing soft-fix approaches to embalming.

13.4 13.4.1

The Newcastle Experience Educational and Technical Context

Gross anatomy education at Newcastle University (NU) is delivered within multiple undergraduate and postgraduate medical, healthcare, and

medical sciences programmes. These courses include medicine, dentistry, physician associate studies, biomedical sciences and speech and language therapy. We have described the current post-2017 and previous pre-2017 iterations of the undergraduate medical curriculum at NU in detail in our previous work (Keenan and Powell 2020; Ben Awadh et al. 2021; Keenan and Ben Awadh 2019; Backhouse et al. 2017). Practical anatomy teaching at NU is typically delivered using prosected human specimens as the core learning resource, supported by physical and digital anatomical models, cross-sectional anatomy and clinical imaging activities, and remote learning exercises. Opportunities for cadaveric dissection are limited to student-selected projects and postgraduate surgical anatomy courses. The provision of in-person practical cadaveric anatomy teaching has been disrupted by the Covid19 pandemic but is expected to return to a similar mode of delivery in due course. Since 2015, we have adopted and developed a novel approach for the preservation of human cadaveric material for the purposes of anatomical education. We describe our approach as the Newcastle-WhitWell technique, as the method is based on a protocol originally devised by Ben Whitworth and Craig Caldwell (MazWell Group Ltd., Whitchurch, Hampshire, U.K.). The WhitWell technique involves the usage of Dodge embalming products (Dodge Co., Billerica, MA, USA) and appropriate tools and instruments, and comprises a consistent procedure, with embalming solutions modified as necessary to provide distinct mixtures that fulfil the needs and requirements of individual anatomy education facilities. Our rationale for introducing the Newcastle-WhitWell protocol has been to improve the efficiency and reliability of established embalming processes and the utility of preserved cadavers. Having done so, our new process not only provides reduced embalming times, but also has positive implications for financial, safety and sustainability considerations. Moreover, our approach supports the enhanced delivery of anatomy education through improving practical benefits and specific pedagogically important elements of preserved specimens.

13

A Novel Cadaveric Embalming Technique for Enhancing Visualisation of Human Anatomy

13.4.2

Newcastle Formaldehyde-Phenol Mix Embalming

Prior to 2015, hard-fix embalming procedures at NU had been established for several decades. The conventional protocol consisted of a pressurised gravity feed method and femoral artery cannulation of either an in-house or commercially sourced formaldehyde-phenol mix (Vickers Laboratories, United Kingdom). Embalming fluid was introduced until swelling and bleaching of the donor could be visibly observed. This process could require 3–4 days of gravity feed perfusion, followed by manual injection of compartments and extremities. Furthermore, while 50–75 L fluid was required to achieve the necessary end-result, there was a significant proportion of excess fluid that was not retained by the body, which was consequently consigned to waste. Vascular drainage of donors was not performed during the Formaldehyde-Phenol mix protocol, which caused blood to be displaced by embalming fluid throughout the cadaveric blood vessels until formaldehyde fixation was complete. The flow of embalming fluid was therefore restricted, which often resulted in the venous system and the chambers of the heart being filled with solidified blood on completion of the process. Similarly, the volume of embalming fluid used in this protocol could disrupt the superficial appearance of the cadaver by causing blistering of the skin, which in turn may be detrimental to the visual and tactile experiences of learners. Following washing, the cadaver would require storage in the mortuary chamber for 6 months before use to allow the embalming fluid to perfuse all body tissues and to ensure that any excess fluid could be drained. Beginning in 2011, we performed a critical appraisal of our embalming procedures with the aim of identifying any need or scope for existing protocols to be updated and enhanced. As described above, the available literature has largely focused upon comparisons of the relative strengths and limitations of commercially available and bespoke formaldehyde-based solutions

309

and Thiel embalming (Eisma et al. 2013; Kennel et al. 2018; Balta et al. 2015b) in addition to descriptions of fresh and frozen approaches (Messmer et al. 2010; Reed et al. 2009). Unfortunately, there were few publications that had considered and described novel or improved embalming techniques, delivery methods and fixatives (Coleman and Kogan 1998; Hammer et al. 2012). Initial insights into the development of embalming procedures were therefore gained from connecting with colleagues from technical and academic societies, and from discussions with, and training from, representatives of commercial suppliers. The outcome of our investigations has resulted in the implementation of Newcastle-WhitWell embalming, based on the key factors of efficiency, cost, safety, and improved fixation outcomes.

13.4.3

Newcastle-WhitWell Embalming Protocol

Complete disinfection of cadaveric surfaces and cavities is performed with Dodge solutions (Dodge Co., Billerica, MA, USA), and the limbs are manipulated throughout their full range of motion, to remove rigor mortis and mobilise the shoulder, elbow, wrist, hip, knee, and ankle joints (Table 13.1). This process facilitates the preservation of a cadaver with an authentic range of movements, that are representative of the joints of the living human body. Care is taken to ensure that joints are not manipulated beyond their normal range of movement as this can carry a risk of muscle damage, with the hamstring muscles semitendinosus, semimembranosus, and biceps femoris within the posterior compartment of the thigh particularly being susceptible to injury from over-extension at the hip joint. An incision is made at the mid-point between the clavicle and the border of the mandible along the margin of the sternocleidomastoid muscle to provide access to underlying structures for cannula insertion. The common carotid artery and the internal jugular vein are separated from each other, and from their surrounding fascia, and ligatures are placed

310

B. Thompson et al.

Table 13.1 The Newcastle-WhitWell Embalming method Dodge reagents Cherry fresh (5%) Water Dispray Syncav

Instruments and equipment Cotton wool

Pre-fixation

Metaflow (1 L) Rectifiant (1 L) Water (1 L)

Scalpel Ligature cord Forceps Aneurysm needle Cannulae Dodge pump LPP-EM524

Embalming

Introfiant (3 L) Metaflow (3 L) Rectifiant (2 L) Restorative (1 L) Dispray (1 L)

Dodge pump LPP-EM524

Cavity treatment

Syncav (500 mL)

Purse-string suture Long medium gauge needle syringe

Procedure Disinfection

Limb manipulation

Cannula withdrawal

Disinfection and storage

Cherry fresh (5%) Dispray

Ligature cord Reverse triangular cut needle Artery forceps Toothed forceps Sutures Body bag

Protocol Remove clothes from the cadaver Shave and wash the cadaver with disinfectant Clean and trim fingernails Wash cadaver surfaces with Dispray Pack cadaver orifices (anus and vagina) with Syncav soaked cotton wool Mobilise joints through full range of motion to reduce stiffness from rigor mortis Avoid over-flexion and over-extension Make incision at margin of sternocleidomastoid mid-way between clavicle and mandible Perform blunt dissection to reveal common carotid artery (CCA) and interval jugular vein Separate vessels from surrounding fascia Place ligatures using forceps or aneurysm needle Mix reagents in pump to create pre-embalming fluid Cannulate CCA with superior-facing (‘up’) and inferiorfacing (‘down’) cannulae Secure with ligature Pump 500 mL pre-embalming fluid via ‘up’ cannula Pump remaining pre-embalming fluid via ‘down’ cannula Use pulse pressure 140–150 psi and flow rate 300 ml/min Extend and flex extremities to facilitate fluid movement Mix reagents in pump to create embalming fluid Pump minimum of 1 L of fluid via ‘up’ cannula Use pulse pressure 140–150 psi and flow rate 300 ml/min When tissue colouration of head and neck is observed, switch to the ‘down’ cannula Continue to manipulate the limbs and extremities Embalming should continue until there is colouration and filling of all tissues Additional embalming fluid should be introduced in cases of inadequate perfusion If inadequate perfusion of extremities occurs, a local artery can be cannulated Insert purse-string suture into the abdomen immediately superior to the umbilicus Inject 500 mL of Syncav into the abdomen via the suture as follows: 100 mL Syncav should be injected at the umbilicus in a posterior orientation 100 mL Syncav should be injected into each abdominal quadrant Withdraw cannulae from artery, pull ligature tight and tie off Close incisions with a baseball stich using needle, artery forceps and toothed forceps

Wash body surfaces with disinfectant and Dispray Transfer the cadaver into a body bag that has been cleaned with Dispray Move cadaver to a refrigerated mortuary chamber for storage The cadaver should be stored for 6 weeks before dissection

13

A Novel Cadaveric Embalming Technique for Enhancing Visualisation of Human Anatomy

around each vessel. Cannulae are inserted in superior and inferior directions, to perfuse the head and neck region, and the trunk and limbs, respectively. A pump feed system is utilised in the Newcastle-WhitWell protocol (Table 13.1), with a view to substantially reducing the time taken for perfusion when compared to the gravity feed approach used in our previous protocol. The pre-fixation solution of Metaflow and Rectifiant (Dodge Co., Billerica, MA, USA) is infused into the vascular system to stabilise pH levels; to facilitate decoagulation and detoxification; and for restoration of cell permeability and fluid balance. During pre-fixation, the limbs and extremities are again flexed and extended to facilitate fluid perfusion throughout the circulatory system, and for purposes of vascular drainage. The pre-fixation solution is then allowed to perfuse the tissues for 20 minutes before embalming is performed. The Newcastle-WhitWell preservative is produced by mixing Introfiant, Metaflow, Rectifiant, Restorative and Dispray (Dodge, Billerica, MA, USA) in a 3:3:2:1:1 ratio. The head and neck region of the donor body is perfused initially, again by pump feed, before moving on to the trunk and limbs (Table 13.1). Observation of the changes in expansion and colouration, as the tissues are filled with fixative, are used as visual indicators of the successful embalming of each region. Fluid in the cadaveric vascular system is then allowed to depressurise. The volume of fixative and therefore the time taken is dependent upon individual donor variations in size, with 10–20 L being in the typical range of fluid required, with a flow rate of 350–500 mL per min. The fluid volume and time taken are therefore considerably reduced when compared to our previous formaldehyde-phenol protocol. The limbs and the extremities are again manipulated during the pump feed process to assist the infusion of preservative fluid and the simultaneous draining of blood. On rare occasions that limbs or extremities do not receive adequate perfusion, compartmental injection or local cannulation can be performed.

311

Using a long medium gauge needle and syringe, cavity preservation fluid (Syncav, Dodge, Billerica, MA, USA) is inserted into all quadrants of the abdominal cavity via an injection site in the anterior abdominal wall, which is located immediately superior to the umbilicus (Table 13.1). Incisions can then be repaired, cadaver disinfection is repeated, and the body is transferred to a refrigerated mortuary chamber. We have now identified that a duration of 6 weeks is sufficient for complete preservation and disinfection to occur. This period is significantly shorter than the 6-month timescale that was allocated for fixation in our previous protocol. The result of the Newcastle-WhitWell procedure is the production of cadavers with a more realistic appearance (Fig. 13.1), and that are considerably less logistically challenging to embalm. As of March 2021, we have embalmed 72 donor bodies using the Newcastle-WhitWell method, with technical records demonstrating that all cadavers have achieved good or excellent outcomes, with no significant issues having arisen. The only limitations that have occurred concern the incomplete embalming of the toes and strong odours that can arise on dissection of the abdominal cavity. These issues have been resolved respectively through use of a scoop stretcher and blocks to position the body at the required angle for complete perfusion to occur, and by reducing the volume of cavity preservation fluid that is introduced into the abdomen during the embalming process. Overall, the NewcastleWhitWell method provides a consistently well embalmed cadaver that retains a normative range of motion, tissue colour and texture. Fascial planes can be readily and effectively separated, while the muscles, organs and vasculature can be mobilised in situ, attributes which can enhance the efficient tissue preparation when prosecting cadavers and produces effective and interactive specimens for anatomy learning. Anecdotally, the response from both taught anatomy students and those pursuing dissection projects has been overwhelmingly positive when comparing Newcastle-WhitWell prosections and cadavers with those embalmed using Formaldehyde-Phenol fixation.

312

B. Thompson et al.

Fig. 13.1 A comparison of Formaldehyde-Phenol embalmed thoracic (top left) and abdominal (bottom left) prosections, and WhitWell embalmed thoracic (top right) and abdominal (bottom right) prosections demonstrates differences in tissue appearance and colouration

13.5

Summary, Conclusions, and Implications for Practice

Approaches to the preservation of human cadavers for the purposes of anatomical education are determined by multiple historical, logistical, financial, curricular, and pedagogic concerns, all of which constitute crucial elements that cannot be considered in isolation. However, it is recommended that anatomy departments seeking to introduce new approaches to embalming should consider the visual, haptic, and olfactory impacts on learning of the tissues produced. The implementation of Newcastle-WhitWell embalming has been primarily based upon scholarly underpinnings of these areas of anatomy pedagogy alongside important technical

considerations. As described above, there are several areas that merit further exploration within future research studies, including the development of evidence-based approaches to identifying the most effective embalming approaches and the impact of colour, texture and realistic tissues on the cognitive and emotional learning experiences of anatomy students. Acknowledgements The authors wish to thank Ben Whitworth, Craig Caldwell and MazWell Group Ltd., for sharing their expertise and contributing to the development of the Newcastle-WhitWell embalming process. The authors acknowledge the Anatomy technical team at Newcastle University for preparation and management of Formalin-Phenol and Newcastle-WhitWell embalmed cadavers. The authors acknowledge with gratitude the contribution of body donors to Newcastle University for the purposes of medical education and research.

13

A Novel Cadaveric Embalming Technique for Enhancing Visualisation of Human Anatomy

References Abayomi A, Odiri E (2017) Embalmment: a veritable source of human body preservation. Anat J Afr 6: 982–986 Abrams MP, Eckert T, Topping D, Daly KD (2020) Reflective writing on the cadaveric dissection experience: an effective tool to assess the impact of dissection on learning of anatomy, humanism, empathy, wellbeing, and professional identity formation in medical students. Anat Sci Educ Ajileye A, Esan E, Adeyemi O (2018) Human embalming techniques: a review. Am J Biomed Sci 10:82–95 Aziz MA, Mckenzie JC, Wilson JS, Cowie RJ, Ayeni SA, Dunn BK (2002) The human cadaver in the age of biomedical informatics. Anat Rec 269:20–32 Backhouse M, Fitzpatrick M, Hutchinson J, Thandi CS, Keenan ID (2017) Improvements in anatomy knowledge when utilizing a novel cyclical “observe-reflectdraw-edit-repeat” learning process. Anat Sci Educ 10: 7–22 Balta JY, Cronin M, Cryan JF, O’Mahony SM (2015a) Human preservation techniques in anatomy: a 21st century medical education perspective. Clin Anat 28: 725–734 Balta JY, Lamb C, Soames RW (2015b) A pilot study comparing the use of Thiel- and formalin-embalmed cadavers in the teaching of human anatomy. Anat Sci Educ 8:86–91 Balta JY, Cronin M, Cryan JF, O’Mahony SM (2017) The utility of cadaver-based approaches for the teaching of human anatomy: a survey of British and Irish anatomy teachers. Anat Sci Educ 10:137–143 Balta JY, Twomey M, Moloney F, Duggan O, Murphy KP, O’Connor OJ, Cronin M, Cryan JF, Maher MM, O’Mahony SM (2019) A comparison of embalming fluids on the structures and properties of tissue in human cadavers. Anat Histol Embryol 48(1):64–73 Barton DPJ, Davies DC, Mahadevan V, Dennis L, Adib T, Mudan S, Sohaib A, Ellis H (2009) Dissection of softpreserved cadavers in the training of gynaecological oncologists: report of the first UK workshop. Gynecol Oncol 113:352–356 Bati AH, Ozer MA, Govsa F, Pinar Y (2013) Anxiety of first cadaver demonstration in medical, dentistry and pharmacy faculty students. Surg Radiol Anat 35: 419–426 Belavy D, Ruitenberg MJ, Brijball RB (2011) Feasibility study of real-time three-/four-dimensional ultrasound for epidural catheter insertion. Br J Anaesth 107(3): 438–445 Ben Awadh A, Clark J, Clowry G, Keenan ID (2021) Multimodal three-dimensional visualization enhances novice learner interpretation of basic cross-sectional anatomy. Anat Sci Educ. https://doi.org/10.1002/ase. 2045 Benet A, Rincon-Torroella J, Lawton MT, González Sánchez JJ (2014) Novel embalming solution for

313

neurosurgical simulation in cadavers. J Neurosurg 120:1229–1237 Benkhadra M, Gérard J, Genelot D, Trouilloud P, Girard C, Anderhuber F, Feigl G (2011) Is Thiel’s embalming method widely known? A world survey about its use. Surg Radiol Anat 33:359–363 Bhat D, Chittoor H, Murugesh P, Basavanna PN, Doddaiah S (2019) Estimation of occupational formaldehyde exposure in cadaver dissection laboratory and its implications. Anat Cell Biol 52:419–425 Biasutto SN, Caussa LI, Criado Del Rio LE (2006) Teaching anatomy: cadavers vs. computers? Ann Anat 188: 187–190 Bockers A, Baader C, Fassnacht UK, Ochsner W, Bockers TM (2012) Reduction of mental distress in the dissection course by introducing the body donor experience through anatomical demonstrations of organ systems. Anat Sci Educ 5:321–329 Bogomolova K, Hierck BP, van der Hage JA, Hovius SER (2020) Anatomy dissection course improves the initially lower levels of visual-spatial abilities of medical undergraduates. Anat Sci Educ 13:333–342 Brenner E (2014) Human body preservation – old and new techniques. J Anat 224:316–344 Chai DQ, Naunton-Morgan R, Hamdorf J (2019) Fresh frozen cadaver workshops for general surgical training. ANZ J Surg 89(11):1428–1431 Chen D, Zhang Q, Deng J, Cai Y, Huang J, Li F, Xiong K (2018) A shortage of cadavers: the predicament of regional anatomy education in mainland China. Anat Sci Educ 11:397–402 Chiu HY, Ng KS, Ma SK, Chan CH, Ng SW, Tipoe GL, Chan LK (2012) Voices of donors: case reports of body donation in Hong Kong. Anat Sci Educ 5: 295–300 Chytas D, Piagkou M, Salmas M, Johnson EO (2020) Is cadaveric dissection the “gold standard” for neuroanatomy education? Anat Sci Educ 13:804–805 Coleman R, Kogan I (1998) An improved low-formaldehyde embalming fluid to preserve cadavers for anatomy teaching. J Anat 192(Pt 3): 443–446 Crosado B, Löffler S, Ondruschka B, Zhang M, Zwirner J, Hammer N (2020) Phenoxyethanol-based embalming for anatomy teaching: an 18 Years’ experience with Crosado embalming at the University of Otago in New Zealand. Anat Sci Educ 13:778–793 Dikshith TSS (2010) Handbook of chemicals and safety, 1st edn. CRC Press, Boca Raton Dinsmore CE, Daugherty S, Zeitz HJ (1999) Teaching and learning gross anatomy: dissection, prosection, or “both of the above?”. Clin Anat 12:110–114 Doomernik DE, Kruse RR, Reijnen MMPJ, Kozicz TL, Kooloos JGM (2016) A comparative study of vascular injection fluids in fresh-frozen and embalmed human cadaver forearms. J Anat 229(4):582–590 Dyer GS, Thorndike ME (2000) Quidne mortui vivos docent? The evolving purpose of human dissection in medical education. Acad Med 75:969–979

314 Dzulkifli MA, Mustafar MF (2013) The influence of colour on memory performance: a review. Malays J Med Sci 20:3–9 Eisma R, Wilkinson T (2014) From “silent teachers” to models. PLoS Biol 12:e1001971 Eisma R, Mahendran S, Majumdar S, Smith D, Soames RW (2011) A comparison of Thiel and formalin embalmed cadavers for thyroid surgery training. Surgeon 9:142–146 Eisma R, Lamb C, Soames RW (2013) From formalin to Thiel embalming: what changes? One anatomy department’s experiences. Clin Anat 26:564–571 Elshaer NSM, Mahmoud MAE (2017) Toxic effects of formalin-treated cadaver on medical students, staff members, and workers in the Alexandria Faculty of Medicine. Alexandria J Med 53:337–343 Farley FH, Grant AP (1976) Arousal and cognition: memory for color versus black and white multimedia presentation. J Psychol 94:147–150 Fasel JH, Aguiar D, Kiss-Bodolay D, Montet X, Kalangos A, Stimec BV, Ratib O (2016) Adapting anatomy teaching to surgical trends: a combination of classical dissection, medical imaging, and 3D-printing technologies. Surg Radiol Anat 38:361–367 Fehrenbach MJ (2013) Dental anatomy coloring book. Elsevier Health Sciences Finn GM, White PM, Abdelbagi I (2011) The impact of color and role on retention of knowledge: a bodypainting study within undergraduate medicine. Anat Sci Educ 4:311–317 Fischer MH (1905) The toxic effects of formaldehyde and formalin. J Exp Med 6:487–518 Flack NAMS, Nicholson HD (2018) What do medical students learn from dissection? Anat Sci Educ 11: 325–335 Fox CH, Johnson FB, Whiting J, Roller PP (1985) Formaldehyde fixation. J Histochem Cytochem 33:845–853 Fruhstorfer BH, Palmer J, Brydges S, Abrahams PH (2011) The use of plastinated prosections for teaching anatomy – the view of medical students on the value of this learning resource. Clin Anat 24:246–252 Gartner LP, Hiatt JL (2012) Color atlas and text of histology. Lippincott Williams & Wilkins Getachew D (2014) Reaction of medical students to experiences in dissection room. Ethiop J Health Sci 24:337–342 Ghosh SK (2015) Human cadaveric dissection: a historical account from ancient Greece to the modern era. Anat Cell Biol 48:153–169 Ghosh SK (2017) Cadaveric dissection as an educational tool for anatomical sciences in the 21st century. Anat Sci Educ 10:286–299 Gosling JA, Harris PF, Humpherson JR, Whitmore I, Willan PL (2016) Human anatomy, color atlas and textbook E-Book. Elsevier Health Sciences Granger NA (2004) Dissection laboratory is vital to medical gross anatomy education. Anat Rec B New Anat 281:6–8

B. Thompson et al. Greene TC, Bell PA, Boyer WN (1983) Coloring the environment: hue, arousal, and boredom. Bull Psychon Soc 21:253–254 Guillot A, Champely S, Batier C, Thiriet P, Collet C (2007) Relationship between spatial abilities, mental rotation and functional anatomy learning. Adv Health Sci Educ 12:491–507 Guttmann GD, Drake RL, Trelease RB (2004) To what extent is cadaver dissection necessary to learn medical gross anatomy? A debate forum. Anat Rec B New Anat 281:2–3 Haizuka Y, Nagase M, Takashino S, Kobayashi Y, Fujikura Y, Matsumura G (2018) A new substitute for formalin: application to embalming cadavers. Clin Anat 31:90–98 Hall RH, Hanna P (2004) The impact of web page textbackground colour combinations on readability, retention, aesthetics and behavioural intention. Behav Inform Technol 23:183–195 Hammer N, Löffler S, Feja C, Sandrock M, Schmidt W, Bechmann I, Steinke H (2012) Ethanol-glycerin fixation with thymol conservation: a potential alternative to formaldehyde and phenol embalming. Anat Sci Educ 5:225–233 Hammer N, Löffler S, Bechmann I, Steinke H, Hädrich C, Feja C (2015) Comparison of modified Thiel embalming and ethanol-glycerin fixation in an anatomy environment: potentials and limitations of two complementary techniques. Anat Sci Educ 8:74–85 Hansen JT (2010) Netter’s anatomy coloring book. Saunders/Elsevier Hisley KC, Anderson LD, Smith SE, Kavic SM, Tracy JK (2008) Coupled physical and digital cadaver dissection followed by a visual test protocol provides insights into the nature of anatomical knowledge and its evaluation. Anat Sci Educ 1:27–40 Human Tissue Authority (2021) Anatomy sector guidance [Online]. https://www.hta.gov.uk/anatomy-sector-guid ance. Accessed 26 Mar 2021 Hunter A, Eisma R, Lamb C (2014) Thiel embalming fluid – a new way to revive formalin-fixed cadaveric specimens. Clin Anat 27:853–855 Jansen MM, Hazenberg CE, de Ruiter QM, van Hamersvelt RW, Bleys RL, van Herwaarden JA (2020) Feasibility of fresh frozen human cadavers as a research and training model for endovascular image guided interventions. PLoS One 15(11):e0242596 Jaung R, Cook P, Blyth P (2011) A comparison of embalming fluids for use in surgical workshops. Clin Anat 24:155–161 Jones MG, Minogue J, Tretter TR, Negishi A, Taylor R (2006) Haptic augmentation of science instruction: does touch matter? Sci Educ 90:111–123 Keenan ID, Ben Awadh A (2019) Integrating 3D visualisation technologies in undergraduate anatomy education. Adv Exp Med Biol 1120:39–53 Keenan I, Powell M (2020) Interdimensional travel: visualisation of 3D-2D transitions in anatomy learning. Adv Exp Med Biol

13

A Novel Cadaveric Embalming Technique for Enhancing Visualisation of Human Anatomy

Keenan ID, Hutchinson J, Bell K (2017) Twelve tips for implementing artistic learning approaches in anatomy education. MedEdPublish 6 Kennel L, Martin DMA, Shaw H, Wilkinson T (2018) Learning anatomy through Thiel- vs. formalinembalmed cadavers: student perceptions of embalming methods and effect on functional anatomy knowledge. Anat Sci Educ 11:166–174 Khalil MK, Paas F, Johnson TE, Payer AF (2005) Design of interactive and dynamic anatomical visualizations: the implication of cognitive load theory. Anat Rec Part B 286:15–20 Khan HM, Mirza TM (2013) Physical and psychological effects of cadaveric dissection on undergraduate medical students. J Pak Med Assoc 63:831–834 Klatzky RL, Lederman SJ (2011) Haptic object perception: spatial dimensionality and relation to vision. Philos Trans R Soc Lond Ser B, Biol Sci 366:3097–3105 Korf HW, Wicht H, Snipes RL, Timmermans JP, Paulsen F, Rune G, Baumgart-Vogt E (2008) The dissection course – necessary and indispensable for teaching anatomy to medical students. Ann Anat 190: 16–22 Langlois J, Bellemare C, Toulouse J, Wells GA (2017) Spatial abilities and anatomy knowledge assessment: a systematic review. Anat Sci Educ 10:235–241 Lempp HK (2005) Perceptions of dissection by students in one medical school: beyond learning about anatomy. A qualitative study. Med Educ 39:318–325 Liu JK, Kshettry VR, Recinos PF, Kamian K, Schlenk RP, Benzel EC (2015) Establishing a surgical skills laboratory and dissection curriculum for neurosurgical residency training. J Neurosurg 123:1331–1338 Lloyd-Jones TJ, Nakabayashi K (2009) Independent effects of colour on object identification and memory. Q J Exp Psychol 62:310–322 Lodge JM, Hansen L, Cottrell D (2016) Modality preference and learning style theories: rethinking the role of sensory modality in learning. Learning 2:4–17 Logan BM, Hutchings RT (2011) McMinn’s color atlas of foot and ankle anatomy E-book. Elsevier Health Sciences Loomis JM, Klatzky RL, Giudice NA (2013) Representing 3D space in working memory: spatial images from vision, hearing, touch, and language. In: Lacey S, Lawson R (eds) Multisensory imagery. Springer, New York Lufler RS, Zumwalt AC, Romney CA, Hoagland TM (2012) Effect of visual-spatial ability on medical students’ performance in a gross anatomy course. Anat Sci Educ 5:3–9 Marquet PA, Santoro CM, Latorre C, Standen VG, Abades SR, Rivadeneira MM, Arriaza B, Hochberg ME (2012) Emergence of social complexity among coastal huntergatherers in the Atacama Desert of northern Chile. Proc Natl Acad Sci U S A 109:14754–14760 Mccann S, Wise E (2017) Anatomy coloring book. Simon and Schuster

315

Mcconnell J (2012) The anatomy coloring book. British Medical Journal Publishing Group Mclachlan JC (2004) New path for teaching anatomy: living anatomy and medical imaging vs. dissection. Anat Rec B New Anat 281:4–5 Mclachlan JC, Bligh J, Bradley P, Searle J (2004) Teaching anatomy without cadavers. Med Educ 38:418–424 McLachlan JC, Patten D (2006) Anatomy teaching: ghosts of the past, present and future. Med Educ 40(3):243– 253 Messmer C, Kellogg RT, Zhang Y, Baiak A, Leiweke C, Marcus JR, Levin LS, Zenn MR, Erdmann D (2010) A technique to perfuse cadavers that extends the useful life of fresh tissues: the Duke experience. Anat Sci Educ 3:191–194 Miller M (2016) Use of computer-aided holographic models improves performance in a cadaver dissection-based course in gross anatomy. Clin Anat 29:917–924 Minogue J, Jones MG (2006) Haptics in education: exploring an untapped sensory modality. Rev Educ Res 76: 317–348 Mitchell PD, Boston C, Chamberlain AT, Chaplin S, Chauhan V, Evans J, Fowler L, Powers N, Walker D, Webb H, Witkin A (2011) The study of anatomy in England from 1700 to the early 20th century. J Anat 219:91–99 Miyake S, Suenaga J, Miyazaki R, Sasame J, Akimoto T, Tanaka T, Ohtake M, Takase H, Tateishi K, Shimizu N, Murata H, Funakoshi K, Yamamoto T (2020) Thiel’s embalming method with additional intra-cerebral ventricular formalin injection (TEIF) for cadaver training of head and brain surgery. Anat Sci Int 95:564–570 Moore CW, Wilson TD, Rice CL (2017) Digital preservation of anatomical variation: 3D-modeling of embalmed and plastinated cadaveric specimens using uCT and MRI. Ann Anat 209:69–75 Noël GPJ (2020) Flammability and combustion hazard of preserved human tissues. Clin Anat 33:876–879 O’Sullivan E, Mitchell BS (1993) An improved composition for embalming fluid to preserve cadavers for anatomy teaching in the United Kingdom. J Anat 182(Pt 2): 295–297 Olofsson JK, Ekström I, Lindström J, Syrjänen E, Stigsdotter-Neely A, Nyberg L, Jonsson S, Larsson M (2020) Smell-based memory training: evidence of olfactory learning and transfer to the visual domain. Chem Senses 45:593–600 Pandey P, Zimitat C (2007) Medical students’ learning of anatomy: memorisation, understanding and visualisation. Med Educ 41:7–14 Pawlina W, Lachman N (2004) Dissection in learning and teaching gross anatomy: rebuttal to McLachlan. Anat Rec B New Anat 281:9–11 Plaisant O, Courtois R, Toussaint PJ, Mendelsohn GA, John OP, Delmas V, Moxham BJ (2011) Medical students’ attitudes toward the anatomy dissection room in relation to personality. Anat Sci Educ 4:305–310

316 Pole T (1790) The anatomical instructor: or, an illustration of the modern and most approved methods of preparing and preserving the different parts of the human body, and of quadrupeds, by injection, corrosion, maceration, distention, articulation, modelling, &c, Couchman and Fry Quince TA, Barclay SI, Spear M, Parker RA, Wood DF (2011) Student attitudes toward cadaveric dissection at a UK medical school. Anat Sci Educ 4:200–207 Rasiah R, Manikam R, Chandarsekaran SK, Thangiah G, Puspharajan S, Swaminathan D (2014) The influence of socioeconomic and demographic variables on willingness to donate cadaveric human organs in Malaysia. Medicine (Baltimore) 93:e126 Reed AB, Crafton C, Giglia JS, Hutto JD (2009) Back to basics: use of fresh cadavers in vascular surgery training. Surgery 146:757–762. discussion 762-3 Reid S, Shapiro L, Louw G (2019) How haptics and drawing enhance the learning of anatomy. Anat Sci Educ 12:164–172 Ribatti D (2009) William Harvey and the discovery of the circulation of the blood. J Angiogenes Res 1:3 Richardson R, Hurwitz B (1995) Donors’ attitudes towards body donation for dissection. Lancet 346: 277–279 Saltarelli AJ, Roseth CJ, Saltarelli WA (2014) Human cadavers vs. multimedia simulation: a study of student learning in anatomy. Anat Sci Educ 7:331–339 Shapiro L, Bell K, Dhas K, Branson T, Louw G, Keenan ID (2019) Focused multisensory anatomy observation and drawing for enhancing social learning and threedimensional spatial understanding. Anat Sci Educ Sheikh AH, Barry DS, Gutierrez H, Cryan JF, O’Keeffe GW (2016) Cadaveric anatomy in the future of medical education: what is the surgeons view? Anat Sci Educ 9: 203–208 Shoja MM, Benninger B, Agutter P, Loukas M, Tubbs RS (2013) A historical perspective: infection from cadaveric dissection from the 18th to 20th centuries. Clin Anat 26:154–160 Slon V, Hershkovitz I, May H (2014) The value of cadaver CT scans in gross anatomy laboratory. Anat Sci Educ 7:80–82 Smilek D, Dixon MJ, Cudahy C, Merikle PM (2002) Synesthetic color experiences influence memory. Psychol Sci 13:548–552 Spence I, Wong P, Rusan M, Rastegar N (2006) How color enhances visual memory for natural scenes. Psychol Sci 17:1–6 Srivastava G, Nagwani M (2019) Embalming of human cadavers from Egyptian era to the most modern techniques-a review on preservation of human cadavers. Era’s J Med Res 6(2):94–97 Sugand K, Abrahams P, Khurana A (2010) The anatomy of anatomy: a review for its modernization. Anat Sci Educ 3:83–93 Sweeney K, Hayes JA, Chiavaroli N (2014) Does spatial ability help the learning of anatomy in a biomedical science course? Anat Sci Educ 7:289–294

B. Thompson et al. Sweller J (1988) Cognitive load during problem solving: effects on learning. Cogn Sci 12:257–285 Thiel W (1992) The preservation of the whole corpse with natural color. Ann Anat 174:185–195 Thiel W (2002) Supplement to the conservation of an entire cadaver according to W. Thiel. Ann Anat 184: 267–269 Tomalty D, Pang SC, Ellis RE (2019) Preservation of neural tissue with a formaldehyde-free phenol-based embalming protocol. Clin Anat 32:224–230 Topp KS (2004) Prosection vs. dissection, the debate continues: rebuttal to Granger. Anat Rec B New Anat 281:12–14 Tseng WT, Lin YP (2016) “Detached concern” of medical students in a cadaver dissection course: a phenomenological study. Anat Sci Educ 9:265–271 UK Government (2021) Covid-19 guidance for care of the deceased [Online]. https://www.gov.uk/government/ publications/covid-19-guidance-for-care-of-thedeceased/guidance-for-care-of-the-deceased-withsuspected-or-confirmed-coronavirus-covid-19. Accessed 26 Mar 2021 UK Health and Safety Executive (2018) Managing infection risks when handling the deceased [Online]. https:// www.hse.gov.uk/pubns/books/hsg283.htm. Accessed 26 Mar 2021 Venne G, Zec ML, Welte L, Noel GPJC (2020) Qualitative and quantitative comparison of Thiel and phenol-based soft-embalmed cadavers for surgery training. Anat Histol Embryol 49:372–381 Vernon D, Lloyd-Jones TJ (2003) The role of colour in implicit and explicit memory performance. Q J Exp Psychol Sect A 56:779–802 Veys R, Verpoort P, Van Haute C, Wang ZT, Chi T, Tailly T (2020) Thiel-embalmed cadavers as a novel training model for ultrasound-guided supine endoscopic combined intrarenal surgery. BJU Int 125:579–585 Von Hagens G, Tiedemann K, Kriz W (1987) The current potential of plastination. Anat Embryol 175:411–421 Vorstenbosch MA, Klaassen TP, Donders AR, Kooloos JG, Bolhuis SM, Laan RF (2013) Learning anatomy enhances spatial ability. Anat Sci Educ 6:257–262 Williams AD, Greenwald EE, Soricelli RL, Depace DM (2014) Medical students’ reactions to anatomic dissection and the phenomenon of cadaver naming. Anat Sci Educ 7:169–180 Wilson AB, Miller CH, Klein BA, Taylor MA, Goodwin M, Boyle EK, Brown K, Hoppe C, Lazarus M (2018) A meta-analysis of anatomy laboratory pedagogies. Clin Anat 31:122–133 Winkelmann A (2007) Anatomical dissection as a teaching method in medical school: a review of the evidence. Med Educ 41:15–22 Winkelmann A, Hendrix S, Kiessling C (2007) What do students actually do during a dissection course? First steps towards understanding a complex learning experience. Acad Med 82:989–995 Yiasemidou M, Roberts D, Glassman D, Tomlinson J, Biyani S, Miskovic D (2017) A multispecialty

13

A Novel Cadaveric Embalming Technique for Enhancing Visualisation of Human Anatomy

317

evaluation of Thiel Cadavers for surgical training. World J Surg 41:1201–1207 Zelcer I, Cohen H, Richter-Levin G, Lebiosn T, Grossberger T, Barkai E (2006) A cellular correlate of learning-induced metaplasticity in the hippocampus. Cereb Cortex 16:460–468

Kayleigh S. Scotcher, B.Sc., (Hons), MSc, F.H.E.A is a Lecturer in Anatomy within the School of Medical Education at Newcastle University in Newcastle upon Tyne, United Kingdom. and is a Fellow of Advance HE. Her research interests include the investigation, development, and implementation of clinically orientated teaching approaches in anatomy education.

Brian Thompson is the former Technical Team Leader for the Anatomy and Clinical Skills Centre and School of Medical Education at Newcastle University, Newcastle upon Tyne, United Kingdom. Brian is now Anatomy Technician at the School of Medicine, University of Sunderland, Sunderland, United Kingdom.

Iain D. Keenan, B.Sc., (Hons.), Ph.D., M.Med.Ed., S.F. H.E.A., N.T.F. is a Senior Lecturer in Anatomy within the School of Medical Education at Newcastle University, Newcastle upon Tyne, United Kingdom and is an Advance HE National Teaching Fellow. His research interests include the development and investigation of threedimensional digital visualisation and art-based learning approaches in anatomy education.

Emily Green, B.Sc., (Hons.), MSc., F.H.E.A. is a Lecturer in Anatomy within the School of Medical Education at Newcastle University in Newcastle upon Tyne, United Kingdom and is a Fellow of Advance HE. Her interests are in clinically oriented anatomy teaching as well as science communication and public engagement.

Assessing the Impact of Interactive Educational Videos and Screencasts Within Pre-clinical Microanatomy and Medical Physiology Teaching

14

Alistair Robson, Yarrow Scantling-Birch, Stuart Morton, Deepika Anbu, and Scott Border

Abstract

Modern medical curricula adopt the use selfdirected learning approaches, which frequently include the use of technology enhanced learning resources. Often, students prefer those which are available via mobile devices because it can facilitate flexibility and autonomy with their learning, more so than with traditional modalities. Although the production value of resources may be appealing to users, those that work most effectively for education align well to existing pedagogies. One such pedagogy is the cognitive theory of multimedia learning. It is a framework that can be used to facilitate the construction of educational video, that can benefit learning gain through reducing the cognitive load. Although much research has been conducted on how information should be presented in video resources, there is very limited evidence within the subject of clinical anatomy and physiology or when applying different types of educational video, such as screencasts, or interactive video. In the field of anatomy education recent approaches have

A. Robson · Y. Scantling-Birch · S. Morton · D. Anbu · S. Border (*) Centre for Learning Anatomical Sciences, Primary Care, Population Sciences and Medical Education, Mailpoint 845, University Hospital Southampton, Southampton, UK e-mail: [email protected]

sought to standardize a robust methodology to evaluate digital resources. This procedure utilizes a combination of normalized learning gain and learner perceptions to gain an accurate picture of educational impact. The current study investigated the impact of both interactive educational videos and screencasts compared with traditional teaching techniques in the challenging subjects of histology and pain physiology. A quasirandomized, cross-sectional study was conducted with 135 medical students enrolled at the University of Southampton. Sixty fourth- and fifth-year students assessed the histology resources, and 75 second-year students assessed the pain histology resources. Participants were randomly assigned to either a text-based resource, interactive video, or screencast group. Outcomes measured were: 1. Normalized knowledge gain (and retention) assessed using one-best-answer multiple choice question tests 2. Student perceptions using 1–10 Likertscale style questionnaires. A significant improvement in mean normalized knowledge gain was observed for all teaching modalities. For pain physiology, the means were: Text—49.0% (n ¼ 23), interactive video—70.1% (n ¼ 26), and screencast—53.8% (n ¼ 26). For all learning gains, P < 0.001. For histology, there was a mean normalized learning gain for text—

# The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 P. M. Rea (ed.), Biomedical Visualisation, Advances in Experimental Medicine and Biology 1356, https://doi.org/10.1007/978-3-030-87779-8_14

319

320

A. Robson et al.

80.0% (n ¼ 20), interactive video—74.4% (n ¼ 20), and screencast groups—68.3% (n ¼ 20). For all learning gains, P < 0.001. For pain physiology resources, interactive videos significantly improved learning gain compared to the screencast (P < 0.05) and the text resource groups (P < 0.01). There was no significant difference between those who used the text or screencast resources. There was also no significant difference in knowledge retention between the different teaching methods for each subject. Following teaching, all three teaching modalities had similar effects on student confidence in the subjects, desire for educational channels dedicated to each topic, and preference for locally produced vs. externally produced videos. These findings have the potential to inform educators on which types of resources to create or to select for their students to have the best impact on learning. Keywords

Screencasts · Educational video · Blended learning · Anatomy education · Physiology education · Technology enhanced learning

Abbreviations CLT CTML IV (in graphs) MCQ TEL UoS

14.1

Cognitive load theory Cognitive theory of multimedia learning Interactive video Multiple choice quiz Technology enhanced learning University of Southampton

Introduction

Despite the growing quantity of information doctors must know upon graduation and beyond, time devoted to discipline-based science teaching

has decreased (McHanwell et al. 2019). Less teaching time is committed to core subjects in medicine such as anatomy, histology, and physiology (Prober and Heath 2012; Heylings 2002; Drake et al. 2009). Today’s students have become more diverse in background, ability, and with their learning preferences (Lujan and DiCarlo 2006). They are considered to be skilled multitaskers who are efficient users of technology in their learning (DiLullo et al. 2011). To cater for this diversity, multi-modal educational tools have been developed, which aim to improve learning by enabling a variety of educational experiences. This variety of tools available for students allows them to engage with learning resources they themselves find to be most enjoyable, beneficial and accessible (Drake 2014). Traditional teaching methods have been criticized for not suitably engaging students (Harden 2008; Matheson 2008). Modern innovations in medical education facilitate the development of material that promotes active learning; engaging the participant with the content which is crucial for improving learning (Campbell and Mayer 2009; Draper and Brown 2004). Technology enhanced learning (TEL) resources, such as educational videos have can improve student engagement across a range of subjects such as programming (Sugar et al. 2010), statistics (Lloyd and Robertson 2012), sociology (Auster 2016), and engineering (Green et al. 2012), potentially resulting in an increase in learning, however there remains significant investigation regarding appropriate methodology for investigation learning gain, and the application of TEL to a medical curriculum. Most students own portable multimedia devices with internet connectivity (Barry et al. 2015), enabling access to a plethora of online TEL resources. Students frequently use these in combination with traditional educational tools such as textbooks (Pickering 2017a). This allows flexibility, educational autonomy and enhanced digital literacy skills which is limited through traditional didactic modalities (Pickering 2015). With the rising prevalence of integrated curricula, particularly in the US (Drake et al. 2014) video

14

Assessing the Impact of Interactive Educational Videos and Screencasts. . .

resources might be considered an asset which can complement such interdisciplinary approaches (Trelease 2016). The cognitive theory of multimedia learning (CTML) presets a framework that facilitates the construction of educational videos that should maximize learning gain by reducing cognitive load on the user. Historically, scholarship in medical education has been criticized for being ‘quasi-experimental’ in its approach (Colliver and McGaghie 2008), often featuring methodological flaws with limited control and randomization procedures (Kaestle 1993; Lurie 2003; Todres et al. 2007). As part of a broader evaluation process for digital resources in anatomy education, it has been proposed that a combination of knowledge gain and learner perceptions should be collected to provide robust evidence to adequately inform educators of a resource’s efficacy before deployment (Pickering and Joynes 2016). Additionally, calculating normalized learning gain allows for more reliable comparisons between diverse groups of participants because it takes into account differences in baseline knowledge levels (Pickering 2017a, b). Anatomy is a core subject of the medical curriculum (Heylings 2002; Older 2004; Louw et al. 2009; Wong and Tay 2005), and much research has shown that TEL resources such as screencasts and interactive videos make the subject easier to understand and more digestible (Jaffar 2012; Reid et al. 2018) as well as having a positive impact on learning (Pickering 2015; Saxena et al. 2008). If such approaches are effective for visually challenging subjects like anatomy, it would be reasonable to assume they could benefit subjects that are challenging in slightly different ways, such as medical histology and physiology. Histology can also be described as microanatomy and is highly visual much like gross anatomy, while physiology is more conceptual, and less rooted in physical structures requiring accurate visualization, but frequently benefits from its processes being represented in flow diagrams or schematics to ease understanding by reducing the cognitive load, which will be discussed in further detail in this work.

14.1.1

321

The Use of Video in Clinical Anatomy

TEL is the use of information technology to improve education and knowledge (Ruiz et al. 2006). For medical students, Youtube® (a videosharing platform) is a major open access tool for viewing educational videos, with 78–99% of them using it regularly for anatomy education (Jaffar 2012; Mahmud et al. 2011; Pickering and Swinnerton 2019). This would suggest that preparing video-based resources that can be hosted on Youtube® would be an easy and accessible interface for students. Common structural components have been defined, such as bumpers at the start and end of the videos, and instructional techniques, such as providing a suitable overview of the topic, and highlighting key information points (Sugar et al. 2010). Unfortunately, it is difficult to assess the benefit of using Youtube® directly, since there is variety in the style, quality and length of videos uploaded to the platform (Barry et al. 2015). Although there are commercial enterprises producing high production quality video content, it is also possible for tutors to create bespoke videos that align to curricula learning outcomes and are therefore tailored towards institutions’ own programmes. Saxena et al. demonstrated that the use of faculty-developed videos improved anatomy examination performance by 3.4% (P ¼ 0.007) when compared to the previous year. However, this requires a somewhat cautious interpretation since variation in annual exam results can be explained by a wide range of possible factors (Pickering 2015; Saxena et al. 2008). This can be mitigated by including a control group, but there might be ethical and practical implications if this was introduced within a live curriculum setting such influence on careerinfluencing exams, or the sharing of resources between peers. Furthermore, it is possible that videos appeal to more highly motivated or achieving students, as they may make use of all available resources. To help circumvent this issue, the application of normalizing learning gain has been suggested by Pickering (2017b).

322

A. Robson et al.

To categorize resources, the terms ‘screencast’ (Udell 2005) and ‘interactive video’ (Zhang et al. 2006) have been developed as subgenres of the broader term video. The screencast process can be likened to the types of diagrams created on a visualizer or white board during a live lecture, to encourage active learning (Pickering and Roberts 2018). While interactive video allows opportunities to test knowledge before, after or during playback. This often requires embedding the video within additional software, such as Panopto for example.

14.1.2

The Use of Screencasts in Clinical Anatomy

A screencast is a ‘rough and ready’ digitally recorded playback of drawn images including audio narration (Udell 2005), with ‘bumpers’ either side of a screen recording (Figs. 14.1 and 14.2) slowly building up layers from a blank screen, actively engaging users through stimulating anticipation of the gradual construction of the image, developing a coherent mental model (Mayer 2009). This process actively engages students through the anticipation and gradual construction of the image, imagining what the next stage of the structure will be without overloading working memory, as described by the modality principle, and the temporal-contiguity principle of memory (Moreno and Mayer 1999). In all, screencasts are a flexible recreation of a classroom or lecture style teaching session much like a whiteboard illustration produced spontaneously by the teacher in live teaching environments

Fig. 14.1 An example of bumpers

(Pickering and Roberts 2018). Those who have viewed it from a blank screen through its entirety to the final image will have engaged with each part of the learning as it is layered, piece by piece, onto the drawing (Greene 2018).

14.1.3

The Use of Interactive Video in Anatomy Education

In contrast, an interactive video reflects a wellproduced recorded lecture, where information is presented slide-by-slide, with scripted narration and tidy or complete illustrations that complement the text (Zhang et al. 2006). The application of knowledge retrieval within a teaching resource is a metacognitive technique which has reliably demonstrated improvements in learning (Larsen et al. 2008, 2009; Wormald et al. 2009). There are not currently many digital multimedia tools which can facilitate creation of these videos using standard video file format, however there are ways to use regular video files within other software packages so that interactive elements can be incorporated. This is achievable using HTML5 files that can include video with integrated interactive content. The disadvantage of using this approach is that they cannot be hosted on YouTube. However, they can be hosted on learning management systems such as Blackboard, Panopto or on websites. These resources have proven to be popular with learners (Pickering 2015) potentially leading to improved student usage, engagement and performance (Trowler 2010) (Fig. 14.3).

14

Assessing the Impact of Interactive Educational Videos and Screencasts. . .

323

Fig. 14.2 Screen capture of the gradual drawing process

Fig. 14.3 Example of interactive video slides

14.1.4

How Do These Resources Improve Learning?

Educational videos of any kind, which are developed in line with the cognitive theory of

multimedia learning (CTML), improve learning through minimizing cognitive load and supporting the active engagement of students (Mayer 2009; Moreno and Mayer 1999). CTML is based on three main assumptions: there are

324

separate auditory and visual channels for processing information (also known as a dual channel hypothesis); each channel has a finite capacity; and learning requires the student to actively engage with the information without overload of any one channel (Mayer 2009; Moreno and Mayer 1999). Cognitive load is the strain on an individual’s working memory when processing information for learning (Paas et al. 2003). Cognitive load theory (CLT) suggests the working memory synthesizes new knowledge from recent learning, integrated with previously known information from the long-term memory, resulting in the consolidation and therefore the potential for retrieval of the newly processed information (Moreno and Mayer 1999; Paas et al. 2003; Mayer and Moreno 1998). Overloading the cognitive process, particularly at the working memory stage, results in reduced levels of learning as demonstrated by recall (Mayer et al. 2001). Presumably this is because the working memory modality of memory is the least stable. Three categories of cognitive load have been identified that are thought to place strain on the working memory: intrinsic load, extraneous load, and germane load (Moreno and Mayer 1999). Intrinsic load is the essential minimum information the learner must know to fully understand the topic. Therefore this is a fixed quantity, not affected by teaching modality (Sweller 1988). Extraneous load on working memory is a load burden caused by the presentation of information, which is not essential for understanding the topic, and so it can be manipulated. For example a shape can be described in verbal terms, however showing a picture of the shape allows the concept to be understood in an immediate, simple, and effortless way, reducing extraneous load and benefiting the facilitation of understanding (Sweller 1988). Germane load is the working memory capacity used for mental construction of the framework, which represents the information presented (i.e., a schema). Maximizing this improves learning, since increased capacity for the germane load equals greater capacity for the comprehension of the topic and enhances the construction of

A. Robson et al.

coherent mental models (Mayer et al. 2001) as well as supporting the learning acquired from the intrinsic load (Sweller 1988). CTML suggests that by increasing germane load and reducing the extraneous load, the overload of working memory can be prevented (Moreno and Mayer 1999; Mayer and Moreno 1998). Educational videos made for any subject area which are closely aligned with these principles will ultimately facilitate knowledge acquisition: the approaches by which this can be achieved are dictated by the spatial-contiguity principle, the temporal-contiguity principle, the modality principle, and the dual channel hypothesis (Moreno and Mayer 1999; Mayer and Moreno 1998). These are the aspects of the theory than can be built into the instructional design process of video creation and become part of the resource creation workflow to ensure that the final video stands the best chance of having a positive impact on learning for the user.

14.1.4.1 The Spatial-Contiguity Principle According to this principle, the presentation of text close to, or integrated within an image, improves learning in comparison to when the same information is presented, but the text is spatially separate from the image (Mayer et al. 1995). It has been concluded from a review of ten studies that there is significant evidence supporting the efficacy for its impact on learning (Mayer 1997) (Fig. 14.4). 14.1.4.2 The Temporal Contiguity Principle Learning is improved when visual and aural information are synchronised e.g. the audio narration, text, and the animation or motion graphics all occur at the same time (Fig. 14.5) (Moreno and Mayer 1999; Mayer and Anderson 1992). 14.1.4.3 The Modality Principle This principle suggests that it is preferable for words to be presented as audial narration, rather than on-screen text (Fig. 14.6). This supports the dual-channel theory (Moreno and Mayer 1999; Mayer and Moreno 1998). There is good evidence to suggest that the performance of students

14

Assessing the Impact of Interactive Educational Videos and Screencasts. . .

325

Fig. 14.4 Spatial contiguity principle—Spatially integrating text and animation improves learning gain

Fig. 14.5 Temporal contiguity effect—synchronised presentation of information improves learning gain

in a variety of studies has been shown to be improved through the activation and use of both the audio and visual working memories as outlined by this modality principle (Moreno and Mayer 1999; Mayer and Moreno 1998; Sweller 1988; Mousavi et al. 1995).

14.1.5

Methods for Assessing Learning Gain

In order to appropriately assess learning gain from video or screencast use, Pickering et al. adopted a framework utilising normalised

learning gain (calculated using pre- and post-test scores) to demonstrate a quantitative impact on learning gain (Elmansouri et al. 2020; Pickering et al. 2019). It is calculated by dividing the absolute gain by the maximum possible learning gain, thereby eliminating the influence of the pre-test score (Pickering 2017b). This normalisation allows for the actual change in learning gain to be recorded independent of pre-test scores and permits comparisons between diverse groups of volunteers, allowing a more reliable way to quantify the knowledge improvement. Student perception of the resource using both questionnaires and focus groups should be used in conjunction with

326

A. Robson et al.

Fig. 14.6 Outlining the modality principle

the learning gain to evaluate the perception of the resources, as student enjoyment is crucial to maximise engagement (Pickering and Joynes 2016).

14.1.6

Rationale for a Study in Basic Medical Sciences

There appears to be a paucity of literature on the effect of educational video resources in medical education, despite increasing interest in e-learning in medical education, which is likely a result of more institutions adopting e-learning in their curricula (Jego 2019). By developing video resources aligned with CLT, the dual channel hypothesis, it is possible that newly developed media resources will compare favourability to traditional learning resources, or possibly even enhance student learning beyond the levels that currently exist. The current approach has combined core elements of a robust experimental design, such as a randomized control trial structure, normalized learning gain calculations and a user perception survey. It is thought that the combination of these features will offer a meaningful

and holistic appraisal of educational resources (Pickering and Joynes 2016; Clunie et al. 2018). The evaluation of TEL resources has historically been criticized for lacking comprehensiveness and clarity on the relationships between the intervention, measured outcomes, and the conceptual grounding for the subject or the methods of the evaluation—suggesting a need for a standard, comprehensive evaluation approach to promote the utility of individual evaluations, and facilitate the integration and synthesis of results across studies (Pickering and Joynes 2016; Cook and Ellaway 2015). In seeking to thoroughly assess the impact of TEL resources on learning gain, the preferred methodology of testing a null hypothesis with a valid control group, builds on only one other such piece of TEL research of this nature (Morris and Chikwa 2014). This study concluded a statistically significant academic improvement, with the experimental group outperforming the non-user control group. However, it is was a single-center study that did not include an investigation into interactive video use.

14

Assessing the Impact of Interactive Educational Videos and Screencasts. . .

Further research utilizing robust methodology is necessary to advance this field before this can become more generalizable within medical education. This is particularly important in a post Covid-19 educational landscape, where many institutions will continue to deliver teaching via a blended approach and incorporate this new philosophy into longer term strategies. In terms of deciding upon appropriate content areas for resource creation within this research, it was felt that the subjects that would be most suitable would be those that were moderately complex and required good explanations and visual elements to be fully understood. Medical histology is historically unpopular amongst medical students, difficult to interpret, but also a highly visual subject, while medical physiology is challenging due to its conceptual and more abstract nature, particularly in relation to the understanding of applied biochemistry. To this end, both disciplines have potential to benefit from the application of video resources that can support face to face delivery of the subject.

14.2

Aims

This study aimed to: 1. Investigate the educational impact of interactive videos and screencasts in comparison to traditional text-book style resources on the following outcomes: a. Knowledge gain b. Knowledge retention c. User experience

14.3 14.3.1

Methods Video Production and Design

Histology of cartilage, histology of respiratory epithelium, and pain physiology were the topics chosen for the investigation. Since histology is visual, and physiology is more abstract, these investigations broadly represent the effect of

327

educational video resources in the diverse medical curriculum. The appropriate standard of information to include was guided by GMC requirements in understanding ‘normal human structure and function’ (General Medical C 2018), while the key points for teaching were informed by the University of Southampton Medical School curriculum. Relevant information was organized to produce the script for narration and guide the production of appropriate illustrations using Pixelmator for IOS® (Pixelmator-Team-Ltd.-& Apple-Inc.), an illustration app on the iPad Pro (Apple Inc.). The script and static images were combined to produce a textbook-style resource which would be used as the traditional learning resource in the study. A narration of the script was recorded (using-Blue®, digital Spark USB condenser microphone), and processed in the sound editing app Audacity® (The Audacity Team Inc.). To standardize the structure of both the interactive video and the screencast, a template slideshow was produced, for which all videos were based upon. Using the animation software Doodly® (# Bryxen, Inc), the static images were animated to appear to be hand-drawn live. These animated drawings featured in the screencast, including short, animated annotations accompanying the images, evoke the feel of a live demonstration. The narration and animated slides were combined to produce the screencasts in line with the pre-defined structural features of a screencast (Sugar et al. 2010). Using the learning objectives produced from the original information resources, questions were produced for the interactive video. These questions were integrated with narration, static images, and text information accompanying the images. Approaches to producing the interactive videos improved on previous designs by standardizing the intrinsic load, ensuring that the only difference between the teaching modalities was the modality in which the information is presented. This methodology is shown in the schematic in Fig. 14.7. In all, three pain

328

A. Robson et al.

Fig. 14.7 Schematic to produce different resources while ensuring the intrinsic information remains the same

physiology videos were produced for each modality, ranging from 8:30–11 min in length.

14.3.2

Participant Recruitment

In the pilot study, the mean knowledge gain for the text resource was 37.5%, 3.8%. To detect a 10% variation from this, a power calculation for continuous data of an independent sample study determined each group in this investigation required at least 16 participants (48 at least for each cohort). This recruitment target was achieved in this study. Sixty fourth- and fifth-year BM5 BM/BS medical students were recruited for the histology group while seventy-five second-year students were recruited for the pain physiology component. Students were recruited via social media, poster campaigns, and word of mouth. The histology group were incentivized with a £10 e-voucher for completion of the entire research commitment. For data collection, students were randomly allocated to each group. Students were given a specific project ID number to protect their identities and confidentiality.

14.3.3

Knowledge Testing

Participants completed the pre-teaching multiple choice questionnaire (MCQ) on the subject taught, then completed a pre-teaching perception questionnaire, using a 10-point Likert scale style survey. Participants were subsequently given access to the appropriate teaching resources for their allocated group: either a paper copy of the text resource, or online access details for the different video resources. The total time on task was set as double the total length of the video resource to encourage natural user interaction whilst maintaining a controlled environment. Following this each group completed a post-teaching MCQ test and a post-teaching perceptions questionnaire. This structure is outlined in Fig. 14.8. The perception questionnaires were designed to include a combination of paired question types (in the pre-teaching and post-teaching questionnaires) to assess for a change in perception. Additionally, standalone questions were also included to assess broader themes and attitudes towards the topic. A further MCQ was sent to participants 3 weeks after the teaching session to

14

Assessing the Impact of Interactive Educational Videos and Screencasts. . .

329

Fig. 14.8 Schematic outlining the structure of the teaching sessions Table 14.1 Demographics data of the participants Demographics data Cohort Pain physiology

Histology

Resource Text Screencast Interactive Video Text Screencast Interactive Video

assess knowledge retention. In summary, the data collected were: 1. Pre-teaching, post-teaching, and retention MCQ scores. 2. Pre- and post-teaching perceptions. All research carried out in this study was approved by the faculty of Medicine’s Ethics and Research Governance committee (ID: 32080) University of Southampton. All statistical tests and graph production were performed in GraphPad Prism® (GraphPad Software, LLC). The data sets were normally distributed, as demonstrated by the application of the D’Agostino–Pearson test for normal distribution. To statistically compare the pre-teaching score in comparison to the post-teaching (and their respective retention scores) from each study group, student’s paired t tests were used. To compare the learning gain (and retained learning) across all three groups, an ANOVA was used. For the student perception analysis, a Wilcoxon signed rank test was used to assess for any significant change between the paired pre-perception and post-perception ratings within

Mean age 20.7 20.4 20.5 22.3 22.7 22.4

Gender 9 M 13F 9 M 17F 12 M 14F 10 M 10F 15 M 5F 11 M 9F

each group. To compare these ratings between groups the Mann–Whitney U analysis was performed. Three of the modalities within the survey were statistically compared using Kruskal–Wallis ANOVA. Combining learning gain and student perception data enabled a detailed and correlational assessment of TEL resources.

14.4 14.4.1

Results Demographics

Table 14.1 shows the randomization of participants was suitable, with similar gender and age distributions for each resource in the relative cohorts.

14.4.2

Knowledge Testing

14.4.2.1 Histology Resources: Learning Gain and Retention For histology, all groups showed a significant increase from the pre-teaching score in the post-

330

A. Robson et al.

Fig. 14.9 Graph of learning gain and retention for each modality in the histology cohort

teaching MCQ and retention MCQ, P < 0.001. Figure 14.9 shows the mean post-teaching normalized learning gain was: text—80.0% (n ¼ 20), interactive video—74.4% (n ¼ 20), and screencast—68.3% (n ¼ 20). There was no significant difference between the different modalities in terms of post-teaching learning gain for histology (P > 0.05). The mean normalized retained learning gain was text—36.2% (n ¼ 20), interactive video— 46.7% (n ¼ 19), and screencast—41.4% (n ¼ 16). There was no significant difference between the different modalities in terms of retained learning gain for histology (P > 0.05).

14.4.2.2 Pain Physiology Resources: Learning Gain and Retention For pain physiology, all groups showed a significant increase from the pre-teaching score for the post-teaching MCQ and retention MCQ (P < 0.001). Figure 14.10 shows the mean postteaching normalized learning gain was: text— 51.3% (n ¼ 22), interactive video—70.1% (n ¼ 26), and screencast—53.8% (n ¼ 26). The interactive video group demonstrated a significantly greater normalized learning gain in the post-teaching MCQ than the text resource group (P < 0.01) and the screencast group (P < 0.05). There was no significant difference between the text resource and the screencast (P > 0.05). The mean normalized retained learning gain was: text—35.1% (n ¼ 13), interactive

video—46.0% (n ¼ 12), and screencast—41.4% (n ¼ 17). There was no significant difference between the different modalities in terms of retained learning gain for pain physiology (P > 0.05). Fewer pain physiology participants responded to the retention test compared to the histology group (43.2% compared to 91.7%). This is likely due to the study design, as those who took part in the histology teaching research were offered a £10 e-voucher for completing the retention MCQ, whereas no such reward was available for the pain physiology teaching at the time. Despite this, the post-hoc power of the retention investigation was 88.9%. For all groups, there was some loss of knowledge between the original study session and the retention test.

14.4.3

Student Attitudes, Perceptions, and User Experience

14.4.3.1 Standalone Questions: Post-teaching Perceptions Participants who answered ‘n/a’ to this set of questions were excluded from the analysis. The data from the histology and pain physiology cohorts revealed strong agreement for the item ‘I found this new teaching resource to be more effective for teaching than traditional resources’ for both interactive videos (7.56  0.26) and screencasts (8.04  0.22) (see Fig. 14.6).

14

Assessing the Impact of Interactive Educational Videos and Screencasts. . .

331

Fig. 14.10 Graph of learning gain and retention for each modality in the pain physiology cohort

Mann–Whitney U analysis indicated that there was no significant difference between the video groups (P > 0.05) (Fig. 14.11). When combining the data from both cohorts, perception scores suggested strong agreement for the item ‘I would like to see this resource being integrated with teaching in the future’ for the text resource (7.47  0.39), interactive videos (8.60  0.24) and screencasts (8.89  0.18) (see Fig. 14.7). A Kruskal–Wallis ANOVA indicated a significant difference between each video modality and the text resource (P < 0.05). There was no significant difference between the video modalities (P > 0.05) (Fig. 14.12).

14.4.3.2 Histology Perceptions: Paired Questions Pre- vs. post-teaching changes in perception using 10-point Likert scales were calculated using Wilcoxon Signed rank tests. These data reveal that experience from all study groups had Fig. 14.11 Comparing the video modalities for preference to traditional teaching resources

a similar effect on student perceptions of the subject, TEL resources, and the value of student vs. staff produced resources. Participants felt much more confident with histology following teaching, despite the resource (P < 0.001) and found the subject less challenging to learn (P < 0.005). When comparing this change in perception using Kruskal–Wallis ANOVA, there was no significant difference between the different teaching modalities on the increase in student confidence (P > 0.05). Video resources decreased perception of how hard histology was to learn more than text (P < 0.05). Table 14.2 collates the paired responses and allows for comparison of these for the histology cohort.

14.4.3.3 Physiology Perceptions: Paired Questions Pre- vs. post-teaching changes in perception on 10-point Likert scales were calculated using Wilcoxon Signed rank tests. These data suggest

332

A. Robson et al.

Fig. 14.12 Comparing student perceptions on the desire to integrate their resource with current teaching

participants felt much more confident with pain physiology following teaching despite the resource being used (P < 0.005) and found the subject less challenging to learn (P < 0.005). A Kruskal–Wallis ANOVA revealed no significant difference in either confidence levels (P > 0.05), or difficulty of the topic (P > 0.05) for any of the study groups. Table 14.3 collates the paired responses and allows for comparison of these for the physiology cohort.

14.4.4

Comparison Between Cohorts

One major difference between the perceptions of the cohorts is that there was not a significant change in preference with who explained the videos (students vs. staff) in the histology group (P > 0.05), whereas all groups in the pain physiology cohort had a significant increase in this preference towards student explanations (P < 0.05). For both cohorts, histology and pain YouTube channels/content were considered highly desirable by participants. The strength of this perception increased across all groups after completion of the study. In addition, there was a strong preference for University of Southampton (e.g., in house or bespoke) resources compared to those

created externally in the pre-teaching questionnaire (scoring from 7.82–9.53/10 in all groups). There was no significant change to this view after the completion of the experiment. Furthermore, all participants showed a significant increase in trust in the quality and accuracy of the teaching resource following teaching (P < 0.01).

14.4.4.1 Perceived Confidence To identify a change in perceived confidence between all video resources (both screencast and interactive video) vs. traditional text resources, all confidence perception ratings were pooled. There was no significant difference in confidence gain between those using the traditional text resource (3.15  0.24, n ¼ 58) and video resources (3.40  0.19, n ¼ 127) (see Figs. 14.9 and 14.13).

14.5 14.5.1

Discussion Introduction

The findings from this study further extend the growing examples within the literature which suggest that screencasts and interactive video are directly comparable to traditional text resources, in terms of their impact on learning and their enhancement of the student experience (Pickering

14

Assessing the Impact of Interactive Educational Videos and Screencasts. . .

333

Table 14.2 Paired perception questions for the histology cohort Pre-

Post-

Pre- vs

Histology Perceptions: Paired

Teaching

teaching

teaching

questions

Modality

Likert

Likert

score

score

Text Interactive I found histology hard to learn

video Screencast Text

I trust the quality and accuracy of

Interactive

the resources

video Screencast Text

I would prefer the content of videos be explained by students as much as by staff

Interactive video

6.95

4.32

-2.63

2.17E-03

N/A

6.45

3.45

-3

4.76E-04

0.689

7.5

3.1

-4.4

1.31E-04

0.031

5.53

8.65

3.12

1.55E-03

N/A

5.4

8.95

3.55

4.87E-04

0.608

5.8

9

3.2

1.86E-04

0.909

6.71

6.93

0.21

0.59

N/A

6.16

6.9

0.74

9.09E-02

0.214

0.117

0.343

0.16

0.351

N/A

7.85

8.05

0.2

0.426

0.931

Screencast

8.95

8.95

0

0.856

0.737

Text

8.79

8.68

-0.11

0.792

N/A

8.47

8.84

0.37

5.23E-02

0.108

I would find a histology YouTube

Interactive

educational channel useful

video Screencast Text Interactive video Screencast Text

histology?

text

0.85

video

understanding basic respiratory

difference

compared to

8.53

Interactive

How confident are you

statistical

of difference

8.15

external institutions

histology?

teaching

7.3

I would prefer UoS videos vs

understanding basic cartilage

Difference

Significance

8.37

Screencast Text

How confident are you

Post-

Interactive video Screencast

8.8

9.35

0.55

2.43E-02

0.24

3.21

7.16

3.95

1.34E-04

N/A

3.3

6.9

3.6

2.91E-04

0.591

3

7.7

4.7

8.85E-05

0.117

4.16

7.74

3.58

2.05E-04

N/A

4

7.85

3.85

2.61E-04

0.721

3.75

7.95

4.2

8.93E-05

0.304

This demonstrates the change in different perceptions in paired questions following teaching. Highlighted results suggest a statistically significant effect

2015, 2017a; Saxena et al. 2008). Crucially, the current study has evaluated the role of both interactive video and screencast capture recordings in two separate and challenging subject areas, and in doing so has further expanded the evidence base for designing and implementing multimedia learning tools, which align to the principles of

CTML. Importantly, by choosing to evaluate single multimedia resources in a controlled environment, this investigation has avoided some of the common pitfalls associated with holistic or whole cohort assessments of TEL that tend to be undertaken within a live curriculum setting (Colliver and McGaghie 2008; Pickering and

334

A. Robson et al.

Table 14.3 Paired perception questions for the pain physiology cohort Pre- vs Post-

Pain Physiology: Paired

Teaching

Pre-

Post-

questions

Modality

teaching

Teaching

Difference

teaching statistical difference 1.14E-03

N/A

6.3

4.7

-1.61

3.22E-03

0.543

Screencast

7.07

5.19

-1.89

1.66E-04

0.181

Text

5.11

9.68

4.58

1.31E-04

N/A

5.1

9

3.9

1.99E-04

0.342

Screencast

6.26

9.44

3.19

3.53E-05

0.426

Text

6.33

7.5

1.17

3.32E-02

N/A

6.89

7.83

0.94

1.20E-02

0.683

Screencast

7.04

7.96

0.92

2.05E-03

0.677

Text

8.56

8.44

-0.13

0.667

N/A

8.76

9.05

0.29

0.282

0.308

Screencast

7.93

8.33

0.41

0.169

0.2232

Text

7.82

9

1.18

4.49E-02

N/A

7.95

8.48

0.52

0.176

0.369

Screencast

8.26

8.89

0.63

4.58E-02

0.34

Text

5.48

7.33

1.86

4.39E-04

N/A

5.05

7.09

2.05

2.27E-03

0.777

5.15

6.93

1.78

1.65E-04

0.879

Interactive video

I trust the quality and accuracy

Interactive

of the teaching resource

video

Interactive video

I would prefer UoS affiliated

Interactive

videos vs external institutions

video

I would find a pain physiology YouTube educational channel useful

How confident are you understanding basic pain physiology?

text

-2

learn

as much as by staff

compared to

4.48

I found pain physiology hard to

videos be explained by students

of difference

6.48

Text

I would prefer the content of

Significance

Interactive video

Interactive video Screencast

Demonstrates the change in different perceptions in paired questions following teaching. Highlighted results suggest a statistically significant effect

Joynes 2016). Despite the artificial environment, the strength of this work is that the findings can be wholly attributed to the resource in question and not confounded by unwanted variables.

14.5.2

Learning Gain and Knowledge Retention

The text and video resources in this study significantly raised student scores in the post-teaching

and retention MCQ tests, compared to that of the pre-teaching MCQ. For the subject of histology, there was no significant difference between the different experimental groups in the post-teaching and retention learning gain (P > 0.05). In contrast, interactive video use resulted in a significantly greater increase in learning, compared to both the screencast (P < 0.05) and text resource (P < 0.01) in the pain physiology group. These results are consistent with work reported by Pickering (2015) who demonstrated that the use

14

Assessing the Impact of Interactive Educational Videos and Screencasts. . .

Fig. 14.13 Combining the video and text perception change data from the questions on confidence of the subjects

of anatomy videos caused a mild, but insignificant increase in knowledge gain in a cross-sectional cohort study within a UK medical curriculum. The current study builds on this evidence by reporting that the interactive video group outperformed both the screencast and text group in both of our subject based experiments (significantly so in one of them). This has meaningful implications for the further understanding of the impact of video resources on learning in basic science medical education, particularly when it is considered that retention scores were consistently (but non-significantly) superior in the interactive video group. Although the current study design chose preand post-teaching MCQs as an objective outcome measure, previous work has reported a significant educational impact of anatomy videos on examination performance in comparison to the previous year (Saxena et al. 2008), though interestingly the same study also reported a non-significant effect on the group that used radiology resources. This suggests that the application of video tools does not always yield consistently significant positive outcomes in such experiments. However, it is possible that the findings might be sensitive to the topic of choice, since radiology is a subject already solely taught based on the interpretation of images using mobile devices, and therefore adopting video as a modality for learning may

335

not have had a notable impact compared to current teaching approaches. However, according to the theory of CTML the video format should still have offered some degree of reduced cognitive load and therefore educational enhancement for the learner, if the resource was developed in accordance with its principles (Trelease 2016; Moreno and Mayer 1999). Presumably this would be achieved by reducing extraneous load through the further inclusion of diagrams (Moreno and Mayer 1999) and reducing overload through the appropriate timely positions of both the auditory and visual signals (modality principle) (Mayer and Moreno 1998). Although there have been some concerns raised over the passive nature of student’s learning through the use of video media (Szpunar et al. 2014; Means et al. 2009), this can be countered by promoting engagement through the use of interactive features (Trelease 2016; Pickering and Swinnerton 2019), and the integration of these resources into a diverse curriculum. In the UK, core pre-clinical teaching of these subject areas remains as face-to-face practical classes, alongside associated small group instruction and traditional lectures (Heylings 2002; Older 2004). However, due to the COVID-19 pandemic, medical education is increasing the use of TEL resources which will likely continue (Rose 2020; Chick et al. 2020; Chatterjee and Chakraborty 2020), and so these resources must be scrutinized thoroughly. Importantly, the results of the present study provide some strong evidence to counteract concern regarding a potential negative or ineffectual impact on student’s learning from video/screencast use that some educators believe might be contributing to false levels of increased confidence. This current work, together with several positions amongst the existing literature, have indicated that improvements in learning gain between the video groups compared to traditional methods is at least equitable (Pickering 2017b; Saxena et al. 2008), and this research suggests that retention of knowledge appears to be positively comparable to that of text only groups, with similar findings in related research (Pickering 2017b; Trowler 2010). This conclusion supports the findings from Pickering’s

336

A. Robson et al.

2016 preliminary findings and is therefore encouraged as an area of further exploration in other areas of anatomy and physiology education (Pickering and Joynes 2016). As it stands, an assessment of the current evidence suggests that students who naturally prefer video resources and rely upon them routinely are not being unknowingly deceived into believing that knowledge acquisition is occurring when it is not. It has recently been documented within the literature that a thorough validated framework for the evaluation of TEL resources would be welcomed in the field. Such a tool has been developed which suggests that a combination of analysis on the user experience and by measuring impact on knowledge (via the calculation of normalized knowledge gain) represents a robust and satisfactory assessment (Pickering 2017a; Pickering and Joynes 2016; Pickering et al. 2019). The current investigation adequately fulfils this criterion, demonstrating that two different video resources are both, at least equally as effective as textbook-style resources for learning two relatively difficult subject areas, histology and pain physiology.

14.5.3

Assessing Interactive Video

The key benefit of interactive video resources is their promotion of student engagement in a resource, and the benefit of questions to reinforce knowledge. The benefit of questions to encourage knowledge retrieval within a teaching resource has reliably demonstrated improvements in learning (Larsen et al. 2008, 2009; Wormald et al. 2009), but the benefits of engagement requires further discussion. Within an educational context, engagement is seen as the quality of effort students themselves devote to educationally purposeful activities that contribute directly to desired outcomes (Krause and Coates 2008), where students construct their own knowledge and personal mental schema by engaging with multiple educational resources (Mayer et al. 2001; Dixson 2015; Krahenbuhl 2016). Student engagement with resources is broken down further into three domains: emotional,

cognitive, and behavioral engagement (Trowler 2010). Emotional engagement relates to enjoyment and interest in the information, and cognitive refers to how a student is intellectually stimulated by and invested in the resource, and so these are influenced by the core information contained within the resource. However behavioral engagement requires active participation which is encouraged by interactive TEL tools. The findings from the current study serve to extend existing knowledge in the field by demonstrating the efficacy of interactive video use alongside that of a screencast and textbook group. In addition, our research suggests that the TEL resources produced improved both student performance in learning gain, and their confidence in the subject suggesting there was also an emotional and cognitive engagement with our TEL resources that was at least equal to that of the text-based resource. To date, the body of prior research has only aimed at comparing a standard screencast/video format with a non-video control group (Lloyd and Robertson 2012; Green et al. 2012; Pickering 2015; Saxena et al. 2008), in a single subject area (usually anatomy) (Barry et al. 2015; Pickering 2015; Jaffar 2012; Azer 2012). Many of the early adopter studies focused purely on investigating either quantitative or qualitative data on the use of video and therefore offered only limited insights into their educational value (Pickering and Joynes 2016; Clunie et al. 2018). The authors believe that the results from this study support more recent advances associating the blending of TEL into modern curricula with positive outcomes (Pickering 2015; Sung et al. 2016; Alexander et al. 2019).

14.5.4

Curriculum Integration of Video Resources

One view on effective distribution of TEL resources is that although it makes sense to have a central repository for such resources, essentially it is the responsibility of instructors to signpost the resources at the most appropriate time and to integrate them with traditional face to face teaching. One approach has been to adopt the use of

14

Assessing the Impact of Interactive Educational Videos and Screencasts. . .

quick response codes on lecture slides (Elmansouri et al. 2018). Another approach has been to use video resources in a flipped classroom design, so that face to face contact is becomes more about working up and synthesizing, analyzing or problem solving around a concept or principle rather than the pure dissemination of information didactically (Chen et al. 2017). The video resources satisfy students’ learning needs whilst providing equitable knowledge gain to textbook-style resources. We would agree with other authors who have acknowledged the important role that textbooks continue to play in anatomy education and not wish to deter people from utilizing them as a resource. The evidence provided from these studies still very much supports their effective use as a learning tool. To ensure the sustainability of video resource production for medical education and fostering the vertical integration of topics through medical programs, it has been suggested that ‘senior medical students can form partnerships with faculty members to achieve some of the mutual benefits similar to those witnessed by hands-on peer assisted learning’ (Border 2019). Once concern raised was that this may be perceived as risky in trusting students’ ability to produce appropriate tools. However, the success of Soton Brain Hub (largely student-produced videos), and the high perceptions Southampton students have of student-produced resources shown in this research are encouraging and help assuage concerns educators may have when partnering with students to develop these educational tools.

14.5.5

Student Perceptions and Preferences

Only 28.9% of participants knew the difference between a screencast and interactive video and were therefore unlikely to consume videos based on this criterion. It is likely that the majority of students would be unfamiliar with the pedagogy underpinning the resources they select or the esoteric nature of the nuanced differences between them. Recent research suggests that although students regularly utilize digital technologies for

337

their own “personal empowerment and entertainment,” if one considered their use of such technology for learning they are not able to effectively utilize technology to specifically support their own learning, and their true skills can frequently fall short of their own perceived abilities (Kennedy and Fox 2013). This may suggest that digital literacy in this generation of students is overestimated, and further education on TEL resources could be useful in helping students maximize their benefits.

14.5.6

Screencasts and Standard Video Formats

It is unknown exactly how participants interact with screencasts, but this particular type of multimedia product has strong links to the generative drawing principle, suggesting that students would follow the gradual step-by-step creation of the taught concept and engage by anticipating each incremental change as the images are produced (Sugar et al. 2010; Terrell 2006). The inherent diversity within a student population suggests that different teaching approaches will appeal to different students and greater knowledge of instructional design techniques in resource creation would enable educators to produce a variety of effective resources. In the current investigation, 37.8% of medical students confirmed that they used YouTube® as a supplementary teaching resource which was lower than anticipated based on reports within the existing literature (Jaffar 2012). This pre-covid figure is likely to have changed since this research was undertaken because of the heavy reliance on e-learning resources in higher education as a result of several national lock downs and social distancing measures put in place across the world (Evans et al. 2020; Hall and Border 2020; Ravi 2020). In the current investigation non-paired perception-based questions determined influences on the decision to watch a video. The number of views was perceived to have a mild positive effect on student’s decision to watch a video (5.92 ¼ slightly agree). In contrast, the comments on published videos were perceived to not have

338

much of an impact (4.29 ¼ disagree-slightly disagree). Interestingly, students showed a strong preference for University of Southampton (UoS)-produced resources in the pre-teaching questionnaire, scoring from 7.82–9.53/10 in all the different groups before the teaching activity took place. This remained high following teaching. This may be due to the students’ desire for content to be tailored and aligned to the learning outcomes of their own curriculum, suggesting that the content is more relevant to their assessments comparted to the available alternatives. As the technology ecosystem facilitating the development of video resources grows, faculty-developed tools can be produced more easily without the need for expertise in animation techniques or software (Pickering 2017a). This infers that the creation of bespoke eLearning tools would be welcomed by students, even if the standard of production is inferior. In fact, it might even suggest that student’s judgement of the quality of a resource is driven more by its anatomical accuracy than it is its professional production value. Another key perception highlighted was whether students preferred videos be explained by students or staff. The faculty of medicine at UoS has a strong reputation for working in partnership with students and co-creating many of its curriculum resources, particularly in anatomy. The benefits of this approach have been discussed elsewhere (Border 2017). In terms of preferring either student or staff produced resources there was a disparity. None of the groups evidenced a significant change in their preference in the histology study (P > 0.05), whereas all groups in the pain physiology cohort showed a significant increase in this preference (P < 0.05). At UoS, an active near-peer team frequently delivers teaching sessions on neurology topics to second year students, therefore these students may be more receptive and comfortable receiving co-created and co-delivered content in this discipline due to their familiarity with student-led teaching (Harrison et al. 2019). It has been reported that aspects of cognitive and social congruence make the narration style of student produced videos more appealing than staff produced

A. Robson et al.

videos by using language and syntax that is relatable and accessible, assisting understanding of the topic (Lockspeiser et al. 2008; Hall et al. 2018).

14.5.7

The Learner Experience

Overall students that used the video resources were very satisfied with them, which is consistent with many studies investigating the student experience of educational video tools (Pickering 2017b; Saxena et al. 2008; Topping 2014; Yilmaz 2017; Evans 2011; Mohorovičić 2012), and an important measurement when holistically appraising the value of TEL resources (Pickering and Joynes 2016; Pickering et al. 2019). Perception scores suggested strong agreement with the statement ‘I found this new teaching resource to be more effective for teaching than traditional resources’ for both interactive videos (7.56  0.26) and screencasts (8.04  0.22). Additionally, students strongly agreed with the statement ‘I would like to see video resources being integrated with teaching in the future’, in the text group (7.47  0.39), interactive video group (8.60  0.24) and screencast group (8.89  0.18), with both video modalities scoring significantly higher than the text resource (P < 0.05), further reinforcing evidence that the video resources were well received, and students engaged with them appropriately. Finally, students strongly agreed with the proposal ‘I will now consider integrating videos into my learning’ for both interactive videos (8.85  0.18) and screencasts (8.38  0.26). In these types of evaluations student satisfaction scores should not be underestimated, but neither should they be used in isolation. The literature has consistently reported that enjoyment and satisfaction leads to greater emotional engagement with and use of teaching tools (DiLullo et al. 2011; Trowler 2010) and this may increase the time ‘spent on task’ and is therefore likely to benefit learning (Brown 2001; Ericsson et al. 1993). So, these data are complementary to knowledge gain information. The fact that these scores tend to be extremely positive have in some part probably contributed to a

14

Assessing the Impact of Interactive Educational Videos and Screencasts. . .

degree of skepticism amongst educators concerning their educational benefits. However, beyond learning gain and satisfaction there are other noteworthy benefits, one of which is the flexibility which they allow (Pickering 2015). When adopted within the field of neuroanatomy, studies have shown that the use of screencasts can lead to a reduction in perceived levels of stress and anxiety for learning the subject and may alleviate the symptoms of neurophobia (Harrison et al. 2019). When such difficult and fast paced topics are taught within a time restricted curriculum, video resources offer students an opportunity to revisit material at their own pace (Geoghegan et al. 2018). For educators, it provides opportunities to deploy blended learning strategies and flipped classroom approaches which have witnessed many benefits (Blair et al. 2016). Such approaches have become increasingly common when deploying teaching strategies during the Covid-19 pandemic (Chatterjee and Chakraborty 2020; Evans et al. 2020). Participants in all video groups reported a significant increase in a feeling that the topic became easier to understand and increased their confidence. These groups indicated a significant increase in their desire for a Youtube® channel dedicated to supporting their learning in subjects they were allocated to. It is likely that their preference would be that the creators of these channels would be from their own institutions rather than commercial companies or external providers.

14.5.8

Methodological Approaches

From a methodological perspective, this investigation adopts the use of use a control group, randomization of the groups (i.e. a randomized controlled trial), recruiting both fourth- and second- year students to act as a representative cohort, and using the same MCQ and teaching time in the investigations for each group. This methodology builds on best practice guidelines recommended for the holistic appraisal of educational resources

339

(Pickering 2017a, b; Pickering and Joynes 2016; Pickering et al. 2019). This improves on previous investigations which would compare different years of medical student examinations, using a previous year’s cohort as a control group. It is acknowledged in literature that differences in the year-on-year exam results may be due to a variety of factors outside the control of the investigators, such as the ability of the student cohort (Pickering and Joynes 2016; Saxena et al. 2008).

14.5.9

Limitations

Although the authors consider this to be a robust measure for assessing the effectiveness of screencast video use in anatomy education the study is not without some limitations. For the period between the teaching session and retention test, there was no effective way to control individual learning or engagement with the material and therefore this may have influenced the retention test scores. Furthermore, the pre/post-test methodology is restrictive since students may more actively pursue knowledge acquisition during the investigation than they would if using the resources privately. Moreover, pre and post test scores offer an artificial experimental environment to gather data and may not be directly comparable to student performances on summative high stakes assessments. However, cohort studies that do compare assessment data in a live curriculum or between curriculums are unable to control for unwanted independent variables which influence student learning outcomes and examination performance. In addition, there is a possibility of a selection bias, as students who chose to come to a teaching session may be more likely to actively engage with the teaching. This had potential to skew the learning gain data, although the use of a control group helps to mitigate this. Finally, this study did not investigate longerterm retention of knowledge, as data collection finished after 5 weeks, so any later differences would not be detected.

340

A. Robson et al.

14.5.10 Future Work The resources produced in this study have been published on YouTube for students to use freely around the world. Analysis of the web data, and feedback from a global audience and on social media can be used to further assess the impact of these videos (Border et al. 2019). Such information would certainly be useful in guiding instructional design principles in the future. Similar research on other subjects, within and outside medicine, using similar methodology could add to the growing body of literature around the use of TEL resources, enabling a deeper understanding of TEL across a variety of disciplines. It might be that screencast/video applications can be compared to other popular media-based deliveries used for anatomy education, such as podcast or virtual patient style eLearning packages. It would be of interest to compare experimental outcomes in relation to their conformity to the theory of CTML. Growing TEL use in higher education must be complemented by robust evaluation strategies to further understand why these resources are so well received by students and improve learning gain to at least the same level as traditional teaching methods. Additionally, suitable methods for integration of these resources need to be developed without hampering current academic achievement. In producing evaluation tools to answer this, students can obtain a robust education that suitably makes use of TEL (Barry et al. 2015; Clunie et al. 2018). A randomized control approach combining student perception and learning gain data should be seen as the optimal standard for the evaluation of individual resources in this area of research (Pickering and Joynes 2016; Pickering et al. 2019; Clunie et al. 2018).

14.6

Conclusion

Collectively, the outcomes from this study support the current trends amongst the literature which consider the use of screencast and

interactive video resources to be at least equal to textbook-style resources in terms of academic improvement and their effects on student perceptions, and that students can effectively utilize appropriate video teaching resources and text-based resources without academic penalties. We hope that this finding, when considered amongst the growing body of literature on the topic, will reassure those tutors who have concerns about the passive nature of video use for learning clinical anatomy topics. The authors advocate for the supplementary application of educational videos when used in a suitable way such as preferably alongside cadaveric material, accompanying large and small group teaching and as a flexible adjunct to encourage engaging, flexible and student-centred learning and flippedclassroom approaches. Funding Partially funded by The Anatomical Society.

References Alexander J, McLachlan S, Barcellona M, Sackley C (2019) Technology-enhanced learning in physiotherapy education: student satisfaction and knowledge acquisition of entry-level students in the United Kingdom. Res Learn Technol 27 Auster CJ (2016) Blended learning as a potentially winning combination of face-to-face and online learning: an exploratory study. Teach Sociol 44(1):39–48. https://doi.org/10.1177/0092055x15619217 Azer SA (2012) Can “YouTube” help students in learning surface anatomy? Surg Radiol Anat 34(5):465–468. https://doi.org/10.1007/s00276-012-0935-x Barry DS, Marzouk F, Chulak-Oglu K, Bennett D, Tierney P, O'Keeffe GW (2015) Anatomy education for the YouTube generation. Anat Sci Educ 9(1):90–96. https://doi.org/10.1002/ase.1550 Blair E, Maharaj C, Primus S (2016) Performance and perception in the flipped classroom. Educ Inf Technol 21(6):1465–1482. https://doi.org/10.1007/s10639015-9393-5 Border S (2017) Working with students as partners in anatomy education. Anat Sci Educ 10(6):613–614. https://doi.org/10.1002/ase.1712 Border S (2019) Assessing the role of screencasting and video use in anatomy education. In: Biomedical visualisation. Springer, pp 1–13 Border S, Hennessy C, Pickering J (2019) The rapidly changing landscape of student social media use in anatomy education. Anat Sci Educ. https://doi.org/10. 1002/ase.1875

14

Assessing the Impact of Interactive Educational Videos and Screencasts. . .

Brown KG (2001) Using computers to deliver training: which employees learn and why? Pers Psychol 54(2):271–296 Campbell J, Mayer RE (2009) Questioning as an instructional method: does it affect learning from lectures? Appl Cogn Psychol 23(6):747–759 Chatterjee I, Chakraborty P (2020) Use of information communication technology by medical educators amid covid-19 pandemic and beyond. J Educ Technol Syst. 0047239520966996 Chen F, Lui AM, Martinelli SM (2017) A systematic review of the effectiveness of flipped classrooms in medical education. Med Educ 51(6):585–597 Chick RC, Clifton GT, Peace KM, Propper BW, Hale DF, Alseidi AA et al (2020) Using technology to maintain the education of residents during the COVID-19 pandemic. J Surg Educ 77(4):729–732 Clunie L, Morris NP, Joynes VCT, Pickering JD (2018) How comprehensive are research studies investigating the efficacy of technology-enhanced learning resources in anatomy education? A systematic review. Anat Sci Educ 11(3):303–319 Colliver JA, McGaghie WC (2008) The reputation of medical education research: quasi-experimentation and unresolved threats to validity. Taylor & Francis Cook DA, Ellaway RH (2015) Evaluating technologyenhanced learning: a comprehensive framework. Med Teach 37(10):961–970. https://doi.org/10.3109/ 0142159X.2015.1009024 DiLullo C, McGee P, Kriebel RM (2011) Demystifying the millennial student: a reassessment in measures of character and engagement in professional education. Anat Sci Educ 4(4):214–226 Dixson MD (2015) Measuring student engagement in the online course: the online student engagement scale (OSE). Online Learn 19(4):n4 Drake RL (2014) A retrospective and prospective look at medical education in the United States: trends shaping anatomical sciences education. J Anat 224(3):256–260. https://doi.org/10.1111/joa.12054 Drake RL, McBride JM, Lachman N, Pawlina W (2009) Medical education in the anatomical sciences: the winds of change continue to blow. Anat Sci Educ 2(6):253–259 Drake RL, McBride JM, Pawlina W (2014) An update on the status of anatomical sciences education in United States medical schools. Anat Sci Educ 7(4):321–325. https://doi.org/10.1002/ase.1468 Draper SW, Brown MI (2004) Increasing interactivity in lectures using an electronic voting system. J Comput Assist Learn 20(2):81–94 Elmansouri A, Myers M, Parton W, Harrison CH, Hall S, Stephens J et al (2018) Using quick response codes to combine online digital media with traditional anatomy teaching methods. Solving clinical problems with anatomical solutions. J Anat 232:1038–1064 Elmansouri A, Murray O, Hall S, Border S (2020) TEL methods used for the learning of clinical

341

neuroanatomy. In: Biomedical visualisation. Springer, pp 43–73 Ericsson KA, Krampe RT, Tesch-Römer C (1993) The role of deliberate practice in the acquisition of expert performance. Psychol Rev 100(3):363 Evans DJR (2011) Using embryology screencasts: a useful addition to the student learning experience? Anat Sci Educ 4(2):57–63. https://doi.org/10.1002/ase.209 Evans DJR, Bay BH, Wilson TD, Smith CF, Lachman N, Pawlina W (2020) Going virtual to support anatomy education: a STOPGAP in the midst of the Covid-19 pandemic. Wiley Online Library General Medical C (2018) Outcomes for graduates 2018. GMC London Geoghegan K, Payne DR, Myers MA, Hall S, Elmansouri A, Parton WJC et al (2018) The National Undergraduate Neuroanatomy Competition: lessons learned from partnering with students to innovate undergraduate neuroanatomy education. Neuroscientist 25(3):271–280. https://doi.org/10.1177/ 1073858418788000 Green KR, Pinder-Grover T, Millunchick JM (2012) Impact of screencast technology: connecting the perception of usefulness and the reality of performance. J Eng Educ 101(4):717–737. https://doi.org/10.1002/j. 2168-9830.2012.tb01126.x Greene SJ (2018) The use and effectiveness of interactive progressive drawing in anatomy education. Anat Sci Educ Hall S, Border S (2020) Online neuroanatomy education and its role during the coronavirus disease 2019 (Covid-19) lockdown. World Neurosurg Hall S, Harrison CH, Stephens J, Andrade MG, Seaby EG, Parton W et al (2018) The benefits of being a near-peer teacher. Clin Teach 15(5):403–407 Harden RM (2008) Death by PowerPoint–the need for a ‘fidget index’. Med Teach 30(9–10):833–835 Harrison CH, Elmansouri A, Parton W, Myers MA, Hall S, Stephens JR et al (2019) The efficacy of frontline nearpeer teaching in a modern medical curriculum. Anat Sci Educ 12(3):236–244 Heylings DJA (2002) Anatomy 1999–2000: the curriculum, who teaches it and how? Med Educ 36(8):702–710 Jaffar AA (2012) YouTube: an emerging tool in anatomy education. Anat Sci Educ 5(3):158–164. https://doi. org/10.1002/ase.1268 Jego EH (2019) E-Learning in medical education in Japan and around the world. 日大医学雑誌 78(5):331–333 Kaestle CF (1993) Research news and comment: the awful reputation of education research. Educ Res 22(1):23–31 Kennedy DM, Fox R (2013) ‘Digital natives’: an Asian perspective for using learning technologies. Int J Educ Dev Using ICT 9(1):65–79 Krahenbuhl KS (2016) Student-centered education and constructivism: challenges, concerns, and clarity for teachers. Clearing House 89(3):97–105

342 Krause KL, Coates H (2008) Students’ engagement in first-year university. Assess Eval High Educ 33(5):493–505 Larsen DP, Butler AC, Roediger Iii HL (2008) Testenhanced learning in medical education. Med Educ 42(10):959–966 Larsen DP, Butler AC, Roediger Iii HL (2009) Repeated testing improves long-term retention relative to repeated study: a randomised controlled trial. Med Educ 43(12):1174–1181 Lloyd SA, Robertson CL (2012) Screencast tutorials enhance student learning of statistics. Teach Psychol 39(1):67–71. https://doi.org/10.1177/ 0098628311430640 Lockspeiser TM, O’Sullivan P, Teherani A, Muller J (2008) Understanding the experience of being taught by peers: the value of social and cognitive congruence. Adv Health Sci Educ 13(3):361–372 Louw G, Eizenberg N, Carmichael SW (2009) The place of anatomy in medical education: AMEE guide no 41. Med Teach 31(5):373–386. https://doi.org/10.1080/ 01421590902825149 Lujan HL, DiCarlo SE (2006) First-year medical students prefer multiple learning styles. Adv Physiol Educ 30(1):13–16. https://doi.org/10.1152/advan.00045. 2005 Lurie SJ (2003) Raising the passing grade for studies of medical education. JAMA 290(9):1210–1212 Mahmud W, Hyder O, Butt J, Aftab A (2011) Dissection videos do not improve anatomy examination scores. Anat Sci Educ 4(1):16–21. https://doi.org/10.1002/ ase.194 Matheson C (2008) The educational value and effectiveness of lectures. Clin Teach 5(4):218–221 Mayer RE (1997) Multimedia learning: are we asking the right questions? Educ Psychol 32(1):1–19 Mayer RE (2009) Multimedia learning, 2nd edn. Cambridge University Press, New York Mayer RE, Anderson RB (1992) The instructive animation: helping students build connections between words and pictures in multimedia learning. J Educ Psychol 84(4):444 Mayer RE, Moreno R (1998) Split-attention effect in multimedia learning: evidence for dual processing systems in working memory. J Educ Psychol 90(2):312–320. https://doi.org/10.1037//0022-0663.90.2.312 Mayer RE, Steinhoff K, Bower G, Mars R (1995) A generative theory of textbook design: using annotated illustrations to foster meaningful learning of science text. Educ Technol Res Dev 43(1):31–41 Mayer RE, Heiser J, Lonn S (2001) Cognitive constraints on multimedia learning: when presenting more material results in less understanding. J Educ Psychol 93(1):187–198. https://doi.org/10.1037//0022-0663. 93.1.187 McHanwell S, Davies DC, Morris J, Parkin I, Whiten S, Atkinson M et al (2019) A core syllabus in anatomy for medical students-adding common sense to need to know. Eur J Anat 11(S1):3–18

A. Robson et al. Means B, Toyama Y, Murphy R, Bakia M, Jones K (2009) Evaluation of evidence-based practices in online learning: a meta-analysis and review of online learning studies Mohorovičić S (2012) Creation and use of screencasts in higher education. IEEE. pp 1293–1298 Moreno R, Mayer RE (1999) Cognitive principles of multimedia learning: the role of modality and contiguity. J Educ Psychol 91(2):358–368. https://doi.org/10.1037/ 0022-0663.91.2.358 Morris C, Chikwa G (2014) Screencasts: how effective are they and how do students engage with them? Act Learn High Educ 15(1):25–37. https://doi.org/10.1177/ 1469787413514654 Mousavi SY, Low R, Sweller J (1995) Reducing cognitive load by mixing auditory and visual presentation modes. J Educ Psychol 87(2):319 Older J (2004) Anatomy: a must for teaching the next generation. Surgeon 2(2):79–90. https://doi.org/10. 1016/S1479-666X(04)80050-7 Paas F, Tuovinen JE, Tabbers H, Van Gerven PWM (2003) Cognitive load measurement as a means to advance cognitive load theory. Educ Psychol 38(1):63–71. https://doi.org/10.1207/ s15326985ep3801_8 Pickering JD (2015) Anatomy drawing screencasts: enabling flexible learning for medical students. Anat Sci Educ 8(3):249–257. https://doi.org/10.1002/ase. 1480 Pickering JD (2017a) Developing the evidence-base to support the integration of technology-enhanced learning in healthcare education. Med Sci Educ 27(4):903–905. https://doi.org/10.1007/s40670-0170424-2 Pickering JD (2017b) Measuring learning gain: comparing anatomy drawing screencasts and paper-based resources. Anat Sci Educ 10(4):307–316. https://doi. org/10.1002/ase.1666 Pickering JD, Joynes VCT (2016) A holistic model for evaluating the impact of individual technologyenhanced learning resources. Med Teach 38(12):1242–124 7. h ttps ://doi.org/10.1080/ 0142159X.2016.1210112 Pickering JD, Roberts DJH (2018) Flipped classroom or an active lecture? Clin Anat 31(1):118–121 Pickering JD, Swinnerton BJ (2019) Exploring the dimensions of medical student engagement with technology-enhanced learning resources and assessing the impact on assessment outcomes. Anat Sci Educ 12(2):117–128. https://doi.org/10.1002/ase.1810 Pickering JD, Lazarus MD, Hallam JL (2019) A Practitioner’s guide to performing a holistic evaluation of technology-enhanced learning in medical education. Med Sci Educ 29(4):1095–1102 Prober CG, Heath C (2012) Lecture halls without lectures—a proposal for medical education. N Engl J Med 366(18):1657–1659 Ravi KS (2020) Dead body management in times of Covid-19 and its potential impact on the availability

14

Assessing the Impact of Interactive Educational Videos and Screencasts. . .

of cadavers for medical education in India. Anat Sci Educ 13(3):316–317 Reid S, Shapiro L, Louw G (2018) How haptics and drawing enhance the learning of anatomy. Anat Sci Educ Rose S (2020) Medical student education in the time of COVID-19. JAMA 323(21):2131–2132 Ruiz JG, Mintzer MJ, Leipzig RM (2006) The impact of E-learning in medical education. Acad Med 81(3) Saxena V, Natarajan P, O'Sullivan PS, Jain S (2008) Effect of the use of instructional anatomy videos on student performance. Anat Sci Educ 1(4):159–165. https://doi. org/10.1002/ase.38 Sugar W, Brown A, Luterbach K (2010) Examining the anatomy of a screencast: uncovering common elements and instructional strategies. Int Rev Res Open Distrib Learn 11(3):1–19 Sung Y-T, Chang K-E, Liu T-C (2016) The effects of integrating mobile devices with teaching and learning on students’ learning performance: a meta-analysis and research synthesis. Comput Educ 94:252–275 Sweller J (1988) Cognitive load during problem solving: effects on learning. Cogn Sci 12(2):257–285 Szpunar KK, Jing HG, Schacter DL (2014) Overcoming overconfidence in learning from video-recorded lectures: implications of interpolated testing for online education. J Appl Res Mem Cogn 3(3):161–164. https://doi.org/10.1016/j.jarmac.2014.02.001 Terrell M (2006) Anatomy of learning: instructional design principles for the anatomical sciences. Anat

343

Rec Part B 289B(6):252–260. https://doi.org/10.1002/ ar.b.20116 Todres M, Stephenson A, Jones R (2007) Medical education research remains the poor relation. BMJ 335(7615):333–335 Topping DB (2014) Gross anatomy videos: student satisfaction, usage, and effect on student performance in a condensed curriculum. Anat Sci Educ 7(4):273–279 Trelease RB (2016) From chalkboard, slides, and paper to e-learning: how computing technologies have transformed anatomical sciences education. Anat Sci Educ 9(6):583–602. https://doi.org/10.1002/ase.1620 Trowler V (2010) Student engagement literature review. Higher Educ Acad 11(1):1–15 Udell J (2005) What is screencasting. O’Reilly digitalmedia [Blog]. 16 Wong WC, Tay SS (2005) The teaching of anatomy: the first hundred years (1905-2005). Ann Acad Med Singap 34:72C–78C Wormald BW, Schoeman S, Somasunderam A, Penn M (2009) Assessment drives learning: an unavoidable truth? Anat Sci Educ 2(5):199–204 Yilmaz R (2017) Exploring the role of e-learning readiness on student satisfaction and motivation in flipped classroom. Comput Hum Behav 70:251–260 Zhang DS, Zhou LN, Briggs RO, Nunamaker JF (2006) Instructional video in e-learning: assessing the impact of interactive video on learning effectiveness. Inf Manag 43(1):15–27. https://doi.org/10.1016/j.im. 2005.01.004

Correction to: How Artificial Intelligence and Machine Learning Is Assisting Us to Extract Meaning from Data on Bone Mechanics? Saeed Mouloodi, Hadi Rahmanpanah, Colin Burvill, Colin Martin, Soheil Gohari, and Helen M. S. Davies

Correction to: Chapter 9 in: P. M. Rea (ed.), Biomedical Visualisation, Advances in Experimental Medicine and Biology 1356, https://doi.org/10.1007/978-3-030-87779-8_9 The original version of the book was inadvertently published without listing Colin Burvill as a co-author. The correct group of authors has been updated as below and the updated chapter has been approved by the authors. Saeed Mouloodi, Hadi Rahmanpanah, Colin Burvill, Colin Martin, Soheil Gohari, Helen M. S. Davies

The updated original version for this chapter can be found at https://doi.org/10.1007/978-3-030-87779-8_9 # The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 P. M. Rea (ed.), Biomedical Visualisation, Advances in Experimental Medicine and Biology 1356, https://doi.org/10.1007/978-3-030-87779-8_15

C1