3D Physical and Virtual Models in Fetal Medicine: Applications and Procedures 3031148541, 9783031148545

Table of contents :
Foreword
Introduction
Contents
Contributors
1: The Use of 3D Representations in Fetal Medicine
References
Part I: Digital Input
Digital Input.
2: Imaging Technologies: Ultrasound, Computed Tomography, Magnetic Resonance Imaging, Micro-CT, 3D Scanner
2.1 Ultrasound
2.2 Computed Tomography
2.3 Magnetic Resonance Imaging
2.4 Microtomography
2.5 3D Scanners and Photogrammetry
References
3: Post-processing Images to Generate the 3D Data
3.1 Segmentation
3.2 3D Models
3.2.1 Treatment and Smoothing
3.2.2 Models and Assemblies
3.3 Archives Format
References
Part II: Physical Output
Physical Output.
4: (3D) Printing Technologies
4.1 Fused Deposition Modeling (Solid-Based System) or FDM
4.2 Selective Laser Sintering (Powder-Based System) or SLS
4.3 BJT: Binder Jetting (Joined with Bonding Plaster/Gypsum)
4.4 Material Jetting (MJ)
4.5 Stereolithography (SLA)
4.6 Digital Light Processing/Liquid Crystal Display: DLP/ LCD
4.7 Availability
4.8 Cost per Model
4.9 Productivity
4.10 Ease of Use
4.11 Average Build Volume
4.12 Print Time
4.13 Resolution and Post-Processing
4.14 Material
4.15 How to Apply Each One
4.16 Conclusion
References
Part III: Virtual Output
Virtual Output.
5: Virtual Navigation on Expanded Reality Devices (Virtual Reality, Augmented Reality, and Expanded Reality)
References
6: Artificial Intelligence Techniques for Fetal Medicine
6.1 The Present State of AI
6.2 AI in Fetal Medicine
6.3 Challenges to AI Approaches in Fetal Medicine and General Evaluation Criteria
6.4 Example: Amniotic Fluid Segmentation
6.5 Conclusion
References
7: Metaverse in Fetal Medicine
7.1 From Virtuality Continuum to Internet 4.0
7.2 Expanded Reality for Mediatic Shared Experiences
7.3 Perspectives for Fetal Medicine and Experiments
References
Part IV: Applicability in Clinical Cases
8: Three-Dimensional Printing and Virtual Models in Fetal Medicine
8.1 Study of Fetal Pathologies
8.1.1 First Trimester
8.1.2 Second and Third Trimesters
8.1.3 Central Nervous System
8.1.3.1 Ventriculomegaly
8.1.3.2 Anencephaly
8.1.3.3 Holoprosencephaly
8.1.3.4 Microcephaly
8.1.3.5 Fetal Intracranial Hemorrhage
8.1.3.6 Chiari Malformation
8.1.3.7 Dandy-Walker Malformation
8.1.3.8 Encephalocele
8.1.4 Face
8.1.4.1 Cleft Lip and Palate
8.1.4.2 Tumors (Epignathus Teratoma)
8.1.5 Cervical Masses
8.1.5.1 Fetal Goiter
8.1.5.2 Lymphangioma
8.1.5.3 Teratoma
8.1.6 Chest Anomalies
8.1.6.1 Congenital Diaphragmatic Hernia
8.1.6.2 Congenital High Airway Obstruction Syndrome
8.1.7 Congenital Heart Disease
8.1.8 Abdominal Anomalies
8.1.8.1 Omphalocele
8.1.8.2 Gastroschisis
8.1.8.3 Limb–Body Wall Complex (LBWC)
8.1.8.4 Sacrococcygeal Teratomas
8.1.9 Genitourinary Anomalies
8.1.9.1 Lower Urinary Tract Obstruction
8.1.10 Fetal Musculoskeletal Disorders
8.1.11 Genetics
References
9: 3D Printing and Virtual Models Assisting Fetal Surgeries
9.1 Introduction
9.2 Rational for Fetal Surgery
9.2.1 Bidimensional Ultrasound (2D US) in Fetal Surgeries
9.2.2 Three-Dimensional Ultrasonography (3D US) and Four-Dimensional Ultrasonography (4D US) in Fetal Surgeries
9.2.3 3D Printing Models in Fetal Surgeries
9.2.4 Virtual Reconstruction in Fetal Surgery
9.3 Conclusion
9.4 Assistance for Fetal Surgery
References
10: Postnatal Surgery
10.1 Clinical Study 1 (Neurosurgery): Craniopagus Twins—A Challenge
10.2 Clinical Study 2 (Pediatric Surgery)
10.2.1 Jejunal and Ileal Atresia
10.2.2 Esophageal Atresia with Distal Fistula
10.2.3 Choledochal Cyst
References
11: Multiple Pregnancy
11.1 Introduction
11.2 The Role of Ultrasound
11.3 Computed Tomography and Magnetic Resonance Imaging
11.4 Multiplanar Ultrasound and Model Printing
11.5 Conclusions
References
12: Maternal–Fetal Attachment in Blind Women Using Physical Model
References
Part V: Experimental and Future Applications
13: Haptics/Force Feedback Technologies
13.1 Devices
13.1.1 Actuators
13.1.1.1 Haptic Gloves
13.1.1.2 Force Feedback Pen
13.2 Process
13.2.1 Optimization of the 3D Model
13.2.2 Visualization
13.2.3 Smoothness and Roughness
13.2.4 Tenacity Map
13.2.5 Deformity of the Surface
13.3 Case Study
References
14: Recent Advances in 3D Bioprinting Technologies and Possibilities for the Fetal Medicine
14.1 Introduction
14.2 Recent Advances in 3D Bioprinting Research
14.2.1 Placenta-on-a-Chip
14.2.2 3D Bioprinted Outer-Blood-Retina for Anti-Zika Small-Molecules Discovery
14.2.3 Bone Regeneration with 3D Bioprinting
14.2.4 Bioprinted Autologous Heart Valve Implants with Regenerative Capabilities and Life-Long Durability
14.3 3D Bioprinting and Fetal Medicine
14.4 Considerations
References

Citation preview

3D Physical and Virtual Models in Fetal Medicine Applications and Procedures Heron Werner Gabriele Tonni Jorge Lopes

123

3D Physical and Virtual Models in Fetal Medicine

Heron Werner • Gabriele Tonni Jorge Lopes

3D Physical and Virtual Models in Fetal Medicine Applications and Procedures

Heron Werner Laboratório de Biodesign (Dasa/ PUC-Rio) Alta Excelência Diagnóstica (Dasa) Rio de Janeiro, Brazil

Gabriele Tonni Department of Obstetrics & Gynecology IRCCS Azienda USL Reggio Emilia, Italy

Jorge Lopes Laboratório de Biodesign (Dasa/ PUC-Rio) Instituto Nacional de Tecnologia (INT) Rio de Janeiro, Brazil

ISBN 978-3-031-14854-5    ISBN 978-3-031-14855-2 (eBook) https://doi.org/10.1007/978-3-031-14855-2 © Springer Nature Switzerland AG 2023 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Foreword

On a sunny day in early October 2008, I met Jorge Lopes outside the Royal College of Art in London where he had been accepted to study for a Ph.D. He had asked if he could show me his rapid prototyping apparatus and some of the intrauterine fetal models he had made under the leadership of Dr Heron Werner in Rio de Janeiro. I must admit I was blown away by the originality of his project and the maturity and insight of the young researcher. I agreed to co-supervise his thesis and to provide some 3D ultrasound images for him to transform into physical models. In a short space of time, Jorge had converted my 3D ultrasound images into beautiful detailed fetal models that could be handled and examined. Rapid prototyping or additive manufacturing popularly known as 3D printing is now recognized as an important advance in the imaging of the unborn child. For example, by feeling the physical model within a few minutes of the scan it will provide mothers with a tangible expression of their unborn baby, reduce imagined fears, and improve bonding. This is especially true if the fetus has an abnormality such as cleft palate or spina bifida when frequently mothers have fearful imaginings of the defect, and a physical model will enable them to come to terms with the abnormality before birth. A 3D printed model will also help mothers with impaired vision who cannot see the ultrasound screen, bond with their unborn child. Dr Werner and Jorge Lopes have continued to pioneer the application of 3D printing to explore numerous clinical applications and combine 3D ultrasound with MRI imaging to provide models of the entire fetus. My collaboration and friendship with Heron and Jorge have continued even after Jorge had finished his Ph.D. and returned to Rio de Janeiro. They have continued to pioneer advances in prenatal 3D printing and have extended their research into dynamic 3D virtual video fetoscopy with MRI and micro-­CT which greatly assist in planning fetal and postnatal surgery, and they are now taking us into the metaverse where participants from different departments and geographical locations can interactively navigate through slices of 3D data. Heron Werner and his multidisciplinary team in Rio de Janeiro have now created arguably the foremost imaging center in the world and this book with its associated images and videos will transport you into the exciting future of imaging. It is a “must have” book for all those interested in imaging in fetal medicine and gynecology. Stuart Campbell Create Health Clinic London, UK

v

Introduction

The aim of this book is to demonstrate the current state of the art in terms of the production of 3D virtual and physical models in fetal medicine: the evolution from conventional representations to digital technologies, highlighting the improvements made during a period of continuous evolutionary technological progress including current and diverse topics as the use of artificial intelligence. Basically, 3D virtual models can be developed and combined from three sources: built on software from coordinate input (parametric or not), from 3D scanning surface acquisition technologies (such as photogrammetry, laser scanners, and structured light), and directly from files obtained through noninvasive image technologies (such as 3D ultrasound, magnetic resonance imaging, computed tomography, and micro-CT scanners). In fetal medicine, volumetric images obtained by ultrasound or magnetic resonance imaging are used routinely to provide 3D models for surface/volumetric reconstruction. Having the 3D data files, it is possible also to materialize very accurate and reliable physical models through additive manufacturing technologies. Nowadays, it is possible to physically reproduce features as color, specific gravity, densities, transparency, and even biomaterials, giving real characteristics to the final reproduced parts when compared to the human body. The same data can be also used to develop virtual navigation (VN), giving the possibility to the user to traveling inside realistic 3D environment of the fetus and mother, allowing features as the visualization of fetal malformations, the umbilical cord, and placenta. The VN can be experienced direct on a varied type of screens, ranging from mobile phones to computers, as well from virtual reality (VR) and augmented reality (AR) technologies that use software generated realistic images enriched by other senses, such as sound and tactile sensations through haptic technologies, to replicate an immersive sensation of a real body environment. VR/AR systems are growing in popularity in the medical community as clinicians and researchers become aware of its potential benefits and tend to increase the possibilities with the advent of metaverse also described on this book.

vii

Contents

1 The  Use of 3D Representations in Fetal Medicine������������������������   1 References������������������������������������������������������������������������������������������   6 Part I Digital Input 2 Imaging Technologies: Ultrasound, Computed Tomography, Magnetic Resonance Imaging, Micro-CT, 3D Scanner����������������  11 2.1 Ultrasound��������������������������������������������������������������������������������  11 2.2 Computed Tomography������������������������������������������������������������  15 2.3 Magnetic Resonance Imaging��������������������������������������������������  16 2.4 Microtomography ��������������������������������������������������������������������  19 2.5 3D Scanners and Photogrammetry��������������������������������������������  20 References������������������������������������������������������������������������������������������  21 3 Post-processing  Images to Generate the 3D Data ������������������������  25 3.1 Segmentation����������������������������������������������������������������������������  26 3.2 3D Models��������������������������������������������������������������������������������  28 3.2.1 Treatment and Smoothing��������������������������������������������  29 3.2.2 Models and Assemblies������������������������������������������������  29 3.3 Archives Format������������������������������������������������������������������������  32 References������������������������������������������������������������������������������������������  33 Part II Physical Output 4 (3D) Printing Technologies��������������������������������������������������������������  37 4.1 Fused Deposition Modeling (Solid-Based System) or FDM��������������������������������������������������������������������������������������  39 4.2 Selective Laser Sintering (Powder-Based System) or SLS ��������������������������������������������������������������������������������������  39 4.3 BJT: Binder Jetting (Joined with Bonding Plaster/Gypsum)������������������������������������������������������������������������  40 4.4 Material Jetting (MJ)����������������������������������������������������������������  40 4.5 Stereolithography (SLA)����������������������������������������������������������  41 4.6 Digital Light Processing/Liquid Crystal Display: DLP/ LCD��������������������������������������������������������������������������������  41 4.7 Availability��������������������������������������������������������������������������������  42 4.8 Cost per Model��������������������������������������������������������������������������  43 ix

Contents

x

4.9 Productivity������������������������������������������������������������������������������  44 4.10 Ease of Use ������������������������������������������������������������������������������  45 4.11 Average Build Volume��������������������������������������������������������������  47 4.12 Print Time���������������������������������������������������������������������������������  47 4.13 Resolution and Post-Processing������������������������������������������������  47 4.14 Material ������������������������������������������������������������������������������������  51 4.15 How to Apply Each One ����������������������������������������������������������  55 4.16 Conclusion��������������������������������������������������������������������������������  60 References������������������������������������������������������������������������������������������  60 Part III Virtual Output 5 Virtual  Navigation on Expanded Reality Devices (Virtual Reality, Augmented Reality, and Expanded Reality)����������������������������������������������������������������������������  63 References������������������������������������������������������������������������������������������  68 6 Artificial  Intelligence Techniques for Fetal Medicine������������������  71 6.1 The Present State of AI ������������������������������������������������������������  71 6.2 AI in Fetal Medicine ����������������������������������������������������������������  72 6.3 Challenges to AI Approaches in Fetal Medicine and General Evaluation Criteria������������������������������������������������  72 6.4 Example: Amniotic Fluid Segmentation����������������������������������  73 6.5 Conclusion��������������������������������������������������������������������������������  75 References������������������������������������������������������������������������������������������  75 7 Metaverse  in Fetal Medicine ����������������������������������������������������������  77 7.1 From Virtuality Continuum to Internet 4.0 ������������������������������   77 7.2 Expanded Reality for Mediatic Shared Experiences����������������  80 7.3 Perspectives for Fetal Medicine and Experiments��������������������  81 References������������������������������������������������������������������������������������������  82 Part IV Applicability in Clinical Cases 8 Three-Dimensional  Printing and Virtual Models in Fetal Medicine������������������������������������������������������������������������������  85 8.1 Study of Fetal Pathologies��������������������������������������������������������  85 8.1.1 First Trimester��������������������������������������������������������������  85 8.1.2 Second and Third Trimesters����������������������������������������  86 8.1.3 Central Nervous System ����������������������������������������������  86 8.1.4 Face ������������������������������������������������������������������������������  97 8.1.5 Cervical Masses������������������������������������������������������������  99 8.1.6 Chest Anomalies ���������������������������������������������������������� 111 8.1.7 Congenital Heart Disease���������������������������������������������� 115 8.1.8 Abdominal Anomalies�������������������������������������������������� 118 8.1.9 Genitourinary Anomalies���������������������������������������������� 125 8.1.10 Fetal Musculoskeletal Disorders���������������������������������� 126 8.1.11 Genetics������������������������������������������������������������������������ 129 References������������������������������������������������������������������������������������������ 133

Contents

xi

9 3D  Printing and Virtual Models Assisting Fetal Surgeries���������� 137 9.1 Introduction������������������������������������������������������������������������������ 137 9.2 Rational for Fetal Surgery�������������������������������������������������������� 137 9.2.1 Bidimensional Ultrasound (2D US) in Fetal Surgeries���������������������������������������������������������� 138 9.2.2 Three-Dimensional Ultrasonography (3D US) and Four-Dimensional Ultrasonography (4D US) in Fetal Surgeries���������������������������������������������������������� 139 9.2.3 3D Printing Models in Fetal Surgeries�������������������������� 140 9.2.4 Virtual Reconstruction in Fetal Surgery ���������������������� 140 9.3 Conclusion�������������������������������������������������������������������������������� 141 9.4 Assistance for Fetal Surgery ���������������������������������������������������� 141 References������������������������������������������������������������������������������������������ 145 10 Postnatal Surgery���������������������������������������������������������������������������� 147 10.1 Clinical Study 1 (Neurosurgery): Craniopagus Twins—A Challenge�������������������������������������������������������������������������������� 147 10.2 Clinical Study 2 (Pediatric Surgery) �������������������������������������� 153 10.2.1 Jejunal and Ileal Atresia���������������������������������������������� 153 10.2.2 Esophageal Atresia with Distal Fistula ���������������������� 153 10.2.3 Choledochal Cyst�������������������������������������������������������� 155 References������������������������������������������������������������������������������������������ 159 11 Multiple Pregnancy�������������������������������������������������������������������������� 161 11.1 Introduction���������������������������������������������������������������������������� 161 11.2 The Role of Ultrasound���������������������������������������������������������� 161 11.3 Computed Tomography and Magnetic Resonance Imaging���������������������������������������������������������������� 164 11.4 Multiplanar Ultrasound and Model Printing�������������������������� 167 11.5 Conclusions���������������������������������������������������������������������������� 170 References������������������������������������������������������������������������������������������ 170 12 M  aternal–Fetal Attachment in Blind Women Using Physical Model���������������������������������������������������������������������� 173 References������������������������������������������������������������������������������������������ 175 Part V Experimental and Future Applications 13 Haptics/Force Feedback Technologies�������������������������������������������� 179 13.1 Devices������������������������������������������������������������������������������������ 179 13.1.1 Actuators �������������������������������������������������������������������� 179 13.2 Process������������������������������������������������������������������������������������ 180 13.2.1 Optimization of the 3D Model������������������������������������ 180 13.2.2 Visualization �������������������������������������������������������������� 181 13.2.3 Smoothness and Roughness���������������������������������������� 181 13.2.4 Tenacity Map�������������������������������������������������������������� 182 13.2.5 Deformity of the Surface�������������������������������������������� 183 13.3 Case Study������������������������������������������������������������������������������ 184 References������������������������������������������������������������������������������������������ 185

xii

14 Recent  Advances in 3D Bioprinting Technologies and Possibilities for the Fetal Medicine ���������������������������������������� 187 14.1 Introduction���������������������������������������������������������������������������� 187 14.2 Recent Advances in 3D Bioprinting Research������������������������ 189 14.2.1 Placenta-on-a-Chip ���������������������������������������������������� 189 14.2.2 3D Bioprinted Outer-Blood-­Retina for Anti-Zika Small-­Molecules Discovery ���������������������� 191 14.2.3 Bone Regeneration with 3D Bioprinting�������������������� 191 14.2.4 Bioprinted Autologous Heart Valve Implants with Regenerative Capabilities and Life-Long Durability�������������������������������������������� 192 14.3 3D Bioprinting and Fetal Medicine���������������������������������������� 192 14.4 Considerations������������������������������������������������������������������������ 195 References������������������������������������������������������������������������������������������ 195

Contents

Contributors

Edward  Araujo  Júnior Department of Obstetrics, Paulista School of Medicine, Federal University of São Paulo (EPM-UNIFESP), São Paulo, Brazil Vinícius  Arcoverde Laboratório de Biodesign (Dasa/PUC-Rio), Rio de Janeiro, Brazil Pedro Teixeira Castro  Laboratório de Biodesign (Dasa/PUC-Rio), Rio de Janeiro, Brazil Pedro Augusto Daltro  Alta Excelência Diagnóstica (Dasa), Rio de Janeiro, Brazil Maria  Eduarda  Neves  de Alencar Universidade do Grande Rio (UNIGRANRIO), Rio de Janeiro, Brazil João Victor Correia de Melo  Laboratório de Biodesign (Dasa/PUC-Rio), Rio de Janeiro, Brazil Renato Augusto Moreira de Sá  Universidade Federal Fluminense., Niterói, Brazil Tatiana Fazecas  Alta Excelência Diagnóstica (Dasa), Rio de Janeiro, Brazil Alexandre  Zuquete  Guarato  School of Mechanical Engineering, Federal University of Uberlândia, Uberlândia, Brazil Roberto Imbuzeiro  Instituto de Matemática Pura e Aplicada (IMPA), Rio de Janeiro, Brazil Gabriel Liguori  TissueLabs, Manno, Switzerland Mario  R.  S.  Lima Pontifícia Universidade Católica do Rio de Janeiro  PUC-Rio/Fundação CECIERJ, Rio de Janeiro, Brazil Flávia Paiva Lopes  Laboratório de Biodesign (Dasa / PUC-Rio), Pontifícia Universidade Católica do Rio de Janeiro (PUC-Rio), Rio de Janeiro, Brazil Jorge Lopes  Laboratório de Biodesign (Dasa / PUC-Rio), Instituto Nacional de Tecnologia (INT), Rio de Janeiro, Brazil Miguel  Pereira  Macedo Obstetrics and Gynecology Resident, Centro Hospitalar Universitário Lisboa Norte, Lisboa, Portugal

xiii

Contributors

xiv

Nicanor  Macedo Cirurgia-Urologia-Videocirurgia em Pediatria, Hospital Estadual da Criança, Rio de Janeiro, Brazil Hospital Universitário Gaffrèe e Guinle (HUGG), Rio de Janeiro, Brazil Alexandra  Matias Obstetrics and Gynecology, Faculty of Medicine, University of Porto, Porto, Portugal Obstetrics and Gynecology, Centro Hospitalar Universitário São João, Porto, Portugal Ana  Paula  Pinho  Matos Alta Excelência Diagnóstica (Dasa), Rio de Janeiro, Brazil Gabriel  Mufarrej  Pediatric Neurosurgery, Instituto Estadual do Cérebro., Rio de Janeiro, Brazil Renata  Nogueira Alta Excelência Diagnóstica (Dasa), Rio de Janeiro, Brazil Paulo Orenstein  Instituto de Matemática Pura e Aplicada (IMPA), Rio de Janeiro, Brazil Alberto  Barbosa  Raposo  Department of Informatics, Universidade Católica (PUC-Rio), Rio de Janeiro, Brazil

Pontifícia

Gerson Ribeiro  Laboratório de Biodesign (Dasa/PUC-Rio), Rio de Janeiro, Brazil Rodrigo  Ruano Maternal-Fetal Medicine, University of Miami Miller School of Medicine, Miami, FL, USA UHealth Jackson Fetal Care, Miami, FL, USA Department of Obstetrics, Gynecology and Reproductive Sciences, University of Miami Miller School of Medicine, Miami, FL, USA Maternal-Fetal-Children Services, Americas Brazil, United Health Group Brazil, Sao Paulo, Brazil Gabriele Tonni  Department of Obstetrics & Gynecology, IRCCS Azienda USL, Reggio Emilia, Italy Luiz Velho  Instituto de Matemática Pura e Aplicada (IMPA), Rio de Janeiro, Brazil Cristina Paula Scudieri Paes Werner  Alta Excelência Diagnóstica (Dasa), Rio de Janeiro, Brazil Heron Werner  Laboratório de Biodesign (Dasa / PUC-Rio), Alta Excelência Diagnóstica (Dasa), Rio de Janeiro, Brazil

1

The Use of 3D Representations in Fetal Medicine

The roots of didactic models in medicine remount to Italy from the middle age period. The earliest records of graphical representation of the fetus date back to the year 1500. As examples, artistic drawings are scattered in museums and private collections around the world. Among the artists who achieved a refined quality in terms of visual representation of the fetus is Leonardo da Vinci, who through various anatomical studies displayed the process of fetal development [1] (Fig. 1.1). The use of didactic physical models in fetal medicine began in Italy from the Renaissance onward with the appearance of the colored wax teaching models representing different parts of the human body with visual realism, including the alterations of the woman body during the pregnancy period (Fig. 1.2). The important cities on that period were Bologna and Florence, emphasized by the famous “Florentine Museum,” being the Sicilian Gaetano Zumbo one of the skilled wax modelers who moved to Florence in 1691. He contributed to the development of the medical sciences by carrying out research on the reproduction of anatomical wax models. Many of these didactical models were constructed during the period approximately between 1550 and 1800 reaching its maximum period of technical and

With Contributions by Heron Werner, Gabriele Tonni, and Jorge Lopes

scientific splendor during the eighteenth century. Diverse models still can be appreciated in the permanent exhibition at the Museum of Zoology and Natural History “La Specola” in Florence, which was officially inaugurated in 1775 and until the early years of the nineteenth century was the only scientific museum specifically created for the public. Nowadays the Museum is unique on its collection of anatomic wax models [1, 2]. In France, more specifically on the villages of Royaume, around 1778, a very interesting set of hand-made fabric models called “La machine” were created and produced by Madame Du Coudray and largely used around different villages to teach and disseminate information about the birth process through the use of didactical physical models [1, 2] (Fig. 1.3). The fetus as a real image of human form appeared in the pioneering work of the photographer Lennart Nilsson, who started to take pictures of dead fetus and his works were published on the “Life magazine” in 1965. Within the years and the technological advance of fiber optics, Nilsson developed more advanced photographic techniques, being possible to see very clear bidimensional images of fetus inside the womb [2]. In obstetrics, fetal medicine is a field of science where appearance models for didactic purposes have been extensively used, and today it is possible to identify several companies whose work is dedicated to the construction of didactic physical models, intended for use in medical

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 H. Werner et al., 3D Physical and Virtual Models in Fetal Medicine, https://doi.org/10.1007/978-3-031-14855-2_1

1

2

1  The Use of 3D Representations in Fetal Medicine

Fig. 1.1 Photo reproduction: Leonardo da Vinci, The Foetus and Linings of the Uterus, c. 1510–1512, pen and ink, 30.1 × 21.4 cm. The Royal Collection Her Majesty Queen Elizabeth II

Fig. 1.2  Clemente Susini and workshop, after William Smellie, Fragmented Venus, c. 1770–90, La Specola, Florence

schools and related areas, constituting a visual and tactile aid to information by representing many parts of the human body, including all the gestation phases of pregnancy, from the emergence of the embryo to newborn babies, with examples of normal and healthy development, as well possible diseases [3, 4]. Didactical conventional 3D models in obstetrics are made for teaching and learning purposes, to transmit information for medicine students. The current source of tactile information is the use of real dead bodies of fetus (when available after problems occurred during pregnancy or dur-

1  The Use of 3D Representations in Fetal Medicine

Fig. 1.3 Fetus in the seventh month. https://artsci. case.edu/dittrick/2014/10/09/madame-du-coudraya-midwife-in-a-mans-world-2/

ing the birth process) and models made through conventional processes, made with the assistance of pictures of offspring fetuses and analysis of dead bodies [3–5]. In the last few years, ultrasound (US) diagnosis has undergone spectacular progress, thanks largely to advances in computers and in electronics in general. Currently it is a necessary and efficacious complementary method of exploration, and it can be predicted that its future possibilities, in partnership with technological advancements, will be enormous. US is the best technique for screening fetal malformations. The i­mplementation of 3D technology improved the visualization of the body’s surface and creates numerous possibilities in the evaluation of fetal anatomy. This technique provides a third plane that can be represented using conventional methods. There is also the possibility of permanently storing the images in a hard drive, allowing for further studies of the case in the absence of the patient. However, its limitations, such as small field of view, acoustic attenuation, oligohydramnios, inadequate fetal position, and artifacts, compel us to reach for

3

new technologies in aiding uterine content evaluation [5–7]. Magnetic resonance imaging (MRI) is a good option. It is noninvasive method and offers well-­ defined fetal images. Its most important feature is the excellent tissue contrast. MRI does not substitute US but complement it, offering additional fetal structure imagery. It is essentially a morphologic’ exam and its use should be restricted to cases when US result is dubious. Contrary to US, its diagnostic accuracy improves as the pregnancy progresses. The quality of the images is not significantly affected by the reduction in amniotic fluid, mother’s obesity, or fetal position [8, 9]. Our 3D Fetal project was started in 2007 and focuses on the construction of three-dimensional (3D) and physical fetal models from files generated by MRI, computed tomography (CT), and US, along with quick prototyping systems, with the objective of realistically reproducing fetuses for medical and(or) personal reasons (Fig. 1.4). Why did we use the 3D printing for fetus evaluation? First, we thought of using it for didactical educational purpose in medical schools. Then, we started using it for parent’s counseling, intervention planning, and for blind pregnant patients. Additive manufacturing (AM) in medicine is an emerging technique with a variety of medical applications such as surgical planning, biomedical research, and medical education [10–12]. In fetal medicine, since the models are not used for matrix or implants, there is no major concern with minor dimensional variations that may be negligible when placed in the acceptable error statistics [3–5]. Researchers continue to improve 3D printing technology and to explore new ones. The opportunity to prospect this paradigm shift in knowledge that occurs in the transition from 2D to 3D enables doctors to learn more inductively through the 3D modeling as a learning tool and improve patient’s care [5–7] (Fig. 1.5).

1  The Use of 3D Representations in Fetal Medicine

4

a

b

Fig. 1.4  3D ultrasound of 9-week embryo. Virtual and physical model built in a powder-based system (a). 3D ultrasound of 16-week fetus. Virtual and physical model built in a powder-based system (b). 35-week fetus obtained from CT scan in 2007. In the virtual model, note that the parts were added to keep the structure together when constructing the physical model. We can see the

model inside the envelope area construction of SLA equipment and the laser slice of the file being constructed inside the camera with photo sensible resin. 3D printed model just before the cleaning of the polymer residues and the 3D printed model (c). First twins’ model (26 weeks) printed in SLA technology from MRI file in 2008 (d)

1  The Use of 3D Representations in Fetal Medicine

c

d

Fig. 1.4 (continued)

5

1  The Use of 3D Representations in Fetal Medicine

6

a

b

Fig. 1.5  Virtual navigation in the airway path of a 35-week fetus with cervical teratoma in 2012 (a). 3D printing in Polyjet technology demonstrating a cervical tumor (teratoma) in the fetus (b)

References 1. Werner H, Lopes J. 3D technologies, palaeontology, archaeology, fetology. Revinter, editor. 2009. 2. Lopes J, Brancaglion Jr A, Azevedo SA, Werner Jr H. 3D Technologies. Unveiling the past, shaping the future. Lexikon, editor. 2013.

3. Santos JL, Werner H, Fontes R, et  al. Additive manufactured models of fetuses built from 3D ultrasound, magnetic resonance imaging and computed tomography scan data. In: Hoque ME, editor. Rapid prototyping technology  – principles and functional requirements. Rijeka: InTech; 2011. p. 179–92. 4. Werner H, Dos Santos JR, Fontes R, Gasparetto EL, Daltro PA, Kuroki Y, Domingues RC.  The use

References

7

of rapid prototyping didactic models in the study of Glanc P, Gonçalves LF, Gruber GM, Laifer-Narin fetal malformations. Ultrasound Obstet Gynecol. S, Lee W, Millischer AE, Molho M, Neelavalli J, 2008;32:955–6. Platt L, Pugash D, Ramaekers P, Salomon LJ, Sanz 5. Werner H, Dos Santos JRL, Fontes R, Daltro P, M, Timor-Tritsch IE, Tutschek B, Twickler D, Gasparetto E, Marchiori E, Campbell S.  Additive Weber M, Ximenes R, Raine-Fenning N.  ISUOG manufacturing models of fetuses built from three-­ Practice Guidelines: performance of fetal magnetic dimensional ultrasound, magnetic resonance imagresonance imaging. Ultrasound Obstet Gynecol. ing and computed tomography scan data. Ultrasound 2017;49:671–80. Obstet Gynecol. 2010;36:355–61. 10. Armillotta A, Bonhoeffer P, Dubini G, et  al. Use of 6. Werner H, Dos Santos JRL, Fontes R, Daltro P, rapid prototyping models in the planning of percutaGasparetto E, Marchiori E, Campbell S. Virtual bronneous pulmonary valve stent implantation. Proc Inst choscopy in the fetus. Ultrasound Obstet Gynecol. Mech Eng H. 2007;221:407–16. 2011;37:113–5. 11. Robiony M, Salvo I, Costa F, et  al. Virtual reality 7. Werner H, Lopes dos Santos JR, Fontes R, Belmonte surgical planning for maxillofacial distraction osteoS, Daltro P, Gasparetto E, Marchiori E, Campbell genesis: the role of reverse engineering rapid protoS.  Virtual bronchoscopy for evaluating cervical typing and cooperative work. J Oral Maxillofac Surg. tumors of the fetus. Ultrasound Obstet Gynecol. 2007;65:1198–08. 2013;41:90–4. 12. Lopes J, Azevedo SA, Werner Jr H, Brancaglion Jr 8. Hellinger JC, Epelman M.  Fetal MRI in the third A. Seen, unseen – 3D visualization. Rio Book’s, edidimension. Appl Radiol. 2010;39:8–22. tor. 2019. 9. Prayer D, Malinger G, Brugger PC, Cassady C, De Catte L, De Keersmaecker B, Fernandes GL,

Part I Digital Input.

In this section we aim to explain the main noninvasive imaging technologies to obtain 3D files of anatomical parts as well the methods applied to postprocessing them.

2

Imaging Technologies: Ultrasound, Computed Tomography, Magnetic Resonance Imaging, Micro-CT, 3D Scanner

2.1

Ultrasound

onds [6]. From the production of volumetric data in 3D US, images may be extracted in any desired Ultrasound (US) has been used in obstetrics for plane, allowing a critical view that is not possible over 30  years, opening a new window in fetal when using conventional 2D US (Fig. 2.1). The study [1, 2]. It is currently the main routine anatomy examined may be shown with a simultamethod for fetal assessment, given its safety, rel- neous view of orthogonal sectional cuts. atively simple execution, low cost, innocuous- Moreover, the volumetric data may serve to the ness to the fetus/mother, and great availability in visualize the structure’s surface [6, 7]. our midst. Many centers are exploring three-­ In many studies, the 3D US presented benefits dimensional (3D) US as an aid in fetal morphol- in detecting facial anomalies such as cleft lip and ogy evaluation [2, 3]. palate, micrognathia, midline hypoplasia, facial The US exam is useful to confirm or establish tumors, hypo/hypertelorism, facial dysmorphism, the gestational age within a reasonable range and and ear deformities (Fig.  2.2) [6]. The 3D US to determine fetal viability and its presentation. seems to expand the diagnostic capacity providBesides, it allows for early detection of multiple ing extra information about the fetus in addition pregnancies, placenta location, amniotic fluid to the 2D US [7]. Besides being a useful tool in evaluation and monitoring a wide spectrum of the assessment of severe fetal defects, such as in malformations [3]. recurrent surface malformations cases, it also The 3D US emerged in the two lasts decades, provides more convincing elements to confirm having its image based on the capture of a fetal normality (Fig. 2.3). sequence of bidimensional planes. After the The possibility of detecting various types of establishment of the region of interest (ROI), the malformations such as fetal masses and/or tumors block may be evaluated by several methods such is of great importance, as it allows the monitoring as multiplanar, surface, volume rendering, and of pregnancy in tertiary centers [8, 9]. The 3D US color or amplitude Doppler [4, 5]. The real-time brought new perspectives for more adequate processing of 3D US with volume rendering prognosis assessments, as it allows image rotatechnique also helps the evaluation of fetal abnor- tion and acquisition of whichever planes allowmalities, in a processing that takes but a few sec- ing better identification in the relations of the tumor with adjacent structures. It also allows the evaluation of tumor volume, leading to an adeWith Contributions by Ana Paula Pinho Matos, Alexandre quate planning of postnatal surgical correction Zuquete Guarato, Pedro Augusto Daltro, and Heron and better comprehension of the diagnosis by the Werner © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 H. Werner et al., 3D Physical and Virtual Models in Fetal Medicine, https://doi.org/10.1007/978-3-031-14855-2_2

11

12

2  Imaging Technologies: Ultrasound, Computed Tomography, Magnetic Resonance Imaging, Micro-CT…

Fig. 2.1 3D/4D Rendered imaging of the surface anatomy of the fetal heart (27 weeks) (a). 3D ultrasound of the fetal heart at 28 weeks of gestation in multiplanar mode and 3D segmentation; 3D virtual navigation track showing the multiple directions that can be taken; 3D physical model in fused deposition modeling (ABS) and 3D translucid physical model in material jetting (MJ) showing the cavities (b)

a

b

2.1 Ultrasound

13

a

b

Fig. 2.2  3D ultrasound showing cleft lip (27 weeks), virtual and 3D printed model on powder-based system (Z Corp) (arrows) (a). Fetal face on 3D ultrasound

(28  weeks), virtual and 3D printed model on powder-­ based system (Z Corp) (b)

parents, contributing to a greater affective bond [10, 11]. The 3D US offers advantages in fetal studies with bone dysplasia [12, 13]. Specific 3D US studies provide additional diagnostic information for the assessment of skeletal anomalies when compared to 2D US [12]. The use of 3D US facilitates the information on limb extremities malfor-

mations especially in cases of amputation of the hands, feet, and anomalous positions [14]. Epidemiological studies have not shown harmful effects of 2D US and 3D US in their diagnostic use. Even in 3D US/4D US cases that use computerized reconstructions from bidimensional images, the level of energy is not greater than the bidimensional scan [13].

14

2  Imaging Technologies: Ultrasound, Computed Tomography, Magnetic Resonance Imaging, Micro-CT…

Fig. 2.3  3D ultrasound of normal 12-week fetus and 3D printing (a). 3D ultrasound (17 weeks), virtual model and 3D printed model on powder-based system (Z Corp) (b). Fetal face on 3D ultrasound (30 weeks), virtual and 3D printed model on powder-based system (Z Corp) (c)

a

b

c

2.2  Computed Tomography

2.2 Computed Tomography Bone dysplasia diagnosis is often challenging during routine prenatal examinations, especially in the absence of family history. 2D US’s sensitivity is around 60% [15]. For those cases when we have technical difficulties with US evaluation, for example, for obese patients, another option is the use of computed tomography (CT) after the 30th week of pregnancy. This may provide a 3D reconstruction of the fetus’ skeleton with excellent resolution and using a low radiation dose [16–18]. CT is used only in specific cases of suspected fetal malformation, particularly those related to the skeleton, because of potential risks associated with exposure of the fetus to radiation. Its use during pregnancy must be adequately justified and its application is limited to specific pathologies such as bone dysplasia, which can, in some cases, be difficult to diagnose by US, especially in the absence of a family history of the disease [16–19]. CT shows thin imaging slices of tissues and body content, which represent computer assisted mathematical reconstructions. It is comprised of an X-ray that is much more complex than conventional X-ray machines. Its physical principals are based on the amount of radiation absorbed by each body part; that is, tissues with different composition absorb X-rays in different ways. Each image pixel corresponds to the tissue’s average absorption in that region, expressed in Hounsfield units (in honor of its creator Godfrey

15

Hounsfield who developed this technique for producing images in 1972). When penetrated by the X-rays, denser tissues (e.g., the liver) or heavier elements, such as the calcium present in bones, absorb more radiation than less dense tissues, such as lungs. Modern equipment is called multislices. This equipment produces multiple images after an X-ray tube burst. They can produce 16, 64, 128, 256, and 512 channels, allowing for a faster conclusion of the exam. This technology easily permits 3D reconstructions that recreate the image of the fetus, facilitating the study of complex malformations by a team of multidisciplinary specialists [17, 18]. In CT scan, the quality of the image of bony elements has better image resolution and the segmentation process that separates the fetus’ skeleton from the uterine walls is done automatically using medical image treatment software (Fig. 2.4). In this case the project innovation is in creating a series of bone connection elements in the 3D virtual environment in such a way as to keep the skeleton whole, preserving its shape and special positioning, allowing us to later build a physical model without risk of losing the accuracy of the bones’ positioning [17–21]. The CT protocol used was a 64 multislice scanner (Philips, Solingen, Germany) with the following parameters: 40 mAs, 120 kV, 64 slices per rotation, 0.75 pitch, and 0.75 mm slice thickness. This corresponds to a mean radiation dose of 3.12  mGy (CT dose index weighted) to the fetus.

16

2  Imaging Technologies: Ultrasound, Computed Tomography, Magnetic Resonance Imaging, Micro-CT…

a

c

b

d

Fig. 2.4  34-week fetus with left femur hypoplasia. 3D virtual model from computed tomography (a). Stereolithography, final process (b–d)

2.3 Magnetic Resonance Imaging Magnetic resonance imaging (MRI) is a propaedeutic noninvasive method capable of providing well-defined images of the human body [22]. The first study about the use of MRI during pregnancy was done by Smith in the 1980s [23]. His interest was sparked in recent decades, especially for the study of intracranial fetal anomalies, due to its great power of contrast between tissues [22, 24]. The principle of MRI is the digital representation of the chemical composition of various types of tissues exposed to a potent magnetic field. Hydrogen is the most used atom for imaging

because it presents high sensitivity to resonance’s phenomenon, and it is widely distributed in biological matter [22, 25]. The procedure used in MRI consists of subjecting a region to be examined to a magnetic field, disturbing the nuclear balance by a force determined frequency (resonance frequency). The strength of the magnetic field is measured in two units: Gauss and Tesla. A Tesla corresponds to 10.000 Gauss. To give some perspective to the strength of the field used, Earth’s magnetic field has a force between 0.5 and 1.0 Gauss. MRI machines work with magnetic fields of 0.25 to 3.00 Tesla. This way, patients are subjected to magnetic fields 2.500–30.000 times greater than Earth’s magnetic field.

2.3  Magnetic Resonance Imaging

a

17

b

Fig. 2.5  Fetal MRI. The patient positioned in dorsal (a) or left lateral decubitus (b), with her feet entering the magnet first

The most important feature of MRI is the tissue contrast resolution, hence its opportunity for use in obstetrics [24, 26]. Acceptable resolution can be achieved from 18 weeks, but the period considered ideal for performing the exam is from the 20th week of pregnancy [22]. MRI provides relevant information of fetal anatomy and about modifications in maternal organs and tissues during pregnancy. Even though US is still the modality of choice for the routine prenatal exams, due to its low cost, broader availability of machines, safety, good sensitivity, and real-time analysis capability, MRI has a great potential in morphological evaluation of those fetuses harder to be well studied by US [24, 27, 28]. The exam is performed with the patient positioned in dorsal or left lateral decubitus, with her head or feet entering the magnet first (Fig.  2.5). There is no need for preparation prior to the exam. In some cases, such as acute polyhydramnios, it might be necessary to previously sedate the mother with benzodiazepines (5–10  mg) orally, approximately 15 min prior to the exam. The purpose of sedation is to reduce maternal anxiety or possible fetal movements, which are responsible for image degradation. Once the patient is positioned on the magnet, fetal localization is initially performed through multiplanar sequences (axial, coronal, and sagittal planes).

Table 2.1  Main sequences in fetal medicine [27] • T2-weighted fast (turbo) spin-echo (SE) or steady-­state free-precession (SSFP) sequences: Fetal brain and body • T1-weighted contrast is acquired using two-­ dimensional gradient echo (GRE) sequences: Identifies methemoglobin in subacute hemorrhage, calcification, glands, and meconium • Single-shot high-resolution (SSH) GRE echoplanar (EP) sequences: Visualize bony structures, calcification, and the breakdown products of blood, such as deoxyhemoglobin, which suggests a recent bleed, or hemosiderin, which represents an older hemorrhage

The time for performing the exam is approximately 40 minutes. The most used sequences in fetal medicine are in Table 2.1. For 3D reconstructions, we use T2-weighted true fast imaging with steady-state free-precession (TRUFI), oblique sagittal plane (repetition time = 3.02 ms; echo time = 1.43 ms; isotropic voxel  =  1.0  ×  1.0  ×  1.0  mm3; matrix  =  256  ×  256  mm; 136 slices), with a total acquisition time of 26 s [29–31]. In the last few years, MRI has taken an expressive place in fetal exploration [22, 24, 25, 27, 28, 32–38]. It does not come to substitute US, but to complement it, offering additional images of fetal structure (Fig. 2.6). It is an essentially mor-

18

2  Imaging Technologies: Ultrasound, Computed Tomography, Magnetic Resonance Imaging, Micro-CT…

a

b

c

Fig. 2.6  3D virtual reconstruction from fetal MRI of the fetal brain, airway path, and lungs (a). A 27-week-old fetus with a congenital diaphragmatic hernia. The heart is deviated to the right. Note the stomach (gastric volvulus)

inside the chest (arrow) (b). 27-week fetus with right hydrothorax (arrow). 3D reconstruction from ultrasound and magnetic resonance imaging (c)

2.4 Microtomography

phological exam [22, 28, 34, 39]. As it does not emit radiation, it can be used without restrictions in pregnancy [40, 41]. Until this moment, no biological effects of MRI on the fetus are known [41]. Its use must be restricted to cases in which the US result is questionable. Its diagnostic accuracy improves as gestational age increases, not being disturbed by severe oligohydramnios, maternal obesity, or fetal static, that are responsible for the low-quality images in US [22].

2.4 Microtomography Microtomography (micro-CT) is a technology that allows visualization of structures on a microscopic scale in 3 dimensions, without destroying them, using X-ray as an image producer [42]. It is a method broadly used in the parts and ­petrochemical industry, with great development in animal and biological tissues study in the last two decades. As this method is nondestructive and of easy preparation of pieces to form the images, the number of studies using this technology has grown vigorously in recent years. The bigger restriction to this technique is using only ex vivo human samples. Curiously, the study of human tissues began a few years after its development (1979) [43], when in 1984 orthopedic doctors collected a sample of human iliac bone and analyzed it in the newly created technology that was in the parts evaluation laboratory of Ford, in the USA [44]. Between the decades of 1980s and 1990s, technology made little progress in biological tissues, as only mineralized tissues (such as bones and teeth) had contrast between their parts that allowed the formation of 3D images with good definition. But in the middle of the 2000s, studies using biological tissues

19

impregnated with high atomic mass metals managed to demonstrate variations between multiple soft tissues. In 2009, Metcher obtained high-­ definition images and excellent contrast in biological samples with easy execution methods, low cost, and nontoxic contrasts [42]. Since then, the technology development in soft biological tissues, especially in humans, has shown great potential in forming microscopic three-­ dimensional images. To form images in a microscopic scale in 3 dimensions, a few steps are necessary to prepare the sample. In high density samples such as bones and teeth or samples that need fast results, there is no need to impregnate contrast into the tissues. If the goal is to evaluate different soft tissues and their relations, it is necessary to impregnate the samples with atoms or molecules of high atomic mass. The simplest methods use iodine and formalin solutions with immersion of the sample in the solution, for hours or days depending on the volume of the sample. The second step is to fixate the sample inside the machine. Unlike conventional tomography, in micro-CT the sample is on a mobile platform, which rotates 180–360°. With this rotation, we acquire multiple sample images in several angles. These images are reconstructed with software and may be worked on posteriorly. The sample can still be placed in formalin to remove impregnation by iodine and be studied by traditional microscopy without harm to its microscopic characteristics. Among common applications of micro-CT in human tissue, surgical margins assessment during breast cancer surgery and lung cancer stands out, both without the use of contrast [45, 46]. In studies using contrast, the evaluation of fetal-­ embryo malformations and gynecology studies (Fig. 2.7) [47–49].

20

2  Imaging Technologies: Ultrasound, Computed Tomography, Magnetic Resonance Imaging, Micro-CT…

Fig. 2.7  Tubal abortion, the ovum is partially detached from the fallopian tube (FT) lumen (*). Histological section: note the contraction of the blood clot, the fractures, and detachment of the endosalpinx. Micro-CT image demonstrates the site of nidation, with the presence of

blood clot (radiopac image) surrounding the ovum and distending the FT lumen. Note the distention of the FT wall contralaterally to the ovum site. Virtual and 3D printed model

2.5 3D Scanners and Photogrammetry

There also the white light scanners that emit a structured light. These scanners also use triangulation for measurement, but they can acquire millions of points at a time. The light patterns are project into the object and the camera analyzes the light deformations to measure the surface points. Some structured light sensors use more than one camera, and the average result is calculated for both cameras. Light triangulation scanners are commonly used in engineering applications and can be used also for medical purposes (Fig. 2.8) [51–54]. Photogrammetry is a technology that relies on the overllaping of digital images (photographs) of an object, post-processed on a specific software in order to generate a 3D virtual model (with or without the addition of surfaces textures as color).

There are basically three main technologies to obtain a 3D external volume of an object: light triangulation scanners, white light scanners and photogrammetry [50]. Light triangulation scanners generally use at least one light emitter (mostly laser) and one measuring camera (mostly CCD cameras). The light emitter, the measuring camera, and the object to be scanned form a triangle. In this way, the distance is calculated by trigonometric equations. Some scanners project a laser beam to the object and can measure only one point at a time. Other scanners project a laser plane and can measure several points at a time.

References

a

21

b

c

Fig. 2.8  Fetal MRI image fusion with 3D scanning of the maternal body (a). Segmentation of the fetus and mother’s skin. 3D reconstruction obtained by MRI scan data (b).

d

3D scanning of the maternal body using a white light scanner (c). Maternal-fetal model (virtual and physical) (d)

7. Gonçalves LF, Wesley L, Espinoza J, et al. Three- and 4-dimensional ultrasound in obstetric practice. Does it help? J Ultrasound Med. 2005;24:1599–624. 8. Gudex C, Nielsen BL, Madsen M. Why women want 1. Baba K, Okai T, Kozuma S, et  al. Fetal abnormaliprenatal ultrasound in normal pregnancy. Ultrasound ties: evaluation with real-time  – processible three-­ Obstet Gynecol. 2006;27:145–50. dimensional US- preliminary report. Radiology. 9. Kurjak A, Azumendi G, Andonotopo W, et al. Three1999;211:441–6. and four- dimensional ultrasonography for the struc2. Benacerraf BR, Benson CB, Abuhamad AZ, et  al. tural and functional evaluation of the fetal face. Am J Three- and 4- dimensional ultrasound in obstetrics and Obstet Gynecol. 2007;196:16–28. gynecology. J Ultrasound Med. 2005;24:1587–97. 3. Bonilla-Musoles F, Machado LE, Osborne NG. Two- 10. Merz E.  Current 3D/4D ultrasound technology in prenatal diagnosis. Eur Clin Obstet Gynaecol. and three- dimensional ultrasound in malformation 2005;1:184–93. of the medullary canal: report of four cases. Prenat 11. Riccabona M, Pretorius DH, Nelson TR, et al. Three-­ Diagn. 2001;21:622–6. dimensional ultrasound: display modalities in obstet4. Campbell S. 4D, or not 4D: that is the question. rics. J Clin Ultrasound. 1997;25:157–67. Ultrasound Obstet Gynecol. 2002;19:1–4. 5. DeVore GR, Falkensammer P, Sklansly MS, et  al. 12. Ruano R, Molho M, Roume J, et  al. Prenatal diagnosis of fetal skeletal dysplasias by combining Spatio-temporal image correlation (STIC): new techtwo-­ dimensional and three-dimensional ultrasound nology for evaluation of the fetal heart. Ultrasound and intrauterine three-dimensional helical comObstet Gynecol. 2003;22:380–7. puter tomography. Ultrasound Obstet Gynecol. 6. Faure JM, Captier G, Baumler M, et al. Sonographic 2004;24:134–40. assessment of normal fetal palate using three-­ dimensional imaging: a new technique. Ultrasound 13. Tutschek B, Blaas HGK, Abramowicz J, et al. Three-­ dimensional ultrasound imaging of the fetal skull and Obstet Gynecol. 2007;29:159–65. face. Ultrasound Obstet Gynecol. 2017;50(1):7–16.

References

22

2  Imaging Technologies: Ultrasound, Computed Tomography, Magnetic Resonance Imaging, Micro-CT…

14. Werner H, Daltro P, Fazecas T, et al. Prenatal diagnosis of sirenomelia in the second trimester of pregnancy using two-dimensional ultrasound, three-dimensional ultrasound and magnetic resonance imaging. Radiol Bras. 2017;50:201–2. 15. Steiner H, Spitzer D, Weiss-Wichert PH, et al. Three-­ dimensional ultrasound in prenatal diagnosis of skeletal dysplasia. Prenat Diagn. 1995;15(4):373–7. 16. Cassart M, Massez A, Cos T, Tecco L, Thomas D, Van Regemorter N, Avni F.  Contribution of three-­ dimensional computed tomography in the assessment of fetal skeletal dysplasia. Ultrasound Obstet Gynecol. 2007;29:537–43. 17. Werner H, dos Santos JR, Fontes R, Gasparetto EL, Daltro PA, Kuroki Y, Domingues RC.  The use of rapid prototyping didactic models in the study of fetal malformations. Ultrasound Obstet Gynecol. 2008;32:955–6. 18. Werner H, Rolo LC, Araujo Junior E, Dos Santos JRL.  Manufacturing models of fetal malformations built from 3-dimensional ultrasound, magnetic resonance imaging, and computed tomography scan data. Ultrasound Q. 2014;30(1):69–75. 19. Ulla M, Aiello H, Cobos MP, et  al. Prenatal diagnosis of skeletal dysplasias: contribution of three-­ dimensional computed tomography. Fetal Diagn Ther. 2011;29:238–47. 20. Hull AD, Pretorius DH, Lev-Toaff AS, et al. Artifacts and the visualization of fetal distal extremities using three-dimensional ultrasound. Ultrasound Obstet Gynecol. 2000;16:341–4. 21. Werner H, dos Santos JRL, Fontes R, Daltro P, Gasparetto E, Marchiori E, Campbell S.  Additive manufacturing models of fetuses built from three-­ dimensional ultrasound, magnetic resonance imaging and computed tomography scan data. Ultrasound Obstet Gynecol. 2010;36:355–61. 22. Brugger PC, Stuhr F, Lindner C, Prayer D. Methods of fetal MR: beyond T2-weighted imaging. Eur J Radiol. 2006;57:172–81. 23. Smith FW, Adam AH, Phillips WDP. NMR imaging in pregnancy. Lancet. 1983;321:61–2. 24. Leithner K, Prayer D, Porstner E, Kapusta ND, Stammler-Safar M, Krampl-Bettelheim E, Hilger E.  Psychological reactions related to fetal magnetic resonance imaging: a follow-up study. J Perinat Med. 2013;41:273–6. 25. Matos APP, Duarte LB, Castro PT, Daltro P, Werner Júnior H, Araujo Júnior E.  Evaluation of the fetal abdomen by magnetic resonance imaging. Part 1: malformations of the abdominal cavity. Radiol Bras. 2018;51(2):112–8. 26. Salomon LJ, Garel C.  Magnetic resonance imaging examination of the fetal brain. Ultrasound Obstet Gynecol. 2007;30:1019–32. 27. Prayer D, Malinger G, Brugger PC, Cassady C, De Catte L, De Keersmaecker B, Fernandes GL, Glanc P, Gonçalves LF, Gruber GM, Laifer-Narin S, Lee W, Millischer AE, Molho M, Neelavalli J, Platt L, Pugash D, Ramaekers P, Salomon LJ, Sanz M, Timor-Tritsch

IE, Tutschek B, Twickler D, Weber M, Ximenes R, Raine-Fenning N.  ISUOG Practice Guidelines: performance of fetal magnetic resonance imaging. Ultrasound Obstet Gynecol. 2017;49:671–80. 28. Rubesova E.  Why do we need more data on MR volumetric measurements of the fetal lung? Pediatr Radiol. 2016;46(2):167–71. 29. Werner H, Lopes J, Tonni G, Araujo Júnior E. Physical model from 3D ultrasound and magnetic resonance imaging scan data reconstruction of lumbosacral myelomeningocele in a fetus with Chiari II malformation. Childs Nerv Syst. 2015;31(4):511–3. 30. Werner H, Marcondes M, Daltro P, Fazecas T, Ribeiro BG, Nogueira R, Araujo Junior E. Three-dimensional reconstruction of fetal abnormalities using ultrasonography and magnetic resonance imaging. J Matern Fetal Neonatal Med. 2019;32(20):3502–8. 31. Werner H, Ribeiro G, Dos Santos JL, Castro PT, Lopes FP, Daltro P. Cutting-edge 3D image obtained through fusion of three imaging technologies. Ultrasound Obstet Gynecol. 2021;57:354–5. 32. Bulas D, Egloff A. Benefits and risks of MRI in pregnancy. Semin Perinatol. 2013;37(5):301–4. 33. Garel C.  Fetal cerebral biometry: normal parenchymal findings and ventricular size. Eur Radiol. 2005;15:809–13. 34. Hellinger JC, Epelman M.  Fetal MRI in the third dimension. Appl Radiol. 2010;39(7–8):8–22. 35. Teixeira Castro P, Werner H, Matos AP, Daltro P, Araujo Júnior E. Symmetric and ventrally conjoined twins: prenatal evaluation by ultrasound and magnetic resonance imaging and postnatal outcomes. J Matern Fetal Neonatal Med. 2021;34(12):1955–62. 36. Weidner M, Hagelstein C, Debus A, Walleyo A, Weiss C, Schoenberg SO, Schaible T, Busing KA, Kehl S, Neff KW.  MRI-based ratio of fetal lung volume to fetal body volume as a new prognostic marker in congenital diaphragmatic hernia. AJR Am J Roentgenol. 2014;202:1330–6. 37. Werner H, Nogueira R, Lopes FPPL. MR imaging of fetal musculoskeletal disorders. Magn Reson Imaging Clin N Am. 2018;26:631–44. 38. Wright C, Sibley CP, Baker PN.  The role of fetal magnetic resonance imaging. Arch Dis Child Fetal Neonatal. 2010;95:137–41. 39. Shellock FG, Crues JV.  MR procedures: biologic effects, safety, and patient care. Radiology. 2004;232:635–52. 40. Griffiths PD, Sharrack S, Chan KL, Bamfo J, Williams F, Kilby MD.  Fetal brain injury in survivors of twin pregnancies complicated by demise of one twin as assessed by in utero MR imaging. Prenat Diagn. 2015;35:583–91. 41. Malinger G, Werner H, Rodriguez Leonel JC, et  al. Prenatal brain imaging in congenital toxoplasmosis. Prenat Diagn. 2011;31:881–6. 42. Metscher BD.  MicroCT for comparative morphology: simple staining methods allow high-contrast 3D imaging of diverse non-mineralized animal tissues. BMC Physiol. 2009;9:11.

References 43. Herman GT.  Image reconstruction from projections: the fundamentals of computerized tomography. New York: Academic Press; 1980. 44. Feldkamp LA, Davis LC, Kress JW.  Practical cone-­ beam algorithm. J Opt Soc Am. 1984;1:612–9. 45. Tang R, Coopey SB, Buckley JM, et  al. A pilot study evaluating shaved cavity margins with micro-­ computed tomography: a novel method for predicting lumpectomy margin status intraoperatively. Breast J. 2013;19:485–9. 46. Kayı Cangır A, Dizbay Sak S, Güneş G, Orhan K.  Differentiation of benign and malignant regions in paraffin embedded tissue blocks of pulmonary adenocarcinoma using micro-CT scanning of paraffin tissue blocks: a pilot study for method validation. Surg Today. 2021;51(10):1594–601. https://doi. org/10.1007/s00595-­021-­02252-­2. 47. Castro PT, Aranda OL, Matos APP, et al. The human endosalpinx: anatomical three- dimensional study and reconstruction using confocal microtomography. Pol J Radiol. 2019;84:e281–8. https://doi.org/10.5114/ pjr.2019.86824. 48. Castro PT, Matos APP, Aranda OL, et  al. Tuboperitoneal fistula, ectopic pregnancy, and remnants of fallopian tube: a confocal microtomography analysis and 3D reconstruction of human fallopian tube pathologies. J Matern Fetal Neonatal Med. 2019;32:3082–7. 49. Hutchinson JC, Kang X, Shelmerdine SC, et  al. Postmortem microfocus computed tomography for early gestation fetuses: a validation study against

23 conventional autopsy. Am J Obstet Gynecol. 2018;218:445.e1–445.e12. 50. Lartigue C, Quinsat Y, Mehdi-Souzani C, Zuquete-­ Guarato A, Tabibian S. Voxel-based path planning for 3D scanning of mechanical parts. Comp Aided Desig Appl. 2013;11(2):220–7. 51. Guarato A, Loja A, Pereira L, Braga S, Trevilato T.  Qualification of a 3D structured light sensor for a reverse engineering application. In: Proc. SPIE 10151, optics and measurement international conference 2016, 101510C. 11 Nov 2016. https://doi. org/10.1117/12.2257601. 52. Guarato A, Quinsat Y, Mehdi-Souzani C, Lartigue C, Sura E.  Conversion of 3D scanned point cloud into a voxel-based representation for crankshaft mass balancing. Int J Adv Manuf Technol. 2017;95(1–4):1315–24. 53. Werner H, Ribeiro G, Lopes Dos Santos J, Castro P, Lopes F, Daltro P.  Cutting-edge 3D image obtained through fusion of three imaging technologies. Ultrasound Obstet Gynecol. 2021;57(2):354–5. 54. Zuquete-Guarato A, Mehdi-Souzani C, Quinsat Y, Lartigue C, Sabri L.  Towards a new concept of in-­ line crankshaft balancing by contact less measurement: process for selecting the best digitizing system. Volume 4: advanced manufacturing processes; Biomedical engineering; Multiscale mechanics of biological tissues; Sciences, engineering and education; multiphysics; emerging technologies for inspection and reverse engineering; Advanced materials and tribology. 2012.

3

Post-processing Images to Generate the 3D Data

Image processing is the phase following the exam. Files in DICOM format are imported into a medical image processing and reconstruction program. At the moment of import, there might be one or multiple exams. If there are multiple exams, the program displays a list divided by patients, who might have performed one or more exams, called sequences. Each one of these sequences can be arranged by orientation of slices of the patient’s images, or other equipment configurations that result in images that highlight different structures, for example, liquids and soft tissues, etc. It is clear, then, that the data and number of layers—images—in the sequence are necessary to define in which exam to work on the segmentation.

After the import, another processing can be done before initiating segmentation. It is possible to filter the images to reduce noise that generates imperfections and holes in segmentation, which can be time consuming to close manually. There are different types of filters that can be used, obtaining different results. These filters should be applied with caution, as small elements, structures, or deformations may disappear, making the model incompatible with reality. However, these filters are very useful in ultrasound (US) images that have high levels of noise and not many small elements such as magnetic resonance imaging (MRI) or computed tomography (CT) (Fig. 3.1).

Fig. 3.1  Filter to reduce noise

With Contributions by Gerson Ribeiro and Heron Werner Supplementary Information The online version contains supplementary material available at https://doi. org/10.1007/978-­3-­031-­14855-­2_3. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 H. Werner et al., 3D Physical and Virtual Models in Fetal Medicine, https://doi.org/10.1007/978-3-031-14855-2_3

25

26

3.1 Segmentation The process of segmentation uses a few tools to create different masks over each image in the sequence, and each mask corresponds to a structure. For example, a blue mask defines the fetal body; a second pink mask defines the brain; a third, yellow, defines the eyes and umbilical cord and thus create the separations between the structures [1] (Video 3.1). Depending on the quality of the exam’s images this process can be fast or take longer, depending on manual intervention to separate the elements. For that end, the segmentation programs have different tools to allow the selection and editing of the masks, making corrections and adjustments (Fig. 3.2). Segmentation tools: In the following will be discussed the most common tools in different programs available to

3  Post-processing Images to Generate the 3D Data

work on mask selection [2]. The names and some steps may change slightly according to the program, but the logistics in functioning are the same. Interesting to observe that the segmentation process can be also done through the use of a virtual reality medical segmentation software (Video 3.3). The most common tool for this division is the threshold. With the identification of the approximate tonality of the pixels that make up the desired structure, we define the threshold with a minimum and maximum value. For example, from hue 50 to 125, the pixels that fit will be selected. The program identifies in all images of the sequence all the pixels that have a tonality between the two values, selects them creating an overlay mask, and attributes a color. At times it is necessary to exclude a few selected parts, as they belong to other structures, or yet, when the desired

a

b

Fig. 3.2  Segmentation in magnetic resonance imaging, segmented fetus in blue, brain in pink, eyes in yellow, and umbilical cord in green (a). 3D virtual models from segmentations examples (b)

3.1 Segmentation

structure is touching another. In which case, the manual process is crucial to separate both parts. The Paint tool is the simplest one. Present in all programs, it works with a brush shape that can be circular, oval, rectangular, or even spherical and it is possible to adjust the size of the shape. In it all pixels that are within the shape while the user paints are selected regardless of the pixel intensity. In the case of the spherical brush, it allows painting some layers above and below. Erase: This tool works like “Paint tool,” has the same shapes, and does the opposite job. It is able to remove the selections made in pixels by other tools. Another common tool is the Draw or Loop, with it the user draws a line which determines a perimeter and all pixels within will be selected. This tool may be useful for drawing limits or outlines quickly. Islands: This filtering tool works from the continuity analysis of selections inside a layer as well as in the following and previous layers. Which means it also works in volumes. It allows selected objects that are isolated from the main structure to be excluded maintaining only the largest structure, or maintaining only the structure the user selects, also being able to send these fragments to new independent segmentations. It is a great tool for cleaning, in case the selection chooses nearby elements that do not belong to the structure of interest. Seeds: This tool also works in a semi-­ automated form. It is interesting for the selection of structures that have other similarities and same intensity that are close to each other. The user points or draws, with the Paint tool, the regions that make up the structure of interest in numerous layers. In some programs, the user also draws what is not a structure with a second segment (color). In execution, the algorithm expands the selections numerous times considering the intensity of the pixels that were included in the original selection, and so identifying a border, by changing of intensity or finding the second selection that is not a structure, it interrupts the expansion in these limits following the expansion only where it does not find neither of the two hypotheses.

27

Interpolation is one of the fastest ways to select large elements as it has the same versatility and simplicity as the paint or draw tool, however, spares the work of drawing layer by layer. With this tool, the user draws selections, skips a few layers, and selects again. And so, repeating the process until the entire structure is finished. The tool calculates a projection from the selected layers to select the same regions on the remaining layers that were not selected by the user. Finally, after user confirmation all intermediate layers are selected [3]. The logic tools have a few Boolean interaction functions between segments. So, an addition is possible to join both segments, for example, joining the umbilical cord and placenta in a single segment, subtraction is useful to create cavities, and also intersections to select only the common part between segments, etc. Merging of tools: It is possible to blend the tools when using them in sequence and thereby optimize the processes and results, for example, the method we shall call “cocoon” [4], which consists of creating a region in which we draw with the paint tool a number of reduced and spaced layers to outline an area that involves the whole area of interest despite the structures in it. Then, with the interpolation tool—which completes the non-drawn layers—we create a “cocoon.” And so, we apply the threshold tool only within the region, resulting in a cleaner and better controlled selection of structures. Blending also allows quick work to capture different structures within the same area, changing only the threshold interval. For example, a common flaw in fetal segmentation is the head: when we find a threshold that contemplates the body in a satisfactory way, we do not capture the cranial part, which is thinner and has fluid nearby, in addition to having clearer tones that do not enter selection. Using the “cocoon” methodology, we can apply a threshold for the body, and shortly thereafter, apply a different one to the head, posteriorly fusing the two parts and creating a single model (Fig. 3.3). Once the segmentation process is concluded with the structures properly separated in masks,

28

3  Post-processing Images to Generate the 3D Data

Fig. 3.3  The “cocoon” method

we move forward to the 3D model generation process. There are also new image segmentation techniques with semi-automated and fully automated processes. In the semi-automated the tools are equipped with a logical process programmed into the algorithm which will perform the pixel selection assignment according to a few rules. For example, the doctor defines which structure he wishes to segment and, in the sequence, marks are made in specific points in the structure (anatomical references) and so from this marking the algorithm can identify the positioning of the remaining pixels that make up the same structure as well as others. These tools are also capable of identifying divisions such as a liver’s lobes or ramifications of a vessel. There is also the assisted segmentation by artificial intelligence (IA). These are tools that work independently and are capable of identifying the structures and segmenting them with or without human intervention, sufficing loading the image files and executing the task. Furthermore, IA in fetal medicine will be discussed (Video 3.2).

3.2 3D Models Before proceeding, it is important to clarify the difference between a 3D model and an 3D representational image. 3D models are a virtualization which simulates reality as it has measurable dimensions of width, height, and depth. The 3D

representation is the bidimensional form (height and width) of visualizing the 3D model, as printed on paper or disposed on a computer screen [4]. Examples of 3D representation image are the ones generated from the “3D ultrasound” exam that simulates three-dimensional volume, using shading and lighting effects on images to give it a 3D perception, however it is a flat image. There are two types of 3D models—NURBS and polygon mesh—in which the software simulates a three-dimensional spatial ambient, with the width (X), height (Y), and depth (Z) axis represented by values. The type of model used for 3D printing is the polygon mesh, which is composed of several connected triangles, of which each vertex has a position in this space in XYZ.  The set of thousands of these triangles form the mathematically measurable model being possible to extract data such as surface, volume, diameter, height, and others. These measurements are also important to fetal medicine as they help determine the extension of a tumor, for example. Not only in this technique, but in other 3D scanning processes based on sequential image processing, the model is generated from the reconstruction process. Each pixel on the masks is converted to voxel—the squared pixel becomes a cube receiving the same value of width for depth—gaining the Z axis and becoming a volume. Soon after, the program converts the cubes into polygonal mesh, which generates a surface

3.2  3D Models

29

phase in which we use specific software to work with polygonal meshes. This software has tools that help correct imperfections, close holes or absence of triangles in the model, smooth out noise, and enhance details and curves that might get lost. Another important task in the treatment process is adjusting the generated mesh. The raw model, when leaving the segmentation software can have a high number of polygons. In this program we can redo the mesh in an optimal way, maintaining a great number of polygons in regions of high detail, and reduced in regions of simple curves. This contributes to the size reduction of the final file, something of importance when the application is for the output of navigation and interactivity (Fig. 3.5) (Video 3.3).

3.2.2 Models and Assemblies

Fig. 3.4  Polygon mesh (green). Each vertex has a coordinate in “XYZ”

that involves all cubic shapes. This surface is composed of triangles and is called Polygon Mesh becoming the 3D model as described (Fig. 3.4).

3.2.1 Treatment and Smoothing The generated model is not yet the final one, as it can lack refinement, since the noise and interferences in the exam and manual processes in the mask generating phase can create rough and deformed surfaces. To correct such problems the model goes through the treatment process, a

Still regarding treatment, when necessary, we make junctions of three-dimensional models originating from different segmentations, such as joining the external body—skin—with the airways in the internal part, or yet fusing distinct organs, that need to be printed or visualized together. In these cases, each element is produced individually in segmentation, but without altering its positioning in space. So, when being imported into the treatment process, they remain aligned as in the exam. Depending on the occasion, some morphology operations are made to obtain assemblies that help comprehend the final model better. It is possible to unite models such as the union Boolean, in which the model’s meshes join and form a bigger element, excluding the intersection region. For separation, we apply the Boolean subtraction, when a model is removed from another, which is frequently used to obtain cavities. In the Boolean intersection, the region in common

30

3  Post-processing Images to Generate the 3D Data

a

b

Fig. 3.5  Phases, from top left to bottom right: Magnetic resonance imaging, segmentations, raw model, treated model, render, big left model printed in FDM Z Corp z310 (a). Segmentation and virtual model (b). Craniopagus at

26 weeks. Segmentation from magnetic resonance imaging showing the fused brains in the virtual 3D model (c). Preparing the virtual 3D model of 36-week normal fetus (d). Fetal virtual reality segmentation in MRI exam (e)

3.2  3D Models

c

Fig. 3.5 (continued)

31

32

3  Post-processing Images to Generate the 3D Data

d

e

Fig. 3.5 (continued)

between two models is subtracted or maintained, excluding the rest (Fig. 3.6). Editing of model to exhibit the internal parts and structures construction integrated into the model through the Boolean to maintain the elements in position.

3.3 Archives Format

Fig. 3.6  Cervical tumor (teratoma) case with opened face to view the tumor

Once the treatments and assemblies are finalized, there are different formats of exportation according to the output given. In the initial part, the exams need to be saved in DICOM format, which consists of a standard

References

protocol, created to allow communication in a universal manner of imaging exams between the most distinct devices, computer programs, and hospital systems. This format has the ability to hold all data referent to a patient, the equipment and settings used to capture images. The file format is just a part of the whole protocol, the file may be single, with a “.dcm” extension, containing numerous sequential images in it, or yet a set of files, all with “.dcm” extension. Both when imported into the processing program are interpreted the same way [5, 6]. When the processing work is done, the 3D model generated is exported to the mesh treatment program in STL (Standard Triangular Language) format that is also a standard created for model transfer between modeling programs and for 3D printing. The format is created from a simple algorithm that first creates the vertices with mathematical values in three spatial axis X, Y, and Z, arranging them in sequential order, and thereafter creating triangles from the created vertices. In its simple format, binary, it is only capable of creating the vertices and triangles, but in its extended format, asci, it becomes capable of storing different models in a single file, however in a larger size [7–11]. In the post-treatment phase, according to the output given, the file is exported in STL format—as explained above—for 3D printing, or in OBJ format if the goal is navigation and interactivity. OBJ (Object) is an open code format created by Wavefront Technologies. The file is written in a way to associate symbols that define the inserted information in the following manner: to draw a polygon, OBJ draws the vertices and uses the letter “v” before the coordinates in XYZ, after the three vertices, it already determines the triangle, in this case the face of the polygon, with the letter “f”. It also has symbols that define other parameters. It is preferable to

33

use this format, as it is widely compatible with other modeling and animation programs and carries besides the model, texture maps, and the textures themselves.

References 1. Gibson I, Feng W. Advance manufacturing technology for medical applications. West Sussex: Wiley; 2005. 2. Pinter A, Lasso G, Fichtinger. Polymorph segmentation representation for medical image computing. Comput Methods Prog Biomed. 2019;171:19–26. 3. Zukić D, Vicory J, Mccormick M, Wisse L, Gerig G, Yushkevich P, Aylward S.  ND morphological contour interpolation. Insight J. 2016. http://hdl.handle. net/10380/3563. 4. Ribeiro G.  A evolução da representação 3D de imagens: aspectos médicos e de design associados (Dissertação de mestrado). Departamento de Artes e Design. Pontifícia Universidade Católica do Rio de Janeiro; 2020. 153 p. 5. Peck D. Digital Imaging and Communications in Medicine (DICOM): A Practical Introduction and Survival Guide. J Nucl Med. [s.l.], 2009;50(8):1384. Society of Nuclear Medicine. 6. Pianykh OS. Digital imaging and communications in medicine (DICOM): a practical introduction and survival guide. Springer Science & Business Media, 2009. 7. Hiller J, Lipson H. STL 2.0: a proposal for a universal multi-material additive manufacturing file format. In: Proceedings of Solid Free. Fabrication Symposium. SFF’09 Austin, TX, p. 266–78, Feb 16, 2014. 2009. 8. Stroud I, Xirouchakis PC.  STL and extensions. Adv Eng Softw. 2000;31(2):83–95. 9. Fedorov A, Beichel R, Kalpathy-Cramer J, Finet J, Fillion-Robin JC, Pujol S, Bauer C, Jennings D, Fennessy FM, Sonka M, Buatti J, Aylward SR, Miller JV, Pieper S, Kikinis R. 3D slicer as an image computing platform for the quantitative imaging network. Magn Reson Imaging. 2012;30(9):1323–41. PMID: 22770690. PMCID: PMC3466397. 10. Kalaskar DM, editor. 3D printing in medicine. London: Elsevier; 2017. 226 p. 11. Cignoni P, Callieri M, Corsini M, Dellepiane M, Ganovelli F, Ranzuglia G. MeshLab: an open-source mesh processing tool. In: Sixth Eurographics Italian chapter conference. 2008. p. 129–36.

Part II Physical Output.

In this section we intend to present the characteristics of the current 3D printing technologies available in the market.

4

(3D) Printing Technologies

With the advent of three-dimensional (3D) printing, also known as additive manufacturing (AM) technologies, noninvasive fetal imaging techniques can be physically materialized for several purposes. For instance, uterine morphology and fetal anatomy can be better visualized, improving the understanding of various malformations and pathologies. These models can aid in interdisciplinary discussions and help guide medical and surgical interventions. The input data (to be converted into digital 3D files) needs to be obtained typically via US, MRI (combined or not), CT scanner or, in some specific cases through micro-CT scanner as well. The process is carried out by transforming the 3D files into a Standard Triangulation Language (STL) extension (or other formats like .3MF an.OBJ), which consists basically of X, Y, and Z coordinates. Once the STL file is generated, the next step is the horizontal slicing of the whole 3D volumetric file using an appropriate software to the specific hardware being used and calculating the supporting structures when necessary.

The AM method is based on the successive overlapping of thin layers of specific material substances, according to the appropriate technical method. The building process starts with the sequential deposition of layers of material, the layer width ranging from microns to fractions of millimeters, depending on the technology chosen. Then a post-processing step starts, an essential procedure in all current AM technologies, where the model has to be cleaned to remove the support material and/or residues used during the building process. There are currently several different systems of AM technologies on the market which, although using dissimilar material procedures, are based on the same principle of manufacturing by layer deposition. One of the most important features of AM is the possibility of manufacturing parts with significant geometrical complexity, a process in which conventional technologies are lengthy and more expensive, affecting both the time taken to launch the product commercially and the total costs of production.

With Contributions by João Victor Correia de Melo and Jorge Lopes Supplementary Information The online version contains supplementary material available at https://doi. org/10.1007/978-­3-­031-­14855-­2_4. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 H. Werner et al., 3D Physical and Virtual Models in Fetal Medicine, https://doi.org/10.1007/978-3-031-14855-2_4

37

4  (3D) Printing Technologies

38

AM technologies currently can be designated by physical state of the materials to be transformed, as an example: solid based systems, associated with non-powder formats, such as sheets or thermoplastic extruded filaments; powder-­based systems, associated with sintering or agglutination of grain particles; and liquid-­ based systems, associated with photopolymer resins (Fig. 4.1). (a) Solid based systems • FDM/FFF—Fused deposition modeling/ fused filament fabrication (polymer filament) • APF/FGF—Arburg plastic freeforming/ fused granular fabrication (polymer granulate) (b) Powder-based systems • PBF—Powder bed fusion

a

b

• SLS—Selective laser sintering (fused with laser) • BJT—Binder jetting (joined with bonding plaster/gypsum) • MJF—Multi jet fusion (c) Liquid based systems • MJT—Material jetting (photopolymer cured with UV light) • SLA—Stereolithography (photopolymer cured with laser) • DLP/LCD—Digital light processing (photopolymer cured with projector) In order to exemplify the categories, five different AM technologies are described. These technologies are the most used by our research group which can bring to the reader a more accurate explanation, with concrete models used on real cases.

c

Fig. 4.1  Additive manufacturing technologies: solid, powder, and liquid based systems

4.2  Selective Laser Sintering (Powder-Based System) or SLS

4.1 Fused Deposition Modeling (Solid-Based System) or FDM FDM works through the microextrusion of thermoplastic materials, which is sequentially deposited in layers. The thermoplastic filament (the most common being acrylonitrile butadiene styrene or ABS) is melted and extruded through a heated nozzle, which moves according to the X and Y coordinates to produce one slice of the model (a profile). Continuing the process, the elevator moves down and the next layer is built on the top of the previous one until the prototype model is fully built. This technology requires the construction of supports that are matched to the shape of the prototype and then removed in the post-processing phase. In terms of didactic purposes, this is one of the few processes that allow observation of the deposition process through a window, which can be stopped if something becomes visibly wrong with the model (Fig. 4.2).

Fig. 4.2  FDM print

39

4.2 Selective Laser Sintering (Powder-Based System) or SLS SLS uses a range of materials (nylon, tpu, metallic materials, and others) in powder form that generate prototypes by heating material to below its melting point until its particles adhere to each other. In the SLS process, a laser beam is focused on the surface of the material which fuses the powder, creating one slice on the surface of a powder bed. After each layer is built, the powder bed is lowered by the thickness of one layer, and a new layer of material is applied on top, the process being repeated until the model is completed. One of the advantages of SLS is the strength of the prototypes, which can function in some cases as a final product, to be tested in real conditions. In addition, the post-processing phase is very simple. It is only necessary to remove it from the powder bed to vacuum the powder, since the parts are strong enough to be manipulated. The non-fused material can be reused in part, in a refresh rate ranging from 0 to 50%. This happens because of the moisture and the loss of some physical properties when exposed to heat (Fig. 4.3).

40

4  (3D) Printing Technologies

Fig. 4.3  Skull of fetus with ZIKA virus from SLS print

4.3 BJT: Binder Jetting (Joined with Bonding Plaster/ Gypsum) The construction process works by using a print head, to deposit an agglutination liquid on the top layer of a plaster composition; the elevator moves down, allowing another layer of material to be added on the top, and the action is then repeated until the model is finished. This process does not use support structures, since the model is positioned inside the powder, sustaining the model by itself. When the model is ready, it is necessary to vacuum the remaining powder (which often can be reused). After the model is removed and cleaned, it is necessary to add a liquid hardener (cyanoacrylate), in order to give the prototype its final required strength. The final appearance of the models is matte, having no plastic appearance and visually looking similar to bones, which might be helpful to represent body structures. This technology also permits the printing of prototypes in color, and when compared with other systems, the negative points are the resolution of the construction, which can be less refined and fragile than other systems (Fig. 4.4).

Fig. 4.4  12 weeks 3D printed model of twins made on Z Corp technology

4.4 Material Jetting (MJ) The technique is based on small droplets of photopolymer cured by UV-Light when exposed to the UV light source, the photocurable resin changes from a liquid to a solid state, generating

4.6  Digital Light Processing/Liquid Crystal Display: DLP/ LCD

41

Fig. 4.5  Heart of a fetus printed on material jetting print

a physical slice. This procedure is then repeated sequentially until it reaches the final dimension of the physical model to be built. This process also requires the addition of physical supports during the construction of the prototype (removed after it has been built). There is a wide range of available materials utilized with this technology, with the most common being high-performance polymers and acrylic and epoxy resins (Fig. 4.5).

4.5 Stereolithography (SLA) It is a vat polymerization technology which involves a photosensitive resin cured by a light source in order to produce solid layers. The resin is enclosed within a vat, or tank, and is cured against a build platform. This platform slowly rises out of the vat as the layers are cured. SLA uses a laser beam to cure the resin. The beam is reflected by galvanometers which can be thought of as mirrors used to guide it through the transparent vat bottom to a specific point on the build platform. In order to cure each layer, the galvanometers project the laser beam to scan each area to be cured, which takes time and, eventually, wear the tank bottom. SLA has a very good finishing and nowadays a lot of different materials such as high temperature resin, elastic resin, ceramic resin, biocompatible resins, and so on (Fig. 4.6).

Fig. 4.6  Skeletons from SLA print

4.6 Digital Light Processing/ Liquid Crystal Display: DLP/ LCD This technology is very similar to SLA, but, instead of a laser, an UV source projects each layer to cure a photosensitive resin on a tank (VAT). The projector lights all the layer at the

42

Fig. 4.7  DLP/LCD print

same time, which makes this method faster for production: the time used to print one piece is the same as printing the whole build platform (Fig. 4.7). For a better understanding of how each of these techs can contribute to a specific case, below they are compared by some characteristics—ranging from availability to material diversity—which give us more concrete elements in order to choose which one to use.

4.7 Availability The AM technologies have been used for more than 30 years. The stereolithography (SLA) was the first patented method—by Chuck Hull in 1984—and since, one of the most popular techniques [1]. Nevertheless, the accessibility of this technology was very limited. It was a high cost process focused on big industries, which limited the spread of use. It did not happen only with the

4  (3D) Printing Technologies

SLA. All the pioneer technologies had the same beginning, being used very strictly. The end of the twentieth century brought the world two remarkable events: China becoming a key player in the global supply chain and the worldwide connection from the popularization of the internet. The industries, from the most diverse areas, were shaken by a great expansion of offers and falling prices of new machines and products, including AM.  At this time the patents were expired, and the technologies gained public access. An impact factor on this accessibility has been the power of the Internet. Online discussions and exchanges brought new insights on AM technologies, popularizing and bringing it more close to the most diverse areas. The 2004 RepRap project was one of the very emblematic examples of this movement, allowing 3D printers to print another 3D printer, reducing drastically the market entry barrier [2–4]. The democratization of manufacturing by making the technology available to individual entrepreneurs and the general public drove a mindset shift and behavioral change toward more diverse ways of the technology application [2, 5, 6]. The use of AM within networks of hobbyist, designers, and producers serves as a think tank which brings knowledge and creative thinking in order to improve this area. Three decades after the creation of the SLA, several repositories with 3D STL files are available on the Internet. Many of these files are shared for free. Enthusiasts have been using these repositories to make available their solutions for the most diverse things: from daily solutions such as pen holders, to complex fabrications such as 3D printers. As an example, Petersen and Pearce [7] demonstrate that local use of AM results in substantial savings for each product produced compared to its counterpart produced by traditional methods and available in current trade channels. The study points to an average marginal cost reduction of 93.3% and 98.7% when compared to the lowest and highest retail values, respectively, considering manufacturing one product per week.

4.8  Cost per Model

43

Comparing the printed objects with the equivalent lower priced product, the payback time of 2.4  years is given. If compared to the higher priced objects, the payback time was only 0.46 years.

4.8 Cost per Model As simple and spread the technology the cheapest it becomes. The cost per model in additive manufacturing reflects the availability, ease of use, and productivity of a specific process or material. Each printer, no matter which technology is, has its own operational cost. This cost is given by an equation which involves material cost, material volume (related to size and geometry of the piece), and energy consumption. As state by Petersen and Pearce [7]. This operating cost (OL) was calculated as follows:

OL  ECE 

CF m f 1000

 USD



where E is the energy consumed in kWh, CE is the average rate of electricity in USD/kWh, CF is the average cost of the material in USD/kg, and mf is the mass of the material in grams consumed during printing. The total cost (CT) to the average consumer using the selected printer for an average of time is the following: T



CT  OL  CL  USD  0

where the operating cost is summed over T years and CL is the cost of the printer itself [7]. Extras such as fails, finishing materials, and maintenance shall be added when needed. Given this cost formulation, it becomes simple to see the order of costs of the six technologies discussed here. FDM is by far the technology with the lowest cost per model of all. This is due especially to the low cost of purchasing the machines and materials. One point that contributes to this low cost is that most of these machines

and materials use open-source technology, expanding the range of solutions and reducing the costs related to proprietary technologies and intellectual property. Obviously, there are more expensive models of FDM printers, as well as very expensive special materials (such as PEEK, PEKK, and PPSU), but on average this type of printing is the cheapest among those analyzed. Today an entry level printer costs around 200.00 USD and the material around 24.00 USD/ kg. Next to FDM is DLP/LCD.  Following the open-source tech already consolidated by FDM, more and more affordable printers and materials have been launched in the market every year. However, their parts are more complex and their materials require more complicated and expensive chemical elements in their manufacture, which reduces the ease of replication and thus increases the cost of the technology as a whole. Today an entry level printer costs around 200.00 USD and the material around 40.00 USD/ kg. Despite the similar numbers the big difference in cost is in the printing area. While FDM has an average print volume of about 8000 cm3 (20 cm × 20 cm × 20 cm) DLP/LCD has an average volume of 1320 cm3. Unlike FDM and DLP/LCD, the other technologies are all proprietary, reducing the number of players, and therefore the acquisition options. In fact, they are more complex technologies that involve minutiae and manufacturing tolerances that are much tighter, which, as a result, makes both printers and materials much more expensive. Among these, the SLA has the lowest cost per model, followed by the SLS and finally the MJ (which is well above the others). Today an entry level SLA printer costs around 3500.00 USD and the material around 150.00  USD/kg plus 150.00  USD per resin tank (which prints up to 10 L). The SLS technology has been gaining space in the market and recently several Desktop options have emerged, which is cheapening its use and consequently reducing the cost per part. Today an entry level SLS printer costs around

44

15,000.00  USD and the material around 150.00 USD/kg (however, with each print about 30% of the volume of material used in the previous one needs to be replaced with brand new material). Next to SLS, BJT has a similar cost. Although there are several industrial models that are quite expensive, the office models are affordable to buy and run for about $3.00 per printed square inch. Finally, MJ is the technology with the highest cost per part among all those being addressed. Only three players manufacture printers with this technology. Today an MJ printer costs around 40,000.00 USD (multi-material) and the material is something around 1300.00 USD/kg.

4.9 Productivity Compared to standard manufacturing technologies, additive manufacturing is somewhat slower. Couple this with a smaller building volume, and it leads to lower overall productivity of the technologies. However, the ability to manufacture complex geometries and the reduced cost relative to tooling flatten the disparity curve.

Graphic 4.1 Injection molding X 3D printing cost comparison

4  (3D) Printing Technologies

A widely used example is the comparison between injection molding and 3D printing. A case is pointed out by Faludi et al. [8]. If the mold and setup for manufacturing a plastic part cost USD 10,000 and the price per injected part is USD 0.50, then 3D printing, which in the case of that same part costs USD 5.50 per unit, is still economical for less than 2000 units. However, if we ramp up production to 100,000  units, the price of 3D printing should drop to $0.60 per piece to be as economical as injection molding (Graphic 4.1). Different technologies bring different results, so the resulting pieces cannot be associated for cost comparison. Clearly, this example does not consider determining factors such as part geometry, cost per freight mile, and, as in the case of this paper specifically, the printing of parts for particular cases and not aimed at large-scale replication. Conversely, it allows us to visualize and compare the productivity of additive manufacturing technologies to mainstream ones. To talk about productivity in AM is to address ease of use, print volume, print time, resolution, and post-processing. These various aspects overlap, influencing each other in ways that result in quantity, speed, and quality of the final part.

4.10  Ease of Use

4.10 Ease of Use

45

huge diversity of materials, make the FDM and DLP/LCD setup more complex and consequently Ease of use, at a first glance, seems to be present result in less homogeneous parts and a higher only in plugging in and having it printed. learning curve. However, because they are However, this aspect is present in several steps of cheaper and simpler technologies, experimentausing AM technologies. For a part to be printed, tion is more present and easier, presenting itself the work begins well before the play button is as a counterpoint to this complexity. pressed. The process begins with a three-­ A very important point in the ease-of-use dimensional file that is basically the same for all analysis is the part orientation over the print area. technologies. The next step is the slicing of this The orientation of the piece is so important that it three-dimensional electronic model. Here the dif- will reflect on all parts regarding productivity. A ference begins. larger print area allows for a larger amount of The slicing program is responsible for gener- parts per cycle or larger part sizes. However, the ating the file that will be “read” by the machine, way the part is arranged in this volume is a cruin order to perform the actions for the part to be cial element in the quality of the final part. manufactured. Most of them are GCODE files, or There are those who consider additive manuvariations, that at each programming line indicate facturing as a process that would not be 3D but the print head positioning (with the exception of 2.5D, since the raw operation of the technologies SLA/DLP, due to its type of printing) and the is in flat slices (2D) that overlap. In short, it configurations of the materials being printed. It is would be several 2D processes stacked together. in the slicer that the most diverse parameters are Thus, a key element regarding the positioning of set to obtain the best quality printing (resistance the part is related to the height in the Z axis. In all and resolution), using the least amount of mate- technologies, the higher it is, the longer it takes to rial and with the highest possible speed. The pro- print. prietary technologies (SLA, SLS, and MJ) end up As the name implies, additive manufacturing having simpler slicers due to the fact that they adds material rather than subtracting it (as in hold the manufacture of both the machine and the classical processes). Therefore, for the material materials which makes preset profiles much more to stay in place a base is needed. In the layer-by-­ functional than open technologies (FDM and layer build-up there may even be angles at which DLP/LCD). This ensures greater replicability the material can overhang. However, the sharper and a smaller learning curve in the handling of the angle, the less precise the final geometry of such technologies, which guarantees parts with the part becomes. Thus, to maintain the desired more homogeneous characteristics. geometric characteristics, in parts with acute Like the hardware and internal software tech- angles or overhangs and concavity with long nologies of the printers, the slicing software for spans, it is necessary to add a support material, FDM and DLP/LCD technologies, for the most whose sole function is to ensure the geometric part, is the result of open-source programming accuracy of overhang areas. After printing, this and language development. As in the case of the material is removed and discarded. machines, this characteristic makes the quantity, Besides the geometric issue caused by manuspread, and use of this software very large, which facturing in layers, anisotropy is another remarkreturns in an active community, always looking able characteristic of most of the parts for new functionality. This fact ends up bringing manufactured by AM. In this manner, the orientaa positive aspect on the one hand, which is the tion of the part in the printing volume is a fundaconstant updating and the addition of increas- mental element in guaranteeing the final quality ingly advanced features, and a negative aspect of the part to be produced. Like the other paramthat is the exponentially increasing complexity in eters, the orientation must take into account a the use of these programs. Many software number of factors such as: number of parts per options, added to many parameter variables and a processing, reduction of the amount of support

4  (3D) Printing Technologies

46

(which will reflect in the reduction of material and printing time), and the specific need for resistance in a certain direction. This logic will vary greatly from one technology to another. Among the technologies treated, FDM is the one that most generates anisotropic parts, and therefore the orientation of the part in relation to the need for strength in a certain sense is crucial to the use of the technique. It is therefore very important to find a middle ground between mechanical strength and the amount of support when defining the positioning of the part in the print volume. Regarding DLP/LCD, anisotropy is not a key point in the part orientation, because the resin curing process leads to a behavior closer to isotropy. In this case the ideal orientation of the part takes into account two factors: the light projection of the pixels and the contact area of the printed layer with the film of the resin tank, with the angulation of the piece being the most suitable orientation. This is for a few reasons. The light projection of the pixels is important because it improves the surface quality of the final part. For a better use of the pixel size in relation to the

chosen layer height, angling the part reduces the “steps” caused by the light distribution between the pixels forming the image matrix. The optimal angle is given by the ratio of the layer height to the pixel width. This makes the distance between the layers and the contour peaks as small as possible, reducing the stepping effect inherent in the process geometry. The second point, the contact area with the VAT film, is important to analyze because of the peeling forces. The larger the contact area of the layer with the tank film, the greater the suction effect. Thus, reducing the polymerized area in contact with the VAT of each layer makes the forces smaller, reducing vibrations and unwanted movements that cause print failures. In some cases the suction can be so strong that it pulls the part off the plate, making the print impossible (Graphic 4.2). In the case of SLA, because it is a laser and not a pixel array, the orientation is to reduce the height in the Z axis and also to reduce the area in contact with the VAT film in order to reduce peeling forces. On the newer machines some features have been added so that the film is tensioned only in the area of contact with the part, reducing the

Graphic 4.2  DLP/LCD part orientation regarding the pixel matrix

47

4.13  Resolution and Post-Processing

suction forces and facilitating the positioning of the parts. In SLS, MJ, and BJT, orientation is made easier. In the former, the fact that the material itself serves as a support makes orientation a simpler matter. Even, due to this fact, it is possible to print fittings already assembled. In MJ, because they are multi-material, soluble support material is used, also making part orientation easier. MJs use voxel logic (three-dimensional pixels), where each of these cubes is printed individually. However, unlike the DLS/LCD logic, the layer height and voxel resolution is so small that the steps eventually disappear. BJT, on the other hand, has a similar logic to SLS where the material itself serves as the support, however, since this process has no heat, the geometric stability is even greater than that of SLS.

4.11 Average Build Volume Print volume varies greatly across all technologies. Today it is already possible to find the most diverse dimensions in all processes. However, print volume and resolution are usually inversely proportional. This is for a simple reason: The higher the resolution, the longer the printing time, and the larger the volume of the part, the longer the printing time. So, to be viable in time, large format printing must give up resolution. The only case where this does not apply (relative to overall part volume) is in DLP/LCD. Since the whole layer is projected at once, and the curing time of the resin does not vary at each point of the VAT, printing a part of size X or 10X, or even 1 part or 10 parts, takes basically the same time. However, since this process is based on the projection of light onto a matrix of pixels, the resolution is given by the number of pixels and their size. Smaller screens end up with higher resolutions because they have smaller pixels. Larger screens, with the same number of pixels, end up having larger pixels. So, in order to keep the pixel size of small screens and big screens the same (i.e., equal resolutions), big screens end up needing to have many more pixels, thus increasing the cost and power consumption.

FDM, BJT, and SLA are the ones that allow the largest print volume of all, with machines that considerably exceed 1 cubic meter in build volume. DLP/LCD comes next with volumes close to 1 cubic meter. MJ and SLS are close in terms of printing volume, with MJ having slightly higher volumes on average.

4.12 Print Time Print time, like most AM features, is not a closed account. Many variables will contribute to it such as part size, machine resolution, material used… A part produced with the same machine, but with different resolution (layer height), will have different printing time, with the one with the higher resolution taking longer. The same applies to materials. A part also produced with the same machine, but with different materials, will have different printing times due to the processing characteristics of each material such as temperature, fluidity, chemical reactions, etc. On an average DLP/LCD (LCD especially with mono matrix) is the fastest technology, especially since it prints the whole layer at once. FDM, MJ and BJT compare, with MJ being slightly faster than FDM when printing with more than one material at a time and BJT being faster than both because the Binder cure is quicker. Next comes SLA.  Although the laser path is similar to that of SLS, the curing time of the materials is slightly shorter than the sintering of the latter. In addition, the preparation time for the SLS print volume (which needs to be heated) and post-print cooling makes the time greatly extended.

4.13 Resolution and Post-Processing The higher the resolution used on a part, the better the finish quality of the part, and thus, the less post-processing, or finishing, is required. There are two types of resolution in AM: the resolution given by the height of the layers (the most spoken of) and the XY resolution. The first is easily mod-

48

ifiable, the second is given by the machine being used, and both will vary greatly between technologies. Post-processing, on the other hand, is related to the post-printing work required before a part can be used. It involves removal, cleaning the part (such as removing the supports), finishing the surface, and painting. The technology with the highest resolution is MJ, far ahead of the others. It can print layers as thin as 16 microns (0.016 mm) and has XY resolution of up to 200 microns (0.2 mm). In addition, it is possible to print with various materials and colors at the same time. Usually a soluble material is used as the support, so coupling that with the great resolution and colors makes the post-­ processing of MJ parts among the lowest. The part comes virtually ready from the printer (Fig. 4.8). Both DLP/LCD and SLA have very similar resolution and post-processing, the former being a bit less precise. Both achieve very fine prints on the Z axis layers, with 0.25 mm being the thinnest thickness effectively visible. The difference is in the XY resolution. As said before, DLP/ LCD technology is based on an array of pixels, so the smaller and larger the pixels, the higher the resolution. So smaller print areas tend to have more pixels. For example, a printer with an area Fig. 4.8  Two views of a MJ pnrint of a stomach balloon (yellow) compared in size with a regular pen

4  (3D) Printing Technologies

of 130 mm × 82 mm and a 2K LCD, or 1620 pixels by 2560 pixels (standard for entry level printers today) delivers an XY resolution of 51 microns (0.051 mm). A 192 mm × 120 mm area printer, on the other hand, needs a 4K LCD, or 3840 pixels by 2400 pixels, to maintain the same 51 microns of XY resolution. If you had a 2K screen with the same print area, the XY resolution would go up to 75 microns. In SLA, the XY resolution is due to the thickness of the laser beam, which is about 85 microns. However, due to the combination of machine and resins (both proprietary), the adjustments on the slicer always give the best combination, bringing a better resolution even if you have a “bigger dot” (Fig. 4.9). The resolutions treated are the theoretical maximums. In both cases, the size of the pixel or of the laser beam does not alone determine the XY resolution. The properties of the resins (formulation, sensitivity, pigmentation, etc.) also have a determining character in the final resolution. As for post-processing, DLP/LCD and SLA are virtually identical. After printing it is necessary to clean the parts, so that all uncured resin is removed from the surface. Normally 99% alcohol is used for this cleaning, but now there are resins that are water soluble, which makes the

4.13  Resolution and Post-Processing

49

Fig. 4.9  LCD/DLP build plate

cleaning easier and cheaper. After cleaning, the supports are removed from the part, and finally the general curing of the resin on the part is completed. This is necessary so that the part polymerizes equally and in all parts (internal and external). The curing can be done in a chamber with UV light or even under the sunlight. SLS has a resolution close to DLP/LCD and SLA, with a small disadvantage in layer height. The minimum possible is between 60 microns and 100 microns. The XY resolution is close to that of SLA (around 80 microns), because it sinters a powder (a more complex process than resin polymerization), its finish is a bit rougher and uneven compared to SLA.  One element that greatly affects the finish of SLS parts is humidity. Each material has a specific percentage of humidity in order to deliver the best possible resolution. If this percentage is outside the standard, there are risks to the integrity of the part and even to the printer. BJT has a slightly lower resolution than the SLS, with the minimum layer height being

between 90 and 100 microns. Conversely, the resolution in XY, for the most accurate ones, is 300 × 450 DPI, which means a pixel of about 70 microns (0.07 mm) (Fig. 4.10). Although they do not use support in the manufacturing of the part, both SLS and BJT require labor-intensive post-processing. Because the material itself supports the printed part, the volume of material in the print area is very large. The process of removing the part looks like an archeological dig: dust is removed until the part is reached. The remaining material is then “recycled” (sieved and, in some cases, replaced with a percentage of virgin material) and restocked for new use. Once removed, this part still has a lot of dust on its surface and needs to be cleaned. To do this, a blasting station is used (with compressed air, and sometimes abrasive materials) so that the part can finally be used. After these steps, the SLS printed parts are ready for use (Fig. 4.11). BJT printed parts, after this post-processing relative to cleaning, need a finishing for practical use, especially when they are not made only for

50

4  (3D) Printing Technologies

Fig. 4.10  SLS print of internal strucutres

Fig. 4.11  SLS post-process

visual representation. As they can be heavy (plaster), they can be also fragile due to liquid binder joint, being necessary to “reinforce” them. This “reinforcement,” or infiltration, is the process of applying a liquid resin (mainly cyanoacrylate) to a printed part to provide specific strength and/or properties. These resins, or even metals in some cases, are formulated to effectively penetrate the part, allowing the resin to soak into the porous surface without leaving just a film on the surface of the part, reinforcing the print, and bringing new mechanical characteristics. Important characteristics of FDM in relation to other technologies are the low resolution and the high tolerances. While it is possible to achieve layer heights close to those achieved by SLA or SLS (in the case of FDM between 50 microns and 100 microns), the XY resolution is lower and will be constrained by the material used. This is because the XY resolution is determined by the diameter of the nozzle, the most common being 0.4 mm (400 microns), and not all materials are compatible with the smaller diameter nozzles, which are typically around 0.2 mm (200 microns). Thus, the surface of FDM printed parts has a

4.14 Material

rougher finish compared to other technologies, and parts are usually of a slightly larger scale. FDM post-processing is also quite complicated. Since the molten material cannot be deposited in air, it is not possible to directly print spans or overhang areas with very acute angles. Therefore, it is necessary to add support structures. This ends up costing more material and time, and the parts will require more work to remove these supports. Another issue is that the surface quality in the contact area is affected, losing some resolution and finish (Fig. 4.12). One way to minimize some of the disadvantages of using support is to print them using a soluble material. This facilitates their removal and improves the surface quality of the contact area. However, to make this possible, the printer needs to have a multiple-extrusion system: one nozzle prints the part material, and another nozzle prints the support material. Although multiple extrusions make support easier, post-processing

51

still involves work and time, and depending on the support material, some chemicals are required. This multiple-extrusion logic, besides the soluble support, allows printing with more than one material or color, which contributes positively to the final finish of the piece, mixing colors and characteristics in specifically designed areas. Finally, a process often used in the post-­ processing of parts made in FDM is surface smoothing. Because it has a lower resolution, the lines of the layers become more prominent, presenting a very peculiar surface characteristic. To make it smoother, processes can be used, and each one is particular to each material. One of the best known is the acetone vapor finishing, used on ABS parts. Acetone, a solvent, reacts with the plastic causing the surface to soften, “joining” the lines of the layers giving a smooth surface. It is important to remember that this is a dangerous process, as it involves very toxic chemical materials. Furthermore, not all materials have solvents that allow this kind of process, and so they are usually finished with sandpaper and putty, or the part is used exactly as it came out of the printer.

4.14 Material

Fig. 4.12  FDM print with support structure

FDM is by far the technology with the widest range of materials available at a lower cost. These are polymers extruded in filament form and may have a diameter of 1.75  mm or 2.85  mm. The materials can be divided into three categories: commodities, engineering, and high performance (Graphic 4.3). Commodity materials are the most affordable and widely distributed. Poly-lactic acid (PLA) and ABS are the most popular plastics in this category. PLA is obtained from corn, cassava, or sugarcane bagasse and can be recycled or composted in industrial composting plants. It is the easiest plastic to use, which helps to popularize the technology. Conversely, it is very rigid and has little resistance to temperature, being affected by environments above 50 °C. It can be found in the most diverse colors or color compositions

52

4  (3D) Printing Technologies

Graphic 4.3  FDM technology has, by far, the widest range of materials and at a lower cost

(filaments in gradient) and also in transparent (natural color, as they call it), which despite its name delivers translucent prints, not totally transparent. It is also possible to find PLA SOFT, which has some plasticizers that reduce its hardness, getting closer to rubbers, with a value of 98A on the SHORE scale. ABS was the first plastic to be popularized in additive manufacturing. It is cheaper than PLA and more widely used in the plastics industry in general. Because it is an amorphous polymer, it presents greater difficulties when printing. Very sensitive to temperature variations, it needs a heated table and as little air circulation as possible, to avoid the warping effect. It is more flexible than PLA and much more resistant to temperature but suffers a lot from exposure to UV light. Another feature of ABS already discussed here is its ease of finishing, presenting a very smooth surface after treatment with acetone vapor. An important point is that ABS can be sterilized by the STERRAD method, while PLA and PETG only can be sterilized with ethylene oxide.

On printers with more than one extruder, both PLA and ABS can be printed together with a soluble material to facilitate the removal of the supports. This technique allows a lower consumption of these materials, as well as improving the surface finish in the areas of contact with the support. Each has a pair that is best suited. PLA can be printed together with PVA that can later be diluted in water. ABS typically uses HIPS (a polymer that can also be used in final parts) as a soluble support material. This material is diluted in d-limonene. The most used engineering materials are PETG, TPE/TPU, and Nylon (PA). PETG has emerged as an excellent alternative to ABS in terms of ease of printing and final part strength. It is a very versatile material and comes in as many colors as PLA and ABS. It is less rigid than the former and more chemically resistant than the latter, besides presenting a lower warping tendency. TPEs are thermoplastic elastomers, and among them the best known and most used is

4.14 Material

TPU (thermoplastic polyurethane). They are ultra-flexible materials with printable filaments that have a hardness ranging from SHORE 60A to SHORE 98A. The more flexible the more complex its printing and often the printer’s extruder needs adaptations to be able to extrude this material with the required flow and speeds. Nylon (PA) is an excellent material for end-­ use parts, or even for replacement parts. Because it is very hygroscopic, it is important to pay attention to the humidity of the environment where it will be printed. Like ABS, it is a material that has a great tendency to warp and therefore it behaves better in printers with a closed chamber. Its printing temperature is a little higher than the polymers treated above, around 250 °C, which limits the use of simpler printers, whose maximum extrusion temperature is around 240/245 °C. Nylon can be sterilized in all commonly used methods—such as autoclave, STERRAD, ethylene oxide, and 70° alcohol— without dimensional loss. The last category is the newest and features quite complex and expensive polymers, PEEK and PEI being the best known. PEEK filament is one of the best materials on the market for high-­ performance applications. It has unique properties in terms of mechanical, thermal, and chemical resistance, making it the solution for the most demanding parts. Among the properties of the models made with PEEK are high heat resistance (above 270  °C), high chemical resistance, and great tensile and load resistance (comparable to titanium and steel). Among the applications for this material are use in medical implants (the material has biocompatible formulations), parts that work in high temperature or high-pressure environments, as well as in places with high chemical risk. PEI or polyetherimide is a high-performance engineering thermoplastic material. It has an amorphous structure with high temperature resistance and is very suitable for printing functional parts. PEI has properties that ensure good stability in the face of temperature and humidity variations. Its long-term heat resistance allows the filament to be used in the construction of parts that replace metal among other materials in the

53

most varied structural applications. Among its main characteristics are high strength, rigidity when subjected to high temperatures, resistance to chemicals, long-term heat resistance, dimensional stability, and it is inherently flame retardant. Being somewhat cheaper, PEI is an alternative to PEEK.  The two materials have very similar high temperature resistance, although they belong to different thermoplastic families: while PEEK is a thermoplastic of the PAEK (polyaryletherketone) family, PEI has no ketone in its molecular structure. For printing PEEK and PEI it is recommended to use professional printers with high temperature settings (hotend above 450  °C) in addition to a heated bed (above 150  °C) and a heated printing chamber (above 80 °C). In recent years many engineering and high-­ performance materials have been combined with reinforcing fibers. Today it is already possible to find Nylon, PETG, and PEEK, reinforced with carbon fiber or fiberglass, for example. SLA, DLP/LCD, and MJ technologies use light-sensitive thermosetting liquid polymer resins (photopolymers), which allow them to be cured (hardened) by the action of UV light. The simplest materials are usually rigid and brittle. However, a major evolution has been taking place in recent years. New materials have been combining two or more photopolymers, resulting in a third material with hybrid properties (e.g., combining rigidity with flexibility). More and more materials are being optimized for specific applications such as tooling and jewelry casting, dentistry, and biocompatible formulations. Much research has been done in order to expand the range of materials available. In the case of MJ and SLA, because they are proprietary technologies, their manufacturers have been investing heavily in special materials and thus photopolymers composed of metals, ceramics, and silicones have been appearing every year. DLP/LCD technology is a little behind in terms of variety, but this gap is being reduced with the advances and popularization of the technology. An interesting feature of the three technologies is that all allow printing of transparent materials, unlike FDM and SLS, and flexible materials,

54

being MJ and SLA able to print materials with hardness as low as SHORE 50A. SLS technology does not have a wide variety of compatible materials. However, the possibility to use technical materials like different types of nylons (especially PA12 and PA11), or TPE and TPUs (with low hardness like SHORE40A), makes it able to cover most applications. As with MJ and SLA, new material types are emerging every year for use in SLS.  The focus of manufacturers has been on more common polymers such as polypropylene, and on composite materials such as aluminum-filled nylon (alumide), glass-­ filled nylon (PA-GF), and carbon-fiber filled nylon (PA-FR). BJT’s materials have been gaining in number and diversity in recent years. In this tech-

Table 4.1  Sterilization method comparison

4  (3D) Printing Technologies

nology, the materials for printing are divided into two parts: one is the powder and the other is the binder. Initially this technique relied on gypsum-­ based composite powders, bonded with colored resins derived from epoxy. Nowadays the range of powders has grown a lot, and it is possible to use polymers (such as PMMA), sand (widely used for printing molds for direct metal casting or investment casting), and even metals (which generate the so-called green parts that after sintering turn into a full metal part). The binders have also evolved, to allow a better performance of each new powder, as well as allowing a better infiltration in order to adequate the mechanical characteristics of the part to its use (Table 4.1).

4.15  How to Apply Each One

55

4.15 How to Apply Each One

and moisture in the working area must be controlled (Fig. 4.14). Fused deposition modeling (FDM) performed MJ has one of the highest levels of precision on professional machines makes it easy to among AM technologies. Thanks to the precise obtain quality parts with final, approved materi- deposition of tiny droplets of material, layers can als. This additive technology is applied in sec- be printed as thin as 0.013 mm, enabling the protors such as industry, medicine, and aeronautics. duction of parts with very smooth surface finSome of its applications are models for dimen- ishes and allowing the production of parts with sional and functional validation (products, sur- small but highly accurate features. gical cases, evaluation models, etc.), parts with This technology is widely used for color parts approved material for internal use (such as and, most notably, multi-material parts, since the prostheses and surgical guides), short series print head used in the process typically incorpomanufacturing (low volume manufacturing) of rates multiple nozzles. These nozzles can deposit parts, and manufacturing of lower cost parts different materials and/or different colors in a (such as ventilation adapters, spare parts, etc.) single printing process. Objects made by MJ can (Fig. 4.13). therefore possess a range of material properties Some advantages of the FDM process are its such as stiffness, flexibility, opacity, and transluwide variety of materials, allowing the use of cency. It is used extensively for the visualization simple or complex resources for any application. of different tissues and their interactions in the Ease of supports removal and no need of post-­ case of multi-material, and in situations where curing make handling the part easier than in other higher contrast is required, such as for visualizatechnologies. With this tech it is possible to man- tion of tumor irrigation, in which multi-color ufacture very large parts without deformation and printing is used. to combine materials during the manufacturing The medical field is increasingly using MJ to process. produce realistic anatomical models, whether for Some considerations must be considered education, surgical planning, or training. These when using FDM. Its build speed is slower than full-color printed models look very close to the other technologies and requires a good part real human body parts, which gives students a ­orientation for a better support generation. The better learning experience and makes it possible surface of the piece will have some visible layer for surgeons to plan and prepare complex surgerlines. It is not recommended to place the parts ies more efficiently compared to how it is done “one on top of the other” in the build volume dur- with 2D images (Fig. 4.15). ing manufacturing; otherwise, it will be very difMJ also has its limitations. The objects proficult to separate them. And finally, temperature duced are usually more fragile, especially when a

b

Fig. 4.13  Skull of fetus with ZIKA virus printed on FDM (model with internal support (a) and cleaned (b))

4  (3D) Printing Technologies

56

compared to other additive manufacturing techniques. This makes the parts produced by this technology generally less suitable for functional applications. MJ is typically used to produce parts in which appearance is more important than function (Fig. 4.16). SLA and DLP/LCD have very close applications. The high quality of the printing makes them very effective in the manufacture of miniatures, medical models, and jewelry. Castable resins work very well for the fabrication of detailed metal parts by investment casting. These technologies are also used in rapid prototyping and product design because of their capability to generate high-quality 3D models

Fig. 4.14  FDM print of a craniopagus twin Fig. 4.15  Bladder model printed by MJ (a, b)

a

and complex shapes. Prototypes and functional tools can also be made with SLA and DLP/LCD due to their accuracy. Another use of these techniques is in a process known as rapid tooling. In it, injection molds are printed, and parts can be manufactured quickly and cheaply for first prototypes and short runs. This lowers the entry barriers for inventors/manufacturers, facilitating the development of new and innovative products. In medical applications, SLA and DLP/LCD are used to create anatomical models of patients, especially in complex cases. These technologies enable accurate printing of both external and internal areas. For an FDM printer, it is a complex task to print internal geometry accurately, but with an SLA printer, complex patterns can be printed on both surfaces. An example is in cases where visualization of irrigated areas is required. With these technologies it is possible to print the internal cavities, allowing visualization of the fluid flow very accurately (Fig. 4.17). These features are also being applied in drug studies, disease modeling, and personalized medicine [9]. The so-called organ-on-a-chip is a small piece printed with biocompatible resin. This chip contains hollow microfluidic channels that are coated with living human cells along with an interface that lines the inner surface of blood ves-

b

4.15  How to Apply Each One

57

Fig. 4.17  SLA printed model with internal cavity area pigmented Fig. 4.16  MJ model of a pregnant woman with her child inside the womb

sels and lymphatic vessels, known as endothelium. This process creates artificial living organs that mimic the complex, physiological responses of real organs, thus allowing drugs to be tested by precisely manipulating the cells and their microenvironments. The pros of using SLA and DLP/LCD are the high-quality 3D models, the possibility of complex structures, it is relatively a fast process, and the unused resin can be fully reused. The cons are the handling of the resin, which is quite toxic, its propensity to spill, and the need for post-printing cleaning (with isopropyl alcohol or specific solvent) and post-curing (either directly in the sun or in a specific UV chamber) of the parts (Fig. 4.18). One of the main benefits of using SLS technology is that it does not require the use of a support structure. All voids are automatically filled

with unused powder, thus making this type of printing self-supporting. This offers an enormous degree of design freedom. Models with large hollow spaces, overhang structures, and very thin elements are not a problem when printing. SLS also provides a viable solution for printing complex shapes that would require multi-part printing if another technology such as FDM or SLA was used. Because of these characteristics, SLS printing is widely used for complex anatomical models, especially for visualization of cavities and precision fittings. Another important characteristic of SLS is that the adhesion between the printed layers is very strong. Because of this property, SLS prints have practically isotropic mechanical properties. This means that the tensile strength, hardness, and elongation of an object printed using SLS are almost equal in all directions, making it an excellent alternative for printing orthotics, prosthetics, spare parts, and custom tools.

58

4  (3D) Printing Technologies

Fig. 4.18  Same file printed with different technologies. DLP/LCD (left) and FDM (right)

SLS technology has also been successfully used in other fields such as biomedical engineering. By modifying standard machines, it is possible to manufacture tissue engineering scaffolds and drug/biomolecule delivery vehicles [10], advancing the field of personalized medicine. The pros of using SLS are that complex parts can be printed without the need for supports, several models can be printed at once using the entire print volume, excellent adhesion between layers, and it is easy to paint after printing. The cons are the porosity of the surface of the printed part (which limits its use in some fields), the tendency to warping in thin parts, the cleaning of the part after printing (which is quite complicated due to the amount and gramature of the dust), and last but not least, it tends to be a process with a lot of waste. As it is necessary to heat the entire printing chamber to bring the material close to the sintering point, this material loses properties and needs to be replaced with new material in every new print. So, between 25% and 50% this material is left over from each print, generating a considerable amount of waste (Fig. 4.19). Binder jetting is used in a variety of applications, especially for manufacturing colored prototypes (the most developed use to date), positive and negative molds for sand box castings, and also functional metal parts (the latter of which

Fig. 4.19  Model printed with SLS

has been expanding rapidly in recent years). One of the advantages of BJT is that it is a process that does not use heat in curing the part, so the dimensional characteristics, as well as warping and residual stresses are avoided. Another favorable point is that no supports are needed at the time of printing, allowing for more complex geometries. This fact also allows the use of the entire build volume, which allows a good productivity per printing batch. Conversely, since the bond between the powder and the binder is mechanical, the parts tend to have a lower mechanical strength when used directly out of the printer, requiring post-processing that is usually quite costly. In addition, the range of materials (especially for office machines) is low, limiting their applicability (Fig. 4.20, Table 4.2).

4.15  How to Apply Each One

Fig. 4.20  Model printed with binder jetting Table 4.2  Technology comparison

59

60

4.16 Conclusion Different technologies bring different results. Each one has its own specific characteristics that make them more or less adequate for a certain type of application. In this way, access to a greater number of techniques allows a complementation between them and thus a broadening of the application horizon of the AM in the desired area.

References 1. Balletti C, Ballarin M, Guerra F. 3D printing: state of the art and future perspectives. J Cult Herit. 2017;26:172–82. 2. Anderson C. A nova revolução industrial: makers. Rio De Janeiro: Elsevier; 2012. 3. Dos Santos J, Brancaglion Junior A, Werner Junior H, Azevedo S. 3D tecnologias desvendando o passado e modelando o futuro. Rio De Janeiro: Lexikon Editora Digital; 2013.

4  (3D) Printing Technologies 4. Dos Santos J, Werner Junior H, Azevedo S, Brancaglion Junior A. Seen/Unseen. Rio De Janeiro: Rio Books; 2019. 5. Leary M. Design for additive manufacturing. Oxford: Elsevier; 2020. 6. Mostafaei A, Elliott A, Barnes J, Li F, Tan W, Cramer C, et  al. Binder jet 3D printing—process parameters, materials, properties, modeling, and challenges. Prog Mater Sci. 2021;119:100707. https://doi. org/10.1016/j.pmatsci.2020.100707. 7. Petersen E, Pearce J. Emergence of home manufacturing in the developed world: return on investment for open-source 3-D printers. Technologies. 2017;5(7):15. https://doi.org/10.3390/technologies5010007. 8. Faludi J, Cline-Thomas N, Agrawala S. 3D printing and its environmental implications. In: The next production revolution: implications for governments and business. OECD Publishing; 2017. 9. Carvalho V, Gonçalves I, Lage T, Rodrigues R, Minas G, Teixeira S, et  al. 3D printing techniques and their applications to organ-on-a-chip platforms: a ­systematic review. Sensors. 2021;21(9):3304. https:// doi.org/10.3390/S21093304. 10. Duan B, Wang M. Selective laser sintering and its application in biomedical engineering. MRS Bull. 2011;36:998– 1005. https://doi.org/10.1557/mrs.2011.270.

Part III Virtual Output.

In this section we discuss subjects ranging from virtual navigation and artificial intelligence to the use of metaverse.

5

Virtual Navigation on Expanded Reality Devices (Virtual Reality, Augmented Reality, and Expanded Reality)

Recently, the world has seen a lot of enthusiasm about the application of virtual reality (VR) and augmented reality (AR) in many areas, including medicine. This is mainly because the technology is quickly maturing, and the hardware necessary to provide more realistic and astonishing experiences has become accessible to everyone (e.g., powerful graphical boards and HMDs—head mounted displays). Moreover, it is also well-­ known that the major technological companies are investing a lot in these technologies, indicating that this is not only a hype [1]. VR is a high-end user-interface technique, where the user can navigate and interact in a computer-generated three-dimensional (3D) synthetic environment, being completely or partially immersed in the sensation generated by multi-­ sensory channels, vision being the main one [2]. One of the benefits of VR environments is the ability to provide advantageous perspectives not obtainable in the real world, such as navigating inside the human body, analyzing physical simulations, and reviewing of large engineering projects [3]. VR systems are composed of input and output devices (visual, auditory, and/or tactile) capable

With Contributions by Alberto Raposo Supplementary Information The online version contains supplementary material available at https://doi. org/10.1007/978-­3-­031-­14855-­2_5.

to provide the user with immersion, interaction, and involvement. The idea of immersion is connected with the feeling of being inside the environment. The term immersion in VR is derived from the physical experience of being submerged in water or, metaphorically, the sensation of being surrounded by a completely different reality. It involves physical and mental participation and implies getting away from everyday experience, playing a different role or taking on a new identity [4]. In VR, immersion is a product of both the technology surrounding the user and the user’s response to being surrounded by technology [5]. However, there are also other dimensions of immersion, such as narrative immersion, defined as “the feeling of being inside a story, completely involved and accepting the world and events of the story as real” [6]. This notion of immersion is particularly important for videogames. Person–Environment Interaction concerns the computer’s ability to detect user input and instantly modify the virtual world and actions on it. Nowadays, especially with the notion of metaverse spread by big technology players, it is also possible to define person–person interaction, which is the ability of multiple users to socially interact in the virtual environment. Finally, Involvement is linked to the degree of engagement of a person with a certain activity, which can be passive, such a watching television, or active, while participating in an interactive game or simulation. Involvement is closely related to immersion and interaction because it is

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 H. Werner et al., 3D Physical and Virtual Models in Fetal Medicine, https://doi.org/10.1007/978-3-031-14855-2_5

63

64

5  Virtual Navigation on Expanded Reality Devices (Virtual Reality, Augmented Reality…

known that better immersion and a better ­capacity of interaction increase users’ involvement in a VR experience. The concept of AR is closely related to VR, but the main difference is that AR uses a real-­world setting, overlaying virtual elements over it, while VR completely immerses the user in a synthetic world. In other words, AR can be defined as a realtime view of “a physical real-world environment that has been enhanced/augmented by adding virtual computer-generated information to it” [7]. A variant of AR is what is commonly called diminished reality, which adds virtual objects that match the background on top of a real object, giving the impression that the real object is not there. It is important to note that AR is not limited to the sense of sight. It can potentially apply to all senses and be used to augment or substitute a person missing sense, for example, by augmenting hearing for deaf users by the use of visual cues. The term mixed reality (MR) also refers to a blend of physical and digital worlds. This term was coined by Milgram and Kishino [8], which explored the concept of a virtuality continuum, ranging from the completely real to the completely virtual and exploring all possible compositions of real and virtual objects (Fig. 5.1).

In the last few years, players in this field created the term extended reality (XR), which is basically an umbrella bringing VR, AR, and MR together under a single term, trying to simplify things for the general public (Fig. 5.1). The first true VR system was constructed in 1960s at the University of Utah. It was basically a head-mounted display composed of two low-­ resolution cathode ray tubes (one for each eye). The HMD was connected to the ceiling, since it was very heavy, and had some head position sensors to send the head movements to the system. The idea of using VR in medicine appeared approximately at the same time, the 1960s, when the first simulators with 3D images appeared [9]. However, many authors consider that the application of VR in healthcare started in early 1990s, with the goal of visualizing complex medical data, especially for surgery planning [10, 11]. VR technologies have promised physicians the ability to move beyond 2-dimensional screens, allowing them to understand organ anatomy in 3D noninvasively [8]. However, only recently, high-resolution immersive displays and powerful computers became accessible, causing an exponential growth in the application of 3D models in medicine (Figs. 5.2 and 5.3).

Fig. 5.1  Mixed reality resulted from a blend of the physical world with the digital world

5  Virtual Navigation on Expanded Reality Devices (Virtual Reality, Augmented Reality…

Fig. 5.2  Virtual navigation inside the pregnant uterus using oculus Quest 2

65

cessing. Due to the importance and difficulty of the task, there is an extensive bibliography dedicated to the problem of segmentation in digital images [14]. Recently, traditional handcrafted-­ feature-­based segmentation methods have been replaced with advantage by artificial intelligence techniques (more specifically, deep learning techniques). The challenges of image segmentation and 3D reconstruction are especially important for the application of VR in medicine. The accuracy of segmentation and the quality of 3D reconstruction directly affect the fidelity of the content (i.e., the 3D model) of the VR application. Regarding the utility of VR in medicine, at least four main applications are expected: 1. Surgical planning 2. Medical education/training of young doctors 3. Better communication with patients and their families about the problem 4. Treatment of phobias and traumas

Fig. 5.3  Mixed reality using HoloLens 2 from Microsoft

Nevertheless, the visualization of complex medical data remains a challenge, since the extraction of 3D models of internal body parts from US, MRI, or CT images is generally a multi-step process that requires the use of different software [12] and requires human expertise. These steps include the image enhancement through filters, segmentation, extraction of 3D model from successive slices, post-processing (e.g., noise removal), and finally, visualization in VR (Fig. 5.4). Particularly, segmentation is the most important and difficult step of this process. It is defined as the process of identifying in a digital image the positions of pixels belonging to a structure or region of interest [13]. The segmentation task is highly challenging, being automatic segmentation one of the most difficult tasks in image pro-

Surgical planning is one of the most expected uses of VR in medicine. In this context, VR is useful in planning operations beforehand, “as it helps the surgical team walk through the whole surgery and rehearse their planned intervention” [15]. The great advantage of using a 3D model in VR (also valid for printed 3D models) is that “the problem is provided in its real 3D extend, the physician is not making assumptions” [12]. This represents a huge advance when compared to the traditional approach, where physicians need to understand and provide a solution to a “3D problem” based only on 2D images (CT, MRI, or US images). This traditional approach depends on the viewer’s mental ability to transfer images into 3D structures [16]. In a VR-based approach, the viewer has the 3D model “in hands,” being able to move, grasp, navigate through it, and in some cases, even touch it (when haptics simulation and equipment are available). Moreover, VR has the potential to enhance the understanding of crucial anatomical details, contributing to the safety and reducing the duration of surgeries [17]. In addition to surgical planning, XR resources have been used in intraoperative scenarios, indi-

66 Fig. 5.4  Image from the virtual reality model evaluating a case of placenta accreta spectrum and placental invasion (a). In this image, the placenta (purple) goes through the uterus and reaches the bladder muscle and mucosa (pink) (b)

5  Virtual Navigation on Expanded Reality Devices (Virtual Reality, Augmented Reality…

a

b

cating their potential as a “next-generation operation-­ supportive tool in terms of spatial awareness, sharing, and simplicity” [18]. Another area where VR is causing a huge impact in medicine is medical education/training of young doctors. VR has the potential to move the learning out of the classroom, focusing on improving competencies, applying learning to practice, and learning from mistakes. In other words, VR may help emphasizing autonomous and blended learning, which are crucial to the learners nowadays [19, 20]. In this area, more general 3D models can be used, without the necessity of being high-fidelity

3D models of a patient exam. VR in medical education can be used for at least two main purposes: 1. To explore the 3D virtual models of the organs 2. To train skills The first use of VR in medical education is related to the exploration of organs using virtual navigation, i.e., “flying” around, behind, or even inside them. In this sense, VR has the potential to be a revolutionary didactic and experiential ­educational tool, “allowing a deeper understanding of the interrelationship of anatomical struc-

5  Virtual Navigation on Expanded Reality Devices (Virtual Reality, Augmented Reality…

tures that cannot be achieved by any other means, including cadaveric dissection” [11]. The second use of VR in medical education is to train the skills in performing different tasks, mainly surgeries. Typical examples are laparoscopic [21] and orthopaedical surgery simulators [22] used for psychomotor skills training. In this kind of simulator, tactile and sensory feedback are essential, in addition to visualization and the capacity to simulate the physic-dynamic characteristics of organs [9]. An advantage of this kind of simulator is that they can provide varying degrees of difficulty to the surgical trainee and objective assessment resources to measure multiple aspects of a trainee’s psychomotor performance skills [23]. Moreover, the trainee’s performance can be recorded, compared, and analyzed. Another important characteristic of VR simulations is that they are repeatable, allowing “learners to make mistakes safely and then learn through deliberate practice to improve performance” [19]. It is important to highlight, however, that VR simulations are not able to cover all possible areas of medical education. With current technologies, VR is not suitable for training abdominal palpation or cannulation, for instance [19]. Conversely, a creative use of VR may create opportunities to teach aspects currently not included in medicine courses. A good example is to teach empathy with a specific kind of patient, such as the elderly [24]. In this example, age-­ related conditions such as macular degeneration and high-frequency hearing loss are simulated to students, so that they can experience the patient’s perspective. Communication with patients and/or their families has always been an issue in medicine. Patients (or their families) are usually debriefed by their healthcare provider before and after any medical procedure or surgery to discuss the findings, the treatment, the next steps, and so on. However, in many cases, the nature of the presented information is complex for patients without any medical or scientific background. In this context, VR models can help patients visualize and better understand their disease, the surgery procedures, the treatment, among other aspects.

67

Therefore, VR can be a valuable way to “learn about and share complicated medical information” [25]. This can result in a positive impact on the treatment adherence, the patient satisfaction, and the quality of the physician-patient relationship. In the area of psychiatry, there are several works using VR for treatment of disorders, such as phobias, PTSD, and anxiety disorders. It can also be used in cases of pregnancy. VR has been used, mainly, to provide through computer-­ generated environments a systematic exposure to anxiety-causing stimuli [26]. The effectiveness of use of virtual environments in the treatment of anxiety disorders has been associated with the level of the feeling of presence reported by users of these virtual environments. Many studies have showed that both conventional exposure treatments and virtual reality exposure therapy (VRET) have similar results, and earnings were maintained in the period 6 months of follow-up after the end of treatment. Additionally, VRET offers advantages in situations where the costs of exposure patients to the actual stimuli needed for treatment are very high, and in cases where the phobia is so intense that it is impossible for patients to experience real situations. VR technologies are also being used for the rehabilitation of stroke, Parkinson’s disease, and patients with motor deficiencies, showing positive results [27]. VR has the potential to increase the involvement of the patient and makes them more motivated to exercise regularly [6]. Due to its capacity of assessment and recording, VR can be also an important tool for tele-rehabilitation, i.e., the rehabilitation made at the patient’s site, without the presence of a therapist [28]. Taking advantage of the fact that immersive VR experience can successfully capture much of the user’s conscious attention, it has been used for pain management. VR immersive experiences are an intense form of distraction during brief painful procedures, particularly well-adapted for use in children [29]. The spectrum of VR uses in medicine is constantly widening. It is possible to include ­applications such as robotics surgery, recovery from addiction to substances, fitness, combating

68

5  Virtual Navigation on Expanded Reality Devices (Virtual Reality, Augmented Reality…

memory loss, cancer screening, telemedicine, among others [15]. If AR is included in this equation, the possibilities exponentially grow, especially considering its use in operation room and for training. As happens with all newly developed fields, the application of VR in medicine is currently an avenue of research opportunities, and the effect of VR related technologies in the future of medicine is still unknown. At the moment, it is clear that XR has the potential to shape the future of many areas. Particularly in medicine, it offers exciting opportunities in every healthcare areas, and the success of such applications depends on the creativity of parties, and capacity of multidisciplinary work, joining technologists and healthcare professionals.

References 1. Christensen H.  Why now is the time to start your virtual reality business. Entrepreneur. 2021. https:// www.entrepreneur.com/article/369414. 2. Burdea G, Coiffet P.  Virtual reality technology. 2nd ed. Wiley-Interscience; 2003. 3. Raposo A, Santos I, Soares L, Wagner G, Corseuil E, Gattass M.  Environ: integrating VR and CAD in engineering projects. IEEE Comp Graph Appl. 2009. (ISSN 0272-1716).;29(6):91–5. https://doi. org/10.1109/MCG.2009.118. 4. Hudson S, Matson-Barkat S, Pallamin N, Jegou G.  With or without you? Interaction and immersion in a virtual reality experience. J Bus Res. 2019;100:459–68. 5. Nilsson NC, Nordahl R, Serafin S. Immersion revisited: a review of existing definitions of immersion and their relation to different theories of presence. Hum Technol. 2016;12(2):108. 6. Adams E, Rollings A. Fundamentals of game design. Upper Saddle River, NJ: Prentice Hall; 2006. 7. Furht B, editor. Handbook of augmented reality. Springer; 2011. 8. Milgram P, Kishino F.  A taxonomy of mixed reality visual displays. IEICE Trans Inform Syst. 1994;E77-D(12):1321–9. 9. Graur F. Virtual reality in medicine — going beyond the limits. In: Lanyi CS, editor. The thousand faces of virtual reality. IntechOpen; 2014. https://doi. org/10.5772/59277. Available from: https://www.intechopen.com/chapters/47752. 10. Mazurek J, Kiper P, Cieślik B, Rutkowski S, Mehlich K, Turolla A, Szczepańska-Gieracha A. Virtual reality in medicine: a brief overview and future research

directions. Hum Mov. 2019;20(3):16–22. https://doi. org/10.5114/hm.2019.83529. 11. Pensieri C, Maddalena P. Overview: virtual reality in medicine. J Virtual Worlds Res. 2014;7:1. 12. Tsioukas V, Karolos I-A, Tsoulfas G, Suri JS, Pikridas C. Chapter 2 - The long and winding road from CT and MRI images to 3D models. In: Tsoulfas G, Bangeas PI, Suri JS, editors. 3D printing: applications in medicine and surgery. Elsevier; 2020. p. 7–20. 13. Motta F, Hurtado J, Radetic D, Raposo A.  A semi-­ automatic technique for fetus segmentation in 3D ultrasound exams. In: Proceedings of the 2019 8th international conference on computing and pattern recognition (ICCPR’19). New  York, NY: Assoc Comput Mach; 2019. p.  179–86. https://doi. org/10.1145/3373509.3373561. 14. Pal NR, Pal SK.  A review on image segmentation techniques. Pattern Recogn. 1993;26(9):1277–94. 15. Thomas L. Applications of virtual reality in medicine. News-Medical. 2021. Retrieved on Nov 08, 2021 from https://www.news-­medical.net/health/Applications-­ of-­Virtual-­Reality-­in-­Medicine.aspx. 16. Boedecker C, Huettl F, Saalfeld P, Paschold M, Kneist W, Baumgart J, Preim B, Hansen C, Lang H, Huber T.  Using virtual 3D-models in surgical planning: workflow of an immersive virtual reality application in liver surgery. Langenbeck’s Arch Surg. 2021;406(3):911–5. 17. Guerriero L, Quero G, et  al. Virtual reality exploration and planning for precision colorectal surgery. Dis Colon Rectum. 2018;61(6):719–23. 18. Saito Y, Sugimoto M, et  al. Intraoperative 3D hologram support with mixed reality techniques in liver surgery. Ann Surg. 2020;271(1):e4–7. 19. Pottle J. Virtual reality and the transformation of medical education. Fut Healthc J. 2019;6(3):181–5. 20. Silva JN, Southworth M, Raptis C, Silva J. Emerging applications of virtual reality in cardiovascular medicine. JACC Basic Transl Sci. 2018;3(3):420–30. 21. Huber T, Paschold M, Hansen C, Wunderling T, Lang H, Kneist W.  New dimensions in surgical training: immersive virtual reality laparoscopic simulation exhilarates surgical staff. Surg Endosc. 2017;31(11):4472–7. 22. Ruikar DD, Hegadi RS, Santosh KC.  A systematic review on orthopedic simulators for psycho-motor skill and surgical procedure training. J Med Syst. 2018;42(9):1–21. 23. Li L, Yu F, Shi D, Shi J, Tian Z, Yang J, Wang X, Jiang Q. Application of virtual reality technology in clinical medicine. Am J Transl Res. 2017;9(9):3867–80. 24. Dyer E, Swartzlander BJ, Gugliucci MR. Using virtual reality in medical education to teach empathy. J Med Lib Assoc JMLA. 2018;106(4):498–500. 25. Palanica A, Docktor MJ, Lee A, et al. Using mobile virtual reality to enhance medical comprehension and satisfaction in patients and their families. Perspect Med Educ. 2019;8:123–7. 26. Tortella-Feliu M, Botella C, Llabrés J, Bretón-López JM, del Amo AR, Baños RM, Gelabert JM.  Virtual

References

69

reality versus computer-aided exposure treatments for interactive machine learning approach. In: Duffy fear of flying. Behav Modif. 2011;35(1):3–30. https:// V, editor. Digital human modeling and applications doi.org/10.1177/0145445510390801. in health, safety, ergonomics and risk management. 27. Kiper P, Turolla A, Piron L, Agostini M, Baba A, Posture, Motion and Health. HCII 2020. Lecture Notes Rossi S, et al. Virtual reality for stroke rehabilitation: in Computer Science, vol. 12198. Cham: Springer; assessment, training and the effect of virtual therapy. 2020. https://doi.org/10.1007/978-­3-­030-­49904-­4-­25. Med Rehabil. 2010;14(2):23–32. 29. Sharar SR, Miller W, Teeley A, et al. Applications of 28. Palomares-Pecho JM, Silva-Calpa GFM, Sierra-­ virtual reality for pain management in burn-injured Franco CA, Raposo A.  End-user programming patients. Expert Rev Neurother. 2008;8(11):1667–74. architecture for physical movement assessment: an

6

Artificial Intelligence Techniques for Fetal Medicine

Artificial intelligence (AI) is the field within computer science broadly dealing with the automation of tasks achieved through human intelligence. Data-driven AI consists of algorithms that can improve their performance by analyzing vast databases of examples and distilling optimal behavior from them. This approach, sometimes called machine learning (ML), has been immensely successful in a wide array of tasks, from language translation to image captioning and segmentation. This chapter discusses the potential impacts of AI methods on fetal medicine. We first provide a brief overview of AI and discuss its applications. Subsequently, we highlight some challenges to the practical deployment of AI methods. We then consider a concrete example from fetal medicine: how to estimate amniotic fluid volume from magnetic resonance imaging (MRI) scans. We show that recent ideas in ML and statistics allow not only for accurate and fast estimation of the volume, but also valid predictive intervals on these estimates. This combination of precision, speed, and proper uncertainty quantification is crucial to enable widespread adoption of AI tools in fetal medicine. The chapter finishes with some concluding thoughts.

With Contributions by Paulo Orenstein and Roberto Imbuzeiro

6.1 The Present State of AI AI emerged as a scientific discipline in the 1950s. At the time, the prevalent approach to the topic involved programming concepts such as symbolic manipulations, which required laborious tinkering by computer scientists. By contrast, the current wave of AI research is powered by methods that are primarily trained from data, with relatively little human intervention. Of these newer algorithms, artificial neural networks have gained particular attention. Recent increases in computing power and the broad availability of data allow for the training of larger and more complex neural networks, with outstanding results in various tasks such as automatic captioning, self-driving cars, and image recognition and segmentation. A reader with no previous knowledge of neural networks, but with some statistical background, may gain some insight from the following comparison. Traditional statistical methods (such as linear regression) consist of choosing a few parameters to best model the data. These parameters quantify the relationship between a measurement to be predicted (e.g., the propensity to develop diabetes) and the available predictive features (e.g., a patient’s current glucose levels, weight, and number of relatives that developed diabetes). Neural networks and other ML methods also fit parameters to data, but they are much more flexible in how they do it. For example,

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 H. Werner et al., 3D Physical and Virtual Models in Fetal Medicine, https://doi.org/10.1007/978-3-031-14855-2_6

71

72

6  Artificial Intelligence Techniques for Fetal Medicine

while typical linear regression uses a few dozen parameters to model the features linearly, neural networks can easily adjust millions of parameters in a highly non-linear fit. This means they are very successful at predicting hard patterns from data, some of which can elude even humans. In that regard, the role of engineers and scientists applying such methods often involves trying different configurations of neural networks (their so-called architectures) to help these algorithms quickly find optimal predictive arrangements. This task is greatly aided by the availability of specialized computer packages, such as TensorFlow [1] or PyTorch [2]. While modern ML methods have achieved many success stories, they also entail complications. First, they require large amounts of data and processing power to properly learn how to make good predictions. Moreover, because they try to replicate complex relationships by design, interpreting these intricate models can be a daunting task. Generally, a practitioner is left with no clue as to how the input features in the model are weighted to form a final prediction (though there are tools available, e.g., [3, 4]). In spite of these important caveats, ML methods can be excellent aids for medical decision-making and for highlighting tangled patterns in patient data.

specific object lies in an image. The other one is image classification, which consists of assigning an image to one of a few classes. For example, AI tools can aid a fetal radiologist in locating amniotic fluid and the fetal brain in ultrasound scans (segmentation) and classify the scan as suggestive of healthy gestation or of various malformations (classification). Recent work in fetal medicine presents some promising results in this direction [9]. Moreover, related problems have been addressed in other medical contexts, including detection of Alzheimer’s disease from PET [10]; polyp detection during colonoscopies [11]; and the analysis of screening mammograms [12, 13].

6.2 AI in Fetal Medicine The potential of machine learning methods in medicine has been studied at length [5, 6]. In particular, AI algorithms can sometimes achieve better diagnostic accuracy than human doctors [7]. Most promisingly, AI may be incorporated into a doctor’s workflow as a second interpretation or consultation. Commercial AI-based clinical tools are already available [8], and there is an increasing number of publications now devoted to machine learning applications in the health sciences. Recent ML-based advances in image processing are especially relevant to fetal medicine. There are at least two important tasks that can be approached with AI methods. One consists of image segmentation, or simply detecting where a

6.3 Challenges to AI Approaches in Fetal Medicine and General Evaluation Criteria As noted, AI-based methods hold great promise for medical imaging in general, and for fetal medicine in particular. However, there are still missing steps on the way to the widespread adoption of such methods. For instance, [14] discusses the following points: 1. Papers on AI in medicine use widely differing evaluation methodologies, which makes it difficult to compare methods. Additionally, due to privacy concerns, there are few publicly available curated datasets over which different research groups can compare their approaches. 2. Algorithmic and dataset representativity biases may lead to methods that perform poorly with some segments of the patient population. The distribution of patients also tends to change over time. 3. Traditional AI performance metrics do not necessarily reflect clinical performance and are usually computed on retrospectively acquired tests. It is important to test such methods prospectively in clinical contexts, preferably through randomized clinical trials.

6.4  Example: Amniotic Fluid Segmentation

An additional but crucial challenge that can be addressed even in preliminary studies is uncertainty quantification. To understand this issue, note that there are many sources of variation and uncertainty in medical imaging, including image noise, motion artifacts, blurring, variability in the patient population, and examiner decisions. Even when faced with such uncertainty, a typical AI method attempts to make a single prediction. For instance, it may try to provide its best estimate of the size of a fetus’ brain in an image, or which malformations it exhibits. However, this is generally not sufficient because it does not quantify how sure the algorithm is of its own predictions. Uncertainty quantification would mean having the algorithm present a range of possible sizes for the brain in the first example along with their respective probabilities, while in the second example it would entail providing not only the most likely malformations present but also indicating the likelihood of each malformation. In recent years, statisticians have developed methods for uncertainty quantification that can be applied to any prediction method from ML, spurring a field called conformal prediction; see [15] for a textbook treatment. We illustrate these ideas below within the context of our work on amniotic fluid segmentation and volume estimation.

6.4 Example: Amniotic Fluid Segmentation The segmentation of amniotic fluid (AF) using MRI scans is prototypical of a class of medical imaging problems that can be laborious for trained humans but effectively performed by neural networks in seconds. Still, a key challenge for their practical implementation is guaranteeing high-confidence, informative bounds on the performance of different architectures. Our work [16] is a first step in this direction. Abnormal AF volumes are linked to various pregnancy complications and negative outcomes [17–19]. Polyhydramnios, or excessive AF volume, occurs in up to 2% of pregnancies and corresponds to up to a fivefold increase in perinatal

73

morbidity and mortality. Conversely, reduced AF volume, or oligohydramnios, occurs in up to 12% of pregnancies and corresponds to up to a 50-fold increase in perinatal morbidity and mortality. Thus, accurate prediction of AF volume is crucial to ensure a healthy pregnancy. While ultrasound exams are the most common screening method for pregnancies, MRI scans are used to obtain higher-quality information about fetal abnormalities. In our work, we considered the task of segmenting AF in MRI images and quantifying the uncertainty in this task, both in terms of the volume estimate and the actual shape of the AF. Though there has been work on automatically segmenting fetal medicine images [20], developing subsequent uncertainty quantification bounds has not received much attention. Providing confidence sets for ML methods allows doctors to better control and interpret the resulting segmentation masks and volume estimates, detect important anomalies and image artifacts, such as cysts and motion blurs, and be automatically alerted to abnormal levels of AF up to a doctor-specified level of uncertainty. Our work consists of the following contributions. As a preliminary step toward network training and evaluation, we curated a dataset of 680 fetal MRI exams, upon which we are still expanding and will make public. Secondly, we developed and evaluated methods based on different neural network architectures for segmenting AF from MRI exams. The best-performing architecture was the U-Net [21], which was developed for image segmentation in other medical settings. See Fig.  6.1 for a comparison between the neural network’s segmentation and that of a trained medical specialist. Lastly, we analyzed the uncertainty in our U-Net neural network predictions via conformal prediction (CP) [22–24]. CP requires a user-­ specified probability level and produces as output a predictive region that contains the true object being estimated with a probability that is at least as large as the specified level. One may notice that the output of a CP method is closely related to the well-known notion of a confidence interval. However, the latter is used to estimate model parameters, whereas CP tries to prospectively

74

6  Artificial Intelligence Techniques for Fetal Medicine

Fig. 6.1  The region correctly segmented by the U-Net is in magenta, while blue indicates areas marked by a medical specialist missing from the algorithm’s segmentation and red indicates excessive segmentation

quantify the uncertainty in the predictions of a ML method. In our setting, we applied CP first to estimating the volume of AF in each test, and then to predicting actual AF shapes (i.e., the geometric region occupied by AF in the images) with high confidence. This produces intervals and images that can be easily interpreted, such as Fig.  6.2. See [16] for details. Our work illustrates two points that should be important for further progress in AI for fetal med-

icine. The first one of these is widely appreciated: general tools from the machine learning literature, like U-Nets, are of great use in medical contexts. The second point is less commonly discussed: it is both desirable and possible to quantify the uncertainty of the predictions of an AI method, empowering doctors to specify to what degree they want to rely on ML predictions and assess other likely predictions. We envision that this latter point will play a crucial role in the deployment of ML methods in clinical practice.

References

75

Fig. 6.2  Lower and upper segmentation masks (for a confidence level of 90%). Note the segmentation on the left is contained by the one on the right. Magenta indicates the region correctly segmented by the mask, while blue denotes missing segmentation and red indicates excess segmentation

6.5 Conclusion The application of AI in fetal medicine holds great promise, but also faces important obstacles. The medical and ML communities must allow for public datasets and baseline tasks for benchmark evaluation; newer ML methods must address inherent biases in medical datasets; and further studies are important in order to understand what issues occur in applying well-established ML algorithms in practice. Beyond these, we stress the importance of quantifying the uncertainty of different ML methods as a way to give doctors more control and confidence over future AI-based clinical tools. In spite of these difficulties, we are enthusiastic and hopeful about the enormous impact that machine learning methods could have

on the workflow of radiologists and doctors, leading to diagnostics that are faster, more objective, and more accurate.

References 1. Abadi M, et al. Tensorflow: a system for large-scale machine learning. In: 12th {USENIX} symposium on operating systems design and implementation ({OSDI} 16). 2016. 2. Paszke A, et al. Automatic differentiation in pytorch. 2017. 3. Lundberg SM, Lee S-I. A unified approach to interpreting model predictions. In: Proceedings of the 31st international conference on neural information processing systems. 2017. 4. Olah C, et al. The building blocks of interpretability. Distill. 2018;3:3.

76

6  Artificial Intelligence Techniques for Fetal Medicine

5. Esteva A, et al. A guide to deep learning in healthcare. Nat Med. 2019;25(1):24–9. 6. Topol EJ.  High-performance medicine: the convergence of human and artificial intelligence. Nat Med. 2019;25(1):44–56. 7. Hwang EJ, et  al. Development and validation of a deep learning–based automated detection algorithm for major thoracic diseases on chest radiographs. JAMA Netw Open. 2019;2(3):e191095. 8. Mucha T, et  al. Commercial adoption of AI in the healthcare sector: an exploratory analysis of S&P 500 companies. MIE. 2020. 9. Kojita Y, et  al. Deep learning model for predicting gestational age after the first trimester using fetal MRI. Eur Radiol. 2021;31(6):3775–82. 10. Ding Y, et al. A deep learning model to predict a diagnosis of Alzheimer disease by using 18F-FDG PET of the brain. Radiology. 2019;290(2):456–64. 11. Wang P, et  al. Development and validation of a deep-learning algorithm for the detection of ­polyps during colonoscopy. Nat Biomed Engineer. 2018;2(10):741–8. 12. Paul HY, et  al. DeepCAT: deep computer-aided triage of screening mammography. J Digit Imaging. 2021;34(1):27–35. 13. Schaffter T, et  al. Evaluation of combined artificial intelligence and radiologist assessment to interpret screening mammograms. JAMA Netw Open. 2020;3:3. 14. Kelly CJ, et  al. Key challenges for delivering clinical impact with artificial intelligence. BMC Med. 2019;17(1):1–9. 15. Balasubramanian V, Ho S-S, Vovk V, editors. Conformal prediction for reliable machine learning: theory, adaptations and applications. Newnes. 2014.

16. Csillag D, et  al. Uncertainty quantification for amniotic fluid segmentation and volume prediction. 2021. https://www.cse.cuhk.edu.hk/~qdou/ p u b l i c / I M L H 2 0 2 1 _ fi l e s / 5 0 _ C a m e r a R e a d y _ Uncertainty_Quantification_for_Amniotic_Fluid_ Segmentation_and_Volume_Prediction.pdf. 17. Chamberlain PF, et al. Ultrasound evaluation of amniotic fluid volume: I. The relationship of marginal and decreased amniotic fluid volumes to perinatal outcome. Am J Obstet Gynecol. 1984;150(3):245–9. 18. Moore TR.  The role of amniotic fluid assessment in evaluating fetal Well-being. Clin Perinatol. 2011;38(1):33–46. 19. Moore TR, Cayle JE.  The amniotic fluid index in normal human pregnancy. Am J Obstet Gynecol. 1990;162(5):1168–73. 20. Looney P, et al. Fully automated 3-D ultrasound segmentation of the placenta, amniotic fluid, and fetus for early pregnancy assessment. IEEE Trans Ultrason Ferroelectr Freq Control. 2021;68(6):2038–47. 21. Ronneberger O, Fischer P, Brox T.  U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Cham: Springer; 2015. 22. Bates S, et  al. Distribution-free, risk-controlling prediction sets. arXiv preprint arXiv:2101.02703. 2021. 23. Lei J, et al. Distribution-free predictive inference for regression. J Am Stat Assoc. 2018;113:523. 24. Shafer G, Vovk V. A tutorial on conformal prediction. J Mach Learn Res. 2008;9:3.

7

Metaverse in Fetal Medicine

The term “Metaverse” was created in 1992 by Neal Stephenson for his science fiction novel Snow Crash [1] to designate a three-dimensional virtual space in parallel to the real world. It was a “meta-universe,” imagined as the virtual reality successor of the internet. Since its inception the Metaverse has been associated with applications using virtual and augmented reality technologies, as well as virtual worlds social platforms like Second Life [2]. Consequently, these initiatives motivated a general perception of a trend toward an integration between virtual and physical spaces with virtual economies that ultimately would involve the entire industry. The scenario described above, accelerated by global transformations due to the COVID-19 pandemic, influenced the five major Web companies: Google, Apple, Facebook, Amazon, and Microsoft, to proactively promote the Metaverse as their “vision of the future.” For example, in 2021 Facebook changed the company name to Meta. Given these facts, it is reasonable to imagine that the “metaverse wave” will gradually reach all sectors of modern society, including fetal medicine.

With Contributions by Luiz Velho Supplementary Information The online version contains supplementary material available at https://doi. org/10.1007/978-­3-­031-­14855-­2_7.

This chapter gives an account of the conceptual and technological aspects of the Metaverse that informs a prospective view of the influence of gradual development of these innovations in the future of medicine. Here we consider Metaverse in a very broad sense, encompassing all innovations and technologies related to new media, communications, and computing, such as IoT (internet of things), 5G, cryptocurrencies, and machine learning. In fact, we will not attempt to arrive at a precise definition of Metaverse since we believe it would be misleading given the diversity of points of view of such a phenomenon “in-process.” Nonetheless, it is enlightening to contrast virtual and augmented reality with the notion of Metaverse. While the former (VR/AR) can be considered enabling media technologies, the latter (Metaverse) is a general application context in which they operate, together with many other technological innovations.

7.1 From Virtuality Continuum to Internet 4.0 As discussed above, we can think of the Metaverse as a network of 3D virtual worlds focused on social connection, which is related to the real economy and physical spaces. In that sense, the concept of “virtuality continuum” is instrumental.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 H. Werner et al., 3D Physical and Virtual Models in Fetal Medicine, https://doi.org/10.1007/978-3-031-14855-2_7

77

78

The virtuality continuum is a continuous scale ranging between the completely virtual (i.e., virtuality) and the completely real (i.e., reality). It was first introduced by Paul Milgram [3] and opens possibilities for variations and compositions of real and virtual objects (Fig. 7.1). The area between the two extremes, where both the real and the virtual are mixed, is called mixed reality. This in turn is said to consist of both augmented reality, where the virtual augments the real, and augmented virtuality, where the real augments the virtual. Fig. 7.1  The virtuality continuum. (From [3])

Fig. 7.2 Metaverse scenarios. (Adapted from [4])

7  Metaverse in Fetal Medicine

The concept has been described in the context of new media and computer science, but in fact it could also be considered a matter of anthropology in the Metaverse scenario. Besides the virtuality continuum it is enlightening to take into account two other associated continua that also play a role in the unfolding of the Metaverse: the spectrum of technologies ranging from augmentation to simulation and the spectrum of applications ranging from personal to social. The interplay of these two axes is depicted in Fig. 7.2.

7.1  From Virtuality Continuum to Internet 4.0

The augmentation-simulation axis is akin to AR-VR technologies where augmentation creates a layer onto our perception of the physical environment and simulation models a parallel reality of entire digital environments. The personal-social axis is linked to a range of applications that focus either on individual users or the world pertaining to a group of users. The combination of these two axes gives rise to four key components of the Metaverse landscape: Virtual Worlds, Mirror Worlds, Augmented Reality, and Life-logging. To complete the panorama, there are techniques that enable and link these components, such as: networks and interfaces; modeling and immersion; sensors and IoT; interaction and identity. This is the scenario proposed by John Smart in the “Metaverse Roadmap: Pathways to the 3D Web” [4]. Finally, to understand the Metaverse as a complete global context for applications is mister to analyze its relationships with the whole society at different levels. Jon Radoff, in “Measuring the Metaverse” [5], proposes a hierarchy of 7 layers where he moves up the value chain from infrastructure at the bottom to experience at the top, stopping at human interface, decentralization,

Fig. 7.3  The 7 layers of the metaverse. (From [5])

79

spatial computing, creator economy, and discovery along the way. Such conceptual framework employs a standard methodology of computer systems that provides a concrete characterization of the Metaverse as part of our culture, society, and information industry. In summary, as shown in Fig. 7.3, these layers represent: • Experience—what people engage with, e.g., games, shows, conferences, etc.; • Discovery—how people find out about an experience; • Creator Economy—content market for the things in the metaverse; • Spatial Computing—platforms for 3D environments and interaction; • Decentralization—democratized distribution ecosystem; • Human Interface—means of access to the metaverse: mobile, headsets, etc.; • Infrastructure—cloud computing; telecommunication networks. In a way, the picture rendered above gives a prospective view the of what the future Web promises with the Internet 4.0.

80

7.2 Expanded Reality for Mediatic Shared Experiences As we have seen in the previous section, at the top of the Metaverse pyramid lies “the experience.” Arguably, from the user’s point of view, this is the most important element because it is how people relate to the applications context. It is important to highlight what is novel and unique about metaverse experiences. They are essentially “Mediatic Shared Experiences” within an “Expanded Reality.” This scenario involves: mediation, communication, and intelligence. In such experiences the users share content and communicate with each other through an intelligent medium. In this setting we have three levels of abstraction: (1) concrete—the metaverse technology; (2) perceptual—the psychophysiological fruition; (3) symbolic—the involvement with content and activity (Fig. 7.4). Fig. 7.4 Mediatic shared experience

Fig. 7.5 Data/model framework

7  Metaverse in Fetal Medicine

The mechanisms behind a metaverse experience belong to the field of study of “Computational Applied Mathematics for Media.” This emerging branch of science deals with mathematical models that arise from data sources and are experienced by users within an application. These three entities are mediated by computational processes that enable the fulfillment of the experience. As can be seen in Fig. 7.5, the users’ interface with data though sensing and display processes (note that this involves perceptual aspects, and this same mechanism allows users to interface with each other through data mediation). The data can be processed in different ways as a signal according to application requirements. Further, it can also go through a process of analysis to create a mathematical model that can be used for simulations, as well as in a synthesis process to generate new data [6, 7]. The data/model framework is based on many traditional disciplines of science and engineer-

7.3  Perspectives for Fetal Medicine and Experiments

ing. Recently, it has been enhanced by artificial intelligence and machine learning techniques.

7.3 Perspectives for Fetal Medicine and Experiments Medicine is perhaps a field of human activity that mostly benefits from technological and scientific advances. One example is the area of fetal medical imaging that relies on sophisticated sensing devices, such as computer tomography (CT), magnetic resonance (MRI), and ultrasound for diagnostics of patients, as well as, for procedure support. Another example is the early adoption of artificial intelligence with rule-based expert systems [8]. Overall, we can predict a great impact of the Metaverse in the future of medicine, both in health and wellness [9, 10]. These developments will likely contribute equally to research, practice, and education in fetal medicine. To demonstrate the potential of Metaverse developments in the area of fetal medicine, we will show examples of some experiments for education in fetal medicine. The experiments have been developed in the Spatial.io platform

Fig. 7.7  3D reconstruction of a fetus

81

[11] that offers support for a wide range of VR, AR, and mobile devices. The first experiment is a remote lecture in which the instructor presents a case to participants joining in VR from different locations. The group is immersed in a virtual classroom and have as study material the digital medical data generated by various processes. In Fig. 7.6, the participants view the result of fetal MRI that can be manipulated interactively to navigate through the 2D slices of the 3D data. In Fig. 7.7, the instructor shows a 3D reconstruction of a real fetus to the class. Note that the virtual object can be freely inspected spatially

Fig. 7.6  Fetal magnetic resonance

7  Metaverse in Fetal Medicine

82

The second experiment demonstrates the possibility to have a shared meeting virtually inside the 3D structure of a fallopian tube which was micro-CT scanned in high resolution [12] (Fig. 7.9).

References

Fig. 7.8  Segmentation of the fetus organs

Fig. 7.9  3D structure of a fallopian tube

with 6 degrees of freedom (translation and rotation) plus scaling. In Fig. 7.8, the instructor presents a 3D virtual model with a segmentation of the fetus organs to discuss various anatomical aspects.

1. Stephenson N. Snow crash. Rizzoli, editors. 2007. 2. Wikipedia. Second life. 2003. https://en.wikipedia. org/wiki/Second_Life. 3. Milgram P, Takemura H, Utsumi A, Kishino F.  Augmented reality: a class of displays on the reality-virtuality continuum. Proceedings of SPIE  the international society for optical engineering, vol. 2351. 1994. 4. Smart J, Cascio J, Paffendorf J. Metaverse roadmap: pathways to the 3D web. Academia Letters; 2007. 5. Radoff J.  The metaverse value-chain. In Building the metaverse. 2021. https://medium.com/ building-­the-­metaverse 6. Gomes J, Velho L.  Abstraction paradigms for computer graphics. Vis Comput. 1995;11(5):227–39. 7. Gomes J, Costa B, Darsa L, Velho L.  Graphical objects. Vis Comput. 1996;12(6):269–82. 8. Xing L, Giger ML, Min JK. Artificial intelligence in medicine. Academic Press; 2021. 9. McLuhan M.  Understanding media. 2nd ed. Routledge; 2001. 10. Yang D. Expert consensus on the metaverse in medicine. 2021. 11. Spatial.io. URL: https://spatial.io/. 12. Werner H, Ribeiro G, Arcoverde V, Lopes J, Velho L. The use of metaverse in fetal medicine and gynecology. European Journal of Radiology; 2022. 150: 110241. https://doi.org/10.1016/j.ejrad.2022.110241.

Part IV Applicability in Clinical Cases

The objective of this chapter is to describe virtual and physical models of the main fetal anomalies diagnosed by ultrasound (US), magnetic resonance imaging (MRI) and computed tomography (CT). This chapter also describe virtual and physical models assisting fetal surgeries, postnatal surgery, multiple pregnancy and maternal-fetal attachment in cases of blind women. This chapter intends to show the use of 3D technologies applied on real cases in medicine.

8

Three-Dimensional Printing and Virtual Models in Fetal Medicine

8.1 Study of Fetal Pathologies 8.1.1 First Trimester Ultrasonographic assessment in the first trimester has become part of the prenatal routine, especially since the advent of transvaginal probes, which provide high-quality images, enabling an early assessment of embryonic development [1, 2]. The gestational sac can already be identified in the 5th week, whereas the embryo can be seen as a linear structure in the 6th week. At this stage, cardiac activity has already been detected. In the 7th week, it is possible to see the division of embryo into trunk and head and the limb buds. In the 8th week, rhombencephalon can be clearly observed. In the 9th week, the physiological umbilical hernia can be seen (Fig. 8.1). Many structural anomalies can be diagnosed in the first trimester of pregnancy. A good evaluation of fetal morphology can already be performed effectively by US at the end of the first trimester. In this phase, basic structures can be studied in the cephalic pole, face, neck, chest, spine, abdomen, and limbs (Fig.  8.2). The placenta, umbilical cord, and amniotic fluid can also be assessed. With Contributions by Edward Araujo Júnior, Gabriele Tonni, Pedro Teixeira Castro, Tatiana Fazecas, Renata Nogueira, and Heron Werner

The sensitivity of the diagnosis depends on the type of malformation. Thus, anencephaly, exencephaly, and acrania can be identified in 100% of cases (Fig. 8.3). It can be easily identified when the skullcap is not identified in the US evaluation. Ventricular dilations can also be identified in some rare cases at this stage (Fig. 8.4). Most of the time, they are obstructive in nature. Holoprosencephaly is a forebrain cleavage defect. It can be classified as alobar, semilobar, or lobar (Fig.  8.5). The alobar forms can be quite evident at this stage. A single ventricle and thalamic fusion are seen. Development of the telencephalon into cerebral hemispheres is visible at the 7th week and the diagnosis of alobar holoprosencephaly can be seen at the 8th week. One of the most prevalent anomalies in the first trimester is cystic hygromas. Many syndromes are related to this type of malformation. Defects in closing the abdominal wall can already be noticed at this stage. Three-­ dimensional (3D) reconstructions can easily show an omphalocele or gastroschisis. Skeletal dysplasias are a heterogeneous group of disorders related to bone development and growth (Figs.  8.6 and 8.7). In body stalk syndrome, the reduction defects of an extremity and the kyphoscoliosis can be detected at the 9th week. Polydactyly can be seen in an embryo within 8 weeks and 6 days.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 H. Werner et al., 3D Physical and Virtual Models in Fetal Medicine, https://doi.org/10.1007/978-3-031-14855-2_8

85

86

8  Three-Dimensional Printing and Virtual Models in Fetal Medicine

Fig. 8.1  9-week embryo images and 3D printed model made on three-dimensional printer (powder-based system, Z Corp) obtained from three-dimensional ultrasound

a

b

Fig. 8.2  12-week fetal model made on three-dimensional printer (powder-based system, Z Corp). View with the uterine cavity (a) and the isolated fetus (b)

8.1.2 Second and Third Trimesters US assessment of the second and third trimesters offers an in-depth investigation of fetal formation. In some specific situations, MRI can help US.

8.1.3 Central Nervous System Central nervous system (CNS) malformations play a significant role in all fetal malformations. The diagnosis of a fetal brain anomaly is a criti-

cal stressor for pregnant parents, and effective counseling depends on an accurate prenatal diagnosis [3]. Fetal brain malformations will be briefly discussed below.

8.1.3.1 Ventriculomegaly Ventriculomegaly (VMG) is a commonly seen abnormality on prenatal US. Its causes are multiple. In the absence of an associated abnormality, it is sometimes difficult to establish a prognosis. It is defined as an abnormally increased size (>10 mm) of the lateral cerebral ventricles of the fetus. It can be a variant of normal or a warning

8.1  Study of Fetal Pathologies

87

Fig. 8.3  In this 13-week three-­ dimensional virtual model, we can observe an exencephaly (arrow)

Fig. 8.4  Ventricular dilatation in a 13-week fetus from transvaginal ultrasound. The choroid plexuses are clearly visible (arrow)

8  Three-Dimensional Printing and Virtual Models in Fetal Medicine

88

a

b

c

d

Fig. 8.5  Fetus with alobar holoprosencephaly with a proboscis at 11 weeks (a) and 26 weeks’ gestation (b, c, d). Virtual and physical model (built in a powder-based system, Z Corp). Note the similarity with the anatomopathological study

sign of an underlying condition. International prevalence data are very heterogeneous. The term VMG is preferable to hydrocephalus, which reflects the progressive distension of the ventricles due to an increase in their cerebrospinal fluid content, resulting in an increase in the cerebral circumference (Fig. 8.8) [4, 5]. The main causes of fetal VMG are aqueduct stenosis, intrauterine infection, and intracerebral hemorrhage. Fetal brain MRI may be offered as an adjunct to US diagnostic, especially in the third trimester of pregnancy, to study the entire ventricular system and gyration if this could not be done by US alone. The main limitation of the MRI is the poor quality of the images as a consequence of artifacts produced by fetus movements.

8.1.3.2 Anencephaly Anencephaly refers to a defect in the closure of the anterior neural tube, with an overall incidence of approximately 1  in 1000 births and a lethal prognosis. Anencephaly is characterized by the absence of the cranial vault, resulting in the expo-

sure of neural tissue (Fig.  8.9). Early diagnosis can be achieved easily with US in the first trimester. MRI can help in cases of multiple gestations where the assessment of the affected fetus is impaired.

8.1.3.3 Holoprosencephaly Holoprosencephaly is a disorder of early origin, observed even during brain organogenesis, and results from a failure of forebrain cleavage. Depending on the degree of severity, it is classified as alobar, semilobar, and lobar with specific imaging findings for each form. The incidence is approximately 1 in 10,000 births, and the prognosis of the alobar and semilobar forms is poor. The development of the face is related to the formation of the brain; hence defects of the face may occur, ranging from mild hypotelorism to cyclopia. US is a very effective method for the diagnosis of alobar and semilobar holoprosencephaly (Fig. 8.10). MRI appears to provide real benefits in characterizing the lobar form and in confirming previous US findings for the alobar and semilobar forms (Fig. 8.11).

8.1  Study of Fetal Pathologies

89

Fig. 8.6  35 weeks fetus. Amputation of the legs, syndactyly in the right hand and ectrodactyly in the left hand. Virtual model from magnetic resonance imaging and computed tomography

Fig. 8.7  Thanatophoric dwarfism (26 weeks). Three-dimensional ultrasound, magnetic resonance imaging (True FISP) and physical model built in a powder-based system. Note the similarity with the anatomopathological study

8  Three-Dimensional Printing and Virtual Models in Fetal Medicine

90

Fig. 8.8  Ventricular dilation in a 37-week fetus with Walker-Warburg Syndrome (*). Three-dimensional reconstruction from magnetic resonance imaging. Note Z-shaped brainstem (arrow) Fig. 8.9 Three-­ dimensional ultrasound and virtual model showing anencephaly (26 weeks) (a). Magnetic resonance imaging of the same case (26 weeks) showing anencephaly (arrow) and the whole fetal body (b)

a

b

8.1  Study of Fetal Pathologies

91

Fig. 8.10  Fetus with alobar holoprosencephaly with a proboscis at 26 weeks’ gestation. Fetal face printed in a filament three-dimensional printer

8.1.3.4 Microcephaly Microcephaly is defined as a significant reduction in head circumference below the fifth percentile or two standard deviations below average (Fig. 8.12). It is usually detected after 24 weeks, being easier to diagnose with advancing gestation due to the contrast between the size of the head and the fetal body [6]. The main causes are autosomal recessive/dominant disorders, infections (cytomegalovirus, toxoplasmosis, rubella, and zika), radiation, drugs, alcohol, and hypoxia [7]. Gyration anomalies such as polymicrogyria, lissencephaly, or simplified gyral patterns may be associated with microcephaly. In such cases, US analysis of the brain is often limited due to the poor acoustic windows resulting from the ossified calvaria and the poor definition of the brain parenchyma, especially when the extra-axial

fluid space is also small. MRI may be helpful in this situation, enabling more detailed examination of brain development and the parenchyma. 3D virtual and physical models from MRI, US, and computed tomography (CT) of the cranium of both fetuses and newborns have been described, showing in detail the degree of microcephaly, allowing the parents and medical team to better understand the anomaly [8–10].

8.1.3.5 Fetal Intracranial Hemorrhage Fetal intracranial hemorrhage (FIH) and ischemic brain injury are rare findings in fetal examinations (Fig.  8.13). The accurate identification and analysis of the extension of the bleeding can lead to important changes in pregnancy management and parental counseling [11–13]. These changes are primarily based on the parental coun-

8  Three-Dimensional Printing and Virtual Models in Fetal Medicine

92

b

a

c

d

Fig. 8.11  Fetus with alobar holoprosencephaly with a proboscis at 26 weeks’ gestation. Virtual model and physical model built in a plastic filament three-dimensional (3D) printer (a). Fetal profile (b). Physical model built in

plastic filament (c) Polydactyly. Three-dimensional ultrasound, magnetic resonance imaging, and 3D printed model (built in a powder-based system, Z Corp) (d)

seling, which is supported by the literature on neonatal outcomes, which are generally related to the intensity of FIH. It can range from a localized bleeding (usually isolated in the germinal matrix at the caudothalamic groove; grade I) to massive bleeding, extending to periventricular hemorrhagic infarction (grade IV). Recently, we have used new MRI sequences to assess FIH, such as T1-weighted vibe imaging sequence in the sagittal plane and T2-weighted true FISP sequences. Using the T1-weighted sequence, the voxels were selected according to the intensity of their signal, producing a 3D graphical expression of different grades of the IH, which could be quantified and numerically related to other anatomical structures. The T2-weighted sequence was used to improve the anatomical relationship between the hemorrhage and other parts of the brain. The virtual 3D model created can enable easy parental counseling.

8.1.3.6 Chiari Malformation Chiari malformations are congenital abnormalities of the hindbrain, usually associated with hydrocephalus. The most frequent are types I and II. Chiari I is the displacement of the cerebellar tonsils into the upper cervical canal, whereas Chiari II is a herniation of the lower cerebellar vermis and the fourth ventricle. Chiari II is found in 65%–100% of severe spina bifida cases (Fig. 8.14). For this reason, when Chiari II malformation is suspected, an investigation of the presence of associated meningomyelocele must be performed [14]. US findings of Chiari II usually allow the diagnosis of this condition, showing a small posterior fossa, hypoplastic cerebellar hemispheres, and ventricular dilatation [15–17]. However, a more detailed analysis of the associated cord malformations and brain parenchyma abnormalities is sometimes difficult with US due to shadowing of the vertebral bodies and calvaria.

8.1  Study of Fetal Pathologies

93

a

b

Fig. 8.12  Zika virus. Three-dimensional sagittal reconstruction from computed tomography scan and corresponding three-dimensional (3D) printing. The skull has collapsed appearance (arrow) (a). Postnatal image obtained from computed tomography and magnetic reso-

nance imaging with 3D reconstruction and printing of a newborn with Zika virus (b). Microcephaly with occipital encephalocele (28 weeks). 3D reconstruction from magnetic resonance imaging with 3D printing in powder-­ based system (c)

94

8  Three-Dimensional Printing and Virtual Models in Fetal Medicine

c

Fig. 8.12 (continued)

In such cases, fetal MRI may help with complete analysis of CNS development. On MRI, Chiari II may present a small posterior fossa with displacement of the inferior vermis, along with a small fourth ventricle, elongated and displaced inferiorly. Dysgenesis of the corpus callosum, polymicrogyria, and hydrocephalus is present in approximately 90% of cases, and spina bifida is found in almost 100% of cases [18–22]. There is another severe form of Chiari malformation, which is type III. This form is rare, consisting of encephalocele with herniation of the posterior fossa contents and sometimes the occipital lobe (cephalocele). The herniated tissue is always abnormal, showing areas of necrosis, gliosis, and fibrosis. Virtual and physical 3D models from a fetus with Chiari II malformation showing all central and peripherical central nervous system abnormalities have been described, as well as 3D reconstructions of images during endoscopic fetal surgery for meningomyelocele repair (Fig. 8.15).

8.1.3.7 Dandy-Walker Malformation Dandy-Walker malformation is a developmental anomaly of the posterior fossa with an incidence of 1  in 25,000–35,000 live births and a small female predominance (Fig. 8.16). It is characterized by complete or partial agenesis of the vermis, cystic dilatation of the fourth ventricle, and enlargement of the posterior fossa, with superior displacement of the tentorium and the torcular Herophili (confluence of venous sinuses) [23]. Due to the late embryological development of the cerebellum, a diagnosis of vermis agenesis or partial agenesis cannot be suggested until the end of the 19th week, when the cerebellar vermis should be fully developed. Associated malformations are frequent and comprise corpus callosum agenesis (25%), corpus callosum lipoma, malformation of the cerebral gyri, holoprosencephaly (25%), cerebellar heterotopia (25%), dysplasia of the cingulated gyrus (25%), and gray matter heterotopia (5–10%). Moreover, there is a high incidence of karyotypic abnormalities, mainly

8.1  Study of Fetal Pathologies

95

a

b

Fig. 8.13  Magnetic resonance imaging (MRI) T1- and T2-weighted sequences are fused. The area of fetal intracranial hemorrhage (FIH) is selected on three planes in different colors according to the intensity of the voxels, creating a three-dimensional (3D) image. The lesion can also be visually compared to other structures, such as intraventricular fluid and the cerebral surface (a). Segmentation of voxels with different intensities also allows for the quantitative analysis of their volumes. The segments were individualized by color, according to the

c

limits of intensity of the MRI signal (threshold). In this case, the FIH was segmented into 5 different intensities, generating the register of the number of voxels and their total volume (b). 3D reconstruction of the FIH. The FIH was segmented into different colors according to the intensity of the MRI signal for each voxel on the T1-weighted sequence. After segmentation, the volume was texturized for better comprehension by the parents for counseling (c)

96

8  Three-Dimensional Printing and Virtual Models in Fetal Medicine

a

b

Fig. 8.14  Chiari II malformation (29  weeks). Three-­ ventriculomegaly (a). 3D virtual reconstruction (external dimensional (3D) virtual model obtained from magnetic and internal view) after MRI. Note meningocele (arrow) resonance imaging (MRI) files shows meningocele and and ventriculomegaly (*) (b)

trisomy 18 and 13, as well as triploidy (up to 40% of cases). MRI may add important information when US analysis of the vermis malformation is

insufficient, enabling more accurate evaluation of the posterior fossa structures and leading to a more specific diagnosis.

8.1  Study of Fetal Pathologies

97

a

b

Fig. 8.15  Chiari II malformation (23 weeks). Physical model built in a powder-based system (a). Myelomeningocele (arrow). Three-dimensional ultrasound in the rendering mode, virtual navigation, and fetoscopy (b)

8.1.3.8 Encephalocele Encephaloceles are neural tube defects resulting in a midline mass overlying a skull defect (Fig.  8.17). This condition is frequently associated with other brain malformations such as hydrocephalus, facial clefts, cardiac abnormalities, and genital malformations. The occipital location is the most frequent (75% of cases), followed by the frontal (15% of cases) and parietal (10%) areas. The lesion may be purely cystic or may present brain tissue incorporated within the mass. When the lesion is purely cystic, the identification of the skull defect is a target for diagnosis. In such cases, fetal MRI may help to identify the defect [24–26]. Recently, our group described a case of parapagus conjoined twins in which one of the fetuses presented a significant occipital enceph-

alocele using both virtual and physical 3D models from MRI scan data. Both 3D models were important for parental counseling as well as for the neonatology team regarding ventilatory support, as it was an anomaly incompatible with postnatal life.

8.1.4 Face 8.1.4.1 Cleft Lip and Palate The lip forms between the fourth and seventh weeks of pregnancy (Fig.  8.18). The face is formed by the fusion of the mesenchymal products known as prominences (frontonasal, mandibular, and jaw expansion). A cleft lip or palate happens when the structures that form the upper lip or palate fail to join together [27–29].

8  Three-Dimensional Printing and Virtual Models in Fetal Medicine

98

a

b

c

d

Fig. 8.16  Three-dimensional (3D) reconstruction from magnetic resonance imaging (MRI) of a 25-week fetus with Walker-Warburg Syndrome. MRI was helpful to evaluate the cerebral and posterior fossa structures. Note lissencephaly (a) and ventricular dilation (b). Vermian

hypoplasia is differential diagnosis with Dandy-Walker syndrome. 3D virtual model (30 weeks) showing enlarged posterior fossa (arrow) (c). Occipital encephalocele with posterior fossa cyst. Virtual 3D reconstruction from MRI at 30 weeks (arrow) (d)

8.1  Study of Fetal Pathologies

99

Fig. 8.17  Magnetic resonance imaging (sagittal T2-weighted) of a 25th week fetus with an occipital encephalocele. Physical model (built in a powder-based system, Z Corp)

8.1.4.2 Tumors (Epignathus Teratoma) Epignathus is a rare teratoma of the oropharynx (Fig. 8.19). These tumors correspond to an intraoral location (most often formed at the expense of the bony palate, sometimes other intraoral structures) [30, 31]. They fill in and then come out of the mouth. They are often pedicled, sometimes with a wide implantation base. Depending on their degree of vascularization, these tumors can have the same complications as teratomas of other sites. 3D ultrasound and MRI allow increased accuracy of the antenatal diagnosis of teratoma epignathus in relation to location and intracranial extension and may also allow better selection of the case requiring ex utero intrapartum treatment (EXIT).

Cervical masses are rare anomalies with high rate of neonatal morbidity and mortality. In most cases, these masses are diagnosed by routine obstetric US, and the findings include fast-growing masses and peripheral vascularization on color Doppler US. MRI can help US in providing a better assessment of mass margins, infiltration, and relationship with adjacent structures [32]. It can precisely determine whether the mass compresses anatomically important structures, such as the trachea and esophagus. The most common oral and cervical masses by order of frequency are teratoma, cystic lymphatic malformation, and hemangioma. Virtual 3D model reconstruction allows a better understanding of the disease and helps in parental counseling (Fig. 8.21).

8.1.5 Cervical Masses

8.1.5.1 Fetal Goiter Fetal goiter is a rare prenatal finding (Fig. 8.22). It results from a hormonal imbalance in the fetus, potentially deleterious for its future psychomotor development [33, 34]. Excessive iodine exposure

The diagnosis of neck masses is essential for prenatal management, delivery planning, perinatal outcomes, and long-term prognosis (Fig.  8.20).

100

8  Three-Dimensional Printing and Virtual Models in Fetal Medicine

a

Fig. 8.18  Fetus with a cleft lip at 28  weeks’ gestation (arrow). Three-dimensional (3D) ultrasound image, 3D virtual model and physical model (built in a power-based system, Z Corp) (a). Fetus with a cleft lip at 30 weeks’ gestation.

3D virtual model by magnetic resonance imaging (MRI) (b). 32-week-old fetus with cleft lip and palate. Virtual model obtained by MRI (c). Virtual model of the same fetus demonstrating the open palate obtained by MRI (d)

8.1  Study of Fetal Pathologies

b

c

Fig. 8.18 (continued)

101

102

d

Fig. 8.18 (continued)

8  Three-Dimensional Printing and Virtual Models in Fetal Medicine

8.1  Study of Fetal Pathologies

103

a

b

c

Fig. 8.19  27-week-old fetus with an epignathus tumor (*). Three-dimensional (3D) reconstruction from ultrasound. Note the protrusion of the tongue (a). 3D reconstruction from magnetic resonance imaging (MRI). Note the good tissue differentiation of the tumor in the virtual

model (b). MRI (sagittal T2-weighted) of a 25-week-old fetus with a large epignathus tumor (arrow) and 3D reconstruction (c). Virtual model from MRI of a 29-week’s fetus shows a heterogeneous cervical mass extending from the oral cavity to the cervical region (*) (d)

104

8  Three-Dimensional Printing and Virtual Models in Fetal Medicine

d

Fig. 8.19 (continued)

Fig. 8.20  Fetal cervical teratoma at 35 weeks of gestation (arrow). Magnetic resonance imaging at sagittal T2-weighted and three-dimensional (3D) virtual model

can cause fetal goiter and thyroid dysfunction [35]. As an essential element for the synthesis of thyroid hormones, iodine crosses the placenta via active transport and concentrates in the thyroid

gland for fetal utilization. 3D US and MRI can be auxiliary methods to the conventional 2D US in the diagnosis of fetal goiter, allowing better monitoring of fetal treatment [36].

8.1  Study of Fetal Pathologies

105

Fig. 8.21  Facial profile in magnetic resonance imaging showing cervical lymphangioma at 27 gestational weeks. Three-dimensional reconstructions of fetal airway and lungs. Note airway narrowing

8.1.5.2 Lymphangioma Lymphangioma is a benign congenital malformation of the lymphatic system that has the potential to infiltrate the surrounding structures (Fig. 8.23) [37]. It constitutes approximately 5–6% of all the benign lesions in childhood and adolescence and occurs most frequently in the head, neck, or axilla. The prognosis depends on the presence of other associated features, such as skin edema, hydrops and polyhydramnios, abnormal karyotype, and location and extent of the lesion. A key concern of 3D reconstruction is to show high-quality images that can be manipulated with 3D software without losing accuracy. These images clearly show the relationship between lymphangioma and soft tissues in the fetal neck in different views. The quality of the images can facilitate a multidisciplinary discussion and also be a good interface for parental explanation. The technique for preparing the virtual model can be applied at the different stages of pregnancy and serve as an innovative contribution to the research on fetal abnormalities.

New advances in 3D reconstructions from MRI scan data allow high-quality images showing the relationship between lymphangioma and soft tissues in the fetal neck in different views. These images may improve parental counseling and the delivery planning by a multidisciplinary team.

8.1.5.3 Teratoma Cervical teratoma is a rare congenital tumor that tends to be large and is usually solid/cystic (Fig. 8.24). It occurs with an incidence of 1 in 20,000–40,000 live births, accounting for about 6% of all fetal teratomas. Airway obstruction in the newborn as determined by tracheal compression or occlusion is reported to be the reason for an 80–100% mortality rate in the neonatal period [38]. 3D virtual navigation in the upper airways of the fetus with cervical teratoma makes it possible to identify the bronchi obstructions, which is essential for the delivery plan including the EXIT procedure [39].

106

8  Three-Dimensional Printing and Virtual Models in Fetal Medicine

a

b

Fig. 8.22  Three-dimensional ultrasound in the rendering mode reconstruction of fetus at 23rd week showing enlarged thyroid gland and magnetic resonance imaging (MRI) at 28 weeks (T1-weighted) showing enlarged thy-

roid gland with hyperintense signal (arrow) (a). Three-­ dimensional (3D) fetal MRI reconstruction demonstrating cervical teratoma (29 weeks), making differential diagnosis with enlarged thyroid (arrow) (b)

8.1  Study of Fetal Pathologies

107

a

b

Fig. 8.23  Lymphangioma 35  weeks of gestation. Fetal magnetic resonance imaging (MRI), sagittal, coronal, and axial T2-weigthed (a). Three-dimensional (3D) virtual

model from MRI. Lymphangioma (*). Note the permeability of the airway path (arrow) (b)

108

8  Three-Dimensional Printing and Virtual Models in Fetal Medicine

a

b

Fig. 8.24  Three-dimensional (3D) reconstruction (virtual and physical model) showing cervical teratoma (*) at 37 gestational weeks (a). Fetal cervical teratoma at 37  weeks of gestation (*) (b). Note the patency of fetal airways (arrow) (c). 3D printed model (built in a color Polyjet 3D printing system) of the same case showing the teratoma (*) and patency of fetal airways (d). Fetal cervi-

cal teratoma at 34 weeks of gestation (*). Virtual model showing whole fetus and tumor (*) (e) and highlighting the tumor (*) and airways (f). The use of virtual reality to visualize the cervical tumor (g). 3D view of airway path and the image from virtual bronchoscopy, demonstrating patency of the airway (h)

8.1  Study of Fetal Pathologies

c

d

Fig. 8.24 (continued)

109

110

e

f

Fig. 8.24 (continued)

8  Three-Dimensional Printing and Virtual Models in Fetal Medicine

8.1  Study of Fetal Pathologies

111

g

h

Fig. 8.24 (continued)

Rhabdoid tumors is a congenital fetal tumor with few descriptions on the literature. It can be divided into three categories according to the primary site of tumor development: renal, central nervous system, and the remaining areas. The site of occurrence is broad. It can be present in the liver, thymus, genitourinary tract, peripheral nerves, gastrointestinal tract, and soft tissues (Fig. 8.25).

8.1.6 Chest Anomalies 8.1.6.1 Congenital Diaphragmatic Hernia Congenital diaphragmatic hernia (CDH) is a life-­ threatening event in severe forms, and fetuses affected may benefit from in utero treatment by fetoscopic endotracheal occlusion (FETO)

112

8  Three-Dimensional Printing and Virtual Models in Fetal Medicine

a

b

Fig. 8.25  Rhabdoid tumor affecting the left side of the fetal thorax (*). Ultrasound image of the tumor, fetal magnetic resonance imaging (MRI) (sagittal T2-weighted),

and three-dimensional (3D) virtual reconstruction from MRI (36  weeks) (a). Postnatal computed tomography with 3D reconstruction showing the tumor invasion (*) (b)

8.1  Study of Fetal Pathologies

(Fig. 8.26) [40]. It is a complex disease that recognizes a combination of multiple genes, especially mutation in the 15q26 chromosome, and environmental factors. Embryologically, the muscular diaphragm forms between the 6th and the 14th post-­menstrual weeks, and by the end of week eight the primitive diaphragm is intact. CDH may be seen in approximately 2.7 in 10,000 live births, is usually unilateral (97%) and left sided in 75–90% of cases.

113

The lung-to-head ratio (LHR), the percentage predicted lung volume (PPLV), the observed-to-­ expected total lung volume ratio (o/e-TLV), and the amount of liver herniation are among the most used prognostic indices to predict perinatal outcomes and treatment, although only LHR is the single validated predictor. In its severe form (LHR