Surgery and Operating Room Innovation [1st ed.] 9789811589782, 9789811589799

Table of contents :
Front Matter ....Pages i-viii
Front Matter ....Pages 1-1
Lightweight Carbon-Reinforced Resin Surgical Instruments (Eiji Mekata, Atsushi Yamada, Masaaki Shimagaki, Takahiro Kajiyama, Tohru Tani)....Pages 3-16
Forceps-Type Palpation System for Laparoscopic Surgery (Michitaka Fujiwara, Yoshihiro Tanaka, Tomohiro Fukuda)....Pages 17-26
Ultrahigh Definition (8K UHD) Video System and Video-Assisted Surgery in the Near Future (Toshiyuki Mori, Hisae Aoki, Toshio Chiba, Hiromasa Yamashita, Kenkichi Tanioka)....Pages 27-31
Monitoring of Surgeon’s Surgical Skills Using Internet of Things-Enabled Medical Instruments (Yuki Ushimaru, Yuichiro Doki, Kiyokazu Nakajima)....Pages 33-43
Front Matter ....Pages 45-45
Regenerative Medicine in the Operating Room at Present and in the Near Future (Kengo Kanetaka, Susumu Eguchi)....Pages 47-55
Surgery and Operating Room for Restoring Organs: Organ Regeneration by Tissue Engineering in the Near Future (Mitsuo Miyazawa, Masato Watanabe, Yoshihisa Naito, Yasumitsu Hirano, Keizo Taniguchi, Takehiro Okumura et al.)....Pages 57-62
Front Matter ....Pages 63-63
Extended Reality (XR:VR/AR/MR), 3D Printing, Holography, A.I., Radiomics, and Online VR Tele-Medicine for Precision Surgery (Maki Sugimoto)....Pages 65-70
Application of AI in Endoscopic Surgical Operations (Norihito Wada, Yuko Kitagawa)....Pages 71-77
Front Matter ....Pages 79-79
Application of ICG Fluorescent Endoscope Systems in Identifying Small Lung Cancers on the Periphery of the Lungs in Thoracoscopic Surgery (Yasuhiko Ohshio)....Pages 81-91
Novel Multispectral Device for Quantitative Imaging of Tissue Oxygen Saturation and Hemoglobin as Surgical Navigation Device (Yasuhiro Haruta, Ryosuke Tsutsumi, Kuriyama Naotaka, Hajime Nagahara, Tetsuo Ikeda)....Pages 93-106
Clinical Benefit of Mixed Reality Holographic Cholangiography for Image-Guided Laparoscopic Cholecystectomy (Michiko Kitagawa, Maki Sugimoto, Akiko Umezawa, Yoshimochi Kurokawa)....Pages 107-112
Front Matter ....Pages 113-113
Development of Laparoscopic Surgery by Means of Foldable Small Humanoid Robot Hands with Tactile Sensation for Laparoscopic Surgery (Masaya Mukai, Ryu Kato, Hiroshi Yokoi)....Pages 115-123
Robotic Surgery: Currently and in the Near Future (Masaaki Ito)....Pages 125-131

Citation preview

Surgery and Operating Room Innovation Seiichi Takenoshita Hiroshi Yasuhara Editors

123

Surgery and Operating Room Innovation

Seiichi Takenoshita • Hiroshi Yasuhara Editors

Surgery and Operating Room Innovation

Editors Seiichi Takenoshita Fukushima Medical University Fukushima Fukushima Japan

Hiroshi Yasuhara Tokyo Teishin hospital Chiyoda-ku Tokyo Japan

ISBN 978-981-15-8978-2    ISBN 978-981-15-8979-9 (eBook) https://doi.org/10.1007/978-981-15-8979-9 © Springer Nature Singapore Pte Ltd. 2021 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore

Preface

The development of surgery has been associated with the invention of operative procedures, which often occurs through the emergence of novel medical technology or new discoveries in the field of medicine. Until recently, the quality of surgery has mainly depended on the surgeon’s skills, built on his own talent and experience. Because of this, the refinement of surgical procedures has been more important than their reform. Surgery has not changed rapidly or dramatically thus far. These days, however, surgical techniques have become increasingly dependent on sophisticated medical instruments equipped with advanced specifications. As a result, sudden progress or emergence of operative procedures often takes place because the improvement of medical instruments has been accelerated by the advancement of technology. Robotic surgery is only one example among many. In fact, prior to the emergence of robotic surgery, surgical procedures using specific endoscopes and various types of forceps had been introduced in thoracic and abdominal surgery and their use had rapidly prevailed. Consequently, laparoscopic and thoracoscopic surgery replaced conventional laparotomy and thoracotomy over a short period. However, that was only the beginning of innovations in surgery brought about by the invention of medical instruments. After a while, once novel surgical procedures using surgical robots emerged for patients with prostatic or gynecological disease, these surgical procedures have in turn spread like lightning all over the world. Finally, robotic surgery has replaced laparoscopic surgery, which had been performed until then. Because the emerging procedures have also improved outcomes, surgery assisted by surgical robots has become the de facto standard for prostate cancer. Thus, surgical innovation, i.e., the emergence of new medical instruments, has caused a paradigm shift in surgery. Innovation in surgery is not limited to the surgical skills performed using novel medical instruments, but also extends to the environment surrounding the operation in the operating room (OR). Inventive ideas for surgical treatment can change the shape and structure of the OR.  For instance, a much cleaner OR environment is needed than before in association with advancement of biomedical technology, such as tissue engineering and regenerative medicine. Because tissue and cell handling is generally carried out in a clean cell processing facility called a Cell Processing Center (CPC), the OR should be sited in the vicinity of the CPC for tissue and organ repair or its implantation. In this facility, a wide variety of biomaterials are created including v

Preface

vi

b­ ladders, small arteries, skin, and grafts of cartilage and trachea for regenerative medicine performed in the OR. Another example of OR innovation is that currently, thanks to navigation technology, the diagnostic imaging data acquired preoperatively enables surgeons to safely perform neurosurgery and orthopedic surgery. In the OR connected with a cloud database using information and communication technology (ICT), they can use those imaging data more easily with improved accessibility in the OR. More recently, as diagnostic imaging machines such as CT or MRI tend to be sited in the OR, surgeons can obtain information on diagnostic images immediately after their acquisition and use those data for the patient right in front of them. In the near future, they will be able to utilize visual images for the operation even more dynamically, using virtual reality technology. OR innovations can make it possible for surgeons to perform more precise and safer surgery than ever before. The chapters in this book are just a part of the whole story. Nevertheless, the examples presented here illustrate how innovations in surgery and the OR are ongoing and will continue to change surgical procedures and the OR environment. Those changes will inevitably influence both present and future surgery, although it is not easy to predict tomorrow’s surgery accurately. The emerging procedures, such as surgery using virtual reality technology, new surgical instruments and materials, or regenerative medicine, might no longer be considered surgery, but no matter what the novel treatment is called, doctors can take part in innovations in surgery themselves if they would like to. We hope that this book will improve understanding of surgical procedures and OR innovation in the emerging field of surgery. Chiyoda-ku, Tokyo, Japan Fukushima, Fukushima, Japan 

Hiroshi Yasuhara Seiichi Takenoshita

Contents

Part I Surgical Devices and the Operating Room 1 Lightweight Carbon-Reinforced Resin Surgical Instruments ����   3 Eiji Mekata, Atsushi Yamada, Masaaki Shimagaki, Takahiro Kajiyama, and Tohru Tani 2 Forceps-Type Palpation System for Laparoscopic Surgery��������  17 Michitaka Fujiwara, Yoshihiro Tanaka, and Tomohiro Fukuda 3 Ultrahigh Definition (8K UHD) Video System and Video-Assisted Surgery in the Near Future����������������������������������  27 Toshiyuki Mori, Hisae Aoki, Toshio Chiba, Hiromasa Yamashita, and Kenkichi Tanioka 4 Monitoring of Surgeon’s Surgical Skills Using Internet of Things-Enabled Medical Instruments������������������������  33 Yuki Ushimaru, Yuichiro Doki, and Kiyokazu Nakajima Part II Medical Materials and Regenerative Medicine 5 Regenerative Medicine in the Operating Room at Present and in the Near Future������������������������������������������������������  47 Kengo Kanetaka and Susumu Eguchi 6 Surgery and Operating Room for Restoring Organs: Organ Regeneration by Tissue Engineering in the Near Future��������������������������������������������������������������������������������������  57 Mitsuo Miyazawa, Masato Watanabe, Yoshihisa Naito, Yasumitsu Hirano, Keizo Taniguchi, Takehiro Okumura, Kaname Maruno, and Shozo Fujino

vii

viii

Part III Artificial Intelligence and Virtual Reality 7 Extended Reality (XR:VR/AR/MR), 3D Printing, Holography, A.I., Radiomics, and Online VR Tele-­Medicine for Precision Surgery����������������������������������������������  65 Maki Sugimoto 8 Application of AI in Endoscopic Surgical Operations������������������  71 Norihito Wada and Yuko Kitagawa Part IV Navigation Surgery 9 Application of ICG Fluorescent Endoscope Systems in Identifying Small Lung Cancers on the Periphery of the Lungs in Thoracoscopic Surgery ����������������������������������������  81 Yasuhiko Ohshio 10 Novel Multispectral Device for Quantitative Imaging of Tissue Oxygen Saturation and Hemoglobin as Surgical Navigation Device����������������������������������������������������������������������������  93 Yasuhiro Haruta, Ryosuke Tsutsumi, Kuriyama Naotaka, Hajime Nagahara, and Tetsuo Ikeda 11 Clinical Benefit of Mixed Reality Holographic Cholangiography for Image-Guided Laparoscopic Cholecystectomy������������������������������������������������������ 107 Michiko Kitagawa, Maki Sugimoto, Akiko Umezawa, and Yoshimochi Kurokawa Part V Robotic Surgery 12 Development of Laparoscopic Surgery by Means of Foldable Small Humanoid Robot Hands with Tactile Sensation for Laparoscopic Surgery���������������������������������� 115 Masaya Mukai, Ryu Kato, and Hiroshi Yokoi 13 Robotic Surgery: Currently and in the Near Future�������������������� 125 Masaaki Ito

Contents

Part I Surgical Devices and the Operating Room

1

Lightweight Carbon-Reinforced Resin Surgical Instruments Eiji Mekata, Atsushi Yamada, Masaaki Shimagaki, Takahiro Kajiyama, and Tohru Tani

1.1

Introduction

Surgical instruments such as Adson and DeBakey tweezers, forceps, needle holders, and Cooper and Metzenbaum scissors are widely used in common or complex surgical procedures [1–7]. These instruments are generally made of stainless steel and have high stiffness for durability and long-term stability, smooth surface processing for ergonomic design and easy washing, and high heat-resistance for repeated autoclave sterilization [8]. This metal material enables the tweezers to be used conveniently as soft coagulation probes for hemostasis by bringing monopolar electric cautery into contact with them while pinching tissues by the tweezers. Technical challenges for these metal instruments are weight saving and compatibility with some imaging modalities. The weight saving

E. Mekata (*) Department of Comprehensive Surgery, Shiga University of Medical Science, Otsu, Shiga, Japan e-mail: [email protected] A. Yamada · T. Tani Department of Research and Development for Innovative Medical Devices and Systems, Shiga University of Medical Science, Otsu, Shiga, Japan M. Shimagaki Frontier Medtec Co., Ltd., Otsu, Shiga, Japan T. Kajiyama Nissei Industries Ltd., Ome, Tokyo, Japan

obviously reduces the physical load on medical staff when carrying and washing the instruments [9]. It is also crucial for mass transport in critical situations with significantly degraded hospital function such as large-scale natural disasters and infectious diseases [10, 11]. Magnetic resonance imaging (MRI) requires non-ferromagnetic materials for MRI safety [12, 13], and computed tomography (CT) requires reducing metal artifacts [14]. The stainless steel instruments require continuous maintenance to maintain their function although there has been an issue of lacking skilled maintenance staff. Recently, titanium or ceramic materials have been used for them [13, 15, 16] which can partially contribute to these challenges but their mass production cost precludes their stockpile for disasters. Polyamide (PA) resin [17, 18] claimed weight saving and imaging compatibility but unfortunately involved losing their high stiffness, reusability, and the convenience of soft coagulation probing. The goal of this study was to develop nonmetallic surgical instrument prototypes as potential solutions that were lightweight and reusable with wide variation and comparable with stainless steel surgical instruments in terms of usability and performance. We evaluated their mechanical performance compared to conventional surgical instruments and their usability was evaluated by multiple surgeons in an animal study.

© Springer Nature Singapore Pte Ltd. 2021 S. Takenoshita, H. Yasuhara (eds.), Surgery and Operating Room Innovation, https://doi.org/10.1007/978-981-15-8979-9_1

3

E. Mekata et al.

4

1.2

Materials and Methods

1.2.1 Surgical Instruments and Material Prototypes We developed prototypes of resin-based surgical instruments by molding: 120  mm-long

Adson tweezers, 200  mm-long DeBakey tweezers, 180 mm-long forceps, a 180 mm-long needle holder, 140  mm-long Cooper scissors, and 180  mm-long Metzenbaum scissors, as shown in Fig.  1.1. The Adson tweezers had embossed grips to prevent finger slipping and knurled tips with straight line patterns to prevent dropping.

a

b

c

d

e

f

g

h

i

j

Fig. 1.1  Prototypes: (a) from the left side, Adson tweezers, DeBakey tweezers, forceps, needle holder, Metzenbaum scissors, and Cooper scissors; (b) Adson tweezers tip; (c) Adson tweezers’ embossed grip; (d)

DeBakey tweezers tip; (e) forceps tip; (f) needle holder tip; (g) Metzenbaum scissor blades; (h) Cooper scissor blades; (i) enlarged image of the surface around the pivot of the needle holder; (j) enlarged image of the needle holder tip

1  Lightweight Carbon-Reinforced Resin Surgical Instruments

The DeBakey tweezers had embossed grips and tips. The forceps had four-stage latches for static pinching and knurled tips with diamond patterns to prevent dropping. The needle holder had three-­stage latches for static pinching and stainless steel knurled tips with diamond patterns for stable needle holding. The scissors had stainless steel blades. These metal parts were joined to the resin frames by multilayered insert molding (Prototype process, Nissei Industries Ltd., Tokyo, Japan): first, molten resin was injection-molded onto a surface-modified metal part. This first insert-molded part was then embedded on a resin frame by the second insert molding. The weights of the prototypes were measured and compared to conventional stainless steel instruments. As the molding resin material, we developed polyphenylene sulfide (PPS) resin reinforced by short carbon-fiber fillers (A630T-30V prototype material, Toray Industries Inc., Tokyo, Japan). We confirmed that the short carbon-fiber fillers were covered with the resin coating by observing a

5

the surfaces of the prototypes with a microscope as shown in Fig. 1.1i, j. As comparative materials, we estimated the mechanical properties of PA 66 reinforced with 40% glass fiber (GF) and PA 66 reinforced with 60% GF based on ISO178 method. These materials are used for conventional resin-based instruments. The mechanical properties and features of the conventional resin-­ based instruments, stainless steel instruments, and the proposed instruments were tabulated.

1.2.2 Design and Fabrication Process Figure 1.2 summarizes the process of fabricating these prototypes. We designed prototype models using computer-aided design (CAD) software (SolidWorks 2017, Dassault Systemes SolidWorks Corp., Waltham, MA). The CAD models were manufactured by molding or multilayered insert molding using a computer-aided d

Computer-aided Design (CAD)

Numerical model analysis

Finite element models

Feedback

Feedback CAD models

b

Computer-aided Manufacturing (CAM)

c Products Physical model analysis

Molding Multi-layered insert molding

Fig. 1.2 Design and developing process: (a) The computer-­aided designed models were used for molding; (b) The mechanical functions were evaluated; (c) stress

concentration and elastic deformation were evaluated numerically; (d) the modified designs were evaluated before molding

E. Mekata et al.

6

manufacturing (CAM) system. The products’ mechanical functions were assessed and feedback about the results was used to improve the models. Unexpected results of the physical model analysis or invisible properties including stress concentration and the distribution of elastic deformation were assessed numerically using a multi-body dynamics solver (DAFUL ver. 6.2 SP2, VirtualMotion Inc., Korea) that enabled the analysis of large elastic deformations of nonmetallic materials. This solver used finite element models converted from CAD models using finite element modeling pre/post-processor (FEMAP ver. 11.1.1, Siemens PLM software, Plano, TX). The feedback about the simulation results was used to improve the models. The designed and improved models could be evaluated numerically and agilely without CAM to reduce the time cost and repeated molding cost. After these trials and errors, the prototypes were manufactured. The prototypes’ mechanical performances were assessed as follows.

1.2.3 Assessment of Mechanical Properties 1.2.3.1 Forceps and Needle Holder We investigated the pinching force of the forceps prototype. First, a side of the prototype’s finger hole was fixed on a machine workbench so that the prototype tip was horizontal as shown in Fig. 1.3. Then, the side of the other finger hole was pushed vertically with a probe of a digital push–

a

pull gauge (9505, Aikoh Engineering, Co. Ltd., Osaka, Japan) and each compression force to lock the first and third latches was recorded. The pinching force Fp at the tip was a­pproximated using the equilibrium moment: Fp = LlFl/Lp [N], where Fl represents the latch force, Ll and Lp represent the length between the tip and the scissors’ pivot and the length between the probe contact point and the pivot, respectively. Conventional stainless steel forceps and resin-based single-use forceps (Aesculap SUSI PEAN, B. Braun, Germany) were also tested as comparative instruments. The pinching force of the needle-holder prototype was also investigated in the same manner as explained above by comparison with a stainless steel needle holder and a resin-based single-use needle holder (Aesculap SUSI Needle Holder, B. Braun).

1.2.3.2 Tweezers We measured the gripping force when the grip area of the Adson tweezers was pushed by the probe of the push–pull gauge until the entire knurling processing areas at the tips had made contact with each other. The measuring was performed in the same manner as the forceps gripping force measurement explained above. The probe pushed positions were set at 30 mm, 45 mm, and 60 mm from the tip in the range of the embossed grip on the tweezers as shown in Fig.  1.4a. The tip conditions are shown in Fig. 1.4b. The gripping force of the comparative stainless steel tweezers and stainless steel tweezers with a carbide tip was also recorded. In addition, we measured the maximum load of the prototype tweezers.

b

Fig. 1.3  Experimental setup for forceps prototype (a) and needle-holder prototype (b)

1  Lightweight Carbon-Reinforced Resin Surgical Instruments

7

a

b

Initial state

Fig. 1.4  Experimental setup for Adson tweezers prototype: (a) gripping positions, white solid arrows represent probe pushed positions; (b) tip condition when starting

The prototype pinched one side of a cut artery manually and stainless steel forceps pinched the other side. The digital push–pull force gauge was connected to the forceps and the force gauge was pulled manually until the artery was slipped off to record the measurements.

1.2.3.3 Scissors We evaluated the stress distribution of the scissors prototypes numerically based on the assumption that both rotations and bending were applied at the scissors’ pivot during repeated manual cutting motions as shown in Fig. 1.5. To mimic the situation of strongly bending the frame by the thumb and index finger during cutting motions, one scissor frame was fixed and the other scis-

Finish state

grasping force measurement and tip condition when finishing grasping force measurement

q Fs

Fig. 1.5  Numerical simulation setup for Cooper scissors prototype: θ represents the blade opening angle and Fs represents pushing force applied at the finger hole. The simulation setup for Metzenbaum scissors prototype was performed as the same manner

E. Mekata et al.

8

sor frame was pushed by Fs  =  1  N vertically at the finger hole. The opening angle of the scissor blades θ was varied at 10–60° in 10° increments to express cutting motions, and we performed a numerical simulation of the same model with stainless steel material properties, and the PA 66 reinforced with 60% GF for comparison with the prototype mechanical performance. The stress distribution images and the maximum deformation in the vertical direction were recorded.

1.2.4 Durability and Long-Term Stability Test We performed a combination of heat cycle test and accelerated aging test for the Adson and DeBakey tweezers, forceps, and needle-holder prototypes to investigate the prototypes’ autoclave durability and long-term stability. Four test procedures were considered: 1. Baseline: No heat cyclic test and no accelerated aging test were performed. 2. Heat cyclic test: Autoclave heating at 121 °C for 20 min repeated 70 times. 3. Accelerated aging test: Heating at 150  °C for 24 h. 4. Sequential combined test: The heat cyclic test (1) was performed and then the accelerated aging test (2) was performed. After executing each procedure, three-point bending tests were performed using a desktop testing system (Instron Model 5848, Illinois Tool Works Inc., Norwood, MA) to evaluate the consistency of the prototype frame strengths. The maximum breaking loads were recorded.

1.2.5 Feasibility and Sensory Evaluations An animal study was performed by four surgeons to evaluate the performance of the prototypes using a female pig. The study was reviewed and approved by the Ethical Research Committee for Animal Life Science at Shiga University of

Medical Science. Part of the stomach was first cut by a microwave surgical instrument (Acrosurg. Scissors, Nikkiso Co. Ltd., Tokyo, Japan). The cut part was then sutured using sutures (3–0 PDS*II polydioxanone suture and 0, 4–0, and 5–0 coated VICRYL* polyglactin 910 sutures, Ethicon Inc., Guaynabo, PR) using the tweezers, forceps, needle holder, and scissors prototypes. In the experiment, sterilized hospital gauzes (AS4–20, Osaki Medical Corp., Aichi, Japan) and surgical drapes (SR-844, Hogy Medical Co, Ltd., Tokyo, Japan) were used. All procedures were shot by a handtype video camera. The developed Adson tweezers were tiny and particularly lightweight among the prototypes because they were comprised of thin flexible bent frames; it was expected that they would have a large different feeling when used than conventional stainless steel ones. Thus, we validated its usability by questionnaire survey with 13 surgeons, six nurses, and four paramedics. Eight questions were considered: 1. Stability during use (grip position; center of gravity). 2. Tactile feeling during gripping (surface processing; shape). 3. Usability of tip edge (pinch and release). 4. Usability of knurled tip (needle holding function). 5. Restitutive force when gripping. 6. Weight. 7. Tint. 8. Comprehensive evaluation. For each question, six response options were set up: very acceptable, fairly acceptable, either/ or, not very acceptable, not at all acceptable, and no answer.

1.3

Results

1.3.1 Surgical Instrument and Material Prototypes Table 1.1 shows the weights of the prototypes and the stainless steel instruments and each instru-

1  Lightweight Carbon-Reinforced Resin Surgical Instruments

9

Table 1.1  Weights of the prototype instruments, the stainless steel equivalents, and their weight ratios Instrument Adson tweezers DeBakey tweezers Forceps Needle holder Cooper scissors Metzenbaum scissors

Prototype Length [mm] 120 200 180 180 140 180

Weight [g] 6 14 13 17 15 13

Stainless steel equivalent Length [mm] Weight [g] 120 20 205 47 165 41 160 49 185 55 180 34

Weight ratio 0.30 0.30 0.32 0.35 0.27 0.38

Table 1.2  Mechanical properties and features of conventional stainless steel (SUS), prototype, polyamide (PA) 66 reinforced with 40% glass fiber (GF) (PA 66/GF 40), and PA 66/GF 60 instruments Flexural modulus [GPa] Tensile strength [Mpa] Melt temperature [°C] Density [kg/m3] Magnetism Conductivity Chemical resistance Workability Mass production Material price

SUS 197 >520 1400 Paramaginetic High High Middle Middle High

Prototype 27.8 236 278 1460 Non-magnetic Middle High High High Middle

PA 66/GF 40 12.5 210 280–300 1460 Non-magnetic No High High High Middle

PA 66/GF 60 19.5 255 280–300 1670 Non-magnetic No High High High Middle

Though SUS is usually nonmagnetic material, it can be magnetic after cold working

ment weight ratio. The average weight of the prototype variation was 13.0 g while that of the stainless steel instrument variation was 41.0  g. Table  1.2 shows the mechanical properties and features of conventional stainless steel (SUS), prototype, and conventional resin-based instruments. The flexural modulus of the prototype was 27.8  GPa, which was about 42.6% higher than that of the PA 66 reinforced with 60% GF. The tensile strength of the prototype was almost same as the conventional resin-based instruments.

1.3.2 Assessment of Mechanical Properties 1.3.2.1 Forceps and Needle Holder Figure 1.6 shows the measurement results for the prototype forceps. The pinching forces were 7.4  N and 21.9  N, which were calculated from the gripping forces of 4.2 N and 12.5 N when the gripper was locked at the first and fourth latches, respectively. The maximum pinching force for

the prototype was about 24% larger and 22% smaller than that of the comparative resin and stainless steel instruments, respectively. Ll and Lp for the prototype, stainless steel, and comparative resin were 60, 60, 53, 105, 130, and 80 mm, respectively. The figure also shows the measurement results for the needle holder prototype. The pinching forces were 75.6 N and 94.5 N, which were calculated from the gripping forces of 16.8  N and 21.0 N when the gripper was locked at the first and third latches, respectively. The maximum pinching force of the prototype was about 86% larger and 23% larger than that of the comparative resin and stainless steel instruments, respectively. Ll and Lp for the prototype, stainless steel, and comparative resin were 30, 30, 35, 135, 130, and 100 mm, respectively.

1.3.2.2 Tweezers Figure 1.7 shows the gripping force with respect to the grip positions. The figure includes the results for the prototypes, conventional stain-

E. Mekata et al.

10

a

b Prototype

Stainless steel

Comparative resin

12.5

21.9 4

4

7.4 1

13

Latch number

Latch number

28.2 17.7

11.7

4.2 1

14.5

6.7

6 0

4

5

10

15

25

20

0

30

5

10

15

Fp[N]

c

25

30

90

100 110

d Prototype

Stainless steel

Comparative resin

94.5 76.7

3 50.9

75.6 40.7

1

21

Latch number

Latch number

20

Fl [N]

17.7

3

17.8

16.8 1

9.4 13

37.1 0

10

20

30

40

50

60

70

80

90

100 110

Fp[N]

0

10

20

30

40

50

60

70

80

Fl [N]

Fig. 1.6  Experimental results for forceps prototype (a), (b) and needle holder (c), (d): (a) pinching forces calculated by the equilibrium moment of the latch force; (b) the first and the fourth latch forces measured by a push–pull

gauge; (c) pinching forces calculated by the equilibrium moment of the latch force; (d) the first and the third latch forces measured by a push–pull gauge

less steel, and stainless steel with a carbide tip; their maximum gripping forces were 37.8, 32.7, and 28.3 N, respectively. Each gripping pressure needed to close the tips was increased almost linearly when the gripping position became farther from the tip. The slopes of each approximate straight line were about 0.63, 0.33, and 0.33, respectively. The average maximum pinching force was 13.3 N.

of the stainless steel Cooper scissors. The maximum displacements of the Metzenbaum prototype when adding the upper bending forces were 22  mm, which was about 2.5-fold larger than that of the stainless steel Metzenbaum scissors. The maximum stress ­distributions were observed around the pivot positions.

1.3.2.3 Scissors Figure 1.8 shows the numerical simulation results. The maximum displacements of the Cooper prototype when adding the upper bending force were 2.5 mm, which was about 2.6-fold larger than that

1.3.3 Durability and Long-Term Stability Test Figure 1.9 shows the result of the three-point bending test. The average maximum breaking loads for the Adson and DeBakey tweezers, for-

1  Lightweight Carbon-Reinforced Resin Surgical Instruments

11

80

70

60

Gripping pressure [N]

Prototype Stainless steel

50

Stainless steel with carbide tip 40

30

20

10

0 30

45

60

Gripping position [mm]

Fig. 1.7  Gripping pressures needed to close the tips for the Adson tweezers prototype, the comparable stainless steel tweezers, and stainless steel tweezers with a carbide tip

ceps, and needle holder prototypes were 300 N, 300  N, 190  N, and 190  N, respectively. The decreased maximum loads for each prototype between the tests were 10% of the insertion time. The median activation time for each usage of the electrocautery probe was 0.41  s (0.14–0.57  s), with variation observed among surgeons (Fig. 4.5b).

that most industries and businesses are inherently different from medical fields, particularly surgery, because sterilization is needed in the latter [28]. Disinfection and sterilization are indispensable to prevent transmission of infectious pathogens from medical activities or surgical instruments to the patient, and, if the guidelines based on scientific grounds are not followed, disease outbreaks can occur [29–35]. Moreover, there is currently no technique to sterilize sensors capable of surgical application, and this has severely limited the use of sensors in surgical devices. Furthermore, it is possible that the regulatory agencies of each country affect the problems and contribute to the poor awareness of physicians. Another technical hurdle to overcome is crosstalk in the wireless system [36]. To your knowledge, our experiment is the first global success in visualizing specific movements during a surgical procedure. As

4.4

 rend of IoT in Medical T Devices

The IoT trend is also expanding gradually into the medical field and medical devices. However, the above applications in the medical field are only experimental undertakings. The question is why medical and surgical fields are lagging behind. One of the reasons could be

4  Monitoring of Surgeon’s Surgical Skills Using Internet of Things-Enabled Medical Instruments

a

b

Activation rate of electrocautery

The activation time per one time

Sec/time

% 100

8.8

8.6

2.4

9.3

6.9

0.7 0.574

0.6 80

0.5

0.5 60

40

39

0.408

0.4 91.2

91.4

97.6

90.7

93.1

0.3 0.2

20

0.29

0.143

0.1 0

Surgeon A Surgeon B Surgeon C Surgeon D Surgeon E

not driven

drive

0

Surgeon A Surgeon B Surgeon C Surgeon D Surgeon E

Median: 0.41sec/time(range: 0.14-0.57sec/time)

Fig. 4.5  The ON/OFF rates and the activation time of the electrocautery. (a) The activation time rate per overall duration of electrocautery usage by each surgeon. (b) The

activation time per electric scalpel usage by each surgeon (Median: 0.41 s/time (range: 0.14–0.57 s/time))

reported, data on aspects that indicate the surgical instrument behavior, the devices held by the surgeon’s left and right hands, and the manner by which the electrosurgical probe is used could enable live feedback to the surgeon by way of visualization and standardization. Furthermore, this technique can enable the study of the correlational aspects of various factors that affect the performance of surgery, which would help refine surgical performance and increase patients’ safety. One result of this study is that most of the devices used with the right hand were electrocautery probes, in all the operations. However, the actual activation time of the electrocautery probe was short. From this, it was observed that the electrocautery probe of the right hand was used for various purposes, such as blunt dissection and ductal tunneling, in addition to coagulation, hemostasis, and cutting. This is consistent with the fact that, when the surgeon uses energy devices, he/she conducts not only

coagulation and cutting but also blunt dissection and other tissue handling. Furthermore, in this study, surgeon A performed laparoscopic cholecystectomy as a mentor surgeon. He intentionally used different forceps depending on the situation: for example, a Maryland dissector for periductal dissection and a hook-type electrocautery for gallbladder dissection from the liver bed. He had to change instruments frequently, mostly for educational and/or demonstration purposes. As a result, the total use time of the electrocautery probe decreased and the traffic of the surgical devices increased. In contrast, surgeons (B, E) showed more practical performance, demonstrating the full use of the electrocautery probe for multiple purposes. Consequently, the activation time per usage of the electrocautery probe became longer, whereas the traffic decreased. Further “big data” analysis may demonstrate that surgical technique might vary depending on the function of operating surgeons.

Y. Ushimaru et al.

40

In the present experiment, only a small amount of data (n = 5) was studied. However, this experiment paved the way for a large-scale study with a very large dataset and, subsequently, a big data analysis. Accumulating surgical data can help identify collective rules by data analysis. From the perspective of data characteristics, although the data obtained provide few evaluation items for each individual patient, they characteristically contain a large number of samples. Increasing the number of subjects will help to identity diversity in individualized patterns. Such data can be applied as benchmarks in various procedures and used for maintaining the technical quality of procedures and for learning, training, and credentialing purposes. Furthermore, linking such a large dataset to a hospital information system (HIS) and incorporating artificial intelligence (AI) would give rise to several new possibilities in the future. It is believed that the decision-making ability of surgeons is worthy of trust, and that, at present, every important step in surgery is performed at the discretion of the surgeon. Conversely, AI can make judgments prospectively based on knowledge obtained from data. Therefore, if ­ information from AI is included in intraoperative judgments, more appropriate perioperative care can be obtained, as compared with conventional methods. This can be achieved by controlling various risks during surgery and providing suitable advice to the surgeon, while predicting postoperative progress in advance. It has been reported that AI can provide a supporting role to the surgeon [37], and combining AI with a large dataset can be of immeasurable benefit [38].

In recent years, sensor technology has rapidly improved, enabling the development of smaller, multifunctional sensors with higher performance. In the near future, we may be able to ascertain the true meaning of forceps traffic. Furthermore, in the fields of dermatology and gastroenterology, it has been suggested that diagnosis can be made by coordinating image data and AI [37, 39, 40], and soon it may be possible to coordinate surgical imaging with AI.  Moreover, coordinating sensor data with HIS could help correlate procedures with outcomes and might provide high-level clinical evidence. The present study demonstrated a proof of concept, and we are planning a clinical study in the future.

4.5

Prospects for Operating Room Innovation

The IoT trend in surgical devices enables intraoperative events, and the traffic of devices, to be automatically recorded, and suggests the possibility that procedures can be visualized and standardized (Fig. 4.6). Therefore, it is expected that creating big data and including such data for analysis by AI technology will aid in the discovery of the most suitable usage of surgical devices, finding the ideal usage of energy devices, improving the safety of surgery, and making the optimal usage standardized. Moreover, we believe that the combining HIS with sensor data will contribute to the improvement of the quality of surgical treatment overall.

4  Monitoring of Surgeon’s Surgical Skills Using Internet of Things-Enabled Medical Instruments

41

Fig. 4.6  The image of the future operating room

References 1. Ashton K.  That ‘Internet of Things’ thing: in the real world, things matter more than ideas. RFID J. 2009;22(7):97–114. 2. Gershenfeld N, Krikorian R, Cohen D. The internet of things. Sci Am. 2004;291:76. 3. Bundesministerium für Bildung und Forschung. Zukunftsprojekt Industrie 4.0. 2014. 4. Lydon B. The 4th Industrial Revolution, Industry 4.0, Unfolding at Hannover Messe 2014. [online]. 2014. http://www.automation.com/automation-news/article/ the-4th-industrial-revolution-industry-40-unfoldingat-hannover-messe-2014: Automation.com. http:// www.automation.com/automation-news/article/

the-4th-industrial-revolution-industry-40-unfoldingat-hannover-messe-2014. Accessed 19 Feb 2014. 5. Industrial Internet Consortium. Manufacturing. 2015. 6. Atzori L, Iera A, Morabito G. The internet of things: a survey. Comput Netw. 2010;54(15):2787–805. 7. Chui M, Löffler M, Roberts R. The internet of things. McKinsey Q. 2010;2:1–9. 8. Winig L.  GE’s big bet on data and analytics. MIT Sloan Management Review. 2016. 9. Ju J, Kim M-S, Ahn J-H. Prototyping business models for IoT service. Proc Comput Sci. 2016;91:882–90. 10. Dinis H, Zamith M, Mendes PM. Performance assessment of an RFID system for automatic surgical sponge detection in a surgery room. In: Conference Proceedings—IEEE Engineering in Medicine and Biology Society, 2015; 2015. p. 3149–52.

42 11. Egan MT, Sandberg WS. Auto identification technology and its impact on patient safety in the operating room of the future. Surg Innov. 2007;14(1):41–50; discussion 51. 12. Hanada E, Hayashi M, Ohira A.  Introduction of an RFID tag system to a large hospital and the practical usage of the data obtained. In: 2015 9th International Symposium on Medical Information and Communication Technology (ISMICT), 24–26 March 2015; 2015a. p. 108–11. 13. Hanada E, Ohira A, Hayashi M, Sawa T. Improving efficiency through analysis of data obtained from an RFID tag system for surgical instruments. In: 2015 IEEE 5th International Conference on Consumer Electronics—Berlin (ICCE-Berlin), 6–9 Sept. 2015; 2015b. p. 84–7. 14. Nakajima R, Sakaguchi K. Service vision design for smart bed system™ of paramount bed. FUJITSU Sci Tech J. 2018;54(1):9–14. 15. Pasluosta CF, Gassner H, Winkler J, Klucken J, Eskofier BM.  An emerging era in the Management of Parkinson's disease: wearable technologies and the internet of things. IEEE J Biomed Health Inform. 2015;19(6):1873–81. 16. Sawa T, Komatsu H.  Shimane university hospital implements RFID technology to manage surgical instruments. In: 2013 7th International Symposium on Medical Information and Communication Technology (ISMICT), 6–8 March 2013; 2013. p. 90–2. 17. Vilallonga R, Lecube A, Fort JM, Boleko MA, Hidalgo M, Armengol M. Internet of things and bariatric surgery follow-up: comparative study of standard and IoT follow-up. Minim Invasive Ther Allied Technol. 2013;22(5):304–11. 18. Yamashita K, Iwakami Y, Imaizumi K, Yasuhara H, Mimura Y, Uetera Y, Ohara N, Komatsu T, Obayashi T, Saito Y, Komatsu H, Shimada S, Hosaka R, Ino S, Ifukube T, Okubo T. Identification of information surgical instrument by ceramic RFID tag. In: 2008 World Automation Congress, 28 Sept-2 Oct, 2008; 2008. p. 1–6. 19. Kranzfelder M, Schneider A, Fiolka A, Schwan E, Gillen S, Wilhelm D, Schirren R, Reiser S, Jensen B, Feussner H. Real-time instrument detection in minimally invasive surgery using radiofrequency identification technology. J Surg Res. 2013;185(2):704–10. 20. Kranzfelder M, Zywitza D, Jell T, Schneider A, Gillen S, Friess H, Feussner H. Real-time monitoring for detection of retained surgical sponges and team motion in the surgical operation room using radio-­ frequency-­ identification (RFID) technology: a preclinical evaluation. J Surg Res. 2012;175(2):191–8. 21. Ushimaru Y, Takahashi T, Souma Y, Yanagimoto Y, Nagase H, Tanaka K, Miyazaki Y, Makino T, Kurokawa Y, Yamasaki M, Mori M, Doki Y, Nakajima K.  Innovation in surgery/operating room driven by internet of things on medical devices. Surg Endosc. 2019;33:3469. 22. Carlomagno N, Santangelo M, Romagnuolo G, Antropoli C, La Tessa C, Renda A. Laparoscopic cholecystectomy: technical compromise between French

Y. Ushimaru et al. and American approach. Presentation of an original technique. Ann Ital Chir. 2014;85(1):93–100. 23. Dubois F. Laparoscopic cholecystectomy: the French technique. Berlin: Springer; 1995. 24. Kramp KH, van Det MJ, Totte ER, Hoff C, Pierie JP. Ergonomic assessment of the French and American position for laparoscopic cholecystectomy in the MIS suite. Surg Endosc. 2014;28(5):1571–8. 25. Kum CK, Eypasch E, Aljaziri A, Troidl H. Randomized comparison of pulmonary function after the ‘French’ and ‘American’ techniques of laparoscopic cholecystectomy. Br J Surg. 1996;83(7):938–41. 26. Asbun HJ, Rossi RL, Lowell JA, Munson JL.  Bile duct injury during laparoscopic cholecystectomy: mechanism of injury, prevention, and management. World J Surg. 1993;17(4):547–51; 551–2. 27. Strasberg SM, Hertl M, Soper NJ. An analysis of the problem of biliary injury during laparoscopic cholecystectomy. J Am Coll Surg. 1995;180(1):101–25. 28. William A, Rutala DJW. The healthcare infection control practices advisory committee (HICPAC), 2008. In: Guideline for disinfection and sterilization in healthcare facilities; 2008. Last update: 15 Feb 2017. 29. Centers for Disease Control and Prevention (CDC). Pseudomonas aeruginosa infections associated with transrectal ultrasound-guided prostate biopsies— Georgia, 2005. MMWR Morb Mortal Wkly Rep. 2006;55(28):776–7. 30. Kovaleva J, Peters FT, van der Mei HC, Degener JE. Transmission of infection by flexible gastrointestinal endoscopy and bronchoscopy. Clin Microbiol Rev. 2013;26(2):231–54. 31. Lowry PW, Jarvis WR, Oberle AD, Bland LA, Silberman R, Bocchini JA Jr, Dean HD, Swenson JM, Wallace RJ Jr. Mycobacterium chelonae causing otitis media in an ear-nose-and-throat practice. N Engl J Med. 1988;319(15):978–82. 32. Mehta AC, Prakash UB, Garland R, Haponik E, Moses L, Schaffner W, Silvestri G. American College of Chest Physicians and American Association for Bronchology [corrected] consensus statement: prevention of flexible bronchoscopy-associated infection. Chest. 2005;128(3):1742–55. 33. Meyers H, Brown-Elliott BA, Moore D, Curry J, Truong C, Zhang Y, Wallace RJ Jr. An outbreak of Mycobacterium chelonae infection following liposuction. Clin Infect Dis. 2002;34(11):1500–7. 34. Spach DH, Silverstein FE, Stamm WE. Transmission of infection by gastrointestinal endoscopy and bronchoscopy. Ann Intern Med. 1993;118(2):117–28. 35. Weber DJ, Rutala WA.  Lessons from outbreaks associated with bronchoscopy. Infect Control Hosp Epidemiol. 2001;22(7):403–8. 36. Witters D, Seidman S, Bassen H.  EMC and wireless healthcare. In: 2010 Asia-Pacific international symposium on electromagnetic compatibility, 12–16 April 2010; 2010. p. 5–8. 37. Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S. Dermatologist-level classification

4  Monitoring of Surgeon’s Surgical Skills Using Internet of Things-Enabled Medical Instruments of skin cancer with deep neural networks. Nature. 2017;542(7639):115–8. 38. Dilsizian SE, Siegel EL.  Artificial intelligence in medicine and cardiac imaging: harnessing big data and advanced computing to provide personalized medical diagnosis and treatment. Curr Cardiol Rep. 2013;16(1):441. 39. Maroulis DE, Iakovidis DK, Karkanis SA, Karras DA. CoLD: a versatile detection system for colorectal

43

lesions in endoscopy video-frames. Comput Methods Programs Biomed. 2003;70(2):151–66. 40. Saftoiu A, Vilmann P, Gorunescu F, Janssen J, Hocke M, Larsen M, Iglesias-Garcia J, Arcidiacono P, Will U, Giovannini M, Dietrich CF, Havre R, Gheorghe C, McKay C, Gheonea DI, Ciurea T. Efficacy of an artificial neural network-based approach to endoscopic ultrasound elastography in diagnosis of focal pancreatic masses. Clin Gastroenterol Hepatol. 2012;10(1):84–90.e1.

Part II Medical Materials and Regenerative Medicine

5

Regenerative Medicine in the Operating Room at Present and in the Near Future Kengo Kanetaka and Susumu Eguchi

5.1

Introduction

A long-standing standard principle of surgery is the removal of injured or diseased tissue/organs and resection is often followed by reconstruction with a substitute or transplantation. In the past several decades, the advent of surgical technology, immunosuppressive agents, and perioperative management including the control of infection have made it possible to replace diseased organs with graft organs gifted from healthy or brain-dead donors, and advances in kidney and liver transplantation have helped large number of patients achieve a cure. However, organ donation has substantial intrinsic disadvantages, such as a shortage of donors, ethical problems, and issues of graft rejection. Regenerative medicine is an upcoming concept involving the repair or regeneration of tissue/ organ deficit caused by disease, surgical removal, and trauma using processed cells/tissues obtained from the patients themselves or other healthy

K. Kanetaka (*) Tissue Engineering and Regenerative therapeutics in Gastrointestinal Surgery, Nagasaki University Graduate School of Biomedical Sciences, Nagasaki, Japan e-mail: [email protected] S. Eguchi Department of Surgery, Nagasaki University Graduate School of Biomedical Sciences, Nagasaki, Japan

donors. This approach might overcome surgical issues of transplantation by compensating for the lost function and tissue using tissue engineering technology, and in the future, it might come be used to create entire new organs. Regenerative medicine involves promoting the regenerative potential of native tissue stem cells using various drugs or reconstructing tissue components via the growth and differentiation of transplanted processed cells. The transplantation of processed cell includes therapy which expects cytokine or growth factor release to promote the regeneration of native tissue from transplanted cells (trophic effect) and therapy that anticipate replacement of injured tissues with transplanted stem cells and differentiation into various lineages to reconstructed tissue. In a broad sense, it also includes cell therapies administered in daily practice, such as cancer therapies with immune cells and so-called stem cell therapies for cosmetic purposes. Regenerative medicine now permeates every surgical specialty, including orthopedic surgery, cardiac surgery, and general surgery [1]. To achieve these advances in surgical fields, highly qualified, sophisticated multidisciplinary teams and advanced tissue engineering facilities and operating rooms are needed [2]. The typical protocol of regenerative medicine is as follows: a bare minimum number of biopsies are harvested from patients or the selected donor in order to prepare starting cell populations as raw materials

© Springer Nature Singapore Pte Ltd. 2021 S. Takenoshita, H. Yasuhara (eds.), Surgery and Operating Room Innovation, https://doi.org/10.1007/978-981-15-8979-9_5

47

K. Kanetaka and S. Eguchi

48

in the operating room. The cells are then transferred to a tissue engineering facility and are isolated, and these cells are provided to a primary culture for cell acclimation and then, subcultures are repeated in a batchwise manner. After the quantity of cells has been expanded sufficiently, the final products are administered to the patient in the operating room [3]. In the future, collaboration between surgery and regenerative medicine will require the development of a specific operating room capable of on-site cell processing and graft implantation within reach of a multidisciplinary team including general surgeons and practitioners [2].

5.2

Regulatory System for Regenerative Medicine

Because of the lack of a clear definition concerning regenerative medicine and its uncontrolled implementation from academic to private clinics, it had been difficult to determine the actual conditions and number of regenerative medicines used as therapies, which puts patients at risk. Regenerative products, which consist of processed cells, have specific characteristics due to their biological heterogeneity derived from the original cell donors. These products are unstable, and it is difficult to validate their quality and safety. The inadequacy of strict regulatory systems concerning regenerative medicine has allowed the unlimited employment of these processed cell products from academic hospital to private clinics at physicians’ discretion. As a result, lethal adverse effects, with one patient dying from a pulmonary embolism soon after receiving stem cell-based therapy at a Japanese clinic, have been reported [4]. As such, a regulatory system must be established in order to ensure the safety and efficacy of this new therapy. In the US, the Food and Drug Administration (FDA) is the principal agency with regulatory oversight over regenerative products, including human cells, tissues and cellular, and tissue-­based products (HCT/Ps) [5]. In Europe, these products are defined as advanced therapy medicinal products (ATMPs) and the European Medicines

Agency (EMA), which is the European counterpart of the US FDA, governs the market authorization of ATMPs, encompassing gene-, somatic-, and tissue engineered therapies via a compulsory centralized process valid in all European countries as well as some European Economic Area countries today [6]. In Japan, researchers in regenerative medicine had only one major guideline to follow (“Guidelines on Clinical Research Using Human Stem Cells”) when conducting clinical research, which is not legally binding. In 2013, the Japanese cabinet introduced a new legal framework, known as the Regenerative Medicine Promotion Law. In line with this framework, two acts related to the regulation of cell therapy were launched. One was the Act on the Safety of Regenerative Medicine (Safety Act); the other was the Act on Pharmaceuticals and Medical Devices (PMD Act) [7]. The Safety Act covers all medical technologies that use processed cells and whose safety and efficacy have not yet been established. Medical procedures of established safety and efficacy, such as widely performed organ transplantations, are excluded [7]. If hospitals or private clinics intend to perform regenerative medicine, regardless of the use of stem cells or autologous fat tissue, this Act obliges them to submit plans for the provision of regenerative medicine to the Ministry of Health, Labor and Welfare (MHLW). The certified special committee or certified committee for regenerative medicine then reviews these plans for the provision of regenerative medicine in advance. Under this Act, regenerative medical technologies are categorized into three classes, depending on the potential risk to human health and the safety of the individual technologies [8]. Not only induced pluripotent stem cells (iPSC) and embryonic stem cells but also gene-introduced, xenogeneic, and allogeneic cells are classified into Class 1 [9]. Medical technologies using somatic stem cells are included in Class 2. This class includes successful implantation of cell sheet technology in clinical research for the regeneration of tissues such as articular cartilage [10], cornea [11], myocardium [12], and ­esophagus [13]. Class 3

5  Regenerative Medicine in the Operating Room at Present and in the Near Future

includes cancer immunotherapy using autologous somatic cells; given that there is substantial accumulated clinical experience concerning this therapy, it can be regarded as carrying a low risk. It is very important to notice that this Act applies to not only clinical trials performed by academics but also cell therapies administered in private practice, such as cancer therapies with immune cells and so-called stem cell therapies for cosmetic purposes. The PMD Act covers the production and marketing of cell-based products. This Act defines regenerative medical products as a new category and makes it possible to ensure the timely provision of safe regenerative medicines. If clinical trials confirm the safety of regenerative medical products or phase II clinical studies strongly suggest they are likely be effective, the product is given conditional and time-limited marketing approval. This authorization enables the timely provision of products to patients and expedites the approval of regenerative medical products [9]. This unique approval system is quite in line with global regulatory trends to resolve unmet medical needs. Overseas enterprises may also participate in the new Japanese regulatory system [8].

5.3

Regenerative Medical Products

According to the PMD Act, regenerative medical technologies and regenerative medical products are defined as processed live human/animal cells that are intended to be used for either (1) the reconstruction, repair, or formation of structures or functions of the human body or (2) the treatment or prevention of human diseases. Regenerative medical products also include gene therapy products [7]. They are established through cell processing, which requires process control and laboratory sophistication following current good manufacturing practices (cGMP). The GMP regulations were drafted by the FDA to ensure the manufacturing of drugs, biologicals, and devices of uniform purity, potency, and efficacy. A guiding principle of drug manufacturing is that the prevention of contamination is much

49

simpler than the removal of contaminants. This is especially true for biological products, such as regenerative medical products [14].

5.4

Cell Processing Facility (CPF)

A CPF is a laboratory intended to make biological products through cell processing. Cell processing is defined as procedures related to artificial cell growth, differentiation, immortalization, and activation, among other methods. In contrast, the separation and morcellation of tissues, separation of cells, isolation of specific cells, treatment with antibodies, sterilization by gamma rays, refrigeration, and thawing are not regarded as “processing.” Cell processing should be performed in a clean environment because these products are vulnerable to contamination and difficult to purify. There are variable standards of cleanliness. To indicate the cleanliness in the operating room, FED-STD209 and the NASA standard were used in the past, whereas recently, ISO14644-1 has been used around the world. These standards all classify cleanliness based on the concentration of airborne particles and the potential to minimize the introduction, generation, and retention of particles inside the room. In Japan, the cleanliness of an operating room is indicated in the Japanese version of FGI guidelines for design and construction of hospitals [15], which do not classify air cleanliness based on the concentration of airborne particles as ISO 14644-1, but instead it categorized air cleanliness with efficacy of air filter and ventilation times per hour. In this system, bio-clean operating rooms where the implantation of artificial devices into bodies, such as hip joints, is performed are classified as Class 1, and ordinary operating rooms where general surgeries are performed are classified as Class 2. In contrast, the cleanliness of CPFs is described under the standard of the Japanese pharmacopoeia 17th edition, and this standard classifies cleanliness from Grades A to D: cell processing should be performed in a Grade A environment. It is difficult to compare the cleanliness between an operating room and

K. Kanetaka and S. Eguchi

50 Table 5.1  Various standards of cleanliness Hospital’ design guidelines Class I

FED-­ STD209 100

NASA 100

ISO 14644-1 5

– Class II

1000 10,000

1000 10,000

6 7

Class III

100,000

100,000

8

Class IV

100,000

100,000

–

a CPF correctly. However, the Table  5.1 shows a rough comparison among these standards. In brief, a Grade A environment is almost the same as a bio-clean operating room, which is classified as Class 1 in the hospital design guidelines. ISO 14644-1, a more global standard, classifies bio-­clean rooms as Class 5 and general operating rooms as Class 7. Cell processing should be performed in a Grade A environment and at licensed facilities according to GMP [16]. This regulation makes it difficult for many hospitals to establish new CPFs and hampers the prevalence of regenerative medical product production by CPFs in clinical practice. To expand this field, the Safety Act enables hospitals and clinics to commission cell processing from licensed enterprises. Such facilities should be designed based on appropriate cleanroom design and standards of operation, and the facility should be monitored for its air temperature, humidity, and pressure as well as for particle and microbiota counts. The facility should be equipped with a product receipt area, quarantine area, material storage rooms, and testing and quality control laboratories, classified as Grade C or D. The main body of the facility is the cleanliness control area and sterilization area, where the actual cell manipulation and tissue engineering takes place. The ster-

Standard of the Japanese Operating Pharmacopoeia 17th room Grade A Bio-clean operating room – – Grade B Ordinary operating room Grade C ICU, CCU, etc. Grade D Recovery room General ward, etc.

Cell processing facility Safety cabinet Biological isolator – Sterilization area Cleanliness control area Monitoring unit

ilization area (Grade B) should be located in the cleanliness control area (Grade C or D). These aseptic areas should be entered through airlock doors enclosing an anteroom for gowning. One possible design for a CPF is shown in the Fig. 5.1. This is based in large part on the current design of the Cell Processing Center at Nagasaki University Hospital. The CPF must be well organized and not overly crowed for two fundamental reasons: preventing mix-ups and preventing contamination in every aspect of processing [17]. It is important to carefully design the flow of raw materials, such as cells, tissues, reagents, and supplies, by isolating different process areas. There are two important pieces of equipment necessary in Grade A environments for manufacturing regenerative medicine products: biological safety cabinets and biological isolators. Biological safety cabinets (Fig.  5.2) can reduce the risk of extrinsic contamination by airflow control, although floating particles in the room can be brought in with materials or carried on the hands of workers, so this equipment should be installed in a Grade B sterilization area, and practitioners who enter the room should be forced to wear a clean-suit in order to maintain the cleanliness. Biological isolators (Fig.  5.3) also provide a microbiologically sealed space, and the risk of

Isolator

5  Regenerative Medicine in the Operating Room at Present and in the Near Future

Cleanliness Control Area

51

Grade A Class 100

Grade B Class 10,000 Grade C Class 100,000

Safety Cabinet

Gowning Unit

Grade D Class 100,000

Supply Unit

People Monitoring Unit

Safety Cabinet

Sterilization Area

Materials, Products

Degowning Unit

Fig. 5.1  Drawing of cell processing facility

5.5

Fig. 5.2  Safety cabinet

infectious microorganisms being brought in from the outside is extremely low, as workers perform their duties using gloves fixed to the front wall of the chamber. As this equipment can be installed in a Grade C or D cleanliness control area, running costs and required space can be reduced. However, one disadvantage associated with biological isolators is the low manual operability, due to the need for operators to wear gloves, thus leading to a loss of efficacy [18].

 elationship Between CPFs R and Operating Room

While cell processing is performed in the clean environment of a CPF, harvesting biopsy specimens or tissues from patients or a selected donor and transplanting processed tissue and cells into patients are generally performed in an operating room. Therefore, when using processed products in clinical practice, it is very important to smooth the flow of personnel and products between the CPF and the operating room in order to minimize the risk of contamination or mix-ups. Some reports have noted that operating rooms where regenerative medicine is performed should strictly have an ultraclean nature because these processes are required for the aseptic maneuver of regenerative medical products, therefore they claimed the necessity of Class 100 environment for aseptic process compared with conventional Class 10,000 operating room [2]. However, in the actual clinical setting, there are no regulations dictating what kind of regenerative medicine can be performed in what grade operating room.

52

K. Kanetaka and S. Eguchi

Fig. 5.3 Biological isolator

Therefore, when regenerative medical products are used in the operating room, they are treated based on the standards of cleanliness of the operating room, depending on the type of operation in which these products are to be used. For example, processed chondrocyte transplantation into the knee joint is performed in a bio-clean operating room [10], and myoblast heart sheet transplantation for severe heart failure treatment is performed in a general operating room in which cardiac surgery is typically performed [12]. In contrast to cell products using tissue stem cells, gene transfer products, especially those using virus-vector products, carry a risk of environment exposure and adverse effects on the conservation of biological diversity; therefore, specially equipped operating rooms and additional regulations may be required in the future.

5.5.1 Regenerative Medicine Without CPF Regenerative medicine does not always require a CPF if cell processing is not needed for the creation of the product. Platelet-rich plasma (PRP) therapy is widely employed as an adjuvant modality in different surgical procedures, most frequently in fields of dentistry and orthopedics [19]. PRP can be prepared from autologous whole-blood samples. Blood is sampled from the patient in an operating room containing a centri-

fuge for cell isolation. Following centrifugation of the samples to form a concentration, occasionally with the addition of anticoagulants, coagulation factor is performed. PRP is prepared without cell processing at a CPF and administered to patients at various sites, such as the knee joint, in a bio-clean operating room. An important point to note is that the Safety Act also applies to the use of PRP, and operating rooms that perform minimal manipulations, such as the collection of PRP, are also required to file an application in order to be registered as CPFs.

5.5.2 Regenerative Medicine With CPFs In regenerative medicine using CPFs, the flow of materials can be complex, harvested cells/tissues have to be carried from operating room to CPF for cell processing, and regenerative medical products have to be carried from a facility to an operating room for transplantation and thus are exposed to unclean conditions until arriving at the operating room and CPF. To prevent contamination and mix-up of materials, carefully planning the flow of people and materials between operating rooms and the facility is important. This is a key point for not only guaranteeing the quality of raw materials, such as cells and tissues, but also reducing the mental and physical stress of practitioners who engage in cell processing.

5  Regenerative Medicine in the Operating Room at Present and in the Near Future

Therefore, the CPF should be located adjacent to the operating room, or at least in the operative section of the hospital, if possible. In CPFs, cell processing is performed in a Grade A environment, such as safety cabinets and biological isolators. Safety cabinets are placed in a Grade B room. To enter the room classified as more than Grade B, it is necessary for practitioner to wear gown, clean-suit. It seems to be difficult for practitioners in the Grade B environment to pass the product to a person who carries this from CPF to operating room where these products will then be transplanted. In contrast, biological isolators can be placed in Grade C rooms, where wearing a clean-suit is not required to enter. The critical issue is therefore how to manage the flow of practitioners and products between the CPF and operating room. One resolution is to equip the pass box with a decontamination apparatus in order to allow materials such as culture vessels and containers for cells and medium to pass from the Grade B area to the Grade C area without any additional buffer spaces being required [3]. Receiving regenerative medical products through a pass box between the practitioner in the CPF and the person carrying the product to the operating room would be optimal for facilitating the timely generation of the organ tissue and its safe transfer to the patient at the time of surgery.

5.6

Operating Rooms and Regenerative Medicine in the Future

With the spread of regenerative medicine throughout the surgical field [1], it is important for general surgeons to be aware of regenerative medicine and its related topics, such as regenerative products, the relevant regulatory systems, and the operation of CPFs. The combination of surgery and regenerative products processed in a CPF will provide opportunities for modern surgeons to treat patients with tissue or organ deficits using regenerated products. The harvesting of cells and tissues in the operating room and the cell culture and tissue engineering steps should

53

ideally take place in the same institute. However, hospitals at present are poorly equipped to deal with the rapid scientific advances in tissue engineering, cell processing, and regenerative medicine. To help spread regenerative medicine into common clinical practice, many medical centers active in regenerative medicine are working to design and build CPFs capable of performing tissue engineering in order to generate regenerative medical products using cGMP. However, it is difficult for every hospital to decide to establish their own cGMP facility [17]. Several points must be considered when constructing a new facility. First, a CPF should be prepared using the licensed standard operating procedure (SOP) based on the concepts of GMP [16]. However, construction and maintenance of a cGMP facility is expensive, and it is not easy for many clinicians to accept and learn this new regulatory framework because they have never been educated on it or trained to understand GMP and SOP. Therefore, people with knowledge and experience in cGMP cell processing, regulatory requirements, development and execution of validation programs, and SOP development should be involved [20]. Second, to perform regenerative medicine, a multidisciplinary team working in both the CPF and operating room are needed. The smooth flow of personnel and products between the CPF and operating room is very important for minimizing the risk of contamination and mix-up. One way to resolve this is to ensure that the cGMP facility for an operating room is located in reasonable proximity to the transplant hospital. Sometimes a new facility may be established in a renovated space in an existing hospital building. In such situations, it is necessary to determine whether or not the proposed location is feasible for renovation as a cGMP laboratory [17]. Third, as mentioned above, a CPF is very expensive to run, and many general hospitals cannot afford these costs. As cell therapy products can be easily transported, collaboration with an existing cGMP facility may be more cost-­effective for centers with limited resources [20]. However, sharing a facility should also be

K. Kanetaka and S. Eguchi

54

considered, as processed products are transportable, so collaborating with an existing cGMP facility may be the most cost-effective approach for a given hospital [20]. Our group examined the quality of both harvested human tissues and processed oral cell sheets after transportation by air. Transplantation of an oral cell sheet transferred from Tokyo to Nagasaki (>1200 km) for artificial ulceration after endoscopic submucosal dissection for superficial esophageal cancer effectively prevented stenosis (Fig. 5.4) [21]. Fourth, automated cell processing workstations have been developed that enable cell culture and manipulation. The installation of such an automated processing system for cell and tissue culture enables the automation of machinery operations and maintenance of a closed aseptic environment, thus reducing the risk of contamination and increased stability of the processing. Kino-oka’s group described the Tissue Factory

in which several automated modules are combined to perform various culture processes such as manufacturing of multilayered skeletal myoblast sheets. They indicated improvement of manufacturability in terms of lower production cost, improved quality variance, and reduced contamination risks [18]. Finally, regarding operating rooms in the future, the advent of tissue engineering technology will make it possible to create whole organs. Generating larger tissues will require the induction of vessel systems in these tissues as feeder vessels. In addition, for the transplantation of these tissues into a body, anastomosis between graft vessels and recipient vessels should be performed as microsurgery. The operating room should therefore be equipped with a microscope and imaging-assisted navigation systems [21]. To monitor the transplanted tissues and organs, monitoring systems should be installed, such as super-high-resolution computed

Oral mucosa

Hospital 3. Cell Culture

2. Transportation (Oral mucosa and serum) 1. Biopsy of oral mucosa

Oral mucosal epithelial cell sheet

Serum preparation

Cell Culture Facility 4. Transportation (Oral mucosal epithelial cell sheet) 5. Transportation

Fig. 5.4  Airplane transportation of oral epithelial cell sheets [21] (https://doi.org/10.1038/s41598-017-17663-w)

5  Regenerative Medicine in the Operating Room at Present and in the Near Future

tomography and magnetic resonance imaging or bioluminescence capable of visualizing cell and tissue labeling with dyes and nanomarkers, or ICG fluorescence to detect the blood flow [2].

References 1. Orlando G, et al. Regenerative medicine as applied to general surgery. Ann Surg. 2012;255(5):867–80. 2. Fishman JM, et  al. Operating RegenMed: development of better in-theater strategies for handling tissue-engineered organs and tissues. Regen Med. 2014;9(6):785–91. 3. Kino-Oka M, Taya M.  Recent developments in processing systems for cell and tissue cultures toward therapeutic application. J Biosci Bioeng. 2009;108(4):267–76. 4. Cyranoski D.  Korean deaths spark inquiry. Nature. 2010;468(7323):485. 5. Burger SR. Current regulatory issues in cell and tissue therapy. Cytotherapy. 2003;5(4):289–98. 6. De Sousa PA, et al. Development and production of good manufacturing practice grade human embryonic stem cell lines as source material for clinical application. Stem Cell Res. 2016;17(2):379–90. 7. Hara A, Sato D, Sahara Y.  New governmental regulatory system for stem cell-based therapies in Japan. Ther Innov Regul Sci. 2014;48(6):681–8. 8. Konomi K, et al. New Japanese initiatives on stem cell therapies. Cell Stem Cell. 2015;16(4):350–5. 9. Houkin K, et al. Accelerating cell therapy for stroke in Japan: regulatory framework and guidelines on development of cell-based products. Stroke. 2018;49(4):e145–52. 10. Sato M, et  al. Combined surgery and chondrocyte cell-sheet transplantation improves clinical and struc-

55

tural outcomes in knee osteoarthritis. NPJ Regen Med. 2019;4:4. 11. Nishida K, et  al. Corneal reconstruction with tissue-­ engineered cell sheets composed of autologous oral mucosal epithelium. N Engl J Med. 2004;351(12):1187–96. 12. Sawa Y, et al. Safety and efficacy of autologous skeletal myoblast sheets (TCD-51073) for the treatment of severe chronic heart failure due to ischemic heart disease. Circ J. 2015;79(5):991–9. 13. Ohki T, et  al. Prevention of esophageal stricture after endoscopic submucosal dissection using tissue-engineered cell sheets. Gastroenterology. 2012;143(3):582–588 e2. 14. Rowley SD.  Current good manufacturing practices: application to the processing of hematopoietic cell components. Cytotherapy. 2000;2(1):59–62. 15. The Facility Guidelines Institute. Guidelines for design and construction of hospitals. 2018. www. fgiguidelines.org/guidelines/2018-fgi-guidelines/. 16. Kawase T, Okuda K. Comprehensive quality control of the regenerative therapy using platelet concentrates: the current situation and prospects in Japan. Biomed Res Int. 2018;2018:6389157. 17. Burger SR.  Design and operation of a current good manufacturing practices cell-engineering laboratory. Cytotherapy. 2000;2(2):111–22. 18. Kikuchi T, et al. A novel, flexible and automated manufacturing facility for cell-based health care products: tissue factory. Regen Ther. 2018;9:89–99. 19. Marx RE. Platelet-rich plasma: evidence to support its use. J Oral Maxillofac Surg. 2004;62(4):489–96. 20. Arjmand B, et al. The implementation of tissue banking experiences for setting up a cGMP cell manufacturing facility. Cell Tissue Bank. 2012;13(4):587–96. 21. Yamaguchi N, et al. Oral epithelial cell sheets engraftment for esophageal strictures after endoscopic submucosal dissection of squamous cell carcinoma and airplane transportation. Sci Rep. 2017;7(1)):17460.

6

Surgery and Operating Room for Restoring Organs: Organ Regeneration by Tissue Engineering in the Near Future Mitsuo Miyazawa, Masato Watanabe, Yoshihisa Naito, Yasumitsu Hirano, Keizo Taniguchi, Takehiro Okumura, Kaname Maruno, and Shozo Fujino

6.1

Introduction

In the twenty-first century, endoscopic [1] and robotic [2] surgery have been performed, and minimally invasive surgery that minimizes body wall destruction has been pursued. However, in the body cavity, surgery for sewing the defect portion after removal of the diseased portion has been performed, and functional minimally invasiveness has not been acquired. For example, even in the early stages of gastric cancer, extensive gastrectomy is required because the site occupied by the stomach cancer spreads; following surgery, patients often experience a decline in appetite and a reduced quality of life [3]. For surgery to be functional and minimally invasive, it is important that there are no organ defects. Our research interests have focused on minimally invasive organ reconstruction and the restoration of organs by tissue engineering (TE) [4–8]. In the present study, we describe possible methods for the restoration of organs and the req-

M. Miyazawa (*) · M. Watanabe · Y. Naito Y. Hirano · K. Taniguchi · T. Okumura · K. Maruno S. Fujino Department of Surgery, Teikyo University Mizonokuchi Hospital, Takatsu-ku, Kawasaki, Japan e-mail: [email protected]

uisite changes to operating rooms that will enable them, based on the results of our research.

6.2

 rgan Restoration by Tissue O Engineering (TE)

For surgery to be functional and minimally invasive, tissues and organs that retain their original function must be regenerated in the resected defect. Only TE provides a scaffold for the restoration of organ defects. Professor Shinya Yamanaka of Kyoto University has prepared iPS cells [9], and various attempts have been made to induce differentiation in cells in various gastrointestinal areas. It is conceivable that these differentiated cells may be useful in TE, but the tissues have scaffolds with fine and elaborate three-­dimensional structures, and various cell aggregates maintain their functions. Therefore, although cells are only injected, it is currently impossible to determine if cell groups regenerate the extracellular matrix—which is a building element of organs—and migrate to the right place. Even when immature cells are injected into a three-dimensional structure, it is unclear how to control the cellular environment, and it is unlikely that the intended organ can be regenerated.

© Springer Nature Singapore Pte Ltd. 2021 S. Takenoshita, H. Yasuhara (eds.), Surgery and Operating Room Innovation, https://doi.org/10.1007/978-981-15-8979-9_6

57

M. Miyazawa et al.

58

Currently, in the context of actual clinical ­medicine, the most important thing for achieving the clinical application of regenerative medicine is to consider suturing and the regeneration of existing organs within the structure of the body. This requires the arrangement of the environment in such a manner that it can be regenerated (if the organ is missing, restore it by sewing and fixing the scaffold to that part). Three elements are required for TE: (1) cells with the correct tissue phenotype, (2) scaffolds to which cells adhere, and (3) the environment for cells to survive [10]. Research into cells expressing tissue function, including iPS cells, has accelerated. Furthermore, 3D printing technology enables the fabrication of extracellular and artificial matrices as scaffolds, including nano-level three-dimensional microstructures [11]. However, the most important and neglected field of research with regard to tissue regeneration is the study of the environment in which cells express their phenotypes as tissues among three-­ dimensional structures. This is particularly relevant to the regeneration of gastrointestinal tissues and affects complexes regenerated by various environments such as complicated vascular networks, organ circulation, peristaltic movement, the flow of digestive juices, neuronal arrangement, protein synthesis, and immune function. At present, it is difficult to reproduce all these environments in vitro. Tissue regeneration by seeding cells in a scaffold close to the actual skeleton of the tissue to imitate that tissue is considered to be a shortcut. Therefore, it is necessary to consider the kind of scaffold appropriate for the cells that constitute the tissue and the extent to which it is necessary to imitate factors other than cells. Thus, there are still many problems to be clarified at present.

6.3

 rgan Restoration Using O Bioabsorbable Material

After resecting diseased tissue, it is necessary to restore organs with minimal disruption. Therefore, we researched organ reconstruction

and applied TE techniques using bioabsorbable materials. Herein, we briefly present our research on biliary [4], gastric [8], and esophageal [7] regeneration.

6.3.1 Biliary Regeneration Extrahepatic bile ducts fulfill various functions including immune functions. However, the most basic function of bile ducts is the transport of bile from the liver to the duodenum. Several attempts have been made to imitate this function. However, these attempts have failed because they have not provided substitutes for the epithelium on the luminal side of the bile duct, or on the anastomotic part of the native bile duct that could be used in the long term. Following recent developments in TE, we attempted to regenerate a bile duct using a tube comprising a bioabsorbable polymer (artificial bile duct: ABD). More particularly, we prepared a substitute bile duct from a tube-shaped scaffold based on a bioabsorbable polymer and transplanted it into an extrahepatic bile duct. This scaffold comprised a copolymer of lactic acid and caprolactone reinforced with fibers of polyglycolic acid and was designed to have an in  vivo absorption period of 6 to 8 weeks [12]. We used a hybrid pig as a recipient for the ABD. After cutting the common bile duct near the junction of the cystic duct, we ligated the duodenum side of the common bile duct and anastomosed the native common bile duct stump on the liver to the ABD. Next, we drilled a 5-mm hole in the descending leg of the duodenum and sutured the stump end of the ABD to the duodenum hole (Fig. 6.1a). The ABD graft was obtained 6  months after transplantation and was monitored macroscopically and histologically. As a result, the recipient pig of the transplant showed no signs of jaundice, increased in weight, and survived until sacrifice (6  months after transplantation). The neo-bile duct had almost the same morphology as the native common bile duct, both macroscopically and histologically (Fig. 6.1b). The portion considered to comprise bile duct epithelial cells and

6  Surgery and Operating Room for Restoring Organs: Organ Regeneration by Tissue Engineering…

a

b

Fig. 6.1 Biliary regeneration by artificial bile duct (ABD) (modified quotation from [2]). (a) An ABD was used to bypass the liver side bile duct and the duodenum. (b) Regenerated bile duct 6 months after transplantation of artificial bile duct; a bile duct similar to the native bile duct was observed macroscopically in the ABD graft (arrows)

the native bile duct were both positive for cytokeratin-19 (CK 19). These results suggest that in the early stages of transplantation, the ABD used to replace the extrahepatic bile duct can deliver bile to the duodenum without leaking bile into the abdominal cavity, while retaining its tubular shape. Furthermore, after the ABD had degraded and been absorbed in  vivo, the extrahepatic bile duct regenerated at the transplant site and functioned as a native bile duct without stenosis [8]. We have previously demonstrated that it is possible to repair damaged areas of the extrahepatic bile duct using patches comprising the same polymer used in the present study [7] and that a cyclic extrahepatic bile duct of a certain length is native. It is possible to regenerate the connection between the bile ducts [13].

59

6.3.2 Gastric Regeneration Depending on the resection site, closing a suture directly causes deformity and stenosis owing to the morphological features of the stomach. Even if the majority of the stomach can be preserved, the patient’s quality of life is markedly reduced following the operation. Therefore, we began developing a novel treatment regimen by which gastric wall defects can be repaired by substituting stomach walls, minimizing deformation and stenosis without excessive gastrectomy, and preserving biological functions. We created an artificial gastric wall (AGW) from a bioabsorbable polymer sheet comprising the same material used to regenerate the bile duct and transplanted it into a wide range of stomach wall defects. We then monitored the regeneration of the stomach wall. We laparotomized hybrid pigs under general anesthesia, excised 8 × 8 cm of the anterior wall of the middle part of the stomach (approximately one-third of the periphery of the porcine stomach), implanted a patch-shaped AGW of the same size, and closed the incision (Fig. 6.2a). An upper gastrointestinal endoscopy was performed 1 week after transplantation. We removed entire stomachs 6 and 12  weeks after transplantation and examined the transplanted parts of the stomachs macroscopically and histologically. Each pig survived until sacrifice without any reduction in its dietary intake. Five weeks after transplantation, we found that the transplanted areas had formed approximately 2-cm-square ulcers, characterized by a proliferation of connective tissue and infiltrated inflammatory cells. Twelve weeks after transplantation, the thickness of the transplanted gastric wall was macroscopically small, but there was no deformation, and both the serosal and mucosal surfaces were similar to those of the native stomach (Fig.  6.2b). From an histological perspective, 12 weeks after transplantation, the mucosa and submucosa both resembled the native tissue. We proved that it is possible to regenerate the stomach using the AGW, even following resection to treat gastric cancer or the like, and the substitute retained the original shape of the stomach [8].

M. Miyazawa et al.

60

a

a b

b

Fig. 6.2  Gastric regeneration by artificial stomach wall (AGW) (modified quotation from [3]) (a) An AGW (8 × 8 cm) comprising a bioabsorbable sheet was transplanted to the anterior wall of the stomach. (b) 12 weeks after AGW implantation; the mucosal surface of the implanted AGW was indistinguishable from the native stomach (arrows)

6.3.3 Esophageal Regeneration Owing to the anatomical features of the esophagus, even if a defect occurs in a part of the wall, a large invasive operation such as subtotal esophagectomy is required [14]. To avoid such large invasions, it is desirable to develop a material that can repair the esophageal defect. Esophageal wall substitute materials have been reported, but they are imperfect and have not yet been used for clinical applications. We attempted to verify whether it is possible to regenerate esophageal defects without narrowing using the same bioabsorbable polymer as that used for the bile duct

Fig. 6.3  Esophageal regeneration by artificial esophageal wall (AEW) (modified quotation from [4]) (a) transplanted to the a: 4 × 2 cm large AEW was esophagus. (b) 12 weeks after AEW transplantation: the mucosal surface was regenerating mucous membrane that was indistinguishable from the native esophagus (arrows)

and stomach. Therefore, we implanted an artificial esophageal wall (AEW) comprising the polymer sheet into the defective esophagus and monitored the esophageal regeneration process. We thoracotomized pigs under general anesthesia, cut the AEW into ellipses with major axes of 4 cm and minor axes of 2 cm, and then closed the defects by implanting the ellipses, which were the same size as the defective tissues (Fig. 6.3a). Oral ingestion comprising a liquid meal was begun the day after surgery, and on the fifth day, it was consumed as a normal meal. We performed an upper gastrointestinal endoscopy 2  weeks after AEW transplantation, and examined the graft site. After 4 weeks, 8 weeks, and 12 weeks, we subdivided the chests and esophagi of the pigs, and examined the AEW transplant sites macroscopically and histologically. Subsequently, all the pigs survived until sacrifice without significant weight loss. Two weeks after transplantation, the AEWs exhibited granulation sites of approximately

6  Surgery and Operating Room for Restoring Organs: Organ Regeneration by Tissue Engineering… Fig. 6.4  In CPCs, complexes of cells and a scaffold for the intended organ reconstruction are made

Tissue Engineering

• Cells with the correct tissue phenotype

Surgery

• Tissue regeneration using a scaffold

• Scaffolds appropriate for cells

61

• Resection and restore of key organs

Operating Room

CPCs

1 cm in diameter, but there was no stenosis. Four weeks after AEW implantation, we observed generation of the same squamous epithelium as the normal mucosa, but the muscular layer had not regenerated. At 8  weeks, a rough muscular layer appeared (Fig.  6.3b). After 12  weeks, the muscular layer resembled a normal muscle layer, and an esophageal wall that appeared to be similar to the native wall began to regenerate. Our results suggest that the patch-like polymer-based artificial esophageal implant is capable of repairing and regenerating all layers of the esophageal wall without stenosis, and the AEW may be used for minimally invasive therapy [7].

Conflicts of Interest  We do not have any financial interests to disclose in this study.

6.4

References

Establishment of Cell Processing Centers (CPCs)

Cells are required for the restoration of organs by TE.  The body considers bioabsorbable material to be foreign matter, so it is possible to use cells that have migrated in response to such foreign materials. However, in organs in which cells such as bone [15] and cartilage [16] migrate with difficulty, cells prepared outside the body must be transplanted into the body. In such cases, CPCs for preparing cells become necessary. That is, CPCs produce complexes comprising adherent cells and bioabsorbable scaffolds for transplantation into the body. CPCs should be located in the vicinity of the operating room to reduce the risk of infection, prevent changes to cells or bioabsorbable materials, and be ready to respond to emergencies [17, 18] (Fig. 6.4).

6.5

Conclusion

In view of recent developments in surgical procedures, in the near future, surgery should be minimally invasive, after resection of the diseased tissue, and should continue with the restoration of the defective organ. To enable such treatment, the further development of TE and facilities that integrate operating rooms and CPCs are required.

1. Jeong GA, Cho GS, Kim HH, Lee HJ, Ryu SW, Song KY. Laparoscopy-assisted total gastrectomy for gastric cancer: a multicenter retrospective analysis. Surgery. 2009;146:469–74. 2. Berlinger NT. Robotic surgery—squeezing into tight places. N Engl J Med. 2006;354:2099–101. 3. Takahashi M, Terashima M, Kawahira H, Nagai E, Uenosono Y, Kinami S, et al. Quality of life after total vs distal gastrectomy with roux-en-Y reconstruction: use of the postgastrectomy syndrome assessment scale-45. World J Gastroenterol. 2017;23:2068–76. 4. Miyazawa M, Torii T, Toshimitsu Y, Okada K, Koyama I, Ikada Y. A tissue-engineered artificial bile duct grown to resemble the native bile duct. Am J Transplant. 2005;5:1541–7. 5. Toshimitsu Y, Miyazawa M, Torii T, Koyama I, Ikada Y.  Tissue-engineered patch for the reconstruction of inferior vena cava during living-donor liver transplantation. J Gastrointest Surg. 2005;9:789–93. 6. Aikawa M, Miyazawa M, Okamoto K, Toshimitsu Y, Torii T, Okada K, et al. A novel treatment for bile duct

62 injury with a tissue-engineered bioabsorbable polymer patch. Surgery. 2010;147:575–80. 7. Aikawa M, Miyazawa M, Okamoto K, Okada K, Akimoto N, Sato H, et  al. A bioabsorbable polymer patch for the treatment of esophageal defect in a porcine model. J Gastroenterol. 2013;48:822–9. 8. Miyazawa M, Aikawa M, Watanabe Y, Takase K, Okamoto K, Shrestha S, et al. Extensive regeneration of the stomach using bioabsorbable polymer sheets. Surgery. 2015;158:1283–90. 9. Takahashi K, Yamanaka S.  Induction of pluripotent stem cells from mouse embryonic and adult fibroblast cultures by defined factors. Cell. 2006;25(126):663–76. 10. Langer R, Vacanti JP.  Tissue engineering. Science. 1993;260:920–6. 11. Gaetani R, Doevendans PA, Metz CH, Alblas J, Messina E, Giacomello A, et  al. Cardiac tissue engineering using tissue printing technology and human cardiac progenitor cells. Biomaterials. 2012;33:1782–90. 12. Watanabe M, Shin'oka T, Tohyama S, Hibino N, Konuma T, Matsumura G, et  al. Tissue-engineered vascular autograft: inferior vena cava replacement in a dog model. Tissue Eng. 2001;7:429–39. 13. Aikawa M, Miyazawa M, Okamoto K, Toshimitsu Y, Okada K, Akimoto N, et  al. An extrahepatic bile

M. Miyazawa et al. duct grafting using a bioabsorbable polymer tube. J Gastrointest Surg. 2012;16:529–34. 14. Saddoughi SA, Reinersman JM, Zhukov YO, Taswell J, Mara K, Harmsen SW, et al. Survival after surgical resection of stage IV esophageal Cancer. Ann Thorac Surg. 2017;103:261–6. 15. Chen M, Xu Y, Zhang T, Ma Y, Liu J, Yuan B, et  al. Mesenchymal stem cell sheets: a new cellbased strategy for bone repair and regeneration. Biotechnol Lett. 2019;41:305. https://doi.org/10.1007/ s10529-019-02649-7. 16. Kobayashi T, Kan K, Nishida K, Yamato M, Okano T.  Corneal regeneration by transplantation of corneal epithelial cell sheets fabricated with automated cell culture system in rabbit model. Biomaterials. 2013;34:9010–7. 17. Ram S, Lewis LA, Rice PA.  Infections of people with complement deficiencies and patients who have undergone splenectomy. Clin Microbiol Rev. 2010;23:740–80. 18. Ghodsizad A, Klein HM, Borowski A, Stoldt V, Feifel N, Voelkel T, et al. Intraoperative isolation and processing of BM-derived stem cells. Cytotherapy. 2004;6:523–6.

Part III Artificial Intelligence and Virtual Reality

7

Extended Reality (XR:VR/AR/MR), 3D Printing, Holography, A.I., Radiomics, and Online VR Tele-­ Medicine for Precision Surgery Maki Sugimoto

7.1

Introduction

Precision surgery aims to customize surgery for individual patients by tailoring therapeutic decisions. Surgical navigation has become essential for surgeons to accurately and safely perform precision surgery. The 3D organ modeling based on individual patient’s medical image data is indispensable for simulation, navigation, and education for accurate and safe surgery. Thus, radiomics is a crucial role in systematically scientizing the enormous amount of radiological image information. It is a pattern analysis of the features and findings of medical image data such as CT and MRI, which is learned mechanically to generate and store inference models. These features are extracted from the unknown data, and pathological conditions and differential diseases are estimated. The important role here is that, accurately extract organ and tumor shapes, converting the coordinates of a three-dimensional space as polygons, which is difficult to understand within a flat monitor, accurately measure the actual space coordinates of the surgical field, and integrate these in a real environment. Furthermore, polygon data in the shape of organs has already been put into practical use for surgical simulation and M. Sugimoto (*) Innovation Lab, Teikyo University Okinaga Research Institute, Tokyo, Japan e-mail: [email protected]; [email protected]

navigation using extended reality (XR), including virtual reality (VR), augmented reality (AR), mixed reality (MR), or 3D-printing technology. Machine learning with artificial intelligence may improve the accuracy of organ extraction. VR is a simulated experience that can be similar to or completely different from the real world. With VR, you are immersing yourself into a virtual environment and closing yourself off completely from the outside world. AR is an interactive experience of a real-world environment where the objects that reside in the real world are enhanced by computer-generated perceptual information. AR is overlaying a digital layer of contextual information into the built environment. With augmented reality, data and/or instructional information are animated over the real-world view, often through smaller devices such as a mobile phone or tablet. MR is mixing aspects of VR and AR, takes virtual objects and overlays them onto the real world. More than two people can be networked into a virtual world where they can interact together with a virtual building on a real site.

7.2

Methods

We have developed an application that digitally analyzes CT and MRI data of individual patients, automatically extracts feature points of the organ shape with artificial intelligence (AI), converts

© Springer Nature Singapore Pte Ltd. 2021 S. Takenoshita, H. Yasuhara (eds.), Surgery and Operating Room Innovation, https://doi.org/10.1007/978-981-15-8979-9_7

65

M. Sugimoto

66

them into polygons, and views these organ shape coordinates as VR [1]. Then, using the mixed reality technology, the coordinates of the organ shape and the position of the real space calculated by the position sensor were integrated and displayed in the real world using a wearable holographic spatial computer. We have practically used these XR for simulation of surgery, support for treatment, surgical training, and medical education (Fig. 7.1). Based on modeling technique using these polygon data as mentioned above, we developed a 3D-printing surgical simulation and navigation system using anatomically accurate bio-elastic wet hemorrhagic organ replica from MDCT data of a patient’s cancer lesion, skin, bones, blood

vessels, and abdominal organs. Our hybrid 3D imaging and 3D-printed injection molding technology allowed to manufacturing bio-elastic abdominal wall replica. Based on patient-specific DICOM data from MDCT, after generating its surface polygons using OsiriX application, the multi-material inkjet 3D printer created life-size copies of the 3D organs such as liver, biliary system, pancreas, blood vessels, fat, and abdominal cavity. The bio-elastic replicas were manufactured by simultaneous jetting of different types of materials and injection molding the polyvinyl alcohol (PVA) and water. Each organ’s mold model was given an injection of a synthetic resin that helps make it feel wetter and more lifelike for the surgeon.

XR:EXtended reality

Radionics Image acquisition

ROI segmentation

VR (Virtual reality) immersingg yourself in a completely artifial world

AR (Augmented reality) overlaying a digital layer of contextual information into the bult environment

MR (Mixed reality) an interactive mix of VR and AR

Online VR tele-medicine

3D modeling

Polygon (.stl .obj)

Fig. 7.1  Radiomics and Extended reality. Region of Interest (ROI): A selected subset of samples within a dataset identified for a particular purpose. Particularly abdominal organs can be segmented as polygonal selections from several 2D CT images. Polygon (.stl, .obj): 3D

model format OBJ is a geometry definition file format, which is open and has been adopted by many 3D graphics application vendors. STL is also a file format native to the stereolithography CAD software created by 3D systems

7  Extended Reality (XR:VR/AR/MR), 3D Printing, Holography, A.I., Radiomics, and Online VR…

We programed a printer to create clear models made from acrylic resins that allowed us to ­visualize and understand the complex internal organ structures and blood vessels or the exact tumor locations.

7.3

Results

7.3.1 VR In VR, the position of the organ and the user’s gaze were corrected by three-dimensional stereoscopic vision using a position sensor and displayed on the VR headset as if it were always in stereoscopic space. When this was linked to the movement of the surgeon, the immersive sense for understanding the patient’s anatomy improved dramatically, and three factors were improved: three-dimensional spatiality, real-time interaction, and self-projection. Particularly, VR for tele-medicine enabled a more intuitive mode of interacting with information and as a flexible environment that enhances the feeling of physical presence during the interaction (Figs. 7.2 and 7.3).

Fig. 7.2  Online VR tele-medicine (real world)

67

7.3.2 AR In AR, we developed a smart application that integrated the position information of the real space and displayed the organ polygons superimposed on the real world using a position sensor and a 3D camera within a smart device. When the surgeon moved, the organs could always be presented and stayed in the air linked to their movement.

7.3.3 MR In spatial holography by MR, 3D data was presented to a translucent wearable glass as a guide to support organs, surgical devices, and procedures. Using multiple sensors that measure the position and the movement inside the wearable glass, the operators could manipulate, rotate, and zoom in and out by their gestures when looking at the organs (Fig. 7.4). During surgery, using a transparent holographic wearable glasses with built-in IR position sensors (HoloLens and Magic Leap One), surgeons could watch the floating organ models beneath the surgical field.

68

M. Sugimoto

Fig. 7.3  Online VR tele-medicine (virtual world)

Fig. 7.4 Holographic XR for surgical planning and simulation

This is very effective in a sterilized environment, and even during surgery, the hologram of the organs could be confirmed from all directions in the air of the operative field (Fig. 7.5). The location information of multiple MR devices was shared under the same Wi-Fi reception and multiple surgeons could view the same anatomy in the same air, improving their communication (Figs. 7.6 and 7.7).

7.3.4 Bio-elastic Wet 3D-Printing Model The bio-elastic wet 3D printing allowed these models realistic stand-in for ultrasonic diagnosis, intervention, and surgical procedure such as cutting, suturing, and ligation. The personalized bio-­ elastic wet organ replicas were useful for visible and tangible surgical simulation and navigation

7  Extended Reality (XR:VR/AR/MR), 3D Printing, Holography, A.I., Radiomics, and Online VR…

to plan and guide the successful surgeries. With the wet model, surgeons can experience the softness of organs and see them bleed, to help us in

practice on lifelike models before stepping into real surgery. Using abdominal cavity replica, there was a place for using synthetic models in realistic surgical situation including laparoscopic and robotic surgeries (Fig. 7.8).

7.4

Fig. 7.5  XR for daVinci robotic surgery

Fig. 7.6  Holographic XR for liver cancer surgery

Fig. 7.7 Holographic XR for pelvic cancer surgery

69

Discussion

These results are expected to share the technical gap between surgeons by formalizing the tacit knowledge of surgical techniques [2–4]. These holographic organ models were able to share and move freely in all directions by gesture interface, and complex procedures could be confirmed with pointing by all surgeons. The ability to spatial awareness for understanding the extent of resection, blood vessel processing, and lymph node dissection was improved during surgery. Our XR navigation system has high accuracy and stability for registration. The use of 3D-printed replicas for guiding surgery reduced the length of the operation and provided better anatomical reference tools for tailor-made navigation surgery, consequently helping to improve training for the operating room staff, students, and trainees. These could overcome the limitations of the conventional image-guided surgical navigation. It combines the advantages of conventional 3D modeling and precise virtual 3D planning in personalized surgical simulation and navigation [5–7].

M. Sugimoto

70

Fig. 7.8  Bio-elastic wet organ replicas related by 3D printing and hybrid molding

These could further develop “personalized medicine” into “precision medicine,” which selects individual analysis and establishes the optimal treatment for each specific population.

7.5

Conclusion

Our patient-specific XR surgical navigation is highly effective and reduce surgical time, blood loss, and adverse event. This system must have value for future surgeons. Acknowledgments This work was supported by JSPS KAKENHI Grant Numbers JP 19152948, JP 19167615.

References 1. Holoeyes Inc. http://holoeyes.jp/en. Accessed 27 Jan 2020. 2. Sugimoto M. Augmented tangibility surgical navigation using spatial interactive 3-D Hologram zSpace with OsiriX and Bio-Texture 3-D Organ Modeling In: IEEE 2015 international conference on com-

puter application technologies (CCATS 2015); 2015. p. 189–194. 3. Saito Y, Sugimoto M, Imura S, Morine Y, Ikemoto T, Iwahashi S, Yamada S, Shimada M. Intraoperative 3D hologram support with mixed reality techniques in liver surgery. Ann Surg. 2020;271(1):e4–7. 4. Yamada T, Osako M, Uchimuro T, Yoon R, Morikawa T, Sugimoto M, Suda H, Shimizu H.  Three-­ dimensional printing of life-like models for simulation and training of minimally invasive cardiac surgery. Innovations. 2017;12(6):459–65. 5. Soejima Y, Taguchi T, Sugimoto M, Hayashida M, Yoshizumi T, Ikegami T, Uchiyama H, Shirabe K, Maehara Y.  Three-dimensional printing and biotexture modeling for preoperative simulation in living donor liver transplantation for small infants. Liver Transpl. 2016;22(11):1610–4. 6. Komai Y, Sugimoto M, Gotohda N, Matsubara N, Kobayashi T, Sakai Y, Shiga Y, Saito N.  Patient-­ specific 3D printed kidney designed for “4D” surgical navigation-a novel aid to facilitate minimally invasive off-clamp partial nephrectomy in complex tumor cases. Urology. 2016;91:226–33. 7. Kusaka M, Sugimoto M, Fukami N, Sasaki H, Takenaka M, Anraku T, Ito T, Kenmochi T, Shiroki R, Hoshinaga K. Initial experience with a tailor-made simulation and navigation program using a 3-D printer model of kidney transplantation surgery. Transplant Proc. 2015;47(3):596–9.

8

Application of AI in Endoscopic Surgical Operations Norihito Wada and Yuko Kitagawa

Abbreviation FESS Flexible endoscopic surgery system

8.1

Preface

Artificial intelligence (AI) is already part of our everyday life through our smartphones and cars. Nowadays, facial recognition and autonomous driving are common applications of AI.  It has now gone on to invade the field of medicine, and many studies showed promising applications of AI in radiology, pathology, endoscopy, and so on. Artificial intelligence (AI) has been a disruptive innovation in all areas of medicine including surgery. Not only has it been applied for research purpose, but also AI can already provide significant solutions in clinical settings. Attention is now turning to the potential of AI in the field of endoscopic surgery. It is essential to understand how machine learning (ML) and deep learning (DL) work, understand current applications in surgery, and prepare for the future of surgery in the era of AI.

N. Wada (*) · Y. Kitagawa Department of Surgery, Keio University School of Medicine, Tokyo, Japan e-mail: [email protected]

In the fields of medicine that rely mainly on pattern recognition, such as endoscopy and radiology, have evolved artificial intelligence models that have become more accurate than human decisions. To date, usefulness of artificial intelligence in the endoscopic diagnosis of gastrointestinal lesions is reported from many institutions. Progress in computing technology including the use of graphical processing units (GPUs) for parallel processing, the accessibility of types of movie data, and the resurgence of interest in neural networks, and other ML approaches has led to recent advances in AI application in endoscopic surgery. In this chapter, we present the current and future applications of AI in the field of endoscopic surgery by highlighting following topics in order: (1) diagnosis, (2) navigation, (3) decision support, (4) preoperative prediction, (5) education, (6) video recognition, and (7) autonomous surgery.

8.2

 opics of AI Application T in Endoscopic Surgery

8.2.1 Diagnostic Support by AI Diagnostic support by AI is actively investigated in the field of endoscopy. Hirasawa et  al. [1] applied AI using a convolutional neural network for detecting gastric cancer in endoscopic

© Springer Nature Singapore Pte Ltd. 2021 S. Takenoshita, H. Yasuhara (eds.), Surgery and Operating Room Innovation, https://doi.org/10.1007/978-981-15-8979-9_8

71

72

images. They developed a convolutional neural network (CNN)-based diagnostic system based on Single Shot MultiBox Detector architecture using 13,584 endoscopic images of gastric cancer as supervised learning dataset. The validation data of 2296 stomach images collected from 69 patients with 77 gastric cancer lesions were analyzed with the system. Within 47 s, the system correctly diagnosed 71 of 77 gastric cancer lesions with an overall sensitivity of 92.2%, and 161 noncancerous lesions were detected as ­gastric cancer, resulting in a positive predictive value of 30.6%. Seventy of the 71 lesions (98.6%) with a diameter of 6 mm or more as well as all invasive cancers were correctly detected. More recently, Luo et  al. conducted a multicenter, case-control, diagnostic study to confirm the validation of real-time artificial intelligence for detection of upper gastrointestinal cancer by endoscopy. A total of 1,036,496 endoscopy images from 84,424 individuals were used to develop and test the diagnostic system. The diagnostic accuracy in identifying upper gastrointestinal cancers was 95.5% in the internal validation set, 92.7% in the prospective set, and ranged from 91.5% to 97.7% in the five external validation sets. This system achieved diagnostic sensitivity (94.2%) similar to that of the expert endoscopist (94.5%) and superior sensitivity compared with competent (85.8%) and trainee (72.2%) endoscopists. The positive predictive value was 81.4% for the AI system, 93.2% for the expert endoscopist, 97.4% for the competent endoscopist, and 82.4% for the trainee endoscopist. The negative predictive value was 97.8 for the system, 98.0% for the expert endoscopist, 95.1% for the competent endoscopist, and 90.4% for the trainee endoscopist. The system has potential to assist community-based hospitals in improving their effectiveness in upper gastrointestinal cancer diagnoses. Diagnostic support by AI accomplished a level of expert accuracy. In the field of surgery, Reismann et  al. [2] presented a method for automatic diagnosis of appendicitis as well as the differentiation between complicated and uncomplicated inflammation using values/parameters which

N. Wada and Y. Kitagawa

are routinely and unbiasedly obtained for each patient with suspected appendicitis. A total of 590 patients were analyzed retrospectively with modern algorithms from ML and AI.  A supervised learning algorithm was used to analyze: the data of 35% of the patients were used for discovery and the remaining 65% were used for validation. The accuracy of the biomarker signature for diagnosis of appendicitis was 90% (93% sensitivity, 67% specificity), while the accuracy to correctly identify complicated inflammation was 51% (95% sensitivity, 33% specificity) on validation data. Such a test would be capable to prevent two out of three patients without appendicitis from useless surgery as well as one out of three patients with uncomplicated appendicitis. The presented method has the potential to change today’s therapeutic approach for appendicitis and demonstrates the capability of algorithms from AI and ML to significantly improve diagnostics even based on routine diagnostic parameters. AI-supported diagnosis is well implemented and generally accepted as a tool to improve the quality and efficiency of patient care in the fields of endoscopy and surgery.

8.2.2 AI-Assisted Navigation in Endoscopic Surgery Recent advances in endoscopic navigation are reviewed in detail by Luo et  al. [3]. In image-­ guided endoscopic surgery, the guidance of an instrument toward a desired target is typically defined as surgical navigation. It provides accurate real-time positioning of in  vivo anatomical structures and organs as well as surgical instruments overlaid on preoperative images in the operating room. There are two ways of tracking in surgical navigation (Fig.  8.1). Vision-based tracking is a navigation method used to register 2D endoscopic video images to preoperative 3D data in real time. Typical application of vision-­based tracking is used in the simulation system for bronchoscopic intervention. On the other hand, external tracking refers to the use of

8  Application of AI in Endoscopic Surgical Operations Multi -detector row CT

73

Endoscopic image Field generator

System control unit

Sensor interface unit

Feature points 3D volume data

Vision-based tracking system

Sensor

Electromagnetic tracking system

Fig. 8.1  Vision-based tracking and external tracking. In the vision-based tracking system, feature points in the endoscopic image are tracked in 3D volume data recon-

structed from multi-detector row CT.  Electromagnetic tracking system detects the position and orientation of small sensor in the magnetic fields of field generator

devices and systems to localize surgical instruments in real time during endoscopic surgery. This tracking technology typically uses external position sensors, such as electromagnetic sensors that are attached to the surgical instruments to measure their movement. Calibration and initial registration are mandatory to determine the spatial relationship between computational anatomy and tracking system. Auloge et al. [4] conducted a pilot randomized clinical trial to assess technical feasibility, accuracy, safety, and patient radiation exposure of a novel navigational tool integrating augmented reality (AR) and AI during percutaneous ­ vertebroplasty of patients with vertebral compression fractures. The navigation system used in the study integrates four video cameras within the flat-panel detector of a standard C-arm fluoroscopy machine. Optical input is automatically calibrated and co-registered with the C-arm’s coordinate system. Several 7  mm fiducial markers are applied to the patient’s skin within the field of view of the cameras, to enable motion compensation using a mesh model interconnecting the markers. Following cone beam CT acquisition of the target volume, the AI software automatically recognizes osseous landmarks, numerically identifies each

vertebral level, and displays 2D/3D planning images on the user interface. The target vertebra is manually selected, and the software suggests an optimal trans-pedicular approach, which may be adjusted in multiple planes. Technical feasibility in Group A was 100%. Accuracy in navigation group was 1.68  ±  0.25  mm (skin entry point), 1.02 ± 0.26 mm (trocar tip) in the sagittal plane, and 1.88 ± 0.28 mm (skin entry point) and 0.86 ± 0.17 mm (trocar tip) in the coronal plane, without any significant difference compared to control group. No complications were observed in the entire population. AR/AI-guided percutaneous vertebroplasty appears feasible, accurate, and safe and facilitates lower patient radiation exposure compared to standard fluoroscopic guidance. This type of navigation for rigid organ is feasible because preoperatively collected anatomy data are valid during surgery. However, in the navigation for endoscopic surgery of soft tissue such as thoracic and abdominal organs, difficult task of medical image registration is to overcome the morphology issues of soft tissues which might shift and deform and cause error to the global rigid registration. Deformation recovery methods for soft tissue are comprehensively discussed in the review article by Lin et al. [5]

74

8.2.3 Preoperative Prediction Using AI AI can also be applied in the preoperative prediction. Maubert et al. [6] conducted a study to predict the respectability of peritoneal carcinomatosis using a machine learning model for decision-­ making support, for eligible patients of hyperthermic intraperitoneal chemotherapy (HIPEC). Propensity score matched two groups of 155 patients were obtained: one group without resection and one group with resection. Nine criteria of non-respectability reflecting the organ involvement have been retained. Four classification ­ algorithms (simple classification tree, conditional tree, support vector machine, and random forest model) were tested. The training data included 218 patients and 92 test data. The random forest model exhibited the highest precision with 97.8%, and only two prediction errors were observed. The largest number of patients will allow the authors to improve the precision prediction of HIPEC candidates. Preoperative application of AI can be used for prediction of socioeconomic factors after surgery. Ramkumar et al. [7] developed and validated an artificial neural network (ANN) that learns and predicts length of stay (LOS), inpatient charges, and discharge disposition before primary total knee arthroplasty (TKA). The secondary objective applied the ANN to propose a risk-based, patient-specific payment model (PSPM) commensurate with case complexity. They used data from 175,042 primary TKAs from the National Inpatient Sample of the Unites States and an institutional database, and an ANN was developed to predict LOS, charges, and disposition using 15 preoperative variables, such as age, gender, race, type of admission, emergency department, and so on. This deep learning model demonstrated “learning” with acceptable validity, reliability, and responsiveness in predicting value metrics, offering the ability to preoperatively plan for TKA episodes of care. They conclude that this study has demonstrated deep learning may be used by knee surgeons to preoperatively predict

N. Wada and Y. Kitagawa

value-based outcomes and that this may improve population health, economics, and professional advocacy not only for TKA but also for other specialty procedures.

8.2.4 Intraoperative Decision Support by AI Intraoperative adverse events are a common and important cause of surgical morbidity. However, little could be done to anticipate these events in the operating room. Advances in both data capture in the operating room and AI technology to process these data for real-time clinical decision support tools can help surgical teams anticipate, understand, and prevent intraoperative adverse events. Although many type of operative risk prediction tools using preoperative data are available, intraoperative real-time prediction of adverse events are limited. Lundberg et  al. [8] developed an ML-based system named “Prescience,” which in real time during general anesthesia predicts the risk of hypoxemia and provides explanations of the risk factors. This system can help improve the clinical understanding of hypoxemia risk during surgical care by providing general insights into the exact changes in risk induced by certain patient or procedure characteristics. This kind alert to anesthesiologist is generated from relatively simple dataset, such as blood pressure, body temperature, SpO2, data from anesthesia machines, medications, fluid dose, laboratory results, text descriptions, ASA physical status, surgical procedures, and so on. Intraoperative real-time alert notifications of adverse event, such as bleeding, for endoscopic surgeons are challenging topics. Theoretically, the navigation system based on the preoperative CT volume data, precise real-time mapping of tip of instruments, and real-time endoscopic video data analyses enable the alert system to show the risk of injury to adjacent major vessels. However, it is very difficult to produce this kind of integrated system.

8  Application of AI in Endoscopic Surgical Operations

8.2.5 A  pplication of AI in Surgical Education Simulation has become important in surgical education, with many programs implementing courses involving animal models, cadavers, benchtop models, and virtual reality (VR) simulators [9]. Repeat practice, risk-free environments, and quantification multiple aspects of psychomotor performance during surgical procedures are the merits of VR simulators. Extensive amounts of information from multivariate data sets are available from current VR simulators. Therefore, application of AI is promising in the field of surgical education using VR simulators. Bissonnette et al. [10] reported that AI defined novel metrics of surgical performance and outlined training levels in a virtual reality spinal simulation procedure. They evaluated 41 participants (22 senior and 19 junior) from 4 universities with the module of virtual reality hemilaminectomy and tested 5 types of ML Algorithm (support vector machine, linear discriminant analysis, k-nearest neighbors, naive Bayes, and decision tree) to differentiate “senior” and “junior” participants. Among them, the support vector machine achieved the highest accuracy, at 97.6%, with use of leave-one-out cross-validation. The k-nearest neighbors reached an accuracy of 92.7%. The linear discriminant analysis achieved an accuracy of 87.8%. The decision tree had a 70.7% accuracy. The naive Bayes reached the lowest accuracy, at 65.9%. They conclude that once algorithms are rigorously validated to recognize expert surgeons, surgical accreditation bodies could employ these techniques to ensure their members’ technical competency. The significance of this study lies in the potential of combining VR simulation and AI to provide safer training and objective assessment of surgical skills, which could lead to improved patient care. Winkler-Schwartz et  al. [11] investigated to identify the surgical and operative factors selected by a machine learning algorithm to accurately classify participants by level of expertise in a virtual reality surgical procedure. They

75

recruited 50 participants from a single university. Individuals were classified a priori as expert (neurosurgery staff), seniors (neurosurgical fellows and senior residents), juniors (neurosurgical junior residents), and medical students. The VR simulator they used was a neurosurgical simulator designed to recreate the visual and haptic experience of resecting a human brain tumor through an operative microscope. Four classifier algorithms were used: k-nearest neighbor, naive Bayes, discriminant analysis, and support vector machine. Among 4 algorithms, the k-nearest neighbor algorithm had an accuracy of 90% (45 of 50), the naive Bayes algorithm had an accuracy of 84% (42 of 50), the discriminant analysis algorithm had an accuracy of 78% (39 of 50), and the support vector machine algorithm had an accuracy of 76% (38 of 50). Performance metrics selected by the algorithms spanned the following four principal domains: (1) movement associated with a single instrument, (2) both instruments used in concert, (3) force applied by the instruments, and (4) tissue removed or bleeding caused.

8.2.6 Video Recognition by AI Computer vision (CV) is the field of study of machine-based understanding of movie data using AI technologies, which allows for a dynamic evaluation of surgical images. Recent advances in information technology allowed researchers to analyze exponentially increased information of video. In still pictures, doubtful interpretations may occur when anatomical structures overlap, perspectives are lost, instruments or smoke occlude the view, the camera is out of focus, is too distant, or when light conditions are poor. Videos allow a more conclusive assessment of CVS even if one or more the frames present with the above limitations. Hashimoto et  al. [12] evaluated the performance of CV in analyzing operative video to identify steps of a laparoscopic sleeve gastrectomy (LSG). Videos were segmented into the seven steps: (1) port placement, (2) liver retrac-

N. Wada and Y. Kitagawa

76

tion, (3) liver biopsy, (4) gastrocolic ligament dissection, (5) stapling of the stomach, (6) bagging specimen, and (7) final inspection of staple line. A random 70% of 88 cases of LSG video were used as training data and the remaining 30% were used as validation data. Accuracy of operative step identification by the AI was determined by comparing to surgeon annotations. Mean accuracy of the AI in identifying operative steps in the test set was 82% with a maximum of 85.6%. This suggests operative video could be used as a quantitative data source for research in intraoperative clinical decision support, risk prediction, or outcomes studies. Mascagni et  al. [13] developed and tested a method to report critical view of safety (CVS) in laparoscopic cholecystectomy using 60-s video clips prior to the division of cystic structures. The aim of this study is to develop and test a method for consistent CVS evaluation and reporting in videos. They could successfully formalize a reproducible method for objective video reporting of CVS in laparoscopic cholecystectomy. This achievement could also be the basis for future developments of intraoperative real-time guidance using AI during surgery.

8.2.7 Autonomous Surgery Using AI Completely autonomous surgery is naturally a long way off for now. The Society of Automotive Engineers defines 6 levels of driving automation ranging from 0 (fully manual) to 5 (fully autonomous) (Table  8.1). As the autonomous driving technology has developed step-by-step from level 1 (driver assistance) to level 5 (full driving automation), partial automation is the first step to achieving the goals. Currently available surgiTable 8.1  Six levels of driving automation (The Society of Automotive Engineers) Level 0 Level 1 Level 2 Level 3 Level 4 Level 5

No driving automation Driver assistance Partial driving automation Conditional driving automation High driving automation Full driving automation

cal robot, such as da Vinci Surgical System from Intuitive Surgical is a “master-slave manipulator” and has no automation (level 0). Nowadays, automated stapling devices are commercially available. However, clinically validated autonomous suturing and knot-tying system is not reported to date. Shademan et al. [14] demonstrated supervised autonomous soft tissue surgery in an open surgical setting by an autonomous suturing algorithm. This suturing system (Smart Tissue Autonomous Robot: STAR) integrates near-infrared fluorescent and 3D plenoptic vision, force sensing, submillimeter positioning, and actuated surgical tools. The suture line can be recognized by observing luminescent NIRF markers on the target tissue. They compared metrics of anastomosis of a longitudinal cut along pig intestine by STAR with those by open hand-sewn, laparoscopic manual, and robotic suturing. Suture spacing, leak pressures, number of mistakes, and completion time were evaluated. Despite dynamic scene changes and tissue movement during surgery, they demonstrated that the outcome of supervised autonomous procedures is superior to surgery performed by expert surgeons and robotic techniques in ex vivo porcine tissues and in living pigs. In STAR suturing, manual stay sutures are necessary and suture strings have to be controlled by assistant during procedures. Despite of these limitations, partially autonomous surgery like suturing promises substantial benefits through improved safety from reduction of human errors, increased efficiency due to procedure time reduction, potential access to optimal surgical techniques, and consistent outcomes independent of surgeon training, condition, or experience. In the field of radiotherapy, level 4 automation is available in clinical setting. The CyberKnife system, invented at Stanford University and now built by Accuray [15], generates and executes preoperative plans to deliver radiotherapy to tumors located on the patient’s body. The human surgeon adjusts the automatically generated plan prior to execution and ensures the system performs the task safely during the procedure. In the research and development of the automation of surgery, patients’ safety is the

8  Application of AI in Endoscopic Surgical Operations

77

J Arthroplast. 2019;34(10):2220–2227 e2221. https:// doi.org/10.1016/j.arth.2019.05.034. 8. Lundberg SM, Nair B, Vavilala MS, Horibe M, Eisses MJ, Adams T, Liston DE, Low DK, Newman SF, Kim J, Lee SI.  Explainable machine-learning predictions for the prevention of hypoxaemia during surgery. Nat Biomed Eng. 2018;2(10):749–60. https://doi. org/10.1038/s41551-018-0304-0. 9. Reznick RK, MacRae H.  Teaching surgical skills--changes in the wind. N Engl J Med. 2006;355(25):2664–9. https://doi.org/10.1056/ References NEJMra054785. 10. Bissonnette V, Mirchi N, Ledwos N, Alsidieri G, 1. Hirasawa T, Aoyama K, Tanimoto T, Ishihara S, Winkler-Schwartz A, Del Maestro RF.  Artificial Shichijo S, Ozawa T, Ohnishi T, Fujishiro M, Matsuo intelligence distinguishes surgical training levels K, Fujisaki J, Tada T.  Application of artificial intelin a virtual reality spinal task. J Bone Joint Surg ligence using a convolutional neural network for Am. 2019;101(23):e127. https://doi.org/10.2106/ detecting gastric cancer in endoscopic images. Gastric JBJS.18.01197. Cancer. 2018;21(4):653–60. https://doi.org/10.1007/ 11. Winkler-Schwartz A, Yilmaz R, Mirchi N, Bissonnette s10120-018-0793-2. V, Ledwos N, Siyar S, Azarnoush H, Karlik B, Del 2. Reismann J, Romualdi A, Kiss N, Minderjahn MI, Maestro R.  Machine learning identification of surKallarackal J, Schad M, Reismann M. Diagnosis and gical and operative factors associated with surgical classification of pediatric acute appendicitis by artifiexpertise in virtual reality simulation. JAMA Netw cial intelligence methods: an investigator-independent Open. 2019;2(8):e198363. https://doi.org/10.1001/ approach. PLoS One. 2019;14(9):e0222030. https:// jamanetworkopen.2019.8363. doi.org/10.1371/journal.pone.0222030. 12. Hashimoto DA, Rosman G, Witkowski ER, Stafford 3. Luo X, Mori K, Peters TM. Advanced endoscopic navC, Navarette-Welton AJ, Rattner DW, Lillemoe KD, igation: surgical big data, methodology, and applicaRus DL, Meireles OR.  Computer vision analysis of tions. Annu Rev Biomed Eng. 2018;20:221–51. https:// intraoperative video: automated recognition of operadoi.org/10.1146/annurev-bioeng-062117-120917. tive steps in laparoscopic sleeve gastrectomy. Ann 4. Auloge P, Cazzato RL, Ramamurthy N, de Marini P, Surg. 2019;270(3):414–21. https://doi.org/10.1097/ Rousseau C, Garnon J, Charles YP, Steib JP, Gangi SLA.0000000000003460. A.  Augmented reality and artificial intelligence-­ 13. Mascagni P, Fiorillo C, Urade T, Emre T, Yu T, based navigation during percutaneous vertebroWakabayashi T, Felli E, Perretta S, Swanstrom L, Mutter plasty: a pilot randomised clinical trial. Eur Spine D, Marescaux J, Pessaux P, Costamagna G, Padoy N, J. 2019;29(7):1580–9. https://doi.org/10.1007/ Dallemagne B.  Formalizing video documentation of s00586-019-06054-6. the critical view of safety in laparoscopic cholecystec 5. Lin B, Sun Y, Qian X, Goldgof D, Gitlin R, You tomy: a step towards artificial intelligence assistance to Y.  Video-based 3D reconstruction, laparoscope localimprove surgical safety. Surg Endosc. 2019;34(6):2709– ization and deformation recovery for abdominal mini14. https://doi.org/10.1007/s00464-019-07149-3. mally invasive surgery: a survey. Int J Med Robot. 14. Shademan A, Decker RS, Opfermann JD, Leonard 2016;12(2):158–78. https://doi.org/10.1002/rcs.1661. S, Krieger A, Kim PC.  Supervised autono 6. Maubert A, Birtwisle L, Bernard JL, Benizri E, Bereder mous robotic soft tissue surgery. Sci Transl Med. JM.  Can machine learning predict resecability of a 2016;8(337):337ra364. https://doi.org/10.1126/sciperitoneal carcinomatosis? Surg Oncol. 2019;29:120– translmed.aad9398. 5. https://doi.org/10.1016/j.suronc.2019.04.008. 15. Moustris GP, Hiridis SC, Deliparaschos KM, 7. Ramkumar PN, Karnuta JM, Navarro SM, Haeberle Konstantinidis KM.  Evolution of autonomous and HS, Scuderi GR, Mont MA, Krebs VE, Patterson semi-autonomous robotic surgical systems: a review BM. Deep learning preoperatively predicts value metof the literature. Int J Med Robot. 2011;7(4):375–92. rics for primary total knee arthroplasty: development https://doi.org/10.1002/rcs.408. and validation of an artificial neural network model.

most important factor. Of course, ethical and legal aspects are also important for clinical application. In order to provide better patient care in endoscopic surgery, future application of autonomous surgery using AI would not be avoided.

Part IV Navigation Surgery

9

Application of ICG Fluorescent Endoscope Systems in Identifying Small Lung Cancers on the Periphery of the Lungs in Thoracoscopic Surgery Yasuhiko Ohshio

9.1

Background

guidance, and the percutaneous injection of dye into lungs. These procedures might cause serious In 1992, Lewis et al. first reported thoracoscopic complications such as bleeding, pneumothorax, surgery for lung cancer in order to reduce the sur- or air embolism. Therefore, it is urgent to develop gical stress [1]. Since then, thoracoscopic surgery new identification method of the small lesion has rapidly expanded. In 2015, according to the which can utilize the benefit of safer and minimal annual report by The Japanese Association for invasive surgery. Thoracic Surgery, 64.4% of lung cancer surgery Indocyanine green (ICG) has been used for and 61.7% of lobectomy were performed by tho- more than 50 years in the measurement of hepatic racoscopy [2]. Meanwhile, advances in imaging functional reserve. Recently, various diagnostic equipment have increased the detection of small-­ methods using the fact that ICG is a fluorescent sized lung cancer. Because definitive diagnosis of compound are applied clinically. For example, small lesions is less likely to be obtained preop- identification of sentinel lymph nodes in breast eratively, a partial resection of the lesion is often cancer or malignant melanoma and blood flow performed to confirm the diagnosis with rapid assessment of blood vessels and tissues using pathologic examination during the course of sur- ICG is already covered by the health care insurgery, followed by additional pulmonary resection. ance in Japan. In practice, once injected into the A problem with thoracoscopic surgery of body, ICG is stabilized by binding to lipoprosmall lesions is that intraoperative location is teins, and the ICG complex is excited through difficult to identify, especially if not accompa- the excitation light (excitation at 805  nm) from nied by pleural changes. At present, the surgical the light source of the endoscope. Then, the emitmethod can be changed to thoracotomy for direct ted infrared fluorescence (845  nm) is observed palpating of the lesions, but this option does not from the outside surface of organs in real time. provide the benefit of minimally invasive surgery. It is now considered that fluorescence present at Preoperative marking is required to avoid thora- depths of up to 10  mm from the organ surface cotomy, including the use of preoperative bron- can be detected. In addition, ICG is characterized choscopy to inject barium or dye into the vicinity by being a relatively safe drug with few adverse of the lesion, the setting of metal hooks under CT reactions (incidence: approximately 0.17%). The frequency of each symptom is reported to be 0.02% for shock, 0.08% for nausea/vomiting, Y. Ohshio (*) Department of Thoracic Surgery, Shiga University of 0.04% for vascular pain, and 0.02% for fever/ Medical Science, Otsu, Japan warmth (package inserts). e-mail: [email protected]

© Springer Nature Singapore Pte Ltd. 2021 S. Takenoshita, H. Yasuhara (eds.), Surgery and Operating Room Innovation, https://doi.org/10.1007/978-981-15-8979-9_9

81

Y. Ohshio

82

Fig. 9.1  D-Light P System (KARL STORZ)

In this context, we thought that ICG might play a role as a contrast agent and that it might be possible to recognize lesions including lung cancer as fluorescence signals during surgery and conducted the following clinical studies with the approval from the ethics committee at Shiga University of Medical Science.

9.2

Methods

Under the approval from the ethics committee at our university, ICG was applied to 38 consenting patients who underwent lung cancer (including suspected lung cancer) surgery. The primary lesion was preoperatively determined to be resectable, and CT scans were used to confirm the location of tumor during surgery. ICG dissolved in sterile water for injection at a concentration of 5 mg/mL (0.25–0.5 mg/kg) was administered intravenously in the peripheral vein from 48 h before the surgery started to the intraoperative observation started. After induction of general anesthesia, thoracoscopic surgery or surgery via thoracoscopic assistance

was initiated, the location of the tumor was confirmed visually or by palpation, and the accumulation of ICG in the lesions was observed with fluorescence endoscopic systems. We used D-Light P System (KARL STORZ) and PINPOINT (Novadaq) as fluorescence thoracoscopic systems (Figs. 9.1 and 9.2). After the observation, lung resection and lymph node dissection were performed as scheduled for lung cancer.

9.3

Results

Between January 2015 and December 2017, 38 patients were registered in this study. There were 28 men and 10 women with a mean age of 72.6 (47–83) years, a mean body weight of 59.5 (31–82.6) kg, a mean ICG dose of 0.47 (0.25–0.5) mg/kg, and a mean time from ICG administration to the intraoperative observation started of 18.4 (0–48) h. The mean size of entire lesions on preoperative CT was 24.3 mm (10–66  mm), the mean distance from the pulmonary pleura to the lesions was 3.7  mm

9  Application of ICG Fluorescent Endoscope Systems in Identifying Small Lung Cancers on the Periphery…

83

① Intravenous injection of ICG ②ICG binds to plasma proteins ③ Visualization of ICG in bloodstream with NIR/ICG System

④ NIR/ICG light source

Fig. 9.2  Pinpoint (Novadaq). Once injected into the body, ICG is stabilized by binding to lipoproteins, and the ICG complex is excited through the excitation light (exci-

tation at 805 nm) from the light source of the endoscope. Then, the emitted infrared fluorescence (845  nm) is observed from the outside surface of organs in real time

(0–34  mm), and the mean CT value at the center of the lesion on plain CT was −176.7 (−889.1 to 221.2) HU (Table 9.1). Of the total 38 cases, details of histopathology were as follows: adenocarcinoma in 22; squamous cell

carcinomas in 8; LCNEC in 2; pleomorphic carcinoma in 2; small cell carcinoma in 1; and benign lesions in 3 (lymphovascular tumor in 1, granuloma in 1, fibrosis in 1) (Table  9.4). Details of clinicopathological factors were as

Y. Ohshio

84

follows. Differentiation was G1 in 10, G2 in 10, G3 in 11, G4 in 4; lymphatic invasion ly0 in 31, Ly1 in 4; vascular invasion V0 in 20, V1 in 15; and pleural invasion pl0 in 26, pl1 in 7, pl3 in 2 (Table 9.5). Of the 38 cases, 33 were observed with only D-Light P System, 4 were with only PINPOINT and 1 were both fluorescent endoscopic systems in the operative field (Table 9.2). Table 9.1  Individual data for patients (n = 38) Characteristics Age (years) Sex

72.6 (47–83) Male Female

Body weight (kg) Dose of ICG (mg/kg) Time from ICG administration to start of intraoperative observation (h) Total size of tumor (mm) Distance from lung pleura to tumor (mm) CT value (HU)

28 10 59.5 (31.0–82.6) 0.47 (0.25–0.5) 18.4 (0–48)

24.3 (10–66) 3.7 (0–34) −176.7 (−889.1 to 221.2)

Table 9.2 Fluorescence endoscopic system used for observation D-Light P System only Pinpoint only Both D-Light P System and Pinpoint

n 33 4 1

Of the 34 cases which were observed with D-Light P System, 3 received intravenous ICG just before the observation. Immediately after ICG administration, the whole normal lung except for the lesioned part rapidly emitted the fluorescence and the lesions did not fluoresce. After more than 10  min, fluorescence of the lesions was observed to be weaker than that of the surrounding normal lung, although localization of the lesions was difficult (Fig.  9.3). The fluorescence in normal lung was attenuated in about 3 h from ICG administration to the intraoperative observation started but completely lost, and finally disappeared over 14 h. Thirty-one cases received intravenous ICG, 3–48 h before surgery. Of these, 20 had no fluorescence in the lesions (Figs. 9.4, 9.6, and 9.8). In 11 cases, the lesions showed slight fluorescence, but the fluorescence was so weak that identification of the lesions by fluorescence was difficult (Fig. 9.5). The mean measurement values in negative and positive fluorescence groups were respectively shown as follows; the time from ICG administration to the intraoperative observation started was 17.5 (3–48) h and 23.5 (14–38) h; the entire sizes of the lesions were 24.8 (12–47) mm and 27.8 (12–66) mm; the distances from the lung pleura to the lesion were 3 (0–13.2) mm and 5 (0–34) mm; the CT values were −225.40 (−889.1 to 221.2) HU and −28.48 (−358.74 to 36.92) HU; and the doses of ICG were 0.49 (0.4–0.5) mg/kg and 0.46 (0.4–0.5) mg/kg (Table  9.3). Details of histopathology were as follows. Of 20 cases with

Fig. 9.3  The case in which ICG was given intravenously just before observation. Immediately after administration, the whole normal lung except for tumor sites showed fluorescence. No fluorescence was observed in tumor

9  Application of ICG Fluorescent Endoscope Systems in Identifying Small Lung Cancers on the Periphery…

85

a

b

c

d

Fig. 9.4  No fluorescence was observed with D-Light P System (a, b), but fluorescence was observed in the cross-­ section of tumors with PDE (c, d)

a

b

Fig. 9.5  Partial fluorescence was observed with D-Light P System (purple) (a), and fluorescence was observed in the cross-section of tumors with PDE (b)

fluorescence negative, there were adenocarcinoma in 12, squamous cell carcinoma in 3, pleomorphic carcinoma in 2, small cell carcinoma in 1, and benign lesions in 3 (lymphovascular tumor in 1 and fibrosis in 1). Of 11 cases with slightly fluo-

rescence positive, there were adenocarcinoma in 6, squamous cell carcinoma in 4, and granuloma in 1 (Table  9.4). Details of clinicopathological factors were as follows. In the group with fluorescence negative (lung cancer in 18 cases), the dif-

Y. Ohshio

86 Table 9.3  Administered ICG and tumor data

Dose of ICG (mg/kg) Time from ICG administration to the start of intraoperative observation (h) Entire size of tumor (mm) Distance from lung pleura to tumor (mm) CT value (HU)

D-Light P System Fluorescence negative (n = 20) 0.49 (0.4–0.5) 17.5 (3–48)

Fluorescence positive (n = 11) 0.46 (0.4–0.5) 23.5 (14–38)

Pinpoint Fluorescence negative (n = 5) 0.50 (0.49–0.5) 22.0 (18–24)

PDE Fluorescence positive (n = 9) 0.50 (0.46–0.5) 18.8 (14–24)

24.8 (12–47) 3 (0–13.2)

27.8 (12–66) 5 (0–34)

22.6 (10–36) 2.4 (0–12)

23.4 (16–36) 4.5 (0–19)

−225.40 (−889.1 to 2221.2)

−28.48 (−358.74 to 36.92)

−123.91 (−637.88 to 23.27)

−139.6 (−637.88 to 49.28)

Pinpoint Fluorescence negative (n = 5) 2 1 1 1 0 0

PDE Fluorescence positive (n = 9) 3 3 0 1 0 2

Table 9.4  Histological subtype

Ad Sq LCNEC Pleomorphic Small Benign

Total (n = 38) 22 8 2 2 1 3

D-Light P System Fluorescence negative (n = 20) 12 3 0 2 1 2

Fluorescence positive (n = 11) 6 4 0 0 0 1

Table 9.5  Clinicopathological findings Cancer (n = 35) G 1 2 3 4 Ly 0 1 V 0 1 pl 0 1 2 3

D-Light P System Fluorescence negative Fluorescence (n = 18) positive (n = 10)

Pinpoint Fluorescence negative (n = 5)

PDE Fluorescence positive (n = 7)

10 10 11 4

6 3 7 2

1 5 4 0

1 2 0 2

1 3 2 1

31 4

16 2

9 1

4 1

6 1

20 15

11 7

4 6

4 1

4 3

26 7 0 2

12 5 0 1

7 2 0 1

4 1 0 0

5 2 0 0

9  Application of ICG Fluorescent Endoscope Systems in Identifying Small Lung Cancers on the Periphery…

ferentiation was G1 in 6, G2 in 3, G3 in 7, G4 in 2; lymphatic invasion was ly0 in 16, Ly1 in 2; vascular invasion was V0 in 11, V1 in 7; and pleural invasion was pl0 in 12, pl1 in 5, pl3 in 1. In the group with fluorescence p­ ositive (lung cancer in 10 cases), the differentiation was G1 in 1, G2 in 5, G3 in 4; lymphatic invasion was ly0 in 9,Ly1 in 1; vascular invasion was V0 in 4, V1 in 6; and pleural invasion was pl0 in 7, pl1 in 2, pl3 in 1 (Table 9.5). No fluorescence was observed in the lesions of all five cases observed with PINPOINT (Fig.  9.6). The mean time from ICG administration to the intraoperative observation started was 22.0 (18–24) h, the mean entire size of the lesions was 22.6 (10–36) mm, the mean distance from the pulmonary pleura to the lesion was 2.4 (0–12) mm, the mean CT value was −123.91 (−637.88 to 23.27) HU, and the mean ICG dose was 0.50 (0.49–0.5) mg/kg (Table  9.3). Details of histopathology included adenocarcinoma in 2, squamous cell carcinoma in 1, LCNEC in 1, and pleomorphic carcinoma in 1 (Table 9.4). Details of clinicopathological factors were as follows. Differentiation was G1  in 1, G2  in 2, G4  in 2; lymphatic invasion was ly0 in 4, Ly1 in 1; vascular invasion was V0 in 4, V1 in 1; and pleural invasion was pl0 in 4, pl1 in 1 (Table 9.5).

87

Of the total 38 cases, 9 were observed extracorporeal with PDE (Hamamatsu) after resection. In all nine cases, the lesions were recognized as fluorescence signals (Figs. 9.4, 9.5, 9.6, 9.7, and 9.8). These included two cases with fluorescence positive (Fig. 9.5) and four cases with fluorescence negative (Fig.  9.4) using D-Light P System, two cases using PINPOINT, and one case with no fluorescence observed using both system (Fig.  9.7). The mean time from ICG administration to the intraoperative observation started was 18.8 (14–24) h, the mean size of entire lesion was 23.4 (16–36) mm, the mean distance from the lung pleura to the lesion was 4.5 (0–19) mm, the mean CT value was −139.6 (−637.88 to 49.28) HU, and the mean ICG dose was 0.50 (0.46–0.5) mg/kg (Table 9.3). Details of histopathology included adenocarcinoma in 3, squamous cell carcinoma in 3, pleomorphic carcinoma in 1, and benign lesion in 2 (granuloma in 1, fibrosis in 1) (Table 9.4). Details of clinicopathological factors of seven cases in lung cancer were as follows. Differentiation of G1  in 1, G2  in 3, G3  in 2, G4  in 1; lymphatic invasion of ly0  in 6, Ly1  in 1; vascular invasion of V0 in 4, V1 in 3; and pleural invasion of pl0 in 5, pl1 in 2.

a

b

Fig. 9.6  Lesions consisting of predominantly ground-glass. No fluorescence was observed with D-Light P System (a), but fluorescence was observed in tumors with PDE (b)

Y. Ohshio

88

a

b

c

Fig. 9.7  No fluorescence was observed with Pinpoint (a) and D-Light P System (b), but fluorescence was observed in tumors with PDE (c)

a

b

Fig. 9.8  Benign lesions (granuloma). No fluorescence was observed with D-Light P system (a), but fluorescence was observed in the cross-section of the lesions with PDE (b)

9.4

Discussion

Although advances in imaging technologies and widespread use of CT screenings have led to an increase in the detection of small-sized lesions

in lungs less than 1 cm in diameter, the thoracoscopic surgery often makes it difficult to locate the lesion intraoperatively, especially if it is not accompanied by pleural changes. In such cases, thoracoscopic surgery has been changed to open

9  Application of ICG Fluorescent Endoscope Systems in Identifying Small Lung Cancers on the Periphery…

surgery to palpate the lesions directly, bronchoscopy has been used preoperatively, or marking has been performed under CT guidance. Because these procedures involve increased invasiveness and the risk of serious complications such as bleeding, pneumothorax, and air embolism, there is an urgent need to develop new methods for identifying lesions that are safer and can benefit from minimally invasive surgery. Meanwhile, the technology of navigation surgery using intraoperative fluorescence imaging is also remarkably advanced. ICG fluorescence imaging technology has already been implemented in endoscope systems and has been used to evaluate blood flow in the various organs. This study examined whether the location of the lung lesions could be identified as fluorescence signals using the ICG fluorescence endoscopy system, D-Light P System and PINPOINT, which have already been used in operating rooms. In addition to intraoperative real-time observation with D-Light P System and PINPOINT, we also performed PDE evaluation after resection in some cases. These results confirmed the accumulation of ICG in lung cancer tissues. ICG accumulation in cancer tissues has been reported. Since Ishizawa et al. first demonstrated intraoperative visualization of hepatic metastases from colorectal cancer and hepatectomy from hepatocellular carcinoma in 2009 [3], various reports have been published [4–9]. ICG is a dark green-blue, water-soluble compound with a molecular weight of 774.96 and rapidly binds to plasma proteins after intravenous injection. This ICG complex, which is 4–6  nm in size, tends to leak out of vascular endothelium from areas of increased permeability or stagnation of blood flow into tissues and is difficult to remove rapidly from tissues [5, 10, 11]. Therefore, it is considered that the fluorescence of normal lung tissue, which is rich in blood flow, was emitted immediately after intravenous injection and then decreased rapidly, while the fluorescence of tumor tissue took a long time to be emitted, and the fluorescence continued to be observed for a while after the fluorescence of normal lung disappeared. Our results suggested that ICG should be given intravenously

89

at least 14  h before the intraoperative observation started. However, many investigators in lung surgery have been reported that ICG was given 24 h before the intraoperative observation started [6, 8, 9]. Because the mechanism by which ICG is taken up by tumors is due to nonspecific extravasation and the clearance was delayed, such fluorescence signals may also be observed in tissues with similar characteristics including inflammatory tissues. Our study showed fluorescence in one of three benign lesions observed with D-Light P System and in two benign lesions with PDE. Therefore, ICG fluorescence imaging has been reported to be inappropriate for differentiating between benign and malignant lesions and for delineating tumor tissue boundaries with inflammatory changes [5, 8]. However, in clinical practice, most microlesions in lungs have not been definitively diagnosed preoperatively and require surgical excision, biopsy, and rapid intraoperative pathologic examination. Therefore, ICG imaging, which can recognize lesions as fluorescence signals regardless of whether they are malignant or benign, may be rather useful. In this study, we used D-Light P System and PINPOINT endoscopic fluorescence, which have already been clinically applied. In the field of lung surgery, these systems are useful for setting the extent of resection in segmentectomy [12, 13]. ICG accumulation in lung lesions was ­unfortunately not confirmed in all of the 5 cases using PINPOINT.  However, fluorescence was barely detected in 11 of the 31 cases using D-Light P System, and the contrast to the background lung was not enough to identify the lesion sites. No differences were observed between the groups with negative and positive fluorescence (21 vs.11 cases) in terms of the time from ICG administration to the intraoperative observation started, the entire size of the lesion, the distance from the lung pleura to the lesion, CT value of the lesion on preoperative CT images, or the dose of ICG.  In addition, there were no differences in clinicopathologic factors (degree of ­differentiation, presence or absence of lymphatic invasion, presence or absence of vascular invasion, and degree of pleural invasion) between

Y. Ohshio

90

the two groups, suggesting that these factors are unlikely to influence the integration of ICG into the lesions. Meanwhile, fluorescence was observed in all nine of the examined PDE that could be used ex vivo, providing a contrast to the background lung in which the location of the lesion could be identified. In the past reports that good contrast was obtained, a large high-sensitivity camera was used [5, 8]. Of nine cases we studied, two cases showed slight fluorescence with D-Light P System, and the remaining seven cases did not show fluorescence with the other fluorescence endoscopic systems. Generally, compared with large cameras, endoscope-compatible systems have lower excitation light and lower fluorescence detection capabilities. Furthermore, when an oblique-viewing endoscope is used, it is indicated that irradiation angle of the excitation light and detection angle of the fluorescence are difficult to maintain perpendicular to the lung pleura near the lesion, which could contribute to a decrease in sensitivity. The use of custommade, highly sensitive endoscopic camera has been studied [6, 9]. If a highly sensitive fluorescence endoscopic system can be developed, localization of small-sized lesions in lungs can be achieved. In our study, the dose of ICG was set at 0.5 mg/ mL or less, which is approved in the blood flow evaluation of organs in Japan. Recent reports indicate that high doses of 4–5 mg/mL improve the detection efficiency [14]. They used different fluorescence endoscopic systems, and the effects of high doses on the human body are not clear, but the use of high dosage of ICG may be considered in the future.

9.5

Conclusions

We confirmed that intravenous injected ICG stagnated in lung cancer tissues and emitted fluorescence. In order to recognize lesions as fluorescent signals during thoracoscopic lung surgery, the timing and dosage of ICG administration should be modified, and the sensitivity of fluorescence endoscope systems should be further improved.

It is suggested that the fluorescence thoracoscopic system may become a simpler and safer method for the identification of lesions.

References 1. Lewis RJ, Caccavale RJ, Sisler GE, Mackenzie JW.  One hundred consecutive patients undergoing video-assisted thoracic operations. Ann Thorac Surg. 1992;54:421–6. 2. Masuda M, Endo S, Natsugoe S, Shimizu H, Doki Y, Hirata Y, et  al. Thoracic and cardiovascular surgery in Japan during 2015: Annual report by The Japanese Association for Thoracic Surgery. Gen Thorac Cardiovasc Surg. 2018;66:581–615. 3. Ishizawa T, Fukushima N, Shibahara J, Masuda K, Tamura S, Aoki T, et  al. Real-time identification of liver cancers by using indocyanine green fluorescent imaging. Cancer. 2009;115:2491–504. 4. Okusanya OT, DeJesus EM, Jiang JX, Judy RP, Venegas OG, Deshpande CG, et  al. Intraoperative molecular imaging can identify lung adenocarcinomas during pulmonary resection. J Thorac Cardiovasc Surg. 2015;150:28–35.e1. 5. Holt D, Okusanya O, Judy R, Venegas O, Jiang J, DeJesus E, et  al. Intraoperative near-infrared imaging can distinguish cancer from normal tissue but not inflammation. PLoS One. 2014;9:e103342. 6. Okusanya OT, Holt D, Heitjan D, Deshpande C, Venegas O, Jiang J, et al. Intraoperative near-infrared imaging can identify pulmonary nodules. Ann Thorac Surg. 2014;98:1223–30. 7. Liberale G, Vankerckhove S, Caldon MG, Ahmed B, Moreau M, Nakadi IE, et  al. Fluorescence imaging after indocyanine green injection for detection of peritoneal metastases in patients undergoing cytoreductive surgery for peritoneal carcinomatosis from colorectal cancer: a pilot study. Ann Surg. 2016;264:1110–5. 8. Kim HK, Quan YH, Choi BH, Park JH, Han KN, Choi Y, et al. Intraoperative pulmonary neoplasm identification using near-infrared fluorescence imaging. Eur J Cardiothorac Surg. 2016;49:1497–502. 9. Mao Y, Chi C, Yang F, Zhou J, He K, Li H, et al. The identification of sub-centimetre nodules by near-­ infrared fluorescence thoracoscopic systems in pulmonary resection surgeries. Eur J Cardiothorac Surg. 2017;52:1190–6. 10. Schaafsma BE, Mieog JS, Hutteman M, van der Vorst JR, Kuppen PJ, Löwik CW, et al. The clinical use of indocyanine green as a near-infrared fluorescent contrast agent for image-guided oncologic surgery. J Surg Oncol. 2011;104:323–32. 11. Rosenthal EL, Warram JM, Bland KI, Zinn KR. The status of contemporary image-guided modalities in oncologic surgery. Ann Surg. 2015;261:46–55. 12. Mun M, Okumura S, Nakao M, Matsuura Y, Nakagawa K.  Indocyanine green fluorescence-­ navigated tho-

9  Application of ICG Fluorescent Endoscope Systems in Identifying Small Lung Cancers on the Periphery… racoscopic anatomical segmentectomy. J Vis Surg. 2017;3:80. 13. Pischik VG, Kovalenko A.  The role of indocyanine green fluorescence for intersegmental plane identification during video-assisted thoracoscopic surgery segmentectomies. J Thorac Dis. 2018;10:S3704–S11.

91

14. Newton AD, Predina JD, Corbett CJ, Frenzel-Sulyok LG, Xia L, Petersson EJ, et al. Optimization of second window indocyanine green for intraoperative near-­ infrared imaging of thoracic malignancy. J Am Coll Surg. 2019;228:188–97.

Novel Multispectral Device for Quantitative Imaging of Tissue Oxygen Saturation and Hemoglobin as Surgical Navigation Device

10

Yasuhiro Haruta, Ryosuke Tsutsumi, Kuriyama Naotaka, Hajime Nagahara, and Tetsuo Ikeda

10.1 Introduction In recent years, the necessity of technology allowing surgical navigation has rapidly increased due to the requirements of advanced and subtle surgical procedures. However, at present surgical navigation has not developed from the simulation of the treatment plan based on the stereoscopic positional relationship Y. Haruta · K. Naotaka Department of Surgery and Science, Graduate School of Medical Sciences, Kyushu University, Fukuoka, Fukuoka, Japan R. Tsutsumi Department of Surgery and Science, Graduate School of Medical Sciences, Kyushu University, Fukuoka, Fukuoka, Japan Center of Endoscopy and Endoscopic Surgery, Medical and Dental Hospital, Fukuoka Dental College, Fukuoka, Fukuoka, Japan H. Nagahara Institute for Datability Science Osaka University, Suita, Osaka, Japan T. Ikeda (*) Center of Endoscopy and Endoscopic Surgery, Medical and Dental Hospital, Fukuoka Dental College, Fukuoka, Fukuoka, Japan Center for Advanced Medical Innovation Kyushu University, Fukuoka, Fukuoka, Japan e-mail: [email protected]

between the lesion and vessel [1–3]. It is desirable to develop a navigation device that instantaneously allows appropriate judgments to be made through the application of noninvasive proximity remote sensing technology, allowing clear intraoperative imaging of the oxygen metabolism of biological tissue. Pulse oximeter is a device, which quantifies the oxygen saturation of blood at one point of the living body and reflects the function of the oxygen saturation of blood at only one point in an organism and reflect the pulmonary or cardiac functions [4–7]. The pulse oximeters can only give a quantitative measurement. However, because images of transmitted light and scattered light are captured and used in the device, it is necessary to bring the light source and the sensor into close contact with each other so as to sandwich the objective lens portion, which interferes with surgery [8, 9]. We focused on the reflective characteristics of hemoglobin (Hb). OxyHb and deoxyHb have different reflective characteristics and their reflectances change depending on the total amount of Hb [10]. By monitoring the hemoglobin amount and oxygen saturation of the tissue, it becomes possible to quantify and image the tissue oxygen metabolism in real time [11]. Based on this theory, we developed a device and program for

© Springer Nature Singapore Pte Ltd. 2021 S. Takenoshita, H. Yasuhara (eds.), Surgery and Operating Room Innovation, https://doi.org/10.1007/978-981-15-8979-9_10

93

Y. Haruta et al.

94

quantification and imaging of the oxygen saturation and hemoglobin content of the tissues [12].

10.2 Technological Background Figure 10.1a shows the configuration of the multispectral camera. The camera consists of a sensor unit, lighting unit, and a control and analysis PC unit. Figure 10.1b shows the optical path of the sensor unit. Reflected light from the subject is split into three directions by two beam splitters, and sensing is simultaneously performed by one RGB color CMOS sensor (SENTECH, STC-­SC83POE) and two monochrome CMOS sensors (Sentech, STC-CMB2MPOE-IR). Narrow band filters (EDMOND, 671 nm center

wavelength, 25  mm diameter, 10  nm bandpass filter; EDMOND, 830  nm center wavelength, 25 mm diameter, 10 nm bandpass filter, respectively) were attached to the two monochrome CMOS sensors each. The lighting unit was a halogen lamp (MORITEX MHF-D 100 LR) having a homogeneous spectral distribution in the wavelength range of 200–900 nm, connected to a ring-­shaped light installed coaxially via a converging optical fiber. The personal computer unit was LAN-connected to the three CMOS sensors and could simultaneously perform filming at 24 Fr/s using a dedicated program that we developed. The still images from the three sensors were simultaneously acquired with a cue, image calculation of the three images was performed, and the quantitative imaging of tis-

a Color CMOS ↓

Sensor unit

Monochrome ←CMOS Beam Spritter

LAN cable

Monochrome ←CMOS

PC

Ring light Optical fiber

Fig. 10.1 (a) Configuration of multispectral camera: sensor unit, lighting unit, and control and analysis PC unit. (b) Optical path in the sensor unit: reflected light from the subject was split in three directions using two beam split-

Light source

ters and simultaneously sensed by one RGB color CMOS sensor (SENTECH, STC-SC 83 POE) and two monochrome CMOS sensors (Sentech, STC-CMB 2 MPOE-IR)

10  Novel Multispectral Device for Quantitative Imaging of Tissue Oxygen Saturation and Hemoglob…

95

b 671nm

830nm

Color

Fig. 10.1 (continued)

sue oxygen saturation and hemoglobin amount was performed.

10.2.1 The Principle of Quantifying Tissue Oxygen Saturation The spectral characteristics of Hb are generally measured as absorbance spectra [13, 14]. Hyeonsoo Chang et al. measured reflectance spectra with a specially designed fiber-optic–based probe (Ocean Optics R400-7-UV-VIS, Dunedin, Florida U.S.A) [10, 15]. They adjusted HCT 44% porcine whole blood samples to oxygen saturations of 99%, 70%, and 0% and measured the reflectance. As a result, it was revealed that each blood sample having different oxygen saturation has specific reflection characteristics, and the reflectance increases as O2 Saturation =



10.2.2 Method for Quantification of Tissue Oxygen Saturation On the basis of the Lambert–Beer law, oxyHb and deoxyHb in blood were illuminated with light of two different wavelengths, making it possible to determine their concentrations. When P1R671 is the reflectance of light with a wavelength of 671 nm at point P1 and P1R830 is the reflectance of light with a wavelength of 830 nm at point P1, the following formula can be obtained:

O2 Hb P R671 , = r× 1 O2 Hb + dHb P1 R830

where r is the correlation coefficient. The amount of reflected light at point P1 (P1LR671, P1LR830) that can be actually measured is the value obtained by multiplying the irradiation light quantity (set as LP1671, LP1830) at point P1 by the reflectance as shown in the following formula: P1 LR671 = LP1 671 × P1 R671 P1 LR830 = LP1 830 × P1 R830

the oxygen saturation increases at a wavelength of 671 nm. However, at the wavelength of 830 nm, the higher the oxygen saturation, the lower was the reflectance (Fig. 10.2a).

(10.1)

Because the total amounts of incident light at the 671 and 830  nm wavelengths are equal for the halogen lamp used for the light source in our study, the following equation is satisfied:

LP1 671 = LP1 830

(10.2)

Y. Haruta et al.

a

b 0.6

0.6

Completely De-Oxygenated

Arterial 0.5

40%

44%

36%

0.5 Reflectance (A.U.)

Reflectance (A.U.)

32% 0.4 Venous 0.3 0.2

27%

0.3

0.5 0.4 0.3 0.2 0.1 Siluted by Saline

0.0 20

25 30 35 40 Hematocrit (%)

45

22%

0.2 0.1

0.1 0.0 500

0.4

Reflectance at 627 nm (A.U.)

96

0.0 600

700

800

900

Wavelength (nm)

Fig. 10.2 (a) Reflectance spectra of arterial blood (100% oxygen hemoglobin), venous blood (70% oxygen hemoglobin), and completely deoxygenated blood: at a wavelength of 671 nm, the reflectance increases as the oxygen saturation increases. On the contrary, at the wavelength of 830 nm, the reflectance decreases as the oxygen saturation

500

700

600

10.2.3 Demonstration Experiment Using Blood Arterial blood at deep breath (ADB), arterial blood at breath holding (ABH), and venous blood at rest (VR) were sampled three times each from healthy volunteers. The arterial blood oxygen partial pressure (PaO2) and hematocrit (HCT) were measured from a part of each blood sample. The remaining blood was placed in an acrylic cell (10 × 45 mm; light path: 10 mm) (3.5 mL) and 0.5  mL of heparin Na (1000  units/10  mL) was added and the mixture was sealed with a rubber cap. Filming was performed from a distance of

900

decreases. (b) Reflectance spectrum of arterial blood at different HCTs; when the HTC changed, the reflectance spectra of each HCT also changed; the shape of the graph remained constant and only the height of the spectrum changed ([10], Fig. 10.3)

P L671 × P1 R671 P LR671 O2 saturation = r × 1 . = r× 1 P L 671 × P R 830 P 1 1 1 LR830 The ratio of reflected light between 671 and 830  nm from point P1 correlates with oxygen saturation at P1.

800

Wavelength (nm)

(10.3)

70 cm. The ratio of reflected light at wavelengths of 671 and 830 nm was calculated and the oxygen saturation was quantified (Fig. 10.3a). Each measurement was performed three times, and we examined whether the quantitative oxygen saturation values correlated with the measured PaO2 for the blood in the three groups. The mean PaO2 values were 98 ± 0.8% for ADB, 87.3 ± 2.1 for ABH, and 76.4  ±  2.0% for VR.  The measured HCT was 46  ±  3.4% for ADB, 43  ±  4.3% for ABH, and 46 ± 2.5% for VR. The reflected light amount (luminance value, 8 bit) at a wavelength of 671  nm was as follows: ADB, 165.2  ±  6.1; ABH, 140.3 ± 5.4; and VR, 118.3 ± 3.3; a strong correlation with PaO2 was noted (correlation coefficient = 0.97). In contrast, the reflected light amount at a wavelength of 830  nm was as follows: ADB, 95.5 ± 4.2; ABH, 106.0 ± 3.3; and VR, 111.2  ±  2.3; a negative correlation with

10  Novel Multispectral Device for Quantitative Imaging of Tissue Oxygen Saturation and Hemoglob…

97

a

671nm

170

Reflected light amount (A.U.)

b

830nm

145 ADB 120 ABH VR 95

70 75.

80.

85.

90.

95.

100.

SaO2(%) 670nm / 830nm

c Ratio of the reflected light amount (A.U.)

2.33 ADB 2.1

.88 ABH .65

VR

.43

1.2 75.

80.

85.

90.

95.

100.

SaO2(%)

Fig. 10.3  Experiment to quantify oxygen saturation in blood. (a) Right: filming of blood from a distance of 70  cm. Left: collected blood (from right: ADB, ABH, VR). (b) Reflected light amount for each wavelength of the ADB, ABH, and VR groups. The reflected light amount at a wavelength of 671 nm increases as the oxygen saturation increases, and the reflected light amount at a

wavelength of 830 nm decreases as the oxygen saturation increases. (c) Values of the ratios of the reflected light amounts at a wavelength of 671 and 830 nm of the ADB, ABH, and VR groups are highly correlated with oxygen saturation. (d) PaO2 of collected blood, color image, and pseudo-color image of 671 nm/830 nm

98

Y. Haruta et al.

d

ADB, 1.39 ± 0.09 for ABH, and 1.05 ± 0.10 for VR; a strong correlation with PaO2 was noted (correlation coefficient = 0.93) (Fig. 10.3d, e).

HbO2

66%

94%

100%

PaO2

83%

91%

100%

Color

10.2.4 Quantification of Tissue Oxygen Saturation

High Oxygen saturation

671nm 830nm

Low

Fig. 10.3 (continued)

PaO2 was noted (correlation coefficient = −0.49) (Fig. 10.3c). The ratio between light with wavelengths of 671 and 830  nm was 1.8  ±  0.13 for O2 Saturation = K1

In order to standardize the scaling of the quantitative values and the measured values, it was necessary to determine an appropriate reference value. As a reference value of oxygen saturation, based on the fact that the PaO2 of arterial blood (Point Pa) is controlled to 100% intraoperatively, the coefficient K1 was determined so that the quantitative value (R671  nm/R830  nm) of oxygen ­saturation in the pulsating artery was 100 as the highest value.

O2 Hb P LR671 = K1 1 O2 Hb + dHb P1 LR830

Pa LR671 = 100 Pa LR830

P LR671 K1 = 100 ÷ a Pa LR830

10.2.5 The Principle of Quantification of Hemoglobin Amount Hyeonsoo Chang and colleagues similarly measured the whole blood reflectance spectra by adjusting the oxygen saturation of 99% in blood sampled from pigs to HTCs of 44%, 40%, 36%, 32%, 27%, and 22%. The results revealed that the oxyHb and (oxyHb  +  deoxyHb) ratios (oxygen saturation)

(10.4)

remained the same, and when the HTC (hemoglobin concentration) changed, the reflectance in each spectrum also changed, with the shape of the graph remaining constant and only the height changing (Fig. 10.2b). Therefore, when the ratio of oxyHb and deoxyHb (oxygen saturation) remains constant, the amount of hemoglobin is proportional to the reflectance at each wavelength. Also, if the amount of hemoglobin is constant, the amount of reflected light increases as the amount of incident light increases.

P 1H ( Hb amount ) ∝ Reflection light amount of 671 nm, P 1H ( Hb amount ) ∝ Reflection light amount of 830 nm. P1H is defined as the light amount of the halogen lamp at point P1. However, when comparing Hb amount at two points (or two times) with different oxygen saturation values, comparisons cannot be made based solely on the reflectance in a single spectrum. This is because even if the reflectance at a wavelength of 671 nm remains the same, if the oxygen sat-

uration is different, the reflectance at a wavelength of 830 nm differs. When the reflectance of Hb is measured at a wavelength of 671  nm, the reflectance rises at a constant rate if the oxygen saturation rises. However, in the case of measurement at a wavelength of 830 nm, the reflectance decreases as the oxygen saturation rises. However, the range of

10  Novel Multispectral Device for Quantitative Imaging of Tissue Oxygen Saturation and Hemoglob…

change is different depending on the wavelength. It is possible to calculate the values correlating

to the hemoglobin amount using the abovementioned findings and the following equation:

P1H ( Hb amount ) ∝ ( P1 LR671 + i × P1 LR830 ) , P 1H ( Hb amount ) = a × ( P1 LR671 + i × P1 LR830 ) a : proportional constant where i is the coefficient for the difference in reflectance change due to oxygen saturation at a wavelength of 671 and 830 nm.

10.2.6 Method for Quantifying Hemoglobin Amount From the Eq. (10.5), the sum of the measured values ​​at 671 and 830  nm is proportional to the Hb amount and the incident light amount. When calculating oxygen saturation (the relative concentration of oxyHb and deoxyHb), it was possible to cancel the effect of the changing

99

(10.5)

illumination by taking the ratio of the reflected light at each wavelength, but because the sum was used for the determination of hemoglobin amount, it was also necessary to correct for the difference in illumination caused by the unevenness of the tissue and nonuniform illumination. Of the RGB color components simultaneously capturing light from the same light source, the green component and the blue component, which were less influenced by the changes of Hb, were used for correction. When the blue and green light quantity values​​ measured at the point P1 are defined as B  +  G, P1H and B + G are in a proportional relationship.

P1H ∝ ( B + G ) P1H = b × ( B + G ) b : proportional constant

(10.6)

Therefore, Hb amount obtained by dividing P1H from (5) can be obtained as the following formula. Hbamount = a ×



P LR671 + iP1LR830 P1LR671 + iP1LR830 a P1LR671 + iP1LR830 =r× 1 = × P1H b (B + G) (B + G)

r is a proportional constant defined as a/b. Furthermore, in theory it should be possible to correct for the nonuniformity of illumination and quantify the hemoglobin amount.

10.3 Experimental 10.3.1 Verification Experiment of Luminance Nonuniformity Correction Due to Organ Shape Five porcine organs (small intestine, colon, rectum, uterus, and stomach) of different shapes were placed on a corkboard, and the RGB color, 671 nm, and 830 nm spectral moving images were

(10.7)

filmed from 70 cm directly above. We compared the sum of the quantities of reflected light at wavelengths of 671 and 830 nm (before correction) and the value obtained by taking the ratio of the sum of the quantities of reflected light at wavelengths of 671 and 830  nm to the sum of the reflected light from the B and G components of the image (after correction) (Fig. 10.4c). Measurement was performed on a linear area (width 20 pixels and height 900 pixels) from the center to the left and right sides, and the mean values from 20-pixel wide sections are shown in the graph (Fig. 10.4). The results of calculating the coefficients of variation (standard deviation and mean value) from the mean values and the standard deviations of the height of 900 pixels and the width of 20 pixels revealed that by correcting the ratio by taking the

Y. Haruta et al.

100

sum of B and G components of the color image, the coefficients of variation by organ decreased as follows: small intestine, from 0.174 to 0.045;

stomach, from 0.293 to 0.037; colon, from 0.267 to 0.080; rectum: from 0.213 to 0.049; and uterus, from 0.257 to 0.013 (Table 10.1).

a

Gray Value

150

100

0

200

400

600

800

Distance (pixels)

Fig. 10.4  Verification of correction by B + G using porcine organs. Small intestine samples of different shapes were placed on a corkboard, and RGB color, 671 nm, and 830 nm spectral moving images were filmed from 70 cm directly above. The sum of the reflected light at a wavelength of 671 and 830 nm at a certain moment was compared with the values obtained by taking the ratio by the sum of the reflected light amounts of the B and G compo-

nents of the color image. Measurement was performed in a linear area with width of 20 pixels and height 900 pixels from the center to the left and right, and the mean value for sections with a width of 20 pixels is shown in the graph. The top is a color image of the esophagus. (a) is an image and graph of the sum of reflected light at a wavelength of 671 and 830 nm before correction. (b) Shows a 671 nm + 830 nm/B + G image and graph

10  Novel Multispectral Device for Quantitative Imaging of Tissue Oxygen Saturation and Hemoglob…

b

Gray Value

150

100

0

200

400 Distance (pixels)

Fig. 10.4 (continued)

600

800

101

Y. Haruta et al.

102 Table 10.1  The changes of coefficient of variation (CV) by shading correlation 671 nm + 830 nm Gray value 138.3 ± 15.9 132.6 ± 18.8 102.5 ± 17.1 102.7 ± 17.3 101.7 ± 19.3

Small intestine Stomach Colon Rectum Uterus

(671 nm + 830 nm)/(B + G) Gray value CV 147.9 ± 6.7 0.045 150.6 ± 9.7 0.037 101.6 ± 8.1 0.080 142.3 ± 13.0 0.049 104.3 ± 3.4 0.013

CV 0.174 0.293 0.267 0.213 0.257

10.3.2 Verification Experiment of Correlated Values of Hemoglobin Amount Venous blood from three healthy volunteers was collected and diluted with heparin Na (1000  U/10  mL) to adjust HCT to 40%, 38%, 36%, 34%, and 32%. One drop of each sample of HTC blood was dropped onto a slide glass and sealed with cover glass (Fig.  10.5a). The prepared specimens were arranged side by side in descending order of HTC, and the RGB color, 671 nm, and 830 nm spectral images were taken from 70 cm directly above. The sum of the reflection amounts at a wavelength of 671 and 830 nm at a certain moment was corrected by taking the ratio by the sum of the B and G components of the color image (Fig. 10.5b). The results revealed

that the sum of the measured light amounts of 671 and 830  nm wavelength light of each prepared specimen were correlated with the HTC value (correlation coefficient = 0.96) (Fig. 10.5c). The value of (R671/R830) correlating with oxygen saturation was almost constant, and no correlation with HCT was observed (correlation coefficient = 0.014) (Fig. 10.5d).

10.3.3 Quantification of Hemoglobin Amount In order to unify the scale of the measured values of Hb and the quantitative values, the correction value K2 was determined so that (R671 + R830)/R (B + G) of the pulsating artery (Point Pa) was 100.

Hb amount = O2 Hb + dHb = K 2 K2

P1 R671 + iP1 R830 (B + G)

(10.8)

Pa R671 + iPa R830 = 100 (B + G)

K 2 = 100 ÷

Pa R671 + iPa R830 (B + G)

10.4 Discussion We clinically applied close proximity remote sensing technology and developed a noninvasive, real-time quantitative imaging system for biological tissue and conducted nonclinical and clinical studies to verify its utility. While quantifying the correlation of oxygen saturation with the ratio of oxyHb and deoxyHb due to the shape of the subject, nonuniform illumination, and the influence of ambient light, it

was possible to correct measurements by taking the ratio of the reflected light at the two wavelengths. However, in the quantification of hemoglobin amount, which is the total amount of oxyHb and deoxyHb, corrections were performed by taking the ratio by the sum of the reflected light amounts of B and G light in the RGB color image being filmed. In order to unify the scale of the measured values and quantitative values, a method of determining the correction coefficient using values

10  Novel Multispectral Device for Quantitative Imaging of Tissue Oxygen Saturation and Hemoglob…

103

a

b

670

(670

830

+

830)

/

(G +

B)

Gray Value

160

G

B

140 120 100 0

200

400 Distance (pixels)

600

Color

Fig. 10.5  Verification experiment of quantification of hemoglobin amount. (a) Filming condition from 70  cm directly (left) above the five prepared specimens containing blood with different HCT (center, right). (b) The sum of the reflected light amounts at a wavelength of 671 and 830 nm (upper left) was taken for B + G (lower left); as shown in the graph, a correlation between the amount of reflected light of the resulting image (right) and HCT was

noted. (c) Graph (left) and table (right) showing changes in the light amount of (671 nm + 830 nm)/(B + G) due to HCT values of the blood enclosed in the prepared specimens. (671 nm + 830 nm)/(B + G) values were correlated with HCT value. (d) Graph showing changes in the value of (R671/R830) due to HCT values of the blood enclosed in the prepared specimens. No correlation between HTC values and oxygen saturation was noted

Y. Haruta et al.

104

c Change of reflection (671 nm+830nm)/(B+G) by HCT 140 Reflected light amount

HCT 32%

120 r=0.96 100

(671nm+830nm) / (B+G) 70.1 ± 3.6

34%

86.8 ± 1.9

36%

99.5 ± 10.4

38%

115.3 ± 6.1

40%

129.0 ± 3.96

80

60 32

34

36

38

40

Hematocrit (%)

d

Change of reflection (R671/R830) by HTC

Ratio of the reflected light amount

1.

0.8

0.6

0.4

0.2

0. 32

34

36

38

40

Fig. 10.5 (continued)

extracted from the reflected light of the pulsating artery as a reference value (100) was used. With regard to the oxygen saturation correction coefficient, in principle, intraoperative arterial oxygen saturation while the patient received general anesthesia was maintained at the highest value of 100. However, in an environment in which oxygen saturation could not be maintained, it was necessary to change the setting, for example, to reset the correction coefficient to have a maximum value of 90. In principle, the HTC value of pulsating arterial blood is a constant value throughout the body; therefore, this value was taken as the reference value for quantifying the

Hb amount. Obviously, when HTC rapidly fluctuates even in the same patient, or when comparison is made among patients with different HTC values, it is necessary to recalculate the correction coefficient. By combining tissue oxygen saturation and hemoglobin amount, it becomes possible to understand the hemodynamics and oxygen metabolism of the tissue (Fig.  10.5). For example, if the artery supplying oxygen as oxyHb to the tissue becomes narrow, as long as the supply decreases and the oxygen metabolism of the tissue does not decrease, the tissue oxygen saturation decreases along with the decrease of

10  Novel Multispectral Device for Quantitative Imaging of Tissue Oxygen Saturation and Hemoglob…

the tissue hemoglobin amount (ischemia). In contrast, blood inflow is not hindered, and if the venous return decreases due to capillary failure or venous obstruction, the hemoglobin amount increases and the oxygen saturation decreases (congestion). If the congestion persists, blood inflow should eventually decrease. Furthermore, hemodynamics and oxygen metabolism of tumors differ from those of normal tissue. In many malignant tumors, angiogenesis cannot catch up with the rate of proliferation; hemoglobin amount and oxygen saturation simultaneously decrease, causing ischemia [16–20]. However, in the case of hemangioma with prominent arterial proliferation, hemoglobin amount and oxygen saturation simultaneously increase. In any case, even if the tumor has the same shape as the normal tissue, it can be expected that the boundary with the normal tissue will be clearly depicted by imaging the hemoglobin amount and oxygen saturation of the tissue. In the past, in order to clearly intraoperatively visualize the site where a tumor existed, it was necessary to inject a contrast medium into a blood vessel and to place a large-sized apparatus such as a radiological device into the operative field, creating temporal and spatial obstacles to operative treatment [21, 22]. Evaluation of tissue inflammation was performed mainly on the basis of macroscopic findings and changes in temperature revealed by palpation [23, 24]. During the acute phase of inflammation, cytokines cause vasospasm or vasodilation, and the tissue hemoglobin amount and oxygen saturation change. Furthermore, during the chronic phase, the tissue hemoglobin amount and oxygen saturation differ depending on whether the tissue is necrotic or is in a state of recovery accompanied by vascular growth.

References 1. Zheng P, Xu P, Yao Q, Tang K, Lou Y. 3D-printed navigation template in proximal femoral osteotomy for older children with developmental dysplasia of the hip. Sci Rep. 2017;7:44993. 2. Jermakowicz WJ, Diaz RJ, Cass SH, Ivan ME, Komotar RJ. Use of a mobile intraoperative computed tomography scanner for navigation registration dur-

105

ing laser interstitial thermal therapy of brain tumors. World Neurosurg. 2016;94:418–25. 3. Nishino H, Hatano E, Seo S, Nitta T, Saito T, Nakamura M, Hattori K, Takatani M, Fuji H, Taura K, Uemoto S. Real-time navigation for liver surgery using projection mapping with indocyanine green fluorescence: development of the novel medical imaging projection system. Ann Surg. 2018;267(6):1134–40. 4. Westhorpe RN, et  al. The pulse oximeter. Anaesth Intensive Care. 2008;36:767. 5. Movahedian AH, Mosayebi Z, Sagheb S. Evaluation of pulse oximetry in the early detection of cyanotic congenital heart disease in newborns. J Tehran Heart Cent. 2016;11(2):73–8. 6. Welsh EJ, Carr R.  Pulse oximeters to self monitor oxygen saturation levels as part of a personalized asthma action plan for people with asthma. Cochrane Database Syst Rev. 2015;27(9):CD011584. 7. Colas-Ribas C, Signolet I, Henni S, Feuillloy M, Gagnadoux F, Abraham P. High prevalence of known and unknown pulmonary diseases in patients with claudication during exercise oximetry: a retrospective analysis. Medicine. 2016;95(40):e4888. 8. Severinghaus JW, Astrup PB.  History of blood gas analysis. I.  The development of electrochemistry. J Clin Monit Comput. 1985;1(3):180–92. 9. Watanabe E, Yamashita Y, Maki A, Ito Y, Koizumi H.  Non-invasive functional mapping with multi-­ channel near infra-red spectroscopic topography in humans. Neurosci Lett. 1996;205(1):41–4. 10. Chang H, Kim YL, Hassan A, Fitzgerald PJ.  Whole blood reflectance for assessment of hematologic condition and detection of angiographic contrast media. Appl Opt. 2009;48(13):2435–43. 11. Peters RA. Chemical nature of specific oxygen capacity in haemoglobin. J Physiol. 1912;44(3):131–49. 12. Tsutusmi R, Ikeda T, Nagahara H, Saeki H, Nakashima Y, Oki E, Maehara Y, Hashizume M.  Efficacy of novel multispectral imaging device to determine anastomosis for esophagogastrostomy. J Surg Res. 2019;242:11–22. 13. Rubinstein DL, Ravikovich HM.  Absorption spectrum of haemoglobin in red cells. Nature. 1946;158(4026):952. 14. Horecker BL. The absorption spectra of hemoglobin and its derivatives in the visible and near infra-red regions. J Biol Chem. 1943;148(1):173–83. 15. Yasuda T, Saito T, Kihara T, Takatani S, Funakubo A. Development of a reflected optical fiber system for measuring oxygen saturation in an integrated artificial heart-lung system. Artif Organs. 2008;32(3):229–34. 16. Gorski DH, Beckett MA, Jaskowiak NT, Calvin DP, Mauceri HJ, Salloum RM, Seetharam S, Koons A, Hari DM, Kufe DW, Weichselbaum RR. Blockade of the vascular endothelial growth factor stress response increases the antitumor effects of ionizing radiation. Cancer Res. 1999;59(14):3374–8. 17. Abdollahi A, Lipson KE, Han X, Krempien R, Trinh T, Weber KJ, Hahnfeldt P, Hlatky L, Debus J, Howlett AR, Huber PE.  SU5416 and SU6668 attenuate the

106 angiogenic effects of radiation-induced tumor cell growth factor production and amplify the direct anti-­ endothelial action of radiation in  vitro. Cancer Res. 2003;63(13):3755–63. 18. Mallidi S, Luke GP, Emelianov S.  Photoacoustic imaging in cancer detection, diagnosis, and treatment guidance. Trends Biotechnol. 2011;29(5):213–21. 19. Welter M, Fredrich T, Rinneberg H, Reiger H.  Computational model for tumor oxygenation applied to clinical data on breast tumor hemoglobin concentrations suggests vascular dilatation and compression. PLoS One. 2016;11(8):e0161267. 20. Hong BJ, Kim J, Jeong H, Bok S, Kim YE, Ahn GO.  Tumor hypoxia and reoxygenation: the yin and yang for radiotherapy. Radiat Oncol J. 2016;34(4):239–49. 21. Wagner F, Hakami YA, Warncock G, Fischer G, Hueller MW, Veit-Haibach P. Comparison of contrast-­

Y. Haruta et al. enhanced CT and [18F]FDG PET/CT analysis using kurtosis and skewness in patients with primary colorectal cancer. Mol Imaging Biol. 2017;19(5):795–803. 22. Bhunchet E, Shibata T. Proposal for two strategies to prevent remnants of gastric cancers after endoscopic mucosal resections: fluorescein electronic endoscopy and rapid stump diagnosis based on pit patterns. Gastric Cancer. 2004;7(4):221–32. 23. Mohammed N, Subramanian V. Clinical relevance of endoscopic assessment of inflammation in ulcerative colitis: can endoscopic evaluation predict outcomes? World J Gastroenterol. 2016;22(42):9324–32. 24. Hasmann A, Wehrschuetz-Sigl E, Kanzler G, Gewessler U, Hulla E, Schneider KP, Binder B, Schintler M, Guebitz GM. Novel peptidoglycan-based diagnostic devices for detection of wound infection. Diagn Microbiol Infect Dis. 2011;71(1):12–23.

Clinical Benefit of Mixed Reality Holographic Cholangiography for Image-Guided Laparoscopic Cholecystectomy

11

Michiko Kitagawa, Maki Sugimoto, Akiko Umezawa, and Yoshimochi Kurokawa

11.1 Introduction With recent advancements in information and communication technology, three-dimensional reconstructions of medical images from techniques such as computed tomography (CT) and magnetic resonance imaging (MRI) are being utilized as images for guiding surgery in the operating room. By enabling an accurate understanding of the three-dimensional anatomical relationship between a target pathology and its surrounding organs, these images support smooth, safe operations through their use in preoperative procedural considerations or simulations as well as intraoperative navigation or communication between the surgeon and assistant. But these images can only be viewed on flat, rectangular monitors of limited size. Gaining a correct understanding of anatomy and organ depth requires a high level of spatial awareness, but this is not easy to acquire.

M. Kitagawa (*) · A. Umezawa · Y. Kurokawa Department of Endoscopic Surgery, Yotsuya Medical Cube, Tokyo, Japan e-mail: [email protected]; [email protected]; [email protected]

Endoscopic surgeries such as laparoscopy and thoracoscopy are performed while looking at a video camera display, but in exchange for a noninvasive procedure the surgeon must work by viewing a small screen that dulls the sense of depth, all while imagining three-dimensional structure of the organs. Therefore, in one of the most commonly performed laparoscopic surgeries, laparoscopic cholecystectomy, misidentification of the complex anatomy and inflammatory changes still leads to a certain probability that serious complications such as bile duct injuries will occur. To avoid such errors, there has been a need to develop a system capable of accurately identifying an individual patient’s pathology and three-­dimensional anatomy and allowing this structure to be viewed in three dimensions during surgery. Recent reports have described the utility of technologies such as virtual reality (VR) and mixed reality (MR) that use medical images in three-­dimensional diagnosis, surgical planning, and education [1, 2]. We have developed and used a VR/MR system for laparoscopic surgery to improve spatial awareness while maintaining noninvasiveness and avoiding misidentification, and clinical applications and outlook of the system are explained in this chapter.

M. Sugimoto Innovation Lab, Teikyo University Okinaga Research Institute, Tokyo, Japan e-mail: [email protected] © Springer Nature Singapore Pte Ltd. 2021 S. Takenoshita, H. Yasuhara (eds.), Surgery and Operating Room Innovation, https://doi.org/10.1007/978-981-15-8979-9_11

107

108

M. Kitagawa et al.

11.2 Image-Guided Laparoscopic Cholecystectomy: Current Trends and Issues An imaging technology used to guide laparoscopic cholecystectomy include computed tomography (CT), magnetic resonance cholangiopancreatography (MRCP), endoscopic retrograde cholangiography, and intraoperative cholangiography using fluoroscopy or indocyanine green (ICG), but these images have two problems that need to be overcome. Because images that have been used in intraoperative guidance are viewed as is, in two dimensions, the anatomy cannot be understood intuitively, and some images make it difficult to avoid vascular damage other than bile duct injury. All images used in intraoperative guidance have been planar, two-dimensional (2D) images that require the surgeon to imagine three-­ dimensional structures to understand the anatomy. Therefore, when performing a laparoscopic operation, the surgeon must first view the two-­dimensional image, imagine the three-­ dimensional (3D) image, and then perform the operation while viewing the surgical field in the monitor; a 2D  →  3D  →  2D conversion process is required. To facilitate the conversion from 2D to 3D, three-dimensional MRI or CT images are sometimes used as intraoperative guides, but these are planar images stereoscopically delineated with contrast and shading to appear three-­ dimensional and are not true 3D.  And although the ability to rotate images by using a mouse or touch panel may provide a degree of directional freedom, it is not possible to fully ascertain qualities such as organ depth and volume. Therefore, a degree of experience is necessary to achieve instant spatial awareness from the 2D image and understand the anatomy, leaving the possibility that a relatively inexperienced surgeon might fail to accurately understand the anatomy and cause bile duct injury. In addition to the anatomical variation of the cystic artery, inflammation can narrow the relative proximity of the cystic artery, the cystic duct, and the common bile duct, making it difficult to understand the anatomy and thereby increasing

Fig. 11.1  Conventional image-guided laparoscopic cholecystectomy with intraoperative cholangiography

the risk of hemorrhage or damage to the cystic duct [3–7]. Intraoperative cholangiography using fluoroscopy or ICG is therefore very effective for confirming the path of the bile duct in real time, but because the path of the cystic artery cannot be ascertained, the risk of hemorrhage remains even if the risk of bile duct injury can be avoided (Fig. 11.1). Exploration into VR and MR image guidance is moving forward to find a method of overcoming these two problems.

11.3 VR for Surgery in the Operating Room VR is defined by the American Heritage Dictionary as a computer simulation of a real or imaginary system that enables a user to perform operations on the simulated system and shows the effects in real time. It is also a generic term given to technology that artificially produces an environment that feels like reality. In images created with this technology, organs can be seen in three dimensions with a natural sense of depth. Therefore, the surgeon viewing the image can intuitively understand the anatomy without having to imagine the three-dimensional structure. And because the organs displayed can be observed in 360°, from all directions, the positional relationship with respect to surrounding organs can also be instantly understood. VR can

11  Clinical Benefit of Mixed Reality Holographic Cholangiography for Image-Guided Laparoscopic…

create images that assist spatial recognition and enable an accurate understanding of the anatomy, which would have previously depended on the experience of the surgeon with a flat image. However, using VR in image-guided surgery presents a problem. VR images are not linked to the real world, and the operating environment is not visible without removing the VR headset. So, it is impossible to check the anatomy against the actual organs displayed on the surgical monitor. Therefore, although VR images are highly effective tools for understanding the anatomy before an operation and for surgical simulations, they are not enough for intraoperative guidance. For this reason, image-guided surgery using mixed reality (MR) has attracted attention in recent years as a means of displaying stereoscopic images without hindering the progress of an operation.

109

can reinforce the surgeon’s spatial awareness and enable an intuitive understanding of the anatomy while the operation is being performed.

11.5 Characteristics of the MR Terminal One example of a commercially available MR terminal is the HoloLens (Microsoft), a head-­ mounted wearable computer sold since 2016 [9]. The central feature of this terminal is a self-­ contained holographic computer that projects a holographic image onto a semi-transmissive lens so that the wearer can view the image projected on the lens through goggles as if it exists in real space. Other features of the HoloLens are as follows:

• It is possible to view individualized holograms based on data extracted from patients’ CT images, so no special imaging is required. 11.4 MR for Surgery • The superimposed data display is adjusted by the system in real time according to the wearin the Operating Room er’s movements to maintain its superimposed In MR, real space is combined with virtual space alignment in the surgical field. Thus, the to construct a new space in which the real and image can be viewed without blurring, irrevirtual can be mutually affected in real time. spective of the direction that the wearer is Using this technology, a VR image can be fused facing. with the real world. • Wi-Fi sharing allows all HoloLens wearers to When this technology is used in image-guided observe the same images. surgery, the organ of interest is displayed as a • Because the central processing unit, graphics hologram in the surgical field. This allows the processing unit, and holographic processor are surgeon to observe a three-dimensional image self-contained, the goggle-type device can be while performing an operation. used alone with no delay in data response. Using MR to display images during orthope- • The system is wireless, which makes wire dic surgery has been reported to be more effective placement and sterilization of equipment than displaying conventional planar images [8]. unnecessary. When MR is used for intraoperative image-­ • The wearer can manipulate images by using guided laparoscopic cholecystectomy, the organs gestures, which allows the surgeon to comnecessary to the surgery—the gallbladder, bile fortably perform an operation in a sterile enviduct, and cystic artery—are displayed in three ronment without requiring sterilization of a dimensions in the surgical field. Therefore, the control panel or having an assistant use the surgeon can perform the operation while viewcontrols. ing images in the context of the actual organs displayed on the surgical monitor. Because the HoloLens is a stand-alone device Displaying images through MR can be called that takes up little space and does not require an ideal image-guided surgery technique, which sterilization, it is optimal for use in the operating

110

Fig. 11.2  Mixed reality head mount display: HoloLens®

room, where a clean, sterile environment must be maintained in a limited space with many pieces of anesthesia and surgical equipment. In the next section, the practice of MR-guided surgery using HoloLens is explained (Fig. 11.2).

M. Kitagawa et al.

Fig. 11.3  3D reconstruction image after detecting cystic duct, cystic artery, liver, and bone from CT (white arrow: cystic artery)

11.6 M  ethods of MR for Surgery in the Operating Room Before MR-based image-guided surgery can be performed, the patient’s organ information must be loaded into the HoloLens. Only the data necessary for the operation, such as biliary tract and cystic artery data, are extracted from drip infusion cholangiography with computed tomography (DIC-CT) or contrast-­enhanced CT images taken for the patient’s preoperative evaluation, and the images are polygonized by processing through the existing digital imaging and communications in medicine (DICOM) viewer. Polygonized data are prepared for application by using a commercial web service [10] and downloaded to the HoloLens as patient information (Fig. 11.3). Before beginning the operation, the surgeon and assistant put on the HoloLens to position the 3D images of the patient’s biliary tract and biliary arteries downloaded to the HoloLens as a hologram visible in the surgical field space. At this time, it was possible for all those wearing a device to zoom in and confirm parts of the anatomy that would require caution during the operation, which was an effective means of sharing information just before the operation.

Fig. 11.4  Image-guided laparoscopic cholecystectomy using mixed reality

Fig. 11.5  Image manipulation through gesture control

After displaying the 3D images in the optimal location, laparoscopic cholecystectomy was performed as usual. Surgeons were also able to use gestures to turn the image display on or off as necessary (Figs. 11.4 and 11.5).

11  Clinical Benefit of Mixed Reality Holographic Cholangiography for Image-Guided Laparoscopic…

11.7 Discussion Previously, if inflammation had made the path of the cystic duct difficult to understand during an operation, a certain level of experience was needed to quickly and accurately understand the anatomy by using conventional planar images from DIC-CT or MRCP. But MR’s holographic function allows images of the gallbladder and surrounding vessels to be visualized as if they were there, so that the anatomy can be understood intuitively, and the spatial relationships can be grasped instantaneously. Therefore, complications such as bile duct injury caused by a­ natomical misidentification could be reduced, regardless of the surgeon’s level of experience. For example, if the cystic duct were twisted dorsally at the gallbladder neck as it joined the common hepatic duct or if the cystic duct joined the right hepatic duct, an inexperienced physician using conventional planar images would have difficulty accurately ascertaining the cystic duct’s path and positional relationship with surrounding organs. In MR, the cystic duct can be viewed from all angles and compared with the actual organ, so the operation can be performed without damaging the common bile duct. Wearing the HoloLens allows the surgeon to understand the anatomy with a sense of immersion because the gallbladder and cystic artery appear to float in space. And because MR can uniformly reinforce spatial awareness, which previously depended on the surgeon’s level of experience, it is possible to perform safer operations with reduced risk of complications such as bile duct and arterial injury caused by anatomical misidentification.

111

image-guided techniques. The advantages and disadvantages of MR-based image guidance are summarized below.

11.8.1 Advantages The most noteworthy advantage is that the stereoscopic display allows the organs to appear to float in the surgical field, so that the images can be observed from all directions. This allows the surgeon to intuitively understand the anatomy and the positional relationship between organs. It is therefore possible for less-experienced surgeons to reinforce their spatial awareness and for experienced surgeons to achieve an even deeper understanding of the anatomy than was possible with previous image-guided techniques, which could reduce complications such as bile duct injury. The ability to turn the displayed image off or back on at any time by using gesture controls also reduces time lag and stress during the operation.

11.8.2 Disadvantages

One disadvantage of MR-based image guidance is that it is not a real-time image like intraoperative cholangiography. The displayed stereoscopic image is created from preoperative CT, so it is not possible to confirm the existence of an intraoperative bile duct injury or choledocholithiasis. In addition, because CT imaging of the biliary tract and blood vessels is necessary, holographic images cannot be used for image-guided surgery in patients who cannot tolerate contrast media because of bronchial asthma or an allergy to contrast media. The total weight of the HoloLens and its 11.8 Advantages headband has been reported to be 579 g. Because and Disadvantages this is heavier than a surgical loupe and headof Image-Guided Surgery band, which weighs around 270 g, there is some with MR concern about physical burden, but use of the In laparoscopic cholecystectomy, image-­HoloLens for 90  min without physical fatigue guidance techniques that use MR have different including neck pain or eye strain during or after features than conventional CT- or MRI-based use has been reported [11]. However, there have

112

been a few reports of mild headaches during or after HoloLens use or, after prolonged use, ­episodes of VR sickness characterized by nausea or dizziness [12], so use should be promptly discontinued if the surgeon wearing HoloLens notices signs of physical fatigue. Alternatively, it could be recommended that the device be worn only when checking the anatomy.

11.9 Conclusion MR-based intraoperative image guidance is regarded as a revolutionary surgical support method that reinforces intuitive understanding of the anatomy and spatial awareness. In the future, expected hardware improvements include enhanced resolution, image processing, and sensor performance, reduction in the weight of the device, faster communication speed, and extended operating time. To enable MR devices such as the HoloLens to be widely used in clinical practice, development of software as a medical device and progress toward deregulation are also expected.

References 1. Sugimoto M. Augmented tangibility surgical navigation using spatial interactive 3-D hologram zSpace with OsiriX and bio-texture 3-D organ modeling. In: 2015 international conference on computer application technologies, Matsue, August 2015. Institute of Electrical and Electronics Engineers; 2015. p.  189– 194. https://doi.org/10.1109/CCATS.2015.53.

M. Kitagawa et al. 2. Iannessi A, et  al. A review of existing and potential computer user interfaces for modern radiology. Insights Imaging. 2018;9(4):599–609. https://doi. org/10.1007/s13244-018-0620-7. 3. Nuzzo G, et al. Bile duct injury during laparoscopic cholecystectomy: results of an Italian national survey on 56 591 cholecystectomies. Arch Surg. 2005;140(10):986–92. https://doi.org/10.1001/ archsurg.140.10.986. 4. Navez B, et  al. Surgical management of acute cholecystitis: results of a 2-year prospective multicenter survey in Belgium. Surg Endosc. 2012;26(9):2436– 45. https://doi.org/10.1007/s00464-012-2206-7. 5. Viste A, et al. Bile duct injuries following laparoscopic cholecystectomy. Scand J Surg. 2015;104(4):233–7. https://doi.org/10.1177/1457496915570088. 6. Way LW, et al. Causes and prevention of laparoscopic bile duct injuries: analysis of 252 cases from a human factors and cognitive psychology perspective. Ann Surg. 2003;237(4):460–9. https://doi.org/10.1097/01. SLA.0000060680.92690.E9. 7. Japanese Society of Hepato-Biliary-Pancreatic Surgery. 2018. Tokyo Guidelines 2018. http://www. jshbps.jp/modules/en/index.php?content_id=47. Accessed 21 Mar 2019. 8. Lee SC, et  al. Multi-modal imaging, model-based tracking, and mixed reality visualisation for orthopaedic surgery. Healthc Technol Lett. 2017;4(5):168–73. https://doi.org/10.1049/htl.2017.0066. 9. Microsoft. 2019. HoloLens. https://www.microsoft. com/en-us/hololens. Accessed 21 Mar 2019. 10. Holoeyes, Inc. 2019. Holoeyes. http://holoeyes.jp. Accessed 21 Mar 2019. 11. Cometti C, et  al. Effects of mixed reality head-­ mounted glasses during 90 minutes of mental and manual tasks on cognitive and physiological functions. PeerJ. 2018;6:e5847. https://doi.org/10.7717/ peerj.5847. 12. Tepper OM, et al. Mixed reality with HoloLens: where virtual reality meets augmented reality in the operating room. Plast Reconstr Surg. 2017;140(5):1066–70. https://doi.org/10.1097/PRS.0000000000003802.

Part V Robotic Surgery

Development of Laparoscopic Surgery by Means of Foldable Small Humanoid Robot Hands with Tactile Sensation for Laparoscopic Surgery

12

Masaya Mukai, Ryu Kato, and Hiroshi Yokoi

Abbreviations

ing a port for the camera, and a small incision (i.e., 30–40 mm) is made for the removal of the COL Conventional open laparotomy resected tissue. During standard laparotomy, the HALS Hand-assisted laparoscopic deep part of the pelvic floor cannot be observed, surgery but laparoscopic surgery allows a magnified view pure-LAP Pure laparoscopic surgery to be visualized on a monitor by the surgical team. Robot HALS Robot hand-assisted laparoThus, procedures can be performed more safely scopic surgery [1–4]. However, pure-LAP involves manipulation of four to five forceps, so at least two experienced surgeons are required. In addition, the long operating time results in increased utiliza12.1 Recent Use of HALS tion of anesthesiologists, operating theaters, and staff. Furthermore, the increasing cost of purefor Colorectal Cancer: LAP education/training and the escalating price Advantages of materials have been challenging for small to and Disadvantages medium-sized hospitals in Japan [1–4]. While a In recent years, pure laparoscopic surgery (pure-­ multiple micro-forceps–type robot seems to be LAP) has become very popular. In Japan, a effective for very tight operating fields like that 5- to 6-port system is usually employed, includ- available during prostatic surgery, it is thought that robotic surgery may not be suitable for the gastrointestinal tract including the colon or stomM. Mukai (*) ach, since manipulation of the abdominal organs Department of Surgery, Tokai University Hachioji Hospital, Tokyo, Japan across a larger operating field is required [5, 6]. e-mail: [email protected] HALS is positioned between pure-LAP and R. Kato conventional open laparotomy (COL) and has Department of Mechanical Engineering, Materials long been employed with excellent results in Science, and Ocean Engineering, Yokohama National Western countries and the Middle East [7, 8]. In University, Yokohama, Kanagawa, Japan Japan, HALS was rapidly replaced by pure-LAP e-mail: [email protected] for gastrointestinal surgery. However, Japanese H. Yokoi interest in HALS has recently revived for the Department of Mechanical Engineering and Intelligent Systems, The University of Electro-­ fields of liver and pancreatic surgery. Compared Communications, Tokyo, Japan with pure-LAP, HALS is characterized by (1) a e-mail: [email protected]

© Springer Nature Singapore Pte Ltd. 2021 S. Takenoshita, H. Yasuhara (eds.), Surgery and Operating Room Innovation, https://doi.org/10.1007/978-981-15-8979-9_12

115

116

shorter operating time because of its similarity to laparotomy, (2) good palpation and tactile sensation with smooth and protective maneuvers even when the tumor is large and heavy, (3) a shorter time to achieve proficiency, (4) lower costs, and (5) a lower rate of conversion to open laparotomy and an incision of about 10–20 mm in size. HALS is comparable to COL in many respects, including the perioperative hospitalization period and complication rates [1–4]. Accordingly, we have actively utilized HALS for colorectal surgery since July 2007 and we have reported favorable outcomes in more than 600 patients. We first make an incision of about 50  mm. We use a total of two ports (5 mm/5 mm) for surgery on the colon and three ports (5 mm/12 mm/5 mm) for the rectum. Including Miles’ operation on the rectum and total colectomy, we have used HALS for all procedures in patients with seven types of colorectal cancer [1–4]. Compared with pure-­LAP, the operating time is much shorter for HALS and the rate of open conversion is lower, being only 5.1% (5/98 patients) in our series. Furthermore, many studies comparing HALS and COL have shown that the former is associated with a smaller volume of blood loss and shorter hospital stay. In our study, blood loss was significantly smaller when HALS was performed in patients with stage I/II disease and the hospital stay was significantly shorter in stage III [1–4]. Thus, it is considered that HALS is an excellent surgical procedure and should be revisited in the current Japanese medical environment where the number of surgeons and anesthesiologists is decreasing. However, the left hand is inserted into the abdominal cavity where electrocautery and sealing devices are used during HALS, so surgeons are at risk of burn injury to the hands and fingers. In addition, the range of movement of the left hand is restricted during HALS and the left arm may become painful. Moreover, it is difficult for the surgeon to understand detailed movements of the left hand since it is not visible on a monitor. In addition to these issues, since a hand is inserted into the abdominal cavity, the incision size cannot be smaller than 35 mm. Accordingly, various improvements to HALS are anticipated. So far, prototypes of three-finger

M. Mukai et al.

robot hands and assembly type robot hands have been developed. However, with such robot hands it is expected to be difficult to gently place a spherical object on the palm, grasp it, and invert it by 180° [9, 10]. It is also expected to be difficult to move and displace organs by using these robots. In order to address these shortcomings, we concluded that a humanoid robot hand the size of a child’s hand should be designed with five multi-joint fingers, which could serve as a substitute for the surgeon’s left hand inside the abdominal cavity and could also provide tactile sensory information (Fig. 12.1).

12.2 R  obot Hand with a Master-­ Slave System Developed by Yokoi’s Laboratory First, this project made some modifications to a robot hand system (humanoid robot hand) invented and developed by Professor Yokoi at the University of Electro-Communications, which was originally designed as a myoelectric hand for patients with upper limb amputation [11]. The five fingers of this robot hand can be manipulated independently by using surface myoelectric signals from the upper limb (Fig. 12.2a, b). As a substitute for surface myoelectric signals, we developed a master-slave system in which movement of the robot hand is controlled by a device called a data glove that measures the finger joints (Figs.  12.2c, d and 12.3). The robot hand is made of nylon, and the fingers are connected by springs at the joints. The fingers are moved by a computer-­controlled electric motor via wires connected to fingertips that act like tendons. This allows precise simulation of human hand movements such as pinching/grasping and rotation with the thumb and index finger. In addition, the fingers are soft enough to grasp organs gently without causing damage [11]. However, since the finger joints are soft and the fingers are manipulated by wires, the fingertip force is weak and finger movements may not be precise. Consequently, it may not be possible to hold heavy objects and maneuvers with organs may not be performed adequately.

12  Development of Laparoscopic Surgery by Means of Foldable Small Humanoid Robot Hands… Fig. 12.1 We envisioned that the surgeon’s left hand in the abdominal cavity could be replaced by a small humanoid robot hand with five jointed fingers equipped with palpation and tactile sensation functions

117

Lt. hand

An operator Assistants

Finger movement & positioning evaluation

Data grove Compression or vibration

Master slave control systems

Monitor

devices

Laparoscopy

Motor control

Small robot-hand like a child

5-fingered robot arm

Feed back system for tactile sensation

Pressure sensor

The next future of robot hand HALS

a

b

grasping

c

d rock

paper rock

Fig. 12.2  The robot hands developed by Yokoi et al. at the University of Electro-Communications consisted of two prototypes. These were an electronic prosthetic hand

for patients with upper limb amputation (a, b) and a hand with a data glove that allowed measurement of hand and finger movements (c, d)

M. Mukai et al.

118

12.3 Development of a Foldable Small Humanoid Robot Hand by Kato’s Laboratory

vertically (Figs. 12.5a, b and 12.6a b). The parts of the robot hand were made of ABS resin and A data grove

A robot hand

To overcome the deficiencies mentioned above, Kato et  al. from Yokohama National University modified the Yokoi robot hand so that the finger movements required for HALS could be performed more effectively (Fig.  12.4) [12]. They inserted two direct drive micro-motors to control the carpometacarpal joint of the first finger (thumb) and the corresponding joints of the second and third fingers for performance of precise movements such as pinching. For the other necessary joints and movements, metacarpophalangeal joint of the thumb, carpometacarpal joints of the fourth and fifth fingers, flexion and extension, and pronation and supination of the wrist, wires similar to the Yokoi model are used. In order to allow insertion through a smaller incision, we enabled the palm of the robot hand to be folded

Fig. 12.3  This model can pinch, grasp, and rotate objects with the first and second fingers in a manner similar to the human hand. Since the fingertips are soft, organs can be held safely without damage. However, heavy organs cannot be held, and surgical procedures are restricted accordingly

Fig. 12.4  In this robot hand, only the joints required for HALS procedures are moveable. Joints that are required for precise movements, such as pinching, are directly

driven by motors while other joints are moved by wires. This approach enables subtle and strong grasping maneuvers

Industrial patents with the master slave control systems ・ University of Electro-communications, Yokoi’s labo. ・ Yokohama National University, Kato’s labo.

12  Development of Laparoscopic Surgery by Means of Foldable Small Humanoid Robot Hands…

119

b

a

foldable

c

d

grasping & rotation pinching

Fig. 12.5  The robot hand folds vertically at the palm (a, b). Delicate dynamic movements can be performed, such as pinching, grasping, and rotation quickly (c, d)

were manufactured on a 3D printer. This hand achieves improved allows precision of finger movements and finger force along with insertion through a smaller incision [12]. Moreover, dynamic maneuvers are feasible, such as pinching, grasping, and rotation quickly (Fig. 12.5c, d and 12.6c, d).

12.4 R  obot HALS Project Since 2013: Collaboration of Medical and Engineering Departments with Other Academics The robot HALS project was initiated in 2013 and modifications to the robot hands were made by the Mechanical Engineering departments at two universities. At present, the hand has reached the fourth generation, and we have conducted

more than 30 dry box experiments to investigate its effectiveness using pseudo-organs of chicken meat. In those experiments, the dry box simulated the patient’s abdominal cavity, and a 50 mm wound retractor (Alexis S for small incision surgery, Applied Medical Co. Ltd., USA) was installed in the cover of the box, which represented the abdominal wall (Fig. 12.6a, b). Then 300 g of chicken breast meat was placed inside the box. A surgeon inserted the robot hand system (No. 3 prototype) with a 5.5 size glove. The folded hand was inserted into the box and was opened inside it. Next, the skin of the 300 g piece of chicken breast was pinched between the first and second fingers, and the meat was lifted by at least 5 cm. Subsequently, the meat was grasped with all 5 fingers, lifted by at least 5  cm, and rotated by at least 90°. As a result, the success rate of the pinching maneuver was 90% (18/20) and the success rate of the grasping maneuver

M. Mukai et al.

120

a

b

foldable

c

d

pinching

dissection

Fig. 12.6  The robot hand folds vertically at the palm, allowing it to be used with a smaller incision (a, b). Experiments on use of electronic devices and incision and detachment procedures have been performed in an operating theater (c, d)

was 80% (16/20) (Fig.  12.5c, d). As the surgeon becomes more familiar with the procedure through training, a further increase of the success rate is anticipated.

12.5 Development of Pseudo-­ tactile Sensation Using Pressure Sensors in the Fingertips and Next-­ Generation Prototype Models In order to further reduce the diameter of the robot hand, we modified the shape of the hand joints and investigated a more precise driving method than use of wires to move the hand joints. As a result, the longest diameter of the wound retractor required has been reduced from 50  mm (Alexis S, Applied Medical Co. Ltd.,

USA) to 30 mm (Alexis XS for appendectomy, Applied Medical Co. Ltd., USA) (Fig.  12.7a, c). The current master-slave system using a data glove is unable to precisely and accurately reproduce subtle hand movements (Fig.  12.7b, d). Accordingly, we have been developing an exoskeleton-­typed input device using a potentiometer, but we have encountered restriction in the movement of the first finger, specifically opposition movements, and modifications are ongoing (Fig. 12.8a, b). Moreover, a system is being developed that transmits pressure or vibration from the robot hand as pressure sensation or pseudo vibration sensation via press device or vibrators attached to the fingertips of the surgeon (Fig. 12.8c, d). At present, modifications and further development of the robot hand are being conducted along with education/training toward implementation of an experiment using large animals in Aug 2019. In

12  Development of Laparoscopic Surgery by Means of Foldable Small Humanoid Robot Hands…

a

121

b grasping

Robot hand

c

d

rotation

Robot hand

Fig. 12.7  The hand joints were reduced in size, allowing the use of a 30 mm wound retractor (Alexis XS for appendectomy, Applied Medical Co. Ltd., USA, a, c, arrow

heads) instead of the 50 mm retractor (Alexis S for small incision surgery, Applied Medical Co. Ltd., USA). The range of hand joint rotation is about 90° (b, d)

order to conduct a large animal study of splenectomy and cholecystectomy as well as colorectal surgery, more than 20 training sessions have been conducted using electronic devices in an operating theater. Approval has already been obtained from the Institutional Animal Care Committee of Tokai University School of Medicine (Approved number; 17,053).

ity. To overcome problems with operability, such as breakage and other defects during surgery, a back up hand seems to be needed. Moreover, despite use of double gloves and a sterile surgical gown, sterilization of the robot hand itself is required, but this has not been attempted yet. Since the shaft is made of aluminum and the fingers are made of reinforced plastic, EOG gas sterilization (5–6  min) after disassembling the hand is being considered rather than autoclaving (135 °C/8–10 min). When we were surgical residents in the late 1980s, our supervisors and senior surgeons stressed that surgery must be performed while utilizing the left hand. That was more than 30 years ago, and HALS was first reported more than 20 years ago [7, 8]. Thereafter, pure-LAP became more popular and replaced HALS.  In

12.6 Future Perspective Before achieving clinical use of robot HALS, two major issues must be addressed. The first is durability of the robot hand, which must not be prone to be defects. Since a double-glove eversion method is employed, it is very unlikely that a ­broken device would be left in the abdominal cav-

M. Mukai et al.

122

a

b

c

d

Fig. 12.8  An exoskeleton system is being developed. At present, the movement of the first finger, specifically rotation, is restricted and this is currently undergoing modification (a, b). Pressure and vibration sensors were installed

at multiple locations of fingertips (c, d, arrow heads) and pressure waves were measured (c, d, arrows). To provide pseudo-tactile sensation, a system that allows fingertip pressure to be perceived is being developed in parallel

recent years, we have published many articles reporting the advantages of HALS as a desirable less invasive surgical method [1–4]. HALS is positioned between pure-LAP and COL and is regarded as a safe and reliable, low-cost hybrid surgical method. To overcome the current shortcomings of HALS, it will evolve into Robot HALS based on use of a small humanoid robot hand. The incision required for inserting the folded hand is only 20–30  mm in length. When a button is pushed, the robot hand unfolds

instantaneously in the abdominal cavity and functions as a left hand while performing surgery (Fig.  12.9). An electric scalpel and aspirator are installed at the fingertips and argon beam irradiation from the index finger can be employed for cauterization. The wrist joint is equipped with a power assist function and can be rotated by about 360°. However, palpation and tactile sensation must be provided as well, so many aspects need further consideration and development (Fig. 12.9).

12  Development of Laparoscopic Surgery by Means of Foldable Small Humanoid Robot Hands… Fig. 12.9  Approval was obtained from the Institutional Animal Care Committee of Tokai University School of Medicine, and we are currently conducting training in an operating theater for a large animal study of splenectomy, cholecystectomy, and colorectal resection using the robot hand

123

Pseudo tactile sensation by pressure sensors or vibrations at the finger tips

Finger tips compression or vibration systems

Lt. hand

Pressure sensors Argon plasma coagulation

Acknowledgments This work was supported by JSPS KAKENHI Grant Number JP17K10656 and Grant Number AS2018A000706914 from Taiho Pharmaceutical Co., Ltd. (Tokyo, Japan). The authors sincerely appreciate cooperation and support with research and development from the Japan Society for the Promotion of Science (Tokyo, Japan), as well as graduate students and research associates at the University of Electro-Communications (Yokoi Laboratory) and Yokohama National University (Kato Laboratory). Conflict of Interest: All authors of this chapter have no conflict of interest.

References 1. Mukai M, Tajima T, Hoshikawa T, et al. Efficacy of hybrid 2-port hand-assisted laparoscopic surgery (Mukai’s operation) in patients with colorectal cancer. Oncol Rep. 2009;22:893–9. 2. Tajima T, Mukai M, Yamazaki M, et al. Comparison of hand-assisted laparoscopic surgery and conventional laparotomy for colorectal cancer: interim results from a single institution. Oncol Lett. 2014;8:627–32. 3. Tajima T, Mukai M, Yokoyama D, et al. Comparison of hand-assisted laparoscopic surgery (HALS) and conventional laparotomy in patients with colorectal cancer: final results from single center. Oncol Lett. 2017;13:4953–8. 4. Tajima T, Mukai M, Koike T, et  al. Better survival after hand-assisted laparoscopic surgery than conventional laparotomy for rectal cancer: five year results from a single center in Japan. Clin Surg. 2017;2:1368.

5. Ruurda JP, Van Vroonhoven TJMV, Broeders IAMJ.  Robt-assisted surgical systems: a new era in laparoscopic surgery. Ann R Coll Surg Engl. 2002;84:223–6. 6. Simorov A, Otte RS, Kopietz CM, et  al. Review of surgical robotics user interface: what is the best way to control robotic surgery? Surg Endosc. 2012;26:2117–25. 7. Kusminsky RE, Boland JP, Tiley EH, et  al. Hand-­ assisted laparoscopic splenectomy. Surg Laparosc Endosc. 1995;5:463–7. 8. Ou H.  Laparoscopic-assisted mini-laparotomy with colectomy. Dis Colon Rectum. 1995;38:324–6. 9. Oshima R, Takayama T, Omata T, et al. Assemblable three fingered five-DOF hand for laparoscopic surgery. J Robot Soc. 2008;26:453–61. 10. Oshima R, Takayama T, Omata T, et al. Assemblable three-fingered nine-degree of freedom hand for laparoscopic surgery. In: International conference on intelligent robots and systems, IEEE/RSJ; 2009. p. 5528–5533. 11. Seki T, Kato R, Yokoi H, et al. Development of five-­ finger multi-DoF myoelectric hands with a power allocation mechanism. J Mech Eng Autom. 2014: 97–105. 12. Yoshida K, Yamada H, Kato R, et al. Development of five-finger robotic hand using master-slave control for hand-assisted laparoscopic surgery. In: 38th annual international conference of the IEEE Engineering in Medicine and Biology Society (EMBC2016); 2016. p. 5124–5127.

Robotic Surgery: Currently and in the Near Future

13

Masaaki Ito

13.1 Introduction Endoscopic surgery is one of the greatest advances made in surgical treatment within the past quarter century. Who would have expected that this surgical treatment would advance to the degree that it has since laparoscopic cholecystectomy was performed in Japan in 1990. Furthermore, with the advent of robotic surgery using the da Vinci system in recent years, endoscopic surgery is expected to further evolve. First, we will discuss the transition from the advent of endoscopic surgery to robotic surgery performed within recent years.

13.2 Transition of Endoscopic Surgery: With a Focus on Laparoscopic Colectomy More than 100  years have passed since surgery for gastric cancer was first reported in 1881 by Professor Billroth from the University of Vienna (Fig. 13.1). Surgical procedures at that time, as can be seen in the picture, were performed in “theatrical” surgical settings with a large audience watching the operation table that was placed in the center M. Ito (*) Department of Colorectal Surgery, National Cancer Center Hospital East, Kashiwa-shi, Chiba, Japan e-mail: [email protected]

Fig. 13.1  First gastrectomy for gastric cancer performed by Billroth in 1881

of the operating room. As not only anesthesia techniques but also surgical procedures were not established like current situation, it seemed to be extremely difficult to safely remove malignant tumors from the body and that the patients were in a life-threatening situation. A little over 100 years have passed since then, and it can be

© Springer Nature Singapore Pte Ltd. 2021 S. Takenoshita, H. Yasuhara (eds.), Surgery and Operating Room Innovation, https://doi.org/10.1007/978-981-15-8979-9_13

125

M. Ito

126 Table 13.1  History of laparoscopic colorectal surgery 1991 1993 1996 2002 2004

2008

First laparoscopic colorectal surgery in the world First laparoscopic colorectal surgery in Japan Insurance coverage for early-stage cancer Insurance coverage for advanced colorectal cancer Beginning of large-scale randomized controlled trial (JCOG0404) on stage II/III colorectal cancer Beginning of Japanese phase II trial (LapRC) on rectal cancer

said that surgical treatment has made remarkably significant advances in the past century. In particular, the evolution of surgical treatment over the past 20 years has been rapid and drastic. The application of endoscopic technology has played important roles in the advent of endoscopic surgery. As shown in Table  13.1, endoscopic surgery was performed in the early 1990s for the first time. In Japan, endoscopic surgery for colorectal cancer was reported by Watanabe and colleagues in 1993. Subsequently, laparoscopic surgery was gradually performed in clinical practice, but the expansion in its indication was limited. At the time, laparoscopic colorectal cancer surgery was often limited to procedures that mobilized the colon from its surroundings, and there was a marked difference from the laparoscopic surgery that is common today. Later, laparoscopic surgery for early stage colorectal cancer was approved for insurance coverage in 1996, and its indication was expanded to a large part of colorectal cancer including advanced cancer in 2002. Colorectal cancer surgery under laparoscopy has thus made steady progress in Japan. Now it has reached the next stage where it requires clinical verification of its long-term treatment outcome. Conventional laparotomy had been standard as a treatment for stage II and stage III colorectal cancer. In 2004, a large-scale randomized controlled trial (JCOG 0404) was started to verify that laparoscopic surgery was not inferior to laparotomy in terms of long-term treatment outcome. The target of this trial did not include transverse colon cancer and rectal cancer, but surgical procedures that are

said to be relatively difficult among colorectal cancers. The prognostic outcome of general laparoscopic colorectal cancer surgery was first subjected to verification. The final results of this trial were published last year and showed that the treatment outcome was excellent with a 5-year survival rate of patients with stage II and stage III colorectal cancer exceeding 90% in both laparotomy and laparoscopic surgery. The results of a controlled clinical trial conducted in Japan with more than 1000 cases showing that laparoscopic colorectal cancer resection had a similar survival rate as laparotomy is unlikely to put a brake on the future of laparoscopic surgery.

13.3 W  hat Kind of Surgery Is Endoscopic Surgery? The schema shown in Fig.  13.2 is the basic arrangement of staff and ports for endoscopic surgery for colorectal cancer. Laparoscopic sigmoid colon cancer resection will be used as an example to explain how laparoscopic surgery is actually performed. There are usually three physicians present in the surgery who assume the roles of operator, assistant, and endoscope-holding assistant (scopist). The operator stands to the right of the patient. It is standard to use five ports. The port immediately below the navel is primarily used for inserting the endoscope; the scopist controls this port. The common arrangement for the operator and scopist is to stand side by side on the right of the patient and the assistant to stand on the left of the patient. The right hand of the operator is primarily involved in dissection. The majority of colorectal operators in Japan use spatula-shaped monopolar electric scalpels, while in Europe, hook-shaped electric scalpels are also used. Aside from electric scalpels, ultrasonic coagulation incision devices (such as the Harmonic) and sealing systems (such as the LigaSure) are commonly used. The device held in the right hand of the operator plays a central role in laparoscopic surgery as it is used to dissect and incise the tissue. The clamps going through a total of three ports, one in the

13  Robotic Surgery: Currently and in the Near Future Anesthesia apparatus Scopist Holds the laparoscope and directs it to where vision is required

127

Anesthesiologist

Assistant Assists in exposure by tugging on the organs using clamps

Electric scalpel

Operator Holds the energy device in the right hand and the clamp in the left hand to dissect appropriate areas

el

lp

Scrub nurse

ric

ct

Instrument table

e El

a sc

Fig. 13.2  About laparoscopic surgery

other operator’s left hand and two in the hands of the assistant, are used in grasping maneuvers to clamp the tissue. In other words, the device held in the right hand of the operator that “dissects and incises” plays the leading role, while the other three clamps are responsible for successfully exposing the surrounding tissue. Conversely, if these three clamps fail in exposure, the operator will be unable to successfully perform dissection using his/her right hand. What follows are the interesting results of an analysis of endoscopic operations that were actually performed in our department and neighboring hospitals under various circumstances (Fig. 13.3). There were three settings: • Team 1. The operator was a physician qualified in endoscopic surgical skill (an expert), and the assistants were physicians undergoing training (trainees). • Team 2. The operator was a physician undergoing training (a trainee), and the assistants were physicians qualified in endoscopic surgical skill (experts).

• Team 3. The operator was a physician undergoing training (a trainee), and the assistants were also physicians undergoing training (trainees). We measured and analyzed the exposure time and dissection time of 20 cases of laparoscopic sigmoid colectomy with each of the above endoscopic surgery team configurations. The results showed that the exposure time was similar between team 1 and team 2, while it was prolonged in team 3. Meanwhile, in terms of dissection time, only team 1 completed dissection within a short period of time, whereas time prolongation to a similar extent was observed in both team 2 and team 3. What is suggested by these results is consistent with the characteristic elements of laparoscopic surgery. In other words, exposure in endoscopic surgery may be swiftly performed if there is one expert on the endoscopic surgery team, either as the operator or an assistant. However, this is not the case with dissection. Only highly skilled endoscopic operators can perform adequate and rapid surgical procedures. This is an easy-to-­

M. Ito

128 Total time for exposure 2000

Total time for dissection 6000 5000

1500

4000 1000 3000 500

2000

Team 01 (Expert/Trainee)

Team 03 (Trainee)/Trainee)

Team 01 (Expert/Trainee)

Team 03 (Trainee)/Trainee)

Team 02 (Trainee)/Expert)

Team 02 (Trainee)/Expert)

Fig. 13.3  The importance of exposure and dissection experts

understand result that indicates the characteristics of endoscopic surgery. Only the operator is in charge of dissection in endoscopic surgery, and it is important to improve the operator’s dissection technique to make improvements in this regard. On the other hand, the operator does not necessarily need to be an expert to perform adequate and swift exposure in laparoscopic surgery, and this is dependent on whether an endoscopic surgeon that can lead and guide the present team. To sum up, dissection is an indicator that shows the individual’s proficiency in endoscopic surgical procedures, while exposure is an indicator that reflects the capability of the team led by the leader. Therefore, capability of the individuals and the team must be enhanced separately to proficiently perform endoscopic surgery. The abovementioned analysis results may act as an objective basis for why robotic surgery is feasible. In other words, operators can simultaneously play the part of a “good dissector” and “successfully perform exposure” in current robotic surgeries.

13.4 Advent of Robotic Surgery The da Vinci system, which was developed by the request of the United States Army in the 1980s, was created for the purpose of remote treatment during times of war. Later, the private sector took over the development and completed it in 1999. It was approved by the FDA in the United States in 2000. In the United States, its primary clinical use is in the field of urology, including prostate cancer, and the majority of prostate cancer operations have been performed by robots in recent years. It was introduced to Japan in 2000, and da Vinci S obtained pharmaceutical approval in 2009 after clinical trials. Subsequently, its clinical indication, mostly prostate cancer, in advanced medical care gradually expanded, and its use on total prostatectomy was listed as covered by insurance for the first time in Japan in 2012. Since then, the number of prostate cancer cases in which it has been used increases each year; there were over 8000 patients in 2014 who underwent robotic surgery (Fig. 13.4).

13  Robotic Surgery: Currently and in the Near Future Fig. 13.4  da Vinci. Transition of the number of cases worldwide

129

700,000

201 2014: 7-1 9% Growth

600,000 500,000 400,000 300,000 200,000 100,000 0 2009

2010

Urology

Meanwhile, its application in other fields is limited, and robotic surgery is clinically performed in general gastrointestinal surgery, thoracic surgery, gynecology, and cardiovascular surgery. Although gradual, clinical research on robotic surgery for gastric cancer and rectal cancer is being conducted, and laparoscopic gastric cancer surgery using robotic surgical assistants became available in 2015 as part of advanced medical care (advanced medical care B) approved by the Ministry of Health, Labour, and Welfare. Furthermore, it was decided that laparoscopic partial nephrectomy using the endoscopic surgical robot, “da Vinci,” would be covered by insurance under the FY 2016 Revision of Reimbursement of Medical Fees. In addition, it was listed as covered by insurance in a wide range of fields including gastrointestinal surgery in 2018. In Japan, it is currently used as a primary surgical method in the field of urology, and its application is expanding into other fields as well. The da Vinci Si is the successor to the da Vinci S, which was pharmaceutically approved in Japan in 2009. It obtained pharmaceutical approval in 2012 and recently evolved into the da Vinci Xi. The robotic arms were more compact in the latest model. Redocking of the robotic arms

2011 Gynecology

2012

2013

General Surgery

2014 Other

was required in some cases with previous models for operations where the surgical field spanned a wide range, such as in colorectal cancer surgery. It appears this process can be skipped with the new model. With the advent of the trans-anal total mesorectal excision (taTME), which is a new surgical method for rectal cancer that has progressed primarily in Europe and the United States in the past few years, it has been shown that high-quality rectal cancer surgery can be performed on obese male patients with narrow pelvises and on lower rectal cancer close to the anus. This surgery is performed endoscopically through the anus. Although it requires procedures similar to single-­ port surgery and is relatively difficult, research and development for performing this surgery using robots are also underway.

13.5 Advantages and Disadvantages Brought About by Robotic Surgery Surgery performed by the da Vinci system is performed in a manner similar to endoscopic surgery. The da Vinci system is equipped with

130

four robotic arms and one of them is responsible for holding and controlling the camera. In other words, it plays the role of the scopist in endoscopic surgery. The first arm plays the role of the clamp held in the operator’s right arm in endoscopy surgery, while the remaining two arms play the role of the clamp held by the assistant and the clamp in the operator’s left hand. Therefore, it is an operation that requires exposure with one fewer clamp than the number of ports generally used in laparoscopic surgery. The current da Vinci system is consistent with the laparoscopic surgery widely performed in Europe and the United States where endoscopes and clamps are used through four ports, but five ports are also often used in Japan. Intraoperatively, a robotic-­ surgery-­assisting surgeon near the patient will, more often than not, assist with exposure using clamps for laparoscopic surgery in cases where exposure provided by the da Vinci system is insufficient. The advantages of robotic surgery over endoscopic surgery include: 1. Function of multiarticulate clamps: EndoWrist (Fig. 13.5). 2. Three-dimensional high-quality images. 3. Filtering function for eliminating hand tremors. 4. Scaling function to adjust the operator’s movement.

Fig. 13.5  Advantages of the da Vinci system. Function of multiarticulate clamps (EndoWrist)

M. Ito

As a result of utilizing these functions, it is possible to perform stable and delicate dissections and incisions, while viewing super-enlarged images compared to conventional laparoscopic surgery. Furthermore, it may enable delicate surgery deep down in the pelvis, which is an area difficult to reach in conventional laparoscopic surgery. In addition, as demonstrated in robotic prostate cancer surgery, fine suture procedures can be performed more intuitively. Robotic surgery is unmatched by conventional laparoscopic surgery in this regard. Such delicate robotic surgery has shown that surgery on even blood vessels and nerves can be performed. A report from Korea has indicated excellent preservation of micturition function following rectal cancer surgery. (1) Similarly in Japan, there has been a report of treatment results indicating it may contribute to the preservation of functions, including sexual function, following rectal cancer surgery. (2) Meanwhile, the results of a randomized controlled trial (ROLARR trial) on robotic surgery and laparoscopic surgery for rectal cancer were released recently. (3) The primary end point of this clinical trial was the conversion rate to laparotomy, and this trial was unable to draw the conclusion that robotic surgery was superior to laparoscopic surgery. As we have seen so far, in terms of the clinical superiority of robotic surgery as determined by the patients, an overwhelming advantage may not be demonstrated in the field of rectal cancer. Behind this is the possibility that today, high-­ quality laparoscopic surgery has become available as the procedural skill levels of laparoscopic surgeons have increased. As a result, the widespread of costly robotic surgery may be impeded in consideration of the cost-benefit balance if it lacks overwhelming superiority to inexpensive laparoscopic surgery. However, the development of robotic devices itself is ongoing worldwide, and we are at a point where the evolution of revolutionary surgery cannot be predicted. Furthermore, when we impassively look back on the history of robotic surgery where it established its position as a standard operation for prostate cancer, we real-

13  Robotic Surgery: Currently and in the Near Future

ize that it has a great advantage in surgery with a limited field of view or surgery requiring fine anastomotic maneuvers. Moreover, it has also been reported that, in prostate cancer treatment, surgeons with experience in laparoscopic surgery can become proficient in robotic surgery in a significantly short period of time, and there is little basis for denying the future potential of robotic surgery. Rather, it is expected that the “demand for new robotic surgery” that is more acceptable in clinical practice will increase as a result of new technological innovations and the appropriate reflection of clinical needs.

13.6 T  he Present and Future of Robot Development As the patents of da Vinci started to expire a few years ago, the development of robotic surgical assistants has become active in many countries. Different robots are being developed by a variety of companies, and the direction of development can be divided into (1) master-slave robots with multiple arms similar to da Vinci, (2) robots for single-port surgery, and (3) nonmaster-slave robots that assist the operator in a simpler manner. Robots that are to be released after da Vinci with characteristics that it lacks are being developed in different countries. TransEntrix’s ALF-X is a master-slave robot characterized by tactile feedback and a 5-mm surgical instrument. However, based on the released footage, it appears the 5-mm instrument does not bend. CMR Surgical from the United Kingdom is developing a robot called Versius. It is assumed that Versius is equipped with a bendable clamp that can be used in 5-mm ports. Master-slave robots are being developed in China and South Korea as well. Tianjin University’s Micro Hand S in China and Meere’s Eterne in Korea are some examples. Information about them is limited at present and their characteristics remain unknown. A typical example of a robot for single-port surgery is the successor model to the da Vinci. Although there is little information, footage

131

released at academic conferences shows that it is highly refined. Titan Medical is vying with Intuitive to develop robots for single-port surgery. It is alleged that they are working towards releasing it in 2016  in the United States. In addition, TransEntrix, which was mentioned earlier, is developing a robot for single-port surgery, rather than a complete master-slave robot, that can be used by the operator near the patient. While the robots described above are all master-­ slave robots that are operated by the operator away from the patient, people are also developing robots that assist conventional laparoscopic surgery. In Japan, River Field, a venture that was started at the Tokyo Institute of Technology, has developed and released a robot that operates an endoscopy by pneumatic control. The number of robots that operate an endoscope is increasing rapidly. They are sold by Freehand, AKTORmed, ENDOCONTROL, etc., and are no longer uncommon. Moreover, although not a robot, FlexDex has developed a bendable clamp that can be attached to the operator’s hand. The development of this type of robot is believed to become more popular as well. In addition, Olympus, Medicaroid, and A-Traction have entered the development of robotic surgical assistants in Japan, and various robots from and outside of Japan are considered to make an appearance in a few years.

13.7 In Closing Inherently, there is a dilemma within modern-day surgery. It involves the developmental direction that promotes innovative surgical procedures equipped with new technologies and a direction of standardization that aims for the majority of surgeons to provide safe procedures. Development in a manner that satisfies both these elements is exactly what is required of the innovation in surgical treatment that leads to robotic surgery from laparoscopic surgery.