Telepresence for surgical assistance and training using eXtended reality during and after pandemic periods

Telepresence for surgical assistance and training using eXtended reality during and after pandemic periods

Wisotzky, Eric L. (Humboldt-Universitat zu Berlin; Fraunhofer Heinrich Hertz Institute, Berlin; Rostock University Medical Center)
Rosenthal, Jean Claude (Fraunhofer Heinrich Hertz Institute, Berlin; Rostock University Medical Center)
Meij, S.E.P. (TU Delft Medical Instruments & Bio-Inspired Technology)
van den Dobbelsteen, J.J. (TU Delft Medical Instruments & Bio-Inspired Technology)
Arens, Philipp (Charité Universittsmedizin Berlin)
Hilsmann, Anna (Fraunhofer Heinrich Hertz Institute, Berlin)
Eisert, Peter (Humboldt-Universitat zu Berlin; Fraunhofer Heinrich Hertz Institute, Berlin)
Uecker, Florian Cornelius (Charité Universittsmedizin Berlin)
Schneider, Armin (Munich Surgical Imaging, Munich; Jade Hochschule Wilhelmshaven)

2023

Existing challenges in surgical education (See one, do one, teach one) as well as the COVID-19 pandemic make it necessary to develop new ways for surgical training. Therefore, this work describes the implementation of a scalable remote solution called “TeleSTAR” using immersive, interactive and augmented reality elements which enhances surgical training in the operating room. The system uses a full digital surgical microscope in the context of Ear–Nose–Throat surgery. The microscope is equipped with a modular software augmented reality interface consisting an interactive annotation mode to mark anatomical landmarks using a touch device, an experimental intraoperative image-based stereo-spectral algorithm unit to measure anatomical details and highlight tissue characteristics. The new educational tool was evaluated and tested during the broadcast of three live XR-based three-dimensional cochlear implant surgeries. The system was able to scale to five different remote locations in parallel with low latency and offering a separate two-dimensional YouTube stream with a higher latency. In total more than 150 persons were trained including healthcare professionals, biomedical engineers and medical students.

annotations
ear–nose–throat
live streaming
mixed reality
pandemic
surgical training
telehealth
Telepresence

http://resolver.tudelft.nl/uuid:2c0ab1b0-ceb5-4193-a2a4-271555d7c584

https://doi.org/10.1177/1357633X231166226

1357-633X

Journal of Telemedicine and Telecare

Institutional Repository

journal article

© 2023 Eric L. Wisotzky, Jean Claude Rosenthal, S.E.P. Meij, J.J. van den Dobbelsteen, Philipp Arens, Anna Hilsmann, Peter Eisert, Florian Cornelius Uecker, Armin Schneider