Existing challenges in surgical education (See one, do one, teach one) as well as the COVID-19 pandemic make it necessary to develop new ways for surgical training. Therefore, this work describes the implementation of a scalable remote solution called “TeleSTAR” using immersive, interactive and augmented reality elements which enhances surgical training in the operating room. The system uses a full digital surgical microscope in the context of Ear–Nose–Throat surgery. The microscope is equipped with a modular software augmented reality interface consisting an interactive annotation mode to mark anatomical landmarks using a touch device, an experimental intraoperative image-based stereo-spectral algorithm unit to measure anatomical details and highlight tissue characteristics. The new educational tool was evaluated and tested during the broadcast of three live XR-based three-dimensional cochlear implant surgeries. The system was able to scale to five different remote locations in parallel with low latency and offering a separate two-dimensional YouTube stream with a higher latency. In total more than 150 persons were trained including healthcare professionals, biomedical engineers and medical students.

Existing challenges in surgical education (See one, do one, teach one) as well as the Covid-19 pandemic make it necessary to develop new ways for surgical training. This is also crucial for the dissemination of new technological developments. As today's live transmissions of surgeries to remote locations always come with high information loss, e.g. stereoscopic depth perception, and limited communication channels. This work describes the implementation of a scalable remote solution for surgical training, called TeleSTAR (Telepresence for Surgical Assistance and Training using Augmented Reality), using immersive, interactive and augmented reality elements with a bi-lateral audio pipeline to foster direct communication. The system uses a full digital surgical microscope with a modular software-based AR interface, which consists of an interactive annotation mode to mark anatomical landmarks using an integrated touch panel as well as an intraoperative image-based stereo-spectral algorithm unit to measure anatomical details and highlight tissue characteristics.We broadcasted three cochlea implant surgeries in the context of otorhinolaryngology. The intervention scaled to five different remote locations in Germany and the Netherlands with lowlatency. In total, more than 150 persons could be reached and included an evaluation of a participant's questionnaire indicating that annotated AR-based 3D live transmissions add an extra level of surgical transparency and improve the learning outcome.

perating room planning is a complex task as pre-operative estimations of procedure duration have a limited accuracy. This is due to large variations in the course of procedures. Therefore, information about the progress of procedures is essential to adapt the daily operating room schedule accordingly. This information should ideally be objective, automatically retrievable and in real-time. Recordings made during endoscopic surgeries are a potential source of progress information. A trained observer is able to recognize the ongoing surgical phase from watching these videos. The introduction of deep learning techniques brought up opportunities to automatically retrieve information from surgical videos. The aim of this study was to apply state-of-the art deep learning techniques on a new set of endoscopic videos to automatically recognize the progress of a procedure, and to assess the feasibility of the approach in terms of performance, scalability and practical considerations.