Visual Detection of Clinicians for Surgical Progress Recognition

Abstract

Due to the ageing population and the availability of treatments, healthcare costs keep rising. Surgery is one of the big cost items in a hospital. Improvement of workflow, planning and process efficiency can aid in lowering the costs. Inefficiencies or the effect of improvements can only be detected through measurements. Therefore data is needed for analysis and for systems that improve process efficiency by being aware of surgical context. The workflow includes the entire procedure at the OR, not only the surgical part of the procedure. The presently used method for planning does not take profit from automatic data collection and analysis methods. Multiple techniques have been investigated in prior research, but no solution has been widely adopted in the operating room. A new approach is demonstrated that uses a Computer Vision technique called Human Pose Estimation. This technique detects human poses in image data. The generated data will be used to determine the location of medical personnel within the OR. Inspired by prior research, multiple areas are defined within the OR. The distribution of personnel over these areas is used to analyse the progress of a surgical procedure. This method can be seen as a replacement of dedicated sensors used in prior research. This approach makes the system non-intrusive, it does not disturb the surgical progress, and it is a versatile system that can be exploited further in future research. Next to the technical approach, the non-technical aspects are considered. This includes the laws involved, dealing with privacy and the approvals that are needed to start this type of medical research. For the video data, publicly available datasets are used. The datasets are investigated for their applicability to surgical progress recognition. For surgical progress recognition the performance of two different methods are tested, a handcrafted Decision Tree and a 1D Convolutional Neural Network. The Decision Tree produced a recognition rate for five different phases of 93%. This score is promising, but due to limitation by the data, the number of phases limited. A broader set of phases, will include phases that are harder to detect. The 1D CNN scored a recognition rate of 70% for the same phases. The amount of available data for the 1D CNN seemed to negatively affect the network in learning the relation between input features and the output. Further research should be targeted at creating a dataset specifically for the goal of surgical progress recognition. This new dataset should have a higher framerate and contain a larger number of procedures, recorded in a regular OR. Considering the minimalist approach, the results are promising for future research and might be able to drive a dynamic planning system in future applications.