A deep learning approach to (semi-) automatically track bone movement in ultrasound images of patients with a unilateral transtibial prosthesis

Abstract

Background: The procedure to fit a prosthetic socket to a patient, which can assure the patient’s comfort during activities of daily living, is labour intensive. Such a lengthy procedure could benefit from an automated and more efficient data-driven method capable of automatically tracking the relative movement between the patient’s tibia and the prosthetic socket. To investigate such a method, we acquired in-socket bone displacement data during the physical activities of the prosthetic user. Manually tracking the location of the tibia from, e.g., B-mode (imaging) ultrasound (US) sequences might be a solution, but this is time-consuming, and the interpretation of the sequences is highly operator dependent. Therefore, an automated and efficient method to assess socket fit in US sequences is needed.
Methods: We used an existing 3D U-Net with a long short-term memory module (LSTM) and compared its ability to track a landmark location point on the tibia in US recordings by comparing the displacement and similarity in shape with data obtained from a semi-automatic single-point tracker. To evaluate the performance of the developed automated workflow, we obtained experimental data from three participants who performed three repetitive stepping tasks with their prosthetic leg in a sideways, forward, and backward motion. Three deep learning models were trained with a varying hold-out method (66% training data, 34 % test data) to test the ability to track a landmark location on the tibia in unseen data from one participant. To find the similarity of the deep learning models compared to a semi-automated single point tracker, the normalised root mean squared error (NRMSE) was calculated. We also evaluated the normalised maximum cross-correlation (NMCC) to account for the maximum similarity in displacement trajectory when a delay occurred between the true trajectory and that from the automated model. We analysed the repeatability of each step task per participant with the standard deviation from the mean tibia’s landmark location trajectories.
Results: Due to the delay between the semi-automated single-point tracker and the DL model, the NRMSEs ranged between 27% and 90%. The similarity threshold (0,95) was reached for five trajectories of the tracked point on the tibia in the anterior-posterior direction, with a delay between 1,5% and 8,5% of the step duration. The similarity in the anterior-posterior direction of the tibia’s landmark location trajectory was higher than that in the lateral-medial direction. The SD for all participants was around 1 mm but varied proportionally to the amount of movement observed per participant. The SD of the DL models was similar to that of the semi-automated single-point tracker.
Conclusion: We conclude that a DL model from a 3D U-Net with an LSTM module has the potential to assist prosthetists and researchers in tracking in-socket tibial bone movement in the anterior-posterior direction.