Learning image representations for content-based image retrieval of radiotherapy treatment plans

Learning image representations for content-based image retrieval of radiotherapy treatment plans

Huang, Charles (Stanford University)
Vasudevan, Varun (Stanford University)
Pastor Serrano, O. (TU Delft RST/Medical Physics & Technology; Stanford University)
Islam, Md Tauhidul (Stanford University)
Nomura, Yusuke (Stanford University)
Dubrowski, Piotr (Stanford University)
Wang, Jen Yeu (Stanford University)
Schulz, Joseph B. (Stanford University)
Yang, Yong (Stanford University)

2023

Objective. In this work, we propose a content-based image retrieval (CBIR) method for retrieving dose distributions of previously planned patients based on anatomical similarity. Retrieved dose distributions from this method can be incorporated into automated treatment planning workflows in order to streamline the iterative planning process. As CBIR has not yet been applied to treatment planning, our work seeks to understand which current machine learning models are most viable in this context. Approach. Our proposed CBIR method trains a representation model that produces latent space embeddings of a patient’s anatomical information. The latent space embeddings of new patients are then compared against those of previous patients in a database for image retrieval of dose distributions. All source code for this project is available on github. Main results. The retrieval performance of various CBIR methods is evaluated on a dataset consisting of both publicly available image sets and clinical image sets from our institution. This study compares various encoding methods, ranging from simple autoencoders to more recent Siamese networks like SimSiam, and the best performance was observed for the multitask Siamese network. Significance. Our current results demonstrate that excellent image retrieval performance can be obtained through slight changes to previously developed Siamese networks. We hope to integrate CBIR into automated planning workflow in future works.

content based image retrieval
deep learning
representation learning

http://resolver.tudelft.nl/uuid:f5e89339-8124-453b-9144-b891d7745780

https://doi.org/10.1088/1361-6560/accdb0

2023-11-03

0031-9155

Physics in Medicine and Biology, 68 (9)

Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Institutional Repository

journal article

© 2023 Charles Huang, Varun Vasudevan, O. Pastor Serrano, Md Tauhidul Islam, Yusuke Nomura, Piotr Dubrowski, Jen Yeu Wang, Joseph B. Schulz, Yong Yang, More Authors