Text-to-Image Diffusion Model Selection

Abstract

Text-to-image diffusion models have advanced significantly in recent years. Different models show strong performance across various generation tasks. Choosing the right model is becoming increasingly important since no single model consistently outperforms others in all cases. However, existing model selection approaches are typically evaluated only at the dataset level. Such evaluation overlooks prompt-level variation, where different models may excel on different prompts. In this thesis, we investigate diffusion model selection during inference. The goal is to pick the best model for each individual prompt. We first examine the online setting, where model selection occurs adaptively during deployment. In this context, we create a framework for online diffusion model selection and test it against recent methods from the literature. Our findings show that this approach outperforms existing online selection strategies, highlighting the benefits of prompt-aware model selection. In addition to the online setting, we present an offline approach to diffusion model selection, where decisions are made without online interaction. Overall, this thesis claims that diffusion model selection should be viewed as a prompt-level decision rather than a dataset-level comparison. By exploring both online and offline settings and providing empirical results alongside detailed ablations, we aim to promote a more practical and adaptable approach to diffusion model selection.