Optimizing ML Inference Queries Under Constraints

Optimizing ML Inference Queries Under Constraints

Li, Z. (TU Delft Web Information Systems)
Sun, W. (TU Delft Web Information Systems)
Hai, R. (TU Delft Web Information Systems)
Bozzon, A. (TU Delft Human-Centred Artificial Intelligence)
Katsifodimos, A (TU Delft Web Information Systems)

2023

The proliferation of pre-trained ML models in public Web-based model zoos facilitates the engineering of ML pipelines to address complex inference queries over datasets and streams of unstructured content. Constructing optimal plan for a query is hard, especially when constraints (e.g. accuracy or execution time) must be taken into consideration, and the complexity of the inference query increases. To address this issue, we propose a method for optimizing ML inference queries that selects the most suitable ML models to use, as well as the order in which those models are executed. We formally define the constraint-based ML inference query optimization problem, formulate it as a Mixed Integer Programming (MIP) problem, and develop an optimizer that maximizes accuracy given constraints. This optimizer is capable of navigating a large search space to identify optimal query plans on various model zoos.

http://resolver.tudelft.nl/uuid:284b17af-71ee-4e79-b75e-0cc3b9ce6079

https://doi.org/10.1007/978-3-031-34444-2_4

2023-12-25

International Conference on Web Engineering

Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Institutional Repository

conference paper

© 2023 Z. Li, W. Sun, R. Hai, A. Bozzon, A Katsifodimos