Optimizing machine learning inference queries for multiple objectives

Abstract

Machine learning inference queries are a type of database query for databases where a model pipeline is needed to evaluate its boolean predicates. Using a model zoo it is possible to select a variety of models to execute in a sequence rather than using a highly specialized model to answer every query predicate. Machine learning models can have multiple measurements for gauging performance however, and the quality of a query plan therefore is not only dependent on the time needed to compute it. Selecting a query plan of models that balances multiple objectives is not a trivial feat however. This work builds upon existing methods that utilize MIPs for model selection and ordering for machine learning inference queries by extending them with multi-objective optimizing capabilities. The opportunity for adding a third objective, namely memory footprint, to that of accuracy and execution cost is explored. Several methods are then considered and compared on their suitability, and the final chosen method, the Archimedean goal method, can generate Pareto optimal query plans that provide gains over naive, greedy methods. In addition, several methods of cutting down runtime on the original optimizer are explored, leading to a program than can generate higher quality solutions in less time.