Optimizing Machine Learning Inference Queries for Multiple Objectives
Z. Li (TU Delft - Web Information Systems)
Mariette Schonfeld (Student TU Delft)
R. Hai (TU Delft - Web Information Systems)
A Bozzon (TU Delft - Human-Centred Artificial Intelligence)
A Katsifodimos (TU Delft - Web Information Systems)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Given a set of pre-trained Machine Learning (ML) models, can we solve complex analytic tasks that make use of those models by formulating ML inference queries? Can we mitigate different tradeoffs, e.g., high accuracy, low execution costs and memory footprint, when optimizing the queries? In this work we present different multi-objective ML inference query optimization strategies, and compare them on their usability, applicability, and complexity. We formulate Mixed-Integer-Programming-based (MIP) optimizers for ML inference queries that makes use of different objectives to find Pareto-optimal inference query plans.