Oikonomos

An Opportunistic, Deep-Learning, Resource-Recommendation System for Cloud HPC

Conference Paper (2023)
Authors

Jan Harm L.F. Betting (Erasmus MC)

Dimitrios Liakopoulos (National Technical University of Athens)

Max Engelen (Erasmus MC)

C Strydis (Erasmus MC)

Affiliation
External organisation
To reference this document use:
https://doi.org/10.1109/ASAP57973.2023.00039
More Info
expand_more
Publication Year
2023
Language
English
Affiliation
External organisation
Pages (from-to)
188-196
ISBN (electronic)
9798350346855
DOI:
https://doi.org/10.1109/ASAP57973.2023.00039

Abstract

The cloud has become a powerful environment for deploying High-Performance Computing (HPC) applications. However, the size and heterogeneity of cloud-hardware offerings poses a challenge in selecting the optimal cloud instance type. Users often lack the knowledge or time necessary to make an optimal choice. In this work, we propose Oikonomos, a data-driven, opportunistic, resource-recommendation system for HPC applications in the cloud. Oikonomos trains a Multi-layer Perceptron (MLP) to predict the performance of a given HPC application, for different input parameters and instance types. It, then, calculates the cost of executing the application on different instance types and proposes the one best-fitting the user's needs. We deployed Oikonomos on a diverse mix of HPC workloads, and found that for all applications, it approached an optimal policy. The optimal instance type was chosen in 90% of the cases for seven out of eight applications, scoring a Mean Absolute Percentage Error (MAPE) consistently below 20%. This demonstrated that Oikonomos can provide a practical, general-purpose, resource-recommendation system for cloud HPC.

No files available

Metadata only record. There are no files for this record.