Fair multiple-workflow scheduling with different quality-of-service goals

Rezaeian, Amin; Naghibzadeh, Mahmoud; Epema, D.H.J.

doi:10.1007/s11227-018-2604-2

Fair multiple-workflow scheduling with different quality-of-service goals

Title

Fair multiple-workflow scheduling with different quality-of-service goals

Author

Rezaeian, Amin (Ferdowsi University of Mashhad)
Naghibzadeh, Mahmoud (Ferdowsi University of Mashhad)
Epema, D.H.J. (TU Delft Data-Intensive Systems)

Date

2019

Abstract

Cloud schedulers that allocate resources exclusively to single workflows are not work-conserving as they may be forced to leave gaps in their schedules because of the precedence constraints in the workflows. Thus, they may lead to a waste of financial resources. This problem can be mitigated by multiple-workflow schedulers that share the leased cloud resources among multiple workflows or users by filling the gaps left by one workflow with the tasks of other workflows. This solution may even work when users have different performance objectives for their workflows, such as budgets and deadlines. As an additional requirement, we want the scheduler to be fair to all workflows regardless of their performance objectives. In this paper, we propose a multiple-workflow scheduler that is able to target different quality of service goals for different workflows and that considers fairness among different users. To this aim, we propose an unfairness metric and four workflow selection policies. We prove that the resource selection that decides based on a task’s sub-budget, sub-deadline, finish time, and cost on different resources is selecting the best resource based on the given information, while using the smallest number of calculations. Simulations show that there is a trade-off between overall cost, makespan, and fairness. We conclude that the best workflow selection policy to reduce unfairness is the direct policy, which explicitly selects the workflow that minimizes the value of the proposed unfairness metric in each round.

Subject

Budget-constrained workflow
Deadline-constrained workflow
Fairness
Multiple workflows
Scheduling

To reference this document use:

http://resolver.tudelft.nl/uuid:b1e6f57f-c21f-4149-b058-d26dc4ccb230

DOI

https://doi.org/10.1007/s11227-018-2604-2

Embargo date

2019-09-12

ISSN

0920-8542

Source

Journal of Supercomputing: an international journal of high-performance computer design, analysis and use, 75 (2), 746-769