Hierarchize Pareto Dominance in Multi-Objective Stochastic Linear Bandits

Journal article (2024)

Authors

Ji Cheng City University of Hong Kong
Bo Xue City University of Hong Kong
Y. Jiaxiang Team Marcel Sluiter - Mechanical, Maritime and Materials Engineering
Qingfu Zhang City University of Hong Kong
Research Group
Team Marcel Sluiter (Mechanical, Maritime and Materials Engineering) (TU Delft)
Reinforcement Learning Online Learning & Bandits
More Info
expand_more

Published Date

2024

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Mechanical, Maritime and Materials Engineering

Department

Materials Science and Engineering

Research Group

Team Marcel Sluiter

Abstract

Multi-objective Stochastic Linear bandit (MOSLB) plays a critical role in the sequential decision-making paradigm, however, most existing methods focus on the Pareto dominance among different objectives without considering any priority. In this paper, we study bandit algorithms under mixed Pareto-lexicographic orders, which can reflect decision makers’ preferences. We adopt the Grossone approach to deal with these orders and develop the notion of Pareto-lexicographic optimality to evaluate the learners’ performance. Our work represents a first attempt to address these important and realistic orders in bandit algorithms. To design algorithms under these orders, the upper confidence bound (UCB) policy and the prior free lexicographical filter are adapted to approximate the optimal arms at each round. Moreover, the framework of the algorithms involves two stages in pursuit of the balance between exploration and exploitation. Theoretical analysis as well as numerical experiments demonstrate the effectiveness of our algorithms.