An empirical analysis of entropy search in batch bayesian optimisation

A comprehensive study of function shape, batch size, noise level, and dimensionality impact on information-theoretic methods

Bachelor thesis (2023)

Authors

P.A. Hautelman Electrical Engineering, Mathematics and Computer Science

Contributors

J.A. de Vries Algorithmics - EEMCS (supervisor 1)
M.T.J. Spaan Algorithmics - EEMCS (supervisor 1)
C. Lofi Web Information Systems - EEMCS (supervisor 2)
Faculty
Electrical Engineering, Mathematics and Computer Science
Copyright: © 2023 Alex Hautelman
Acquisition Function Bayesian optimization Information Theory Performance analysis
More Info
expand_more

Published Date

28-06-2023

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Abstract

Bayesian optimisation is a rapidly growing area of research that aims to identify the optimum of the black-box function, as it strategically directs the optimisation process towards promising regions. This paper provides an overview of the theoretical background used by the Entropy Search algorithms under study, mainly Predictive Entropy Search, Max-Value Entropy Search, and Joint Entropy Search. Furthermore, we empirically analyse the performance and sensitivity of the algorithms in different environment settings. In particular, we discuss the impact of function shape, batch size, noise level, and the number of input dimensions on the final simple regret metric. The results show the weak spots of the information-theoretic methods. However, the algorithms perform better for batch optimisation, demonstrating the advantage when considering the information on the maximum function value.