An empirical analysis of entropy search in batch bayesian optimisation

An empirical analysis of entropy search in batch bayesian optimisation: A comprehensive study of function shape, batch size, noise level, and dimensionality impact on information-theoretic methods

Hautelman, Alex (TU Delft Electrical Engineering, Mathematics and Computer Science)

de Vries, J.A. (mentor)
Spaan, M.T.J. (mentor)
Lofi, C. (graduation committee)

Delft University of Technology

Delft University of Technology

Computer Science and Engineering

CSE3000 Research Project

2023-06-28

Bayesian optimisation is a rapidly growing area of research that aims to identify the optimum of the black-box function, as it strategically directs the optimisation process towards promising regions. This paper provides an overview of the theoretical background used by the Entropy Search algorithms under study, mainly Predictive Entropy Search, Max-Value Entropy Search, and Joint Entropy Search. Furthermore, we empirically analyse the performance and sensitivity of the algorithms in different environment settings. In particular, we discuss the impact of function shape, batch size, noise level, and the number of input dimensions on the final simple regret metric. The results show the weak spots of the information-theoretic methods. However, the algorithms perform better for batch optimisation, demonstrating the advantage when considering the information on the maximum function value.

Bayesian optimization
Information Theory
Acquisition Function
performance analysis

http://resolver.tudelft.nl/uuid:6d8035dd-f425-46df-8679-dfad91f9ea8d

https://github.com/ahautelman/entropy-seach-batch-global-optimiation-performance

Student theses

bachelor thesis

© 2023 Alex Hautelman