Market Making in Limit Order Books
using Reinforcement Learning
A.A.G. Wiggers (TU Delft - Electrical Engineering, Mathematics and Computer Science)
A. Papapantoleon – Mentor (TU Delft - Applied Probability)
M.A. Sharifi Kolarijani – Mentor (TU Delft - Team Amin Sharifi Kolarijani)
G.N.J.C. Bierkens – Graduation committee member (TU Delft - Statistics)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Market making, the act of providing liquidity to the market by simultaneously buying and selling, is a difficult problem to solve. The use of reinforcement learning to solve for market making is increasing, as academics and practitioners alike look for novel ways to approximate for optimal policies in increasingly complex markets. This thesis examines the use of reinforcement learning to solve the market making problem in limit order books. To this end, we provide the theoretical background on market making, modelling limit order books, and reinforcement learning. Furthermore, we implement and compare ’classical’ algorithms, such as Q-learning and value and policy iteration, and compare their policies with newer techniques involving deep learning, namely deep Q-networks and double deep Q-networks. To train and compare these models, we use two models to simulate the dynamics of an order book. Experiment results and the ultimately learned policies are presented and discussed. We propose several ideas that could be worked on in the future.