Risk-sensitive Reinforcement Learning for Portfolio Allocation
A.A. Sinha (TU Delft - Electrical Engineering, Mathematics and Computer Science)
FA Oliehoek – Mentor (TU Delft - Sequential Decision Making)
Luciano Cavalcante Siebert – Graduation committee member (TU Delft - Interactive Intelligence)
A. Papapantoleon – Graduation committee member (TU Delft - Applied Probability)
M.M. Celikok – Graduation committee member (TU Delft - Sequential Decision Making)
Rob Huisman – Graduation committee member (Robeco B.V.)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
This study explores the application of risk-sensitive Reinforcement Learning (RL) in portfolio optimization, aiming to integrate asset pricing and portfolio construction into a unified, end-to-end RL framework. While RL has shown promise in various domains, its traditional risk-neutral approach is unsuitable for financial contexts where risk sensitivity is crucial. This research focuses on risk-sensitive RL methods that incorporate different risk measures to manage uncertainty and volatility in financial markets better. The project extends existing RL techniques by adapting the cross-sectional approach to risk-sensitive settings and introducing new variants like PPO-CVaR and PPO-Expectile for portfolio management. A comparative study of these methods is conducted to assess their performance with real market data and simulated environments. The research addresses key questions related to how different risk measures impact learned portfolio strategies, the influence of risk appetite on decision-making, and the performance gap between simulated and real market data. The findings aim to provide insights for practitioners looking to implement risk-sensitive RL in financial asset management.