Enhancing Financial Algorithms for Pairs Trading using Reinforcement Learning

Abstract

Pairs trading exploits the mean reversion of a cointegrated spread of two stocks, classically traded with fixed z-score rules. We recast it as a continuous portfolio-optimisation problem and train reinforcement-learning (PPO) agents to size the two legs and a risk-free asset, comparing a constrained agent forced into the market-neutral hedge against a free agent that weights the legs independently. Agents are trained on a generative market model calibrated to each real pair, and evaluated out of sample, with and without transaction costs, against the classical z-rule. The constrained agent learns the spread’s direction but does not beat the z-rule: it over-trades when no arbitrage is available instead of stepping aside. The free agent earns higher but far more variable returns, mixing directional market exposure with some genuine arbitrage. Transaction costs push both toward smoother, more conservative policies. We outline a potential improvement to the constrained agent to leverage its sizing capabilities in a future work.