Symbolic Turbulence Modelling with Multi- Agent Reinforcement Learning
Z.S. Tan Zing Shawn (TU Delft - Aerospace Engineering)
R.P. Dwight – Mentor (TU Delft - Aerodynamics)
A. Eidi – Mentor (TU Delft - Aerodynamics)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Turbulence modelling remains a major challenge in computational fluid dynamics (CFD), as Reynolds-Averaged Navier-Stokes (RANS) closures rely on empirical relations introducing significant model-form uncertainty. Symbolic regression offers a path toward interpretable data-driven closures, but conventional Deep Symbolic Regression (DSR) struggles to efficiently explore the vast space of physically consistent expressions.
We propose a Multi-Agent Deep Symbolic Regression (MADSR) framework that reformulates turbulence model discovery as a cooperative multi-agent reinforcement learning (MARL) problem. Each agent discovers one scalar coefficient function in the tensor-basis expansion of the Reynolds-stress anisotropy tensor, sharing a common reward derived from frozen RANS evaluations. This cooperative setup promotes coordinated learning among model components.
In MADSR, the effectiveness of 2 MARL techniques, proximal policy optimization (PPO) and centralised training decentralised execution (CTDE), is investigated on symbolic turbulence modelling. Several MADSR variants are developed and tested, including a vanilla multi-agent DSR, a proximal policy optimization (PPO) based MADSR, an actor-critic MADSR, and a MAPPO-DSR inspired by multi-agent proximal policy optimization (PPODSR).
Applied to the Explicit Algebraic Reynolds-Stress Model (EARSM) and k-corrective RANS formulations, MADSR outperforms single-agent DSR in frozen RANS evaluations of the periodic-hill benchmark. The multi-agent structure enhances exploration efficiency and enables discovery of more consistent and interpretable turbulence closures. MADSR thus represents a promising step toward fully end-to-end, reinforcement-learning-based symbolic turbulence modelling.