The aim of this study is to discover new active-flow-control (AFC) techniques for separation mitigation in a two-dimensional NACA 0012 airfoil at a Reynolds number of 3000. To find these AFC strategies, a framework consisting of a deep-reinforcement-learning (DRL) agent has been used to determine the action strategies to apply to the flow. The actions involve blowing and suction through jets at the airfoil surface. The flow is simulated with the numerical code Alya, which is a low-dissipation finite-element code, on a high-performance computing system. Various control strategies obtained through DRL led to 43.9% drag reduction, while others yielded an increase in aerodynamic efficiency of 58.6%. In comparison, periodic-control strategies demonstrated lower energy efficiency while failing to achieve the same level of aerodynamic improvements as the DRL-based approach. These gains have been attained through the implementation of a dynamic, closed-loop, time-dependent, active control mechanism over the airfoil.

Correction to: Nature Communicationshttps://doi.org/10.1038/s41467-025-56408-6, published online 07 February 2024 In the version of the article initially published, the table in the lower half of Fig. 7 was missing and is now amended in the HTML and PDF versions of the article.

The control efficacy of deep reinforcement learning (DRL) compared with classical periodic forcing is numerically assessed for a turbulent separation bubble (TSB). We show that a control strategy learned on a coarse grid works on a fine grid as long as the coarse grid captures main flow features. This allows to significantly reduce the computational cost of DRL training in a turbulent-flow environment. On the fine grid, the periodic control is able to reduce the TSB area by 6.8%, while the DRL-based control achieves 9.0% reduction. Furthermore, the DRL agent provides a smoother control strategy while conserving momentum instantaneously. The physical analysis of the DRL control strategy reveals the production of large-scale counter-rotating vortices by adjacent actuator pairs. It is shown that the DRL agent acts on a wide range of frequencies to sustain these vortices in time. Last, we also introduce our computational fluid dynamics and DRL open-source framework suited for the next generation of exascale computing machines

Active flow control strategies for three-dimensional bluff bodies are challenging to design, yet critical for industrial applications. Here we explore the potential of discovering novel drag-reduction strategies using deep reinforcement learning. We introduce a high-dimensional active flow control setup on a three-dimensional cylinder at Reynolds numbers (Re _D) from 100 to 400, spanning the transition to three-dimensional wake instabilities. The setup involves multiple zero-net-mass-flux jets and couples a computational fluid dynamics solver with a numerical multi-agent reinforcement learning framework based on the proximal policy optimization algorithm. Our results demonstrate up to 16% drag reduction at Re _D = 400, outperforming classical periodic control strategies. A proper orthogonal decomposition analysis reveals that the control leads to a stabilized wake structure with an elongated recirculation bubble. These findings represent the first demonstration of training on three-dimensional cylinders and pave the way toward active flow control of complex turbulent flows.

This study presents novel drag reduction active-flow-control (AFC) strategies for a three-dimensional cylinder immersed in a flow at a Reynolds number based on freestream velocity and cylinder diameter of ReD=3900. The cylinder in this subcritical flow regime has been extensively studied in the literature and is considered a classic case of turbulent flow arising from a bluff body. The strategies presented are explored through the use of deep reinforcement learning. The cylinder is equipped with 10 independent zero-net-mass-flux jet pairs, distributed on the top and bottom surfaces, which define the AFC setup. The method is based on the coupling between a computational-fluid-dynamics solver and a multi-agent reinforcement-learning (MARL) framework using the proximal-policy-optimization algorithm. This work introduces a multi-stage training approach to expand the exploration space and enhance drag reduction stabilization. By accelerating training through the exploitation of local invariants with MARL, a drag reduction of approximately 9% is achieved. The cooperative closed-loop strategy developed by the agents is sophisticated, as it utilizes a wide bandwidth of mass-flow-rate frequencies, which classical control methods are unable to match. Notably, the mass cost efficiency is demonstrated to be two orders of magnitude lower than that of classical control methods reported in the literature. These developments represent a significant advancement in active flow control in turbulent regimes, critical for industrial applications.

The control efficacy of classical periodic forcing and deep reinforcement learning (DRL) is assessed for a turbulent separation bubble (TSB) at Reτ=180 on the upstream region before separation occurs. The TSB can resemble a separation phenomenon naturally arising in wings, and a successful reduction of the TSB can have practical implications in the reduction of the aviation carbon footprint. We find that the classical zero-net-mas-flux (ZNMF) periodic control is able to reduce the TSB by 15.7%. On the other hand, the DRL-based control achieves 25.3% reduction and provides a smoother control strategy while also being ZNMF. To the best of our knowledge, the current test case is the highest Reynolds-number flow that has been successfully controlled using DRL to this date. In future work, these results will be scaled to well-resolved large-eddy simulation grids. Furthermore, we provide details of our open-source CFD–DRL framework suited for the next generation of exascale computing machines.