Real-time single-step deep reinforcement learning framework for control of Tollmien-Schlichting waves

Abstract

A real-time adaptive control framework based on the single-step deep reinforcement learning (SDRL) algorithm is proposed and implemented in numerical simulations for attenuating Tollmien-Schlichting (TS) wave instabilities in a two-dimensional incompressible boundary layer over a flat plate. The perturbation formulation of the Navier-Stokes equations is solved in the time domain using a finite-volume discretization scheme. The controller utilizes volume-distributed body forces as proxies for dielectric barrier discharge plasma actuators, with instantaneous wall-pressure feedback in a closed-loop configuration to dynamically adjust the amplitude of the controlling body force. By learning an opposition control strategy—generating TS waves with the same amplitude but opposite phase to the incoming TS instabilities—the SDRL-based controller achieves effective suppression of both single- and multifrequency TS waves. The performance of the SDRL-based controller is evaluated against a classical model-free adaptive approach, the filtered-x least mean square– (FXLMS) based controller, focusing on effectiveness in terms of TS wave suppression and convergence speed. The SDRL-based controller achieves up to 94.81%
suppression of TS waves, demonstrating superior suppression performance and faster convergence than the FXLMS-based controller in most control cases. Additionally, the robustness of the controllers against measurement and actuation noises is examined. These results establish the SDRL framework as a low-cost, model-free control approach for robust real-time control of convective flow instabilities in the presence of noise sources. Relying on compact neural networks and minimal sensor feedback, the methodology enables a practical and experimentally feasible artificial intelligence-based controller for convective instabilities.