IM-TD3
A Reinforcement Learning Approach for Liquid Rocket Engine Start-Up Optimization
Yuwei Liu (National University of Defense Technology)
Yang Li (Chinese Academy of Sciences)
Yuqiang Cheng (National University of Defense Technology)
Wei Pan (TU Delft - Robot Dynamics)
Jianjun Wu (National University of Defense Technology)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
With advancements in reusable liquid rocket engine technology to meet the diverse demands of space missions, engine systems have become increasingly complex. In most cases, these engines rely on stable open-loop control and closed-loop regulation systems. However, due to the high degree of coupling and nonlinear dynamics within the system, most transient adjustments still depend on open-loop control. Open-loop control often fails to provide the optimal control strategy when encountering external disturbances. To address this issue, we introduce the intrinsically motivated twin delayed deep deterministic (TD3) algorithm, specifically designed for the startup process of LOX/Kerosene high-pressure staged combustion engine. This approach leverages intrinsic motivation to enable the algorithm to adapt to the abrupt parameter changes during the start-up process. A series of comprehensive experiments were conducted to verify the effectiveness of our method. The experimental results demonstrate that our method outperforms both the PID method and previous researchers' reinforcement learning methods based on the TD3 algorithm and DDPG, achieving a faster and more stable start-up process and significantly enhancing engine performance.