Print Email Facebook Twitter Symbolic Regression Methods for Reinforcement Learning Title Symbolic Regression Methods for Reinforcement Learning Author Kubalik, Jiri (Czech Technical University) Derner, Erik (Czech Technical University) Zegklitz, Jan (Czech Technical University) Babuska, R. (TU Delft Learning & Autonomous Control; Czech Technical University) Date 2021 Abstract Reinforcement learning algorithms can solve dynamic decision-making and optimal control problems. With continuous-valued state and input variables, reinforcement learning algorithms must rely on function approximators to represent the value function and policy mappings. Commonly used numerical approximators, such as neural networks or basis function expansions, have two main drawbacks: They are black-box models offering little insight into the mappings learned, and they require extensive trial and error tuning of their hyper-parameters. In this paper, we propose a new approach to constructing smooth value functions in the form of analytic expressions by using symbolic regression. We introduce three off-line methods for finding value functions based on a state-transition model: Symbolic value iteration, symbolic policy iteration, and a direct solution of the Bellman equation. The methods are illustrated on four nonlinear control problems: Velocity control under friction, one-link and two-link pendulum swing-up, and magnetic manipulation. The results show that the value functions yield well-performing policies and are compact, mathematically tractable, and easy to plug into other algorithms. This makes them potentially suitable for further analysis of the closed-loop system. A comparison with an alternative approach using neural networks shows that our method outperforms the neural network-based one. Subject genetic programmingnonlinear optimal controlpolicy iterationReinforcement learningsymbolic regressionvalue iteration To reference this document use: http://resolver.tudelft.nl/uuid:02411602-2f70-4deb-8ede-29f98ff8f91b DOI https://doi.org/10.1109/ACCESS.2021.3119000 ISSN 2169-3536 Source IEEE Access, 9, 139697-139711 Part of collection Institutional Repository Document type journal article Rights © 2021 Jiri Kubalik, Erik Derner, Jan Zegklitz, R. Babuska Files PDF Symbolic_Regression_Metho ... arning.pdf 2.71 MB Close viewer /islandora/object/uuid:02411602-2f70-4deb-8ede-29f98ff8f91b/datastream/OBJ/view