A Multi-step and Eligibility Trace Approach to Incremental Dual Heuristic Programming for Flight Control
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Incremental Dual Heuristic Programming (IDHP) is a successor to the Dual Heuristic Programming (DHP) algorithm that uses an online identified incremental system model, this algorithm showed promising online learning and fault tolerance in simulated flights. This paper studies the potential for extending IDHP through augmenting the computation of agent updates and returns, more specifically, by using eligibility trace updates and multi-step temporal difference error. This results in the IDHP, multi-step IDHP (MIDHP), and MIDHP variants, which are compared against IDHP in simulated flight scenarios with faults introduced mid-flight. The results demonstrate that flight controllers derived from the proposed variants have improved reference tracking & fault tolerance over the baseline IDHP, with the most improvement observed in MIDHP.