Incremental Dual Heuristic Programming (IDHP) is a successor to the Dual Heuristic Programming (DHP) algorithm that uses an online identified incremental system model, this algorithm showed promising online learning and fault tolerance in simulated flights. This paper studies the potential for extending IDHP through augmenting the computation of agent updates and returns, more specifically, by using eligibility trace updates and multi-step temporal difference error. This results in the IDHP, multi-step IDHP (MIDHP), and MIDHP variants, which are compared against IDHP in simulated flight scenarios with faults introduced mid-flight. The results demonstrate that flight controllers derived from the proposed variants have improved reference tracking & fault tolerance over the baseline IDHP, with the most improvement observed in MIDHP. ... Read more