Online Adaptive Flight Control is interesting in the context of growing complexity of aircraft systems and their adaptability requirements to ensure safety. An Incremental Approximate Dynamic Programming (iADP) controller combines reinforcement learning methods, optimal control and Online identified incremental model to achieve optimal adaptive control suitable for Nonlinear Time-Varying systems. The main contribution of this thesis is twofold. Firstly, the iADP controller is designed to achieve automatic Online rate control to track pilot commands via setpoints provided by the manual outer loop on Citation II Aircraft model. Secondly, to assess the controller performance in the presence of sensor dynamics and actuator dynamics, an analysis is carried out to identify causes of any performance degradation. The simulation results from iADP longitudinal control using full state feedback indicate that the discretization of sensor signals, sensor bias and transport delays did not have any significant effect on the controller performance or on the incremental model identification. However noisy signals and sensors delays are found to cause controller performance degradation. Appropriate filtering of signals resulted in better estimation of the incremental model subsequently improving the controller performance due to noisy signals. Control performance degradation due to sensor delays should be addressed in future before conducting flight tests on Citation II Aircraft. ... Read more

In recent years Adaptive Critic Designs (ACDs) have been applied to adaptive flight control of uncertain, nonlinear systems. However, these algorithms often rely on representative models as they require an offline training stage. Therefore, they have limited applicability to a system for which no accurate system model is available, nor readily identifiable. Inspired by recent work on Incremental Dual Heuristic Programming (IDHP), this paper derives and analyzes a Reinforcement Learning (RL) based framework for adaptive flight control of a CS-25 class fixed-wing aircraft. The proposed framework utilizes Artificial Neural Networks (ANNs) and includes an additional network structure to improve learning stability. The designed learning controller is implemented to control a high-fidelity, six-degree-of-freedom simulation of the Cessna 550 Citation II PH-LAB research aircraft. It is demonstrated that the proposed framework is able to learn a near-optimal control policy online without a priori knowledge of the system dynamics nor an offline training phase. Furthermore, it is able to generalize and operate the aircraft in not previously encountered flight regimes as well as identify and adapt to unforeseen changes to the aircraft’s dynamics. ... Read more

The control of aircraft can be carried out by Reinforcement Learning agents; however, the difficulty of obtaining sufficient training samples often makes this approach infeasible. Demonstrations can be used to facilitate the learning process, yet algorithms such as Apprenticeship Learning generally fail to produce a policy that outperforms the demonstrator, and thus cannot efficiently generate policies. In this paper, a model-free learning algorithm with Reinforcement Learning in the loop, based on Apprenticeship Learning, is therefore proposed. This algorithm uses external measurement to improve on the initial demonstration, finally producing a policy that surpasses the demonstration. Efficiency is further improved by utilising the policies produced during the learning process. The empirical results for simulated quadrotor control show that the proposed algorithm is effective and can even learn good policies from a bad demonstration. ... Read more

Higher levels of autonomy in aerospace systems is an urgent requirement, considering the increase in control task difficulties, and the need for adaptability of the complex systems. Reinforcement learning (RL) control is one of the promising approaches for adaptive control of air vehicles that are designed for automation. Conventional discrete reinforcement learning methods fail in providing satisfactory performance for flight control systems (FCSs), especially for a complex configuration of a tailless over-actuated aircraft. The lack of efficiency of the discrete controller in exploration for finding the optimal policy, the so-called problem of 'curse of dimensionality', results in an approach that is not suitable for online implementation. Also, the achieved discrete non-smooth control policy usually does not apply to the real world control surfaces. This paper studies the experiments with Heuristic Dynamic Programming (HDP), a method obtained from adaptive critic design (ACDs), as a continuous reinforcement learning approach. ACD methods can capture the nonlinearities in the complex dynamics of the aircraft while solving the control problem computationally efficient by using continuous states and action spaces. Such qualities make ACDs suitable for online FCS design for unstable systems like tailless aircraft. In this paper, the ACD-based controller is developed and implemented for the Innovative Control Effector (ICE) aircraft, a highly maneuverable aircraft with redundancy in its control effectors suite. The coupled control effectors configuration has strong interactions and, therefore, proposes a need for proper control allocation. The online simulation results show the accuracy of the designed continuous RL controller in the longitudinal control of the aircraft using different sets of control effectors. The proposed approach also shows significant improvements in the tracking performance and control policy smoothness (e.g., compared to discrete methods). ... Read more

In this paper an adaptive version of the incremental nonlinear control allocation (INCA), which is able to account for sudden changes in the aerodynamic configuration of an aircraft, is investigated. The controller is designed for the highly maneuverable and tailless innovative control effectors (ICE) aircraft, which has a control suite of 13 nonlinear, interacting and axis-coupled effectors. The least mean squares (LMS) method is used to estimate a delta control effectiveness Jacobian (CEJ) based on the difference between the expected and measured accelerations of the aircraft. This delta CEJ model is then added to the onboard spline CEJ model to achieve fault tolerance. By keeping the nominal spline model intact the nonlinearities and interactions of the effectors remain modeled, while the LMS estimator allows for fast adaptation. Simulations for four different maneuvers and failure cases showed that the estimator is able to stabilize the aircraft for the most demanding maneuver. For two less demanding maneuvers the adaptive controller greatly reduced the control effort while keeping the tracking error similar to the non-adaptive controller. For the remaining fourth maneuver, which operates in a flight region with the most significant interactions and nonlinearities, the adaptive controller had a reduced performance compared to the nonadaptive controller. A sensitivity analysis showed that the choice of design parameters greatly influences the results, and no general set of best performing parameters was found. ... Read more

As of April 2019, upset prevention and recovery training in flight simulation training devices is a mandatory practice for commercial and civil aircraft pilots. Aircraft stalls are a well-known upset type, therefore simulation of aircraft stall behavior is required. A key characteristic of stalls is the buffeting component, which in current stall models is still insufficiently modeled. In this research, a new methodology to more accurately model stall buffet behavior using swept
wing flight test data is presented. Buffet effects occur after exceeding the critical angle of attack, with an aircraft type-specific buffet onset duration to fully develop maximum buffet intensity. The buffet transient behavior is modeled with a frequency response fit and a multivariate second-order polynomial to capture aircraft eigenmode-shape frequencies and buffet intensity respectively. Aircraft recovery and thus receding buffet effects occur at an increasing angle
of attack, which is used as buffet offset. Generalization of the results was shown with the validation of data set of a straight wing aircraft, which indicates a step towards a more generic stall buffet model methodology. ... Read more

Loss of control is considered as the primary cause of fatal accidents in aviation, which occurs when the aircraft has left the safe flight envelope. To reduce loss-of-control-related accidents, it is important to estimate the safe flight envelope at the current flight condition and integrate it into flight control system design. This task is known as envelope estimation and protection. This project investigates this task on the Innovative Control Effectors aircraft, an over-actuated tailless fighter aircraft with complex aerodynamic coupling between control effectors. It has been observed that this aircraft can easily steer outside the flight envelope and lose control due to its huge control authority.

This thesis proposes a novel and practical framework for safe flight envelope estimation and protection, in order to reduce loss-of-control-related accidents. Despite that multiple envelope estimation methods exist in literature, conventional analytical estimation methods fail to function efficiently for systems with high dimensionality and complex dynamics, which is often the case for high-fidelity aircraft models. In this way, this paper develops a probabilistic envelope estimation method based on Monte Carlo simulation. This method generates a probabilistic estimation of the flight envelope with kernel density estimation by simulating a sample of flight trajectories with extreme control effectiveness, which describes the envelope more practically with fuzzy sets instead of conventional crisp sets. It is shown that this method can significantly reduce the computational load compared with previous optimization-based methods and guarantee feasible and conservative envelope estimation of no less than seven dimensions. This method was applied to the Innovative Control Effectors aircraft developed by Lockheed Martin. The estimation results are demonstrated by comparing different flight conditions and covariance analysis.

The estimated probabilistic flight envelope is used for online envelope protection by a database approach, which estimates the flight envelope offline and carries the results onboard for protection. Both a conventional state-constraint-based and a novel predictive probabilistic flight envelope protection systems were implemented on a multi-loop nonlinear dynamic inversion controller by extending the concept of pseudo control hedging. No systematic framework was available to apply envelope protection to such controller. Real-time simulation results prove that the proposed framework can protect the aircraft within the estimated envelope and save the aircraft from maneuvers that otherwise would result in loss of control. Possibilities were also explored to employ parametric models in envelope protection to simplify the database.

This work, however, is still limited to offline estimation with open-loop commands. Future work can extend this framework to aircraft damage models and closed-loop commands. ... Read more

In this work interval analysis is applied to the thirteen control effector Innovative Control Effectors model to find its trim set. The method to find trim states is based on interval box consistency. At low speed the method is capable of finding interval enclosures of single trim points with a high accuracy if the minimum number of required control effectors is used. At higher speeds the found accelerations are larger. When looking for a full trim set the method finds continuous bounds on the control effectors for the entire input range in one run. This is a good demonstration of the advantages that interval analysis has over conventional methods that generally can only find one trim point at a time. The found bounds are a maximum of 1 deg wide for each control effector, but despite this the remaining accelerations can be up to 0.5 m/s^2 for linear accelerations and up to 10 deg/s^2 for rotational accelerations. Because of these large accelerations the found solutions are not acceptable as trim conditions. On the other hand the potential that interval analysis has as a trimming method is demonstrated, since continuous bounds on trim sets have been found in a single run. This is a feat that no other trimming method has yet accomplished. Further research is needed to exploit the full potential of interval trim methods so that the results can be used for other purposes such as flight envelope prediction. ... Read more

Flight control of the DelFly is challenging, because of its complex dynamics and variability due to manufacturing inconsistencies. Machine Learning algorithms can be used to tackle these challenges. A Policy Gradient algorithm is used to tune the gains of a Proportional-Integral controller using Reinforcement Learning. Furthermore, a novel Classification Algorithm for Machine Learning control (CAML) is presented, which uses model identification and a neural network classifier to select from several predefined gain sets. The algorithms show comparable performance when considering variability only, but the Policy Gradient algorithm is more robust to noise, disturbances, nonlinearities and flapping motion. ... Read more

A new aviation legislation makes it mandatory for air-carrier pilots to go through stall recovery training on simulators. As a result, new aerodynamic modeling techniques are required to model complex non-linear behavior of the aircraft flight envelope. In 2005, the Multivariate Simplex B-Splines method was developed. MSBS are a true general function approximator and are easily integrated in standard identification routines. Their downside is that the basis functions and B-coefficients, forming the B-net, do not have a straightforward physical interpretation. Also creating the triangulation is not a trivial process. Parts of the triangulation domain can require higher approximation power and continuity. The consequence is an overall dense triangulation and high order basis functions. This introduces problems such as over fitting the model and divergent behavior on triangulation boundaries. Physical-Splines make use of a linear transformation that transforms from the barycentric coordinate space to the Cartesian coordinate space, giving the MSBS a physical interpretation. The physical transformation is introduced to the optimization process in the form of equality and inequality constraints. This way a-priori aerodynamic information can form a bound on the stability derivatives. Promising results show that they are robust, prevent over-fitting, prevent propagation of erroneous data, remove divergent behavior on triangulation boundaries, and that they can be used for extrapolation of sparse datasets. Also a stepwise orthonormalization can create physical model structure constraints and set unimportant physical model terms to zero. Overall, the physical constraints make it possible to adjust model approximation power locally and alter the B-net via the physical parameters without breaking them. ... Read more

Loss of Control is the primary contributor to aviation fatalities. To prevent this type of accident, flight envelope protection is considered to be a necessary development. The calculation of the Safe Flight Envelope provides a bound on the states that can safely be approached by the aircraft. Although theoretically accurate, some states may not be reachable under the influence of disturbances (e.g. turbulence). In this thesis a stochastic extension to the reachability analysis is applied to a simplified aircraft model. The probabilistic reachability analysis yields the transition probability from a state to the target set. By comparing the deterministic and probabilistic Safe Flight Envelope, it becomes clear that the Safe Flight Envelope can shrink considerably under the influence of turbulence. It is shown that for a 3 sigma (99.7%) confidence interval, the envelope can shrink by as much as 50.8% compared to the deterministic envelope. Furthermore, it is found that for high roll angles, some parts of the deterministic envelope have a 0% transition probability under the influence of turbulence, further emphasizing the importance of probabilistic envelopes. ... Read more

Objects ingeostationary transfer orbit (GTO) can collide with operative satellites in lowEarth orbit (LEO) and geostationary orbit (GEO). Various organisationshave laid down debris-mitigation guidelines that will be enforced by law forfuture launchers. One of the guidelines entails proving that the generateddebris will re-enter in less than 25 years with a 90% probability. Naturalperturbations can be exploited to meet this requirement without the use ofextra propellant or complex de-orbiting systems, which is especially attractivefrom an economic point of view. Objects in GTO can undergo a resonancetriggered by an interplay between perturbations caused by the Sun’s gravity andthe irregularities in Earth’s gravity field, leading to a sudden re-entryor making the object stay in orbit for decades. This effect is very sensitiveto initial conditions because it depends on the relative positions of theperigee and the Sun when the semi-major axis is close to 15000 km.By simulating the orbital evolution of a representative GTO object—ballistic coefficient of 0.011 m²/kg, initial orbital inclination of 10degrees and initial perigee altitude of 200 km— for several initialepochs, it was found that favourable launch conditions take place twice perday during most part of the year, while for epochs close to the equinoxesof March and September they only happen once per day or not at all. Giventhe high sensitivity to initial conditions, the problem was studied from astatistical perspective, taking into account the uncertainties in thevalues of the relevant parameters. Semi-analytical techniques were used topropagate the mean equinoctial elements instead of the osculatingCartesian elements, which reduced computation times by a factor of 45 whilestill keeping proper levels of accuracy. Current practice for GTO launches fromKourou is to launch at around 6-7 PM. It was found that the launchtime leading to the highest probability of compliance with debris-mitigationguidelines for GEO launches from the European spaceport in Kourou isslightly later, regardless of the day of the year, although the value of theoptimal lifetime does vary slightly throughout the year. Thus, achange in procedures would be required in order to reach a higher degreeof compliance with debris-mitigation guidelines, which was below 10% for GTOlaunches carried out with Ariane 5 from in the period 2004-2012. ... Read more