Sample efficiency is a critical metric in intelligent control systems as it directly influences the feasibility and effectiveness of learning-based approaches. This paper presents the study of how Randomized Ensemble Double Q-Learning (REDQ), a sample-efficient model free algorithm, can be used in flight control applications. Three controllers were developed for: pitch, roll and combined biaxial attitude tracking tasks and tested on a high fidelity Cessna Citation 550 model. For each control task, three agents were trained offline: two using REDQ enhanced Soft Actor Critic (SAC) architectures and one using a standard SAC architecture for comparison. REDQ agents showed statistically significant improvements in sample efficiency during initial learning. Average accurate tracking convergence (error < 1◦) occurred within 5,500 training steps for pitch, 6,400 for roll and 11,500 for biaxial control. The gains in sample efficiency were shown to have drawbacks in learning stability and robustness when deviated too far from nominal conditions.

This study explored the application of particle subswarm optimisation - a non-gradient-based swarming intelligence method - in neuroevolution for solving the powered descent landing problem. It was compared to a gradient-based Soft Actor-Critic (SAC) reinforcement learning algorithm under sparse and non-sparse reward settings. A simulation environment was developed incorporating aerodynamic modelling, grid fins, thrust vector control, and variable inertia. Results show that a Soft Actor Critic struggled with non-sparse rewards due to its inability to generalise across the long-horizon powered descent task due to non-informative gradients from high-variance state-action values. In contrast, learning with the sparse rewards became trapped in a local optimum due to a weak reward signal. Particle subswarm optimisation achieved rapid convergence to feasible solutions and reduced propellant consumption, outperforming SAC due to an episodic gated reward structure, local minima avoidance, and gradient independence. This study establishes particle subswarm optimisation neuroevolution as a viable alternative to reinforcement learning for the assignment of throttle during powered descent of a launch vehicle's first stage.

Demand for fully autonomous VTOL aircraft with high agility has exposed the shortcomings of Euler‐angle models: limiting factors such as the presence of singularities and the need for expensive trigonometric operations impact the reliability and safety of autonomous systems and impair the execution of highly-aggressive manoeuvres. Quaternions offer a singularity‐free alternative, yet their use on full‐scale helicopters remains underexplored. Addressing this gap in research, this paper introduces a methodology for converting existing Euler‐based linear models of a Bo105 helicopter to quaternion form, preserving model fidelity. A hierarchical flight control system is then designed, controlling attitude with a novel implementation of Linear Quadratic Integral (LQI) control with quaternions, and commanding velocity and position using PI and P controllers. The controller is tuned using a structured approach implementing the Particle Swarm Optimization (PSO) algorithm. The system is then augmented with a Finite State Machine (FSM) autopilot to follow offline-generated trajectories. The implemented system was evaluated by simulating a slalom and a pop-up manoeuvre: the simulations demonstrated maximum tracking errors of 5 m for slalom and 7 m for pop‐up, primarily due to velocity‐loop delays, while achieving zero steady‐state error in the position. These results confirm the effectiveness of quaternion‐based modelling and control for agile, autonomous VTOL operations.

Incremental Dual Heuristic Programming (IDHP) is a successor to the Dual Heuristic Programming (DHP) algorithm that uses an online identified incremental system model, this algorithm showed promising flight control performance and tolerance of faults in simulation experiments. This paper studies the potential for extending IDHP through augmenting the computation of agent updates and returns, more specifically, by using eligibility trace updates and multi-step temporal difference error. This results in the IDHP(𝜆), MIDHP, and MIDHP(𝜆) algorithms, which are compared against IDHP in several simulated flight control scenarios with faults introduced mid-flight. The results demonstrate that the proposed algorithms have improved flight control performance and fault tolerance in terms of tracking errors when controlling a nominal aircraft and an aircraft with faults introduced, with the most improvement observed in MIDHP(𝜆)

Exploring planetary bodies using robot swarms can potentially increase the value of the exploration missions; enabling the execution of novel measurements and explorations previously deemed impractical or unattainable. Despite its potential, the technology readiness level of planetary swarms is not very mature. This work uses multi-agent reinforcement learning to find control policies that allow swarms to autonomously explore unknown areas in a decentralized manner, contributing towards the technology readiness of the field. A multi-agent proximal policy optimization (MAPPO) algorithm is proposed for this end, where the policy uses LIDAR perception information, and the input of the value function contains local and global environment information. The algorithm finds control policies that achieve cooperation behaviors and generalize to unseen swarm sizes and environments learning with simple, sparse reward functions. Moreover, different types of reward functions, value inputs, and environment configurations are investigated. Compared with the state-of-the-art in the field, MAPPO can learn with a larger number of agents, more complicated environments, and using sparse rewards instead of dense ones.

Reinforcement Learning applied to flight control has shown to have several benefits over classical, linear flight controllers, as it eliminates the need for gain scheduling and it could provide fault-tolerance. The application to civil aviation in practice, however, is non-existent as there are multiple safety concerns. This research demonstrates the evaluation of longitudinal Handling Qualities of the Soft Actor-Critic Deep Reinforcement Learning framework with the aim to translate the unpredictable black box of Reinforcement Learning into classical flight control terminology. The framework is applied to a pitch rate command system of a jet aircraft and shows robustness to off-nominal flight conditions, center of gravity shifts and biased sensor noise. Accurate tracking performance is achieved, while adhering to Level 1 longitudinal Handling Qualities for all conditions.

The Flying-V is a tailless, V-shaped flying-wing type aircraft that promises to offer significant increases in aerodynamic efficiency. Due to its configuration, the Flying-V faces some control and stability related issues. These include limited control authority, pitch break tendencies and non-ideal handling qualities. To enhance the handling qualities of the Flying-V, Incremental Nonlinear Dynamic Inversion (INDI)-based flight control systems have been proposed. INDI, a sensor-based alternative to conventional Nonlinear Dynamic Inversion (NDI), is rooted in the principle of feedback linearization. Unlike NDI, INDI does not depend heavily on accurate on-board models (OBM), thereby offering increased robustness to aerodynamic uncertainties. However, singular perturbations—such as time delays, aeroelastic effects, and additional unmodeled or unknown dynamics—have been identified as challenges for INDI-based control laws. Various strategies have been explored to improve the overall robustness of INDI-based flight control systems, including outer-loop tuning and inversion loop augmentation strategies. In this research a multi-loop µ-optimal approach for designing robust inversion-based flight control laws is explored for the design of an explicit model-following pitch-rate control system for a short-period approximation of the Flying-V’s longitudinal dynamics. The design problem takes into account both regular and singular perturbations. To assess the robust stability and performance of the proposed control systems, a structured singular value analysis was performed. It was concluded that a multi-loop synthesis approach is capable of achieving better robust stability and performance levels when compared to either strictly inner-loop or outer-loop synthesis. As such, it can be concluded that multi-loop synthesis approaches are best capable of leveraging the robustness functionalities of multi-loop inversion-based control systems.

The Flying-V aircraft could revolutionize commercial aviation, boasting a potential 25% increase in aerodynamic efficiency. Due to inherent design limitations regarding static stability, the need for a proper Flight Control System (FCS) is essential for the development of the aircraft. The concept of Hybrid Incremental Nonlinear Dynamic Inversion (INDI) was introduced to mitigate the insufficient stability margin encountered in existing sensor-based INDI systems due to sensor time delays, to achieve Level 1 Handling Qualities (HQ). Furthermore, the research introduces an exponential potential function-based command limiting Flight Envelope Protection (FEP) to enhance safety compared to the currently implemented linear-based FEP. The study compares and evaluates the effectiveness of the updated system under various flight conditions and parametric uncertainties. Results show improved stability margins and a safer FEP. However, additional research is required into actuator saturation and control allocation issues during the approach condition and to enhance fault tolerance.

In the rapidly evolving aviation sector, the quest for safer and more efficient flight operations has historically relied on traditional Automatic Flight Control Systems (AFCS) based on high-fidelity models. However, such models not only incur high development costs but also struggle to adapt to new, complex aircraft designs and unexpected operational conditions. As an alternative, deep Reinforcement Learning (RL) has emerged as a promising solution for model-free, adaptive flight control. Yet, RL-based approaches pose significant challenges in terms of sample efficiency and safety assurance. Addressing these gaps, this paper introduces Returns Uncertainty-Navigated Distributional Soft Actor-Critic (RUN-DSAC). Designed to enhance the learning efficiency, adaptability, and safety of flight control systems, RUN-DSAC leverages the rich uncertainty information inherent in the returns distribution to refine the decision-making process. When applied to the attitude tracking task on a high-fidelity non-linear fixed-wing aircraft model, RUN-DSAC demonstrates superior performance in learning efficiency, adaptability to varied and unforeseen flight scenarios, and robustness in fault tolerance that outperforms the current state-of-the-art SAC and DSAC algorithms.

Digital fly-by-wire (FBW) control technology has had - and continuous to make - a great impact on modern-day aviation. In particular, it enables stability and control augmentation of the dynamic characteristics of the bare airframe and brings opportunities for substantial automation of tasks related to flight. At the heart of FBW control technology are the control laws, which are embedded in a flight control computer. The so-called divide-and-conquer paradigm long prevailed in the development of flight control laws, which is based on trim-and-linearization of the nonlinear dynamics over a wide range of conditions over the flight envelope. This process enables the use of powerful Linear Time-Invariant (LTI) control design and synthesis tools. Gain scheduling is performed in the subsequent nonlinear implementation step. This is a critical task, for which different techniques can be used to arrive at successful designs. However, the process can be intensive and time-consuming, especially in the case of highly nonlinear aircraft.

Nonlinear Dynamic Inversion (NDI) was developed as an alternative to the divide-and-conquer strategy. Instead of subdividing the operating domain into many different local regions, NDI brings the notional benefit of automatic gain scheduling. This drastically simplifies the nonlinear implementation step. Moreover, as it enables a decoupling of different parts of the control design, NDI brings advantages in terms of design modularity. In its classical form, these aspects are achieved through the use of an on-board model embedded in the control law. Alternatively, in an effort to reduce this model-dependency, a sensor-based incremental variant of NDI (INDI) was proposed in the past. This form aims to increase control law robustness in the face of parametric modeling offsets by relying more directly on sensor measurements instead. Accordingly, different inversion strategies (model-based, sensor-based, or combinations of these) lead to vastly different robust stability and performance characteristics. However, a systematic understanding of these robustness implications has long been missing. In this thesis, this problem is approached using H_∞-based multivariable analysis and synthesis techniques.

In addition to the question of robustness, this thesis also focuses on multi-objective control design in the context of control allocation for input-redundant plants. This concerns over-determined control problems, for which secondary performance criteria can be addressed in addition to the primary motion control task. In particular, it is investigated how the framework of incremental control allocation (INCA) can be used in such multi-objective design scenarios.

The critical challenge for employing autonomous control systems in aircraft is ensuring robustness and safety. This study introduces an intelligent and fault-tolerant controller that merges two Reinforcement Learning (RL) algorithms in a hybrid approach: the Distributional Soft Actor-Critic (DSAC) and the Incremental Dual Heuristic Programming (IDHP). The integration combines the strengths of DSAC in learning a robust control strategy and IDHP in allowing real-time control adaption. Compared to earlier controllers, such as a hybrid using the Soft Actor-Critic (SAC) algorithm and strictly offline DSAC and SAC, our hybrid demonstrates enhanced robustness against changing flight conditions and in the face of sensor noise and bias. During fault tolerance tests, it maintains superior control even when the effectiveness of the aircraft’s ailerons and elevators is compromised. By demonstrating the potential of RL-based controllers to provide robustness and fault tolerance, this research advances the feasibility of safe and autonomous flight control operations.

Power output during flight operation of multi-megawatt airborne wind energy systems is substantially affected by the mass of the airborne subsystem, resulting in power fluctuations. In this paper, an approach to control the tether force using the airborne subsystem is presented that improves the quality of the power output. This kite tether force control concept is implemented on the 3DOF dynamic simulation of the MegAWES reference model. First, the winch of MegAWES is resized because an analysis of winch inertia and radius shows its effect on power output and tether force overshoot. Second, the power consuming sections during the traction phase are eliminated by using a feedforward winch controller. Finally, the peak power is substantially reduced by implementing the kite tether force controller which uses a measurement of the tether force, angle of attack, and airspeed to keep the tether force constant when the system is at its power limit. This reduces the range between minimum and maximum power output by 75%.

With the alignment of the planets in the 2030s, a perfect opportunity is presented to explore the outer reaches of the Solar System. With this, studying Uranus would become possible. With it being one of the least studied planets in the Solar System, the demand for accurate scientific data is high. This is where Ouranos comes in. This project has the specific purpose to design a scientific mission to Uranus, where measurements will be taken of its atmosphere. The summary below highlights the important parts and results of this report.

This research presents a comprehensive modeling approach for the flight dynamics of a hybrid compound helicopter, employing classical mechanics methods. The derived non-linear mathematical model encompasses the individual components of the aircraft, including the rotor, propellers, wings, fuselage, and empennage, which are then integrated into a unified dynamics framework through the final Equations of Motion. Notably, the model incorporates a conventional first-order steady flapping motion and quasi-dynamic inflow modeling for both the rotor and propellers. The resulting model represents a complete 9 Degree-of-Freedom system, augmented with 6 mechanical ones and an additional 3 for the inflows of the rotor and the two propellers. As an over-actuated coupled system, it offers 7 available controls; rotor collective pitch, rotor longitudinal and lateral cyclic pitch, port-side and starboard-side propeller pitch, elevator deflection, and rudder deflection. The primary objective of this study is to identify operating trim points during forward flight, ranging from hover to the observed maximum airspeed of 255 knots. Achieving trim through numerical optimization involves a control allocation process utilizing a mapping function that prioritizes the auxiliary propulsion. The research findings demonstrate the successful implementation of a slowed-rotor strategy at high speeds, effectively mitigating compressibility effects, and a linearized mathematical system is derived, providing essential insights for future stability analysis and control system design. These outcomes contribute valuable information toward the advancement of hybrid compound helicopter technology.

This paper investigates the performance of an autonomous navigation system to navigate a spacecraft in the proximity of a binary asteroid system using optical and laser ranging measurements. The knowledge about the binary asteroid is limited to its orbital parameters and ellipsoid shape models. The accelerometer bias random walk is included in the estimation process. Over a four-hour landing maneuver starting from 6770 m altitude and ending at 550 m, the mean position estimation uncertainty is 41.6 m (3σ). It is shown that the navigation accuracy is sensitive to the Sun phase angle, the irregularity of the asteroid shape, and the goodness of fit of the ellipsoid shape model. The paper demonstrates that the navigation system is robust to large errors in the initialization of the extended Kalman filter state. The impact of image distortion and two types of image noise on the navigation performance are investigated.

Control of a helicopter with a deployed dipping sound navigation and ranging (SONAR) is no trivial task due to the complex dynamics of the suspension cable. The cable can change shape, which influences the effect of water, wind and the motion of the helicopter itself. To control such a system, a control method is needed that can cope with these complex dynamics.
In this paper, an investigation is performed on how to model and control a helicopter and a dipping SONAR through the suspension cable using incremental nonlinear dynamic inversion (INDI) as well as a comparison between the use of INDI and linear proportional integral derivative (PID) control. The controller uses only the cable states at the attachment point of the cable to the helicopter in order to determine the control inputs to the helicopter. This means that only these measurements are needed and no complex model of the cable is required for control. This reduces model dependency of the controller.
The control system is simulated in different wind conditions in order to analyse its performance in turbulent conditions, where the cable changes shape constantly. It was found that, although INDI performs better than PID control at low wind speeds, an INDI position hold controller appeared to perform better than an INDI cable controller at high wind speeds.

Recent research in bio-inspired artificial intelligence potentially provides solutions to the challenging problem of designing fault-tolerant and robust flight control systems. The current work proposes SERL, a novel Safety-informed Evolutionary Reinforcement Learning algorithm, which combines Deep Reinforcement Learning (DRL) and neuro-evolutionary mechanisms. This hybrid method optimises a diverse population of non-linear control policies through both evolutionary mechanisms and gradient-based updates. We apply it to solve the attitude tracking task on a high-fidelity non-linear fixed-wing aircraft model. Compared to a state-of-the-art DRL solution, SERL achieves better tracking performance in nine out of ten cases, remaining robust against faults, changes in initial conditions and external disturbances. Furthermore, the work shows how evolutionary mechanisms can balance performance with the smoothness of control actions, a feature relevant for bridging the gap between simulation and deployment on real flight hardware.