Driven by the need to enhance safety, improve efficiency and address labour shortages, autonomous vessel operations are increasingly moving beyond open-water navigation toward more complex missions demanding integration of multiple control strategies. Dock-to-dock operations represent a critical mission encompassing the full spectrum of motion control challenges: long-distance transit, switching between control phases, and precise low-speed maneuvering in confined waters under environmental disturbances.

This thesis evaluates control strategies for autonomous dock-to-dock sailing on inland waterways. A benchmark system using industry standard PID controllers is compared to an all-in-one Reinforcement Learning (RL) controller and a third, hybrid system is proposed trading off the improved performance of the RL controller with the inherent stability guarantees of the benchmark system. Simulation results show all three controllers can successfully perform the mission. The RL controller docks significantly faster while rejecting higher lateral wind forces but struggles to generalise to unseen docking scenarios, while the hybrid system improves interpretability at the cost of performance. Furthermore, initial real-life testing of the benchmark system validates the simulation results.

This thesis presents a simulation of a Martian lander using Reinforcement Learning. The objective is to train an agent to land on Mars using the Proximal Policy Optimization (PPO) method. The spacecraft is controlled by control allocation, where the translations and rotations are controlled independently. Also a PD controller is made to control the same lander.

The PD controller is found to be more accurate in comparison with the Reinforcement learning controller. A reinforcement learning thruster model is also made. This spacecraft is controlled by five thrusters and the motions coupled. This lander needs more control effort, and is less accurate in tracking a reference velocity. It is also less accurate with landing on the designated landing spot.

Advancements in deep reinforcement learning (RL) open the door to the development of robust flight control systems (FCS) that have the potential to improve both safety and performance during off-nominal flight conditions. Simulation-based work on offline-RL FCS has already demonstrated robustness to adverse weather conditions, mechanical failures, and a wide range of operational conditions. However, it has neglected important dynamical phenomena that limit its applicability to reality. In anticipation of a future flight testing campaign of similar RL-based FCS, this research emulates the transition from simulation to reality by modelling prevalent sensor and actuator dynamics, and introduces a method to incorporate a long short-term memory (LSTM) artificial neural network (ANN) into the policy of a Soft Actor-Critic (SAC) agent. The approach is found to largely diminish the sensitivity of the controller to sensor noise and actuator dynamics, while increasing its robustness to delays in comparison with the ubiquitous feed forward deep neural network (DNN) and a traditional linear controller.

Sample efficiency is a critical metric in intelligent control systems as it directly influences the feasibility and effectiveness of learning-based approaches. This paper presents the study of how Randomized Ensemble Double Q-Learning (REDQ), a sample-efficient model free algorithm, can be used in flight control applications. Three controllers were developed for: pitch, roll and combined biaxial attitude tracking tasks and tested on a high fidelity Cessna Citation 550 model. For each control task, three agents were trained offline: two using REDQ enhanced Soft Actor Critic (SAC) architectures and one using a standard SAC architecture for comparison. REDQ agents showed statistically significant improvements in sample efficiency during initial learning. Average accurate tracking convergence (error < 1◦) occurred within 5,500 training steps for pitch, 6,400 for roll and 11,500 for biaxial control. The gains in sample efficiency were shown to have drawbacks in learning stability and robustness when deviated too far from nominal conditions.

This study explored the application of particle subswarm optimisation - a non-gradient-based swarming intelligence method - in neuroevolution for solving the powered descent landing problem. It was compared to a gradient-based Soft Actor-Critic (SAC) reinforcement learning algorithm under sparse and non-sparse reward settings. A simulation environment was developed incorporating aerodynamic modelling, grid fins, thrust vector control, and variable inertia. Results show that a Soft Actor Critic struggled with non-sparse rewards due to its inability to generalise across the long-horizon powered descent task due to non-informative gradients from high-variance state-action values. In contrast, learning with the sparse rewards became trapped in a local optimum due to a weak reward signal. Particle subswarm optimisation achieved rapid convergence to feasible solutions and reduced propellant consumption, outperforming SAC due to an episodic gated reward structure, local minima avoidance, and gradient independence. This study establishes particle subswarm optimisation neuroevolution as a viable alternative to reinforcement learning for the assignment of throttle during powered descent of a launch vehicle's first stage.

Demand for fully autonomous VTOL aircraft with high agility has exposed the shortcomings of Euler‐angle models: limiting factors such as the presence of singularities and the need for expensive trigonometric operations impact the reliability and safety of autonomous systems and impair the execution of highly-aggressive manoeuvres. Quaternions offer a singularity‐free alternative, yet their use on full‐scale helicopters remains underexplored. Addressing this gap in research, this paper introduces a methodology for converting existing Euler‐based linear models of a Bo105 helicopter to quaternion form, preserving model fidelity. A hierarchical flight control system is then designed, controlling attitude with a novel implementation of Linear Quadratic Integral (LQI) control with quaternions, and commanding velocity and position using PI and P controllers. The controller is tuned using a structured approach implementing the Particle Swarm Optimization (PSO) algorithm. The system is then augmented with a Finite State Machine (FSM) autopilot to follow offline-generated trajectories. The implemented system was evaluated by simulating a slalom and a pop-up manoeuvre: the simulations demonstrated maximum tracking errors of 5 m for slalom and 7 m for pop‐up, primarily due to velocity‐loop delays, while achieving zero steady‐state error in the position. These results confirm the effectiveness of quaternion‐based modelling and control for agile, autonomous VTOL operations.

Incremental Dual Heuristic Programming (IDHP) is a successor to the Dual Heuristic Programming (DHP) algorithm that uses an online identified incremental system model, this algorithm showed promising flight control performance and tolerance of faults in simulation experiments. This paper studies the potential for extending IDHP through augmenting the computation of agent updates and returns, more specifically, by using eligibility trace updates and multi-step temporal difference error. This results in the IDHP(𝜆), MIDHP, and MIDHP(𝜆) algorithms, which are compared against IDHP in several simulated flight control scenarios with faults introduced mid-flight. The results demonstrate that the proposed algorithms have improved flight control performance and fault tolerance in terms of tracking errors when controlling a nominal aircraft and an aircraft with faults introduced, with the most improvement observed in MIDHP(𝜆)

Exploring planetary bodies using robot swarms can potentially increase the value of the exploration missions; enabling the execution of novel measurements and explorations previously deemed impractical or unattainable. Despite its potential, the technology readiness level of planetary swarms is not very mature. This work uses multi-agent reinforcement learning to find control policies that allow swarms to autonomously explore unknown areas in a decentralized manner, contributing towards the technology readiness of the field. A multi-agent proximal policy optimization (MAPPO) algorithm is proposed for this end, where the policy uses LIDAR perception information, and the input of the value function contains local and global environment information. The algorithm finds control policies that achieve cooperation behaviors and generalize to unseen swarm sizes and environments learning with simple, sparse reward functions. Moreover, different types of reward functions, value inputs, and environment configurations are investigated. Compared with the state-of-the-art in the field, MAPPO can learn with a larger number of agents, more complicated environments, and using sparse rewards instead of dense ones.

Reinforcement Learning applied to flight control has shown to have several benefits over classical, linear flight controllers, as it eliminates the need for gain scheduling and it could provide fault-tolerance. The application to civil aviation in practice, however, is non-existent as there are multiple safety concerns. This research demonstrates the evaluation of longitudinal Handling Qualities of the Soft Actor-Critic Deep Reinforcement Learning framework with the aim to translate the unpredictable black box of Reinforcement Learning into classical flight control terminology. The framework is applied to a pitch rate command system of a jet aircraft and shows robustness to off-nominal flight conditions, center of gravity shifts and biased sensor noise. Accurate tracking performance is achieved, while adhering to Level 1 longitudinal Handling Qualities for all conditions.

The Flying-V is a tailless, V-shaped flying-wing type aircraft that promises to offer significant increases in aerodynamic efficiency. Due to its configuration, the Flying-V faces some control and stability related issues. These include limited control authority, pitch break tendencies and non-ideal handling qualities. To enhance the handling qualities of the Flying-V, Incremental Nonlinear Dynamic Inversion (INDI)-based flight control systems have been proposed. INDI, a sensor-based alternative to conventional Nonlinear Dynamic Inversion (NDI), is rooted in the principle of feedback linearization. Unlike NDI, INDI does not depend heavily on accurate on-board models (OBM), thereby offering increased robustness to aerodynamic uncertainties. However, singular perturbations—such as time delays, aeroelastic effects, and additional unmodeled or unknown dynamics—have been identified as challenges for INDI-based control laws. Various strategies have been explored to improve the overall robustness of INDI-based flight control systems, including outer-loop tuning and inversion loop augmentation strategies. In this research a multi-loop µ-optimal approach for designing robust inversion-based flight control laws is explored for the design of an explicit model-following pitch-rate control system for a short-period approximation of the Flying-V’s longitudinal dynamics. The design problem takes into account both regular and singular perturbations. To assess the robust stability and performance of the proposed control systems, a structured singular value analysis was performed. It was concluded that a multi-loop synthesis approach is capable of achieving better robust stability and performance levels when compared to either strictly inner-loop or outer-loop synthesis. As such, it can be concluded that multi-loop synthesis approaches are best capable of leveraging the robustness functionalities of multi-loop inversion-based control systems.

The Flying-V aircraft could revolutionize commercial aviation, boasting a potential 25% increase in aerodynamic efficiency. Due to inherent design limitations regarding static stability, the need for a proper Flight Control System (FCS) is essential for the development of the aircraft. The concept of Hybrid Incremental Nonlinear Dynamic Inversion (INDI) was introduced to mitigate the insufficient stability margin encountered in existing sensor-based INDI systems due to sensor time delays, to achieve Level 1 Handling Qualities (HQ). Furthermore, the research introduces an exponential potential function-based command limiting Flight Envelope Protection (FEP) to enhance safety compared to the currently implemented linear-based FEP. The study compares and evaluates the effectiveness of the updated system under various flight conditions and parametric uncertainties. Results show improved stability margins and a safer FEP. However, additional research is required into actuator saturation and control allocation issues during the approach condition and to enhance fault tolerance.

In the rapidly evolving aviation sector, the quest for safer and more efficient flight operations has historically relied on traditional Automatic Flight Control Systems (AFCS) based on high-fidelity models. However, such models not only incur high development costs but also struggle to adapt to new, complex aircraft designs and unexpected operational conditions. As an alternative, deep Reinforcement Learning (RL) has emerged as a promising solution for model-free, adaptive flight control. Yet, RL-based approaches pose significant challenges in terms of sample efficiency and safety assurance. Addressing these gaps, this paper introduces Returns Uncertainty-Navigated Distributional Soft Actor-Critic (RUN-DSAC). Designed to enhance the learning efficiency, adaptability, and safety of flight control systems, RUN-DSAC leverages the rich uncertainty information inherent in the returns distribution to refine the decision-making process. When applied to the attitude tracking task on a high-fidelity non-linear fixed-wing aircraft model, RUN-DSAC demonstrates superior performance in learning efficiency, adaptability to varied and unforeseen flight scenarios, and robustness in fault tolerance that outperforms the current state-of-the-art SAC and DSAC algorithms.

Digital fly-by-wire (FBW) control technology has had - and continuous to make - a great impact on modern-day aviation. In particular, it enables stability and control augmentation of the dynamic characteristics of the bare airframe and brings opportunities for substantial automation of tasks related to flight. At the heart of FBW control technology are the control laws, which are embedded in a flight control computer. The so-called divide-and-conquer paradigm long prevailed in the development of flight control laws, which is based on trim-and-linearization of the nonlinear dynamics over a wide range of conditions over the flight envelope. This process enables the use of powerful Linear Time-Invariant (LTI) control design and synthesis tools. Gain scheduling is performed in the subsequent nonlinear implementation step. This is a critical task, for which different techniques can be used to arrive at successful designs. However, the process can be intensive and time-consuming, especially in the case of highly nonlinear aircraft.

Nonlinear Dynamic Inversion (NDI) was developed as an alternative to the divide-and-conquer strategy. Instead of subdividing the operating domain into many different local regions, NDI brings the notional benefit of automatic gain scheduling. This drastically simplifies the nonlinear implementation step. Moreover, as it enables a decoupling of different parts of the control design, NDI brings advantages in terms of design modularity. In its classical form, these aspects are achieved through the use of an on-board model embedded in the control law. Alternatively, in an effort to reduce this model-dependency, a sensor-based incremental variant of NDI (INDI) was proposed in the past. This form aims to increase control law robustness in the face of parametric modeling offsets by relying more directly on sensor measurements instead. Accordingly, different inversion strategies (model-based, sensor-based, or combinations of these) lead to vastly different robust stability and performance characteristics. However, a systematic understanding of these robustness implications has long been missing. In this thesis, this problem is approached using H_∞-based multivariable analysis and synthesis techniques.

In addition to the question of robustness, this thesis also focuses on multi-objective control design in the context of control allocation for input-redundant plants. This concerns over-determined control problems, for which secondary performance criteria can be addressed in addition to the primary motion control task. In particular, it is investigated how the framework of incremental control allocation (INCA) can be used in such multi-objective design scenarios.

The critical challenge for employing autonomous control systems in aircraft is ensuring robustness and safety. This study introduces an intelligent and fault-tolerant controller that merges two Reinforcement Learning (RL) algorithms in a hybrid approach: the Distributional Soft Actor-Critic (DSAC) and the Incremental Dual Heuristic Programming (IDHP). The integration combines the strengths of DSAC in learning a robust control strategy and IDHP in allowing real-time control adaption. Compared to earlier controllers, such as a hybrid using the Soft Actor-Critic (SAC) algorithm and strictly offline DSAC and SAC, our hybrid demonstrates enhanced robustness against changing flight conditions and in the face of sensor noise and bias. During fault tolerance tests, it maintains superior control even when the effectiveness of the aircraft’s ailerons and elevators is compromised. By demonstrating the potential of RL-based controllers to provide robustness and fault tolerance, this research advances the feasibility of safe and autonomous flight control operations.

Power output during flight operation of multi-megawatt airborne wind energy systems is substantially affected by the mass of the airborne subsystem, resulting in power fluctuations. In this paper, an approach to control the tether force using the airborne subsystem is presented that improves the quality of the power output. This kite tether force control concept is implemented on the 3DOF dynamic simulation of the MegAWES reference model. First, the winch of MegAWES is resized because an analysis of winch inertia and radius shows its effect on power output and tether force overshoot. Second, the power consuming sections during the traction phase are eliminated by using a feedforward winch controller. Finally, the peak power is substantially reduced by implementing the kite tether force controller which uses a measurement of the tether force, angle of attack, and airspeed to keep the tether force constant when the system is at its power limit. This reduces the range between minimum and maximum power output by 75%.

With the alignment of the planets in the 2030s, a perfect opportunity is presented to explore the outer reaches of the Solar System. With this, studying Uranus would become possible. With it being one of the least studied planets in the Solar System, the demand for accurate scientific data is high. This is where Ouranos comes in. This project has the specific purpose to design a scientific mission to Uranus, where measurements will be taken of its atmosphere. The summary below highlights the important parts and results of this report.

This research presents a comprehensive modeling approach for the flight dynamics of a hybrid compound helicopter, employing classical mechanics methods. The derived non-linear mathematical model encompasses the individual components of the aircraft, including the rotor, propellers, wings, fuselage, and empennage, which are then integrated into a unified dynamics framework through the final Equations of Motion. Notably, the model incorporates a conventional first-order steady flapping motion and quasi-dynamic inflow modeling for both the rotor and propellers. The resulting model represents a complete 9 Degree-of-Freedom system, augmented with 6 mechanical ones and an additional 3 for the inflows of the rotor and the two propellers. As an over-actuated coupled system, it offers 7 available controls; rotor collective pitch, rotor longitudinal and lateral cyclic pitch, port-side and starboard-side propeller pitch, elevator deflection, and rudder deflection. The primary objective of this study is to identify operating trim points during forward flight, ranging from hover to the observed maximum airspeed of 255 knots. Achieving trim through numerical optimization involves a control allocation process utilizing a mapping function that prioritizes the auxiliary propulsion. The research findings demonstrate the successful implementation of a slowed-rotor strategy at high speeds, effectively mitigating compressibility effects, and a linearized mathematical system is derived, providing essential insights for future stability analysis and control system design. These outcomes contribute valuable information toward the advancement of hybrid compound helicopter technology.

This paper investigates the performance of an autonomous navigation system to navigate a spacecraft in the proximity of a binary asteroid system using optical and laser ranging measurements. The knowledge about the binary asteroid is limited to its orbital parameters and ellipsoid shape models. The accelerometer bias random walk is included in the estimation process. Over a four-hour landing maneuver starting from 6770 m altitude and ending at 550 m, the mean position estimation uncertainty is 41.6 m (3σ). It is shown that the navigation accuracy is sensitive to the Sun phase angle, the irregularity of the asteroid shape, and the goodness of fit of the ellipsoid shape model. The paper demonstrates that the navigation system is robust to large errors in the initialization of the extended Kalman filter state. The impact of image distortion and two types of image noise on the navigation performance are investigated.