Multi-agent robotic systems could benefit from reinforcement learning algorithms that are able to learn behaviours in a small number trials, a property known as sample efficiency. This research investigates the use of learned world models to create more sample-efficient algorithms. We present a novel multi-agent model-based reinforcement learning algorithm: Multi-Agent Model-Based Policy Optimization (MAMBPO), utilizing the Centralized Learning for Decentralized Execution (CLDE) framework, and demonstrate state-of-the-art performance in terms of sample efficiency on a number of benchmark domains. CLDE algorithms allow a group of agents to act in a fully decentralized manner after training. This is a desirable property for many systems comprising of multiple robots. Current CLDE algorithms such as Multi-Agent Soft Actor-Critic (MASAC) suffer from limited sample efficiency, often taking many thousands of trials before learning desirable behaviours. This makes these algorithms impractical for learning in real-world robotic tasks. MAMBPO utilizes a learned world model to improve sample efficiency compared to its model-free counterparts. We demonstrate on two simulated multi-agent robotics tasks that MAMBPO is able to reach similar performance to MASAC with up to 3.7 times fewer samples required for learning. Doing this, we take an important step towards making real-life learning for multi-agent robotic systems possible. ... Read more

This report provides an overview of the considerations and decisionsmade during the DSE project from the Faculty of Aerospace Engineering, arriving at the final design of the SolidityONE. The goal was to design a vertical take-off and landing vehicle according to the rules from the 37th annual student design competition by the Vertical Flight Society. This design proves the concept of a rotor with disk solidity equal to or larger than 1.0. Additionally, benefits this design has over existing rotorcraft mean it can be tailored to meet the needs of a specific market... ... Read more

With the growing population the agricultural industry needs to find and implement new methods for enhancing food production. Using a Micro Aerial Vehicle (MAV) in Precision Agriculture (PA) offers a large number of benefits such as enabling the farmer to create targeted strategies to increase crop yield, reduced waste and halt the spread of diseases. Despite these advantages, the use of MAVs, particularly in greenhouses, is still very limited. To this end, this thesis seeks to combine, improve and implement existing strategies to solve the persistent surveillance task for a swarm of MAVs operating in a greenhouse environment.
Broadly speaking, the persistent surveillance task seeks to find the optimal paths for a swarm of MAVs such that every point within the Mission Space (MS) is visited and they must minimise the time between successive visits. This will ensure that the MAVs are able fly through the entire greenhouse to collect up-to-date data about all the crops and the local environment. Naturally, on a physical system one has to deal with the limited flight times of the MAVs. This factor becomes very important to the effectiveness of the solution and is critical to the continuous operation of the MAVs.
In literature, many methods have be proposed to solve this task, but the majority are still only tested in simulation. As a result, many works do not consider some physical constraints that will be applied to the system during implementation in a real-world setting. For example, in most cases the authors do not consider the limited fuel available to the agents or they do not consider a practical alternative indoor positioning system to GPS. In this work the problem has been divided into two main sub-tasks, namely; the persistent surveillance task and the refuelling task.
For the persistent surveillance task it was decided to implement a reactive controller, in the form of an evolved Neural Network (NN), which was run on-board the MAVs. The NN used positional information from the other members of the swarm along with limited environmental information to supply its MAV with a command velocity. These NN controllers could achieve coverage levels of over 95% while simultaneously avoiding collisions between 8 MAVs in a 25m x 25m MS. Later, this method was shown to be robust to failures and scalable in terms of both MS and swarm size.
When dealing with the fuel constraints, a Behaviour Tree (BT) was used to determine when the MAV should return to the depot. Surprisingly, when combined with the NN controllers the system experienced an increase in performance across all the defined metrics. No MAV failed due to low fuel levels, coverage increased to 97.41%, average cell age to 52.39s and the number of tests were no collisions were recorded more than doubled. This increase in performance was attributed to the fact that the refuelling periodically drew the MAV towards the centre of the MS. This is counter to the evolved behaviours of the NN where the MAVs would mainly focus their attention around the edges of the MS.
... Read more

The first launch of the Ariane 6 launch vehicle is planned for 2020, however in its current design no significant part of the launcher will be reusable. A current trend in the global space market is decreasing the costs of spacecraft launches through recovery, retrieval and refurbishment of parts of launchers. As a first step towards this market demand, it is to be investigated whether it is cost-effective to recover, refurbish and reuse the key components of the first stage of the Ariane 6, which are contained in the Vulcain Aft Bay (VuAB). This is where the engine, fuel lines, thrust frame and electronics are attached. The team has the task to develop a cost effective way of reusing the Vulcain Aft Bay. In the preceding report, multiple concepts were analysed and one concept was selected to complete the conceptual design. This concept consists of an Inflatable Heat Shield for re-entry, a Parafoil to control the flight at lower altitudes and a Mid-Air retrieval using a helicopter to perform a soft landing. A functional analysis was performed to define concept specific functions. This was done by means of a Functional Flow Diagram and Functional Breakdown Structure. In order to fulfil these functions, simulations were created of the most critical moments of the mission. One set of simulations analyse the trajectory of the system throughout the mission, predicting the location of landing. The other set of simulations is used to predict the critical load cases of the system. The aforementioned simulations were used to design the individual components of the system. By integrating these simulations and managing the iteration process, the overall system characteristics and configuration were established. The system was then analysed for sustainability, reliability, risk, maintainability and safety. The requirement compliance of the system was then updated, detailing which requirements have achieved full compliance, marginal compliance, no compliance and which have not been sufficiently investigated. Proposals to make all requirements fully compliant are included in a feasibility analysis. These include different design approaches for the team and design changes to the VuAB to accommodate the recovery system. From there strategies on future verification and validation activities was set forth together with operational, refurbishment and production plans. These plans and the design of the system were used to create an updated business model with return on investment figures, which predict a significant cost reduction on a per launch basis within five years. Finally a set of future recommendations and plan is proposed for the continuation of the project. ... Read more

The Velo-E-Raptor Design Synthesis Exercise project has the following mission: "Design an electrically assisted, human powered and sustainable method of flight that is accessible and safe for entertainment and sports usage". After the "Bird" concept was chosen in the midterm-report, detailed design of this concept was performed... ... Read more