Current investigations into urban aerial mobility, as well as the continuing growth of global air transportation, have renewed interest in conflict detection and resolution (CD&R) methods. The use of drones for applications such as package delivery, would result in traffic densities that are orders of magnitude higher than those currently observed in manned aviation. Such densities do not only make automated conflict detection and resolution a necessity, but will also force a re-evaluation of aspects such as coordination vs. priority, or state vs. intent. This paper looks into enabling a safe introduction of drones into urban airspace by setting travelling rules in the operating airspace which benefit tactical conflict resolution. First, conflicts resulting from changes of direction are added to conflict resolution with intent trajectory propagation. Second, the likelihood of aircraft with opposing headings meeting in conflict is reduced by separating traffic into different layers per heading–altitude rules. Guidelines are set in place to make sure aircraft respect the heading ranges allowed at every crossed layer. Finally, we use a reinforcement learning agent to implement variable speed limits towards creating a more homogeneous traffic situation between cruising and climbing/descending aircraft. The effects of all of these variables were tested through fast-time simulations on an open source airspace simulation platform. Results showed that we were able to improve the operational safety of several scenarios@en

Current predictions on future drone operations estimate that traffic density orders of magnitude will be higher than any observed in manned aviation. Such densities redirect the focus towards elements that can decrease conflict rate and severity, with special emphasis on airspace structures, an element that has been overlooked within distributed environments in the past. This work delves into the impacts of different airspace structures in multiple traffic scenarios, and how appropriate structures can increase the safety of future drone operations in urban airspace. First, reinforcement learning was used to define optimal heading range distributions with a layered airspace concept. Second, transition layers were reserved to facilitate the vertical deviation between cruising layers and conflict avoidance. The effects of traffic density, non-linear routes, and vertical deviation between layers were tested in an open-source airspace simulation platform. Results show that optimal structuring catered to the current traffic scenario improves airspace usage by correctly segmenting aircraft according to their flight routes. The number of conflicts and losses of minimum separation was reduced versus using a single, uniform airspace structure for all traffic scenarios, thus enabling higher airspace capacity.@en

Current predictions for future operations with drones estimate traffic densities orders of magnitude higher than any observed in manned aviation. Such densities call for further research and innovation, in particular, into conflict detection and resolution without the need for human intervention. The layered airspace concept, where aircraft are separated per vertical layer according to their heading, has been widely researched and proven to increase traffic capacity. However, aircraft traversing between layers do not benefit from this separation and alignment effect. As a result, interactions between climbing/descending and cruising aircraft can lead to a large increase in conflicts and intrusions. This paper looks into ways of reducing the impact of vertical transitions within the environment. We test two reinforcement learning methods: a decision-making module and a control execution module. The former issues a lane change command based on the planned route. The latter performs operational control to coordinate the longitude and vertical movement of the aircraft for a safe merging manoeuvre. The results show that reinforcement learning is capable of optimising an efficient driving policy for layer change manoeuvres, decreasing the number of conflicts and losses of minimum separation compared to manually defined navigation rules.@en

Future operations involving drones are expected to result in traffic densities that are orders of magnitude higher than any observed in manned aviation. Current geometric conflict resolution (CR) methods have proven to be very efficient at relatively moderate densities. However, at higher densities, performance is hindered by the unpredictable emergent behaviour from neighbouring aircraft. Reinforcement learning (RL) techniques are often capable of identifying emerging patterns through training in the environment. Although some work has started introducing RL to resolve conflicts and ensure separation between aircraft, it is not clear how to employ these methods with a higher number of aircraft, and whether these can compare to or even surpass the performance of current CR geometric methods. In this work, we employ an RL method for distributed conflict resolution; the method is completely responsible for guaranteeing minimum separation of all aircraft during operation. Two different action formulations are tested: (1) where the RL method controls heading, and speed variation; (2) where the RL method controls heading, speed, and altitude variation. The final safety values are directly compared to a state-of-the-art distributed CR algorithm, the Modified Voltage Potential (MVP) method. Although, overall, the RL method is not as efficient as MVP in reducing the total number of losses of minimum separation, its actions help identify favourable patterns to avoid conflicts. The RL method has a more preventive behaviour, defending in advance against nearby neighbouring aircraft not yet in conflict, and head-on conflicts while intruders are still far away.@en

In the coming years, aviation will have to cope with an increase in the number of aircraft, and of drones purchased in the open market. The human intervention of Air Traffic Management (ATM) will be overburden with guaranteeing flight safety with hundreds of manned aircraft and Unmanned Aerial Vehicles (UAVs) operating simultaneously. Conflict Resolution (CR) models are under research, aiming at relieving the workload of ATM services and to enable UAVs to fly safely in the civil airspace. However, these are tested using different simulation tools and scenarios, making it impossible for a direct comparison. This paper compared the performance of commonly used CR methods under the same conditions both for manned and unmanned aviation. Disparities with previously conducted research using different scenarios show the importance of creating a standardized simulation library. Additionally, under the traffic scenarios considered, velocity obstacles (VO) based methods obtained a better performance safety-wise. @en

Increasing delays and congestion reported in many aviation sectors indicate that the current centralised operational model is rapidly approaching saturation levels. Air Traffic Control (ATC) system is not expected to keep pace with the ever-increasing demand for air transportation. Its capacity is still limited by the available controllers, and the number of aircraft that each controller can manage. This system cannot be stretched any further under its current conditions. However, it is expected that the number of aircraft operating simultaneously will continue to increase. On top of this, new unmanned aviation operations promise traffic densities never seen before. The expected future increase in traffic demand has redirected focus into automated tools and alternative approaches. This research has been primarily characterised by a change in the degree of centralisation, more specifically by exploring distributed options, where control is transferred from ground-based Air Traffic Controllers (ATCOs) to each individual aircraft. As each aircraft only takes into account its neighbouring aircraft when resolving conflicts, each distributed resolution system is expected to have only a fraction of the computational strain that a centralised system would have. Nevertheless, a distributed approach has its own challenges. A crucial disadvantage is the lack of global coordination from surrounding traffic, which can affect safety. Without knowledge of the movement of intruders, decentralised solutions cannot guarantee globally optimal solutions when more than two aircraft are involved. Conflict resolution (CR) methods based on geometric solutions have proven to be very successful in achieving a high level of safety for one-to-one conflicts, where a set of rules can be defined which leads to implicitly coordinated optimal behaviour. However, at higher traffic densities, when individual conflict situations can no longer be considered isolated events, successive CR manoeuvres can lead to traffic patterns with a negative effect on the global safety. Knock-on effects of intruders avoiding each othermay result in unforeseen trajectory changes. The latter increases uncertainty regarding intruders’ future movements, decreasing the efficacy of conflict resolution manoeuvres. The goal of this research is to improve upon aircraft self-separation efficacy at higher traffic densities, with an emphasis on employing airspace designs and approaches applicable to future unmanned operations. To do so, we look at a scenario with multi aircraft interacting as a multi-agent problem. Analysis and understanding of emergent behaviour in a multi-agent environment is often almost impossible to the human eye. However, reinforcement learning (RL) techniques are often capable of identifying emerging patterns through training in the environment. We translate successful applications of RL techniques in other areas (e.g., carmobility, lane changing, freeways) to aircraft operational scenarios to mitigate the negative effect on safety...@en

The number of unmanned aircraft operating in the airspace is expected to grow exponentially during the next decades. This will likely lead to traffic densities that are higher than those currently observed in civil and general aviation, and might require both a different airspace structure compared to conventional aviation, as well as different conflict resolution methods. One of the main disadvantages of analytical conflict resolution methods, in high-traffic density scenarios, is that they can cause instabilities of the airspace due to a domino effect of secondary conflicts. Therefore, many studies have also investigated other methods of conflict resolution, such as Deep Reinforcement Learning, which have shown positive results, but tend to be hard to explain due to their black-box nature. This paper investigates if it is possible to explain the behaviour of a Soft Actor-Critic model, trained for resolving vertical conflicts in a layered urban airspace, by interpreting the policy through a heat map of the selected actions. It was found that the model actively changes its policy depending on the degrees of freedom and has a tendency to adopt preventive behaviour on top of conflict resolution. This behaviour can be directly linked to a decrease in secondary conflicts when compared to analytical methods and can potentially be incorporated into these methods to improve them while maintaining explainability.@en

The use of drones for applications such as package delivery, in an urban setting, would result in traffic densities that are orders of magnitude higher than any observed in manned aviation. Such densities not only make automated conflict detection and resolution a necessity, but it will also force a reevaluation of aspects such as centralised vs. distributed, coordination vs. priority, or state vs. intent. This paper investigates the use of intent in tactical conflict detection and resolution at high traffic densities in unmanned aviation. Experimental results show that combining both current state and future intent information improved overall safety. Adding intent enables the detection, in advance, of conflicts resulting from future changes of state. A conflict resolution maneuver is optimal for safety when all aircraft deviate only minimally from their current state to solve the conflict. Consequently, they could deviate from the broadcast intent information. Therefore, state projection into the future must still be kept to prevent very short-term conflicts when intruders do not follow their original intent. @en

With the rise of new and innovative Urban Air Mobility solutions, there also arises a need to integrate these into the existing airspace. Current airspace users include conventional civil, commercial and general aviation, military air users, police and emergency services as well as a plethora of avian life. Planned additions to the airspace are electric vertical take-off and landing vehicles such as logistics drones and air taxis. The airspace for conventional users is stringently controlled. Urban Air Mobility operations are expected to mainly take place in individual corridors to be added to the currently uncontrolled low-level airspace. This airspace is also intended for various types of drone operations, out of which, small-scale drones can be non-co-operative. In addition, the operational altitudes of Urban Air Mobility aircraft will strongly expose them to birds. Due to abundance of these non-cooperating airspace users (like hobby-drones and birds), conflicts with Urban Air Mobility aircraft are expected to be inevitable. The aim of this paper is to develop a concept of Urban Air Mobility Collision Avoidance System to reduce the likelihood of collision between air taxis and non-cooperating airspace users. As such, this work proposes the introduction of an additional safety layer to prevent collisions during operations of strong exposure. The concept consists of a conflict detection and resolution method tailored for Urban Air Mobility operations. A three-dimensional safety envelope is designed using the geometric and performance values of the aircraft configurations currently available. Procedures to avoid conflicts prior to as well as during the flights are presented. Finally, the concept is visualized for the common use case of a shuttle service between an airport and a railway station. The results demonstrate the importance of incorporating individual aircraft configuration into conflict avoidance approach and report its effect to avoid collision.@en

Air traffic demand has increased at an unprecedented rate in the last decade (albeit interrupted by the COVID pandemic), but capacity has not increased at the same rate. Higher levels of automation and the implementation of decision-support tools for air traffic controllers could help increase capacity and catch up with demand. The air traffic control problem can be effectively modelled as a Markov game, where a team of aircraft (the agents) interact in the airspace (the environment) and cooperatively take resolution actions to achieve a common goal: safe separation in the most efficient way. As in any Markov game, the optimal policy for the team could be learnt through trial and error in a simulated environment using reinforcement learning algorithms. In this paper, we use the soft actor-critic algorithm to unravel the optimal air traffic control policy. Unlike some previous works, we propose a global (i.e., shared) reward that encourages cooperative behaviour. Furthermore, we propose a versatile policy model capable of performing heading, speed, and/or altitude resolution actions. We also demonstrate that the policy is robust and can maintain safe separation even in the presence of uncertainty regarding aircraft position, delays in implementing resolution actions, and wind. The findings of this paper also suggest that there is still significant room for improvement when controlling three degrees of freedom at the same time.@en

Road traffic delay and urban overcrowding are increasing rapidly all over the world. As a result, several companies have proposed the use of small unmanned aerial vehicles (sUAVs) as an alternative to road-based transportation. These small autonomous drones are expected to operate within a thin airspace band (Very Low Level) in high traffic densities in constrained urban environments. This presents a challenge for ensuring the safe separation and efficient routing of drone flights. Current research has made modest progress towards finding solutions for conflict detection and prevention in highly dense and constrained environments (e.g., in-between buildings). In this paper, the state of the art of urban airspace design and conflict prevention and resolution research are discussed, and their applications to constrained environments. Additionally, fasttime high-fidelity simulations of high-density traffic scenarios are used along a non-orthogonal city layout to identify bottlenecks in the performance of speed-based conflict resolution in a multilayered airspace structure. Results show that the current airspace structure and conflict detection and resolution concepts need to be refined to further reduce conflicts and intrusions that occur in constrained environments. First, additional measures must be adapted to further prevent conflicts during turning and merging. Second, conflict resolution manoeuvres must account for speed limits resulting in turn radii which do not cross physical boundaries. Finally, conflict detection needs to consider the topology of the streets to prevent false-positive conflicts and to prepare in advance for conflicts resulting from heading changes in non-linear streets. @en

The capacity of the current system of air traffic is rapidly reaching a limit with the increasing demand for air transportation. Expected future traffic densities not only make automated conflict detection and resolution a necessity, but also force a re-evaluation of coordination elements to decrease conflict rate and severity. It has been acknowledged that airspace structure plays a positive role by acting as a first layer of conflict protection, reducing the likelihood of aircraft meeting and, consequently, the likelihood of conflicts and losses of minimum separation. In the recent past, different airspace structures have been explored. Research shows that the layered airspace concept, where groups of aircraft with similar headings remain separated by cruising at different altitudes, increases airspace capacity. However, implementation of this concept often employs an evenly distributed heading range per vertical layer, which is not optimal for all traffic scenarios, since it may lead to unevenly distributed numbers of aircraft per layer. In this work, we use supervised learning to determine a heading range distribution per layer adapted to the current traffic. This method resulted in a reduction of both conflicts and losses of minimum separation when compared to an evenly distributed layers concept. Results show that conflicts prevention, with a structure which efficiently segments aircraft through the airspace, may have a greater impact on safety than applying conflict resolution methods. @en

Current estimates show that the presence of unmanned aviation is likely to grow exponentially over the course of the next decades. Even with the more conservative estimates, these expected high traffic densities require a re-evaluation of the airspace structure to ensure safe and efficient operations. One structure that scored high on both the safety and efficiency metrics, as defined by the Metropolis project, is a layered airspace, where aircraft with an intended heading are assigned to a specific altitude layer. However, a problem arises once aircraft start to vertically traverse between these layers, leading to a large number of conflicts and intrusions. One way to potentially reduce the number of intrusions during these operations is by using conventional conflict resolution algorithms. These algorithms however have also been shown to lead to instabilities at higher traffic densities. As recent years have shown tremendous growth in the capabilities of Deep Reinforcement Learning, it is interesting to see how well these methods perform in the field of conflict resolution. This research investigates and compares the performance of multiple Soft Actor Critic models with the Modified Voltage Potential algorithm during vertical manoeuvres in a layered airspace. The final obtained performance of the trained models is comparable to that of the Modified Voltage Potential algorithm and in certain scenarios, the trained models even outperform the MVP algorithm. Overall, the results show that DRL can improve upon the current state of conflict resolution algorithms and provide new insight into the development of safe operations.@en

The use of drones for applications such as package delivery, in an urban setting, would result in traffic densities that are orders of magnitude higher than any observed in manned aviation. Current geometric resolution models have proven to be very efficient at relatively moderate densities. However, at higher densities, performance is hindered by the unpredictable emergent behaviour from neighbouring aircraft. In this paper, we use a hybrid solution between existing geometric resolution approaches and reinforcement learning (RL), directed at improving conflict resolution performance at high densities. We resort to a Deep Deterministic Policy Gradient (DDPG) model to improve the behaviour of the Modified Voltage Potential (MVP) geometric conflict resolution method. By default, the MVP method generates avoidance manoeuvres of a geometrically-defined type, using a fixed look-ahead time. In the current study, we instead aim to use RL to determine the values for these variables, based on intruder position and traffic density. The analysis in this paper specifically addresses the difficulty of training algorithms in a cooperative multi-agent case to converge to optimal values. We prove that finding the right representation of state/rewards in a nonstationary environment is non-trivial and highly influences the learning process. Finally, we show that a variation of resolution manoeuvres can improve the safety of several scenarios at high traffic densities. @en

The use of drones for applications such as package delivery, in an urban setting, would result in traffic densities that are orders of magnitude higher than any observed in manned aviation. Current geometric resolution models have proven to be very efficient. However, at the extreme densities envisioned for such drone applications, performance is hindered by unpredictable emergent behaviour of interacting traffic. This paper describes a study that intends to investigate how reinforcement learning techniques can be used to complement geometric methods, thus improving conflict detection and resolution at high traffic densities. Different hybrid approaches are discussed, and preliminary results are shown for a hybrid model that uses geometric methods in the training phase of a Deep Deterministic Policy Gradient (DDPG) model. @en