Novel wind technologies, in particular airborne wind energy (AWE) and floating offshore wind turbines, have the potential to unlock untapped wind resources and contribute to power system stability in unique ways. So far, the techno-economic potential of both technologies has only been investigated at a small scale, whereas the most significant benefits will likely play out on a system scale. Given the urgency of the energy transition, the possible contribution of these novel technologies should be addressed. Therefore, we investigate the main system-level trade-offs in integrating AWE systems and floating wind turbines into a highly renewable future energy system. To do so, we develop a modelling workflow that integrates wind resource assessment and future cost and performance estimations into a large-scale energy system model, which finds cost-optimal system designs that are operationally feasible with hourly temporal resolution across ten countries in the North Sea region. Acknowledging the uncertainty on AWE systems' future costs and performance and floating wind turbines, we examine a broad range of cost and technology development scenarios and identify which insights are consistent across different possible futures. We find that onshore AWE outperforms conventional onshore wind regarding system-wide benefits due to higher wind resource availability and distinctive hourly generation profiles, which are sometimes complementary to conventional onshore turbines. The achievable power density per ground surface area is the main limiting factor in large-scale onshore AWE deployment. Offshore AWE, in contrast, provides system benefits similar to those of offshore wind alternatives. Therefore, deployment is primarily driven by cost competitiveness. Floating wind turbines achieve higher performance than conventional wind turbines, so they can cost more and remain competitive. AWE, in particular, might be able to play a significant role in a climate-neutral European energy supply and thus warrants further study.