Obstacle detection is a fundamental capability of an autonomous maritime surface vessel (AMSV). State-of-the-art obstacle detection algorithms are based on convolutional neural networks (CNNs). While CNNs provide higher detection accuracy and fast detection speed, they require enormous amounts of data for their training. In particular, the availability of domain-specific datasets is a challenge for obstacle detection. The difficulty in conducting onsite experiments limits the collection of maritime datasets. Owing to the logistic cost of conducting on-site operations, simulation tools provide a safe and cost-efficient alternative for data collection. In this work, we introduce SimuShips, a publicly available simulation-based dataset for maritime environments. Our dataset consists of 9471 high-resolution (1920x1080) images which include a wide range of obstacle types, atmospheric and illumination conditions along with occlusion, scale and visible proportion variations. We provide annotations in the form of bounding boxes. In addition, we conduct experiments with YOLOv5 to test the viability of simulation data. Our experiments indicate that the combination of real and simulated images improves the recall for all classes by 2.9%.

The use of digital twins for the development of Autonomous Maritime Surface Vessels (AMSVs) has enormous potential to resolve the increasing need for water-based navigation and safety at the sea. Aiming at the problem of lack of broad and integrated digital twin implementations with live data along with the absence of a digital twin-driven framework for AMSV design and development, an application framework for the development of a fully autonomous vessel using an integrated digital twin in a 3D simulation environment has been presented. Our framework has 4 layers which ensure that simulation and real-world vessel and the environment are as close as possible. Åboat, an in-house, experimental research platform for maritime automation and autonomous surface vessel applications, equipped with two trolling electric motors, cameras, LiDARs, IMU and GPS has been used as the case study to provide a proof of concept. Åboat, its sensors, and the environment have been replicated in a commercial, 3D simulation environment, AILiveSim. Using the proposed application framework, we develop obstacle detection and path planning systems based on machine learning which leverage live data from a 3D simulation environment to mirror the complex dynamics of the real world. Exploiting the proposed application framework, the rewards across training episodes of a Deep Reinforcement Learning model are evaluated for live simulated data in AILiveSim.