Urban Air Mobility (UAM) has seen remarkable growth over the past decade, especially considering the delivery of goods. Notably, the delivery of medical supplies via Unmanned Aerial Vehicles (UAVs) has emerged as a promising application, which is driven by its societal benefits and has been demonstrated successfully in rural environments. However, expanding these operations to urban areas presents a unique set of challenges, such as high traffic density and the need for online path planning of UAVs. Effective path planning and coordination are of utmost importance to allow for safe and efficient delivery operations in urban environments. Currently, path planning of UAVs is relatively unexplored, and for drone delivery models the trajectories of UAVs are often oversimplified. This research aims to fill the research gap by developing and evaluating an online path planning and coordination mechanism in 3D for a cooperative fleet of autonomous UAVs delivering medical supplies. The Rotterdam Area is selected as a use case, but the model can be adapted to other cities. Utilising an agent-based model formulation, this paper formulates a novel approach for multi-agent pathfinding in 3D and real-world environments, where the travelled paths of UAVs in drone delivery models are computed by taking into account the urban environment and kinematics of UAVs. Windowed Cooperative Safe Interval Path Planning (WC-SIPP) is used as a path planning and coordination mechanism, where kinematic constraints of UAVs and safety separation distances between UAVs are incorporated. Experimental studies show the capability of the algorithm to plan UAVs in different multi-layer urban environments. We establish that including more than one vertical layer is beneficial for UAVs to avoid conflicts by allowing them to change altitudes during flight. Additionally, we show that an increase in the number of layers does not lead to an increase in the delivery time of packages. However, an increase in available layers does come with a decrease in computational performance. Besides, the proposed method is able to plan the trajectories of UAVs and ensure safety separation between UAVs within a dynamic environment, where up to 20 unexpected obstacles are introduced into the environment. Furthermore, we demonstrated that for our operation model, assigning prioritisation based on either mission type or package request time did not influence the delivery time of packages.