A promising option for storing large-scale quantities of green gases (e.g., hydrogen) is in subsurface rock salt caverns. The mechanical performance of salt caverns utilized for long-term subsurface energy storage plays a significant role in long-term stability and serviceability. However, rock salt undergoes non-linear creep deformation due to long-term loading caused by subsurface storage. Salt caverns have complex geometries and the geological domain surrounding salt caverns has a vast amount of material heterogeneity. To safely store gases in caverns, a thorough analysis of the geological domain becomes crucial. To date, few studies have attempted to analyze the influence of geometrical and material heterogeneity on the state of stress in salt caverns subjected to long-term loading. In this work, we present a rigorous and systematic modeling study to quantify the impact of heterogeneity on the deformation of salt caverns and quantify the state of stress around the caverns. A 2D finite element simulator was developed to consistently account for the non-linear creep deformation and also to model tertiary creep. The computational scheme was benchmarked with the already existing experimental study. The impact of cyclic loading on the cavern was studied considering maximum and minimum pressure that depends on lithostatic pressure. The influence of geometric heterogeneity such as irregularly-shaped caverns and material heterogeneity, which involves different elastic and creep properties of the different materials in the geological domain, is rigorously studied and quantified. Moreover, multi-cavern simulations are conducted to investigate the influence of a cavern on the adjacent caverns. An elaborate sensitivity analysis of parameters involved with creep and damage constitutive laws is performed to understand the influence of creep and damage on deformation and stress evolution around the salt cavern configurations. The simulator developed in this work is publicly available at https://gitlab.tudelft.nl/ADMIRE_Public/Salt_Cavern.