In guiding the energy transition efforts towards renewable energy sources, 3D city models were shown to be useful tools when assessing the annual solar energy generation potential of urban landscapes. However, the simplified roof geometry included in these 3D city models and the lack of additional semantic information about the buildings' roof often yield less accurate solar potential evaluations than desirable. In this paper we propose three different methods to infer and store additional information into 3D city models, namely on physical obstacles present on the roof and existing solar panels. Both can be used to increase the accuracy of roof solar panel retrofit potential. These methods are developed and tested on the open datasets available in the Netherlands, specifically AHN3 lidar point-cloud and PDOK aerial photography. However, we believe they can be adapted to different environments as well, based on the available datasets and their precision locally available.