Vegetation indices, especially the normalized difference vegetation index (NDVI), are widely used in urban vegetation assessments. However, estimating the vegetation abundance in urban scenes using the NDVI has constraints due to the complex spectral signature related to the urban structure, materials and other factors compared to natural ground surfaces. This paper employs the 3D discrete anisotropic radiative transfer (DART) model to simulate the spectro-directional reflectance of synthetic urban scenes with various urban geometries and building materials using a flux-tracking method under shaded and sunlit conditions. The NDVI is calculated using the spectral radiance in the red (0.6545 μm) and near-infrared bands (0.865 μm). The effects of the urban material heterogeneity and 3D structure on the NDVI, and the performance of three NDVI-based fractional vegetation cover (FVC) inversion algorithms, are evaluated. The results show that the effects of the building material heterogeneity on the NDVI are negligible under sunlit conditions but not negligible under shaded conditions. The NDVI value of building components within synthetic scenes is approximately zero. The shaded road exhibits a higher NDVI value in comparison to the illuminated road because of scattering from adjacent pixels. In order to correct the effects of scattering caused by building geometry, the reflectance of the Landsat 8/OLI image is corrected using the sky view factor (SVF) and then used to calculate the FVC. Jilin-1 satellite images with high spatial resolution (0.5 m) are used to extract the vegetation cover and then aggregated to 30 m spatial resolution to calculate the FVC for validation. The results show that the RMSE is up to 0.050 after correction, while the RMSE is 0.169 before correction. This study makes a contribution to the understanding of the effects of the urban 3D structure and material reflectance on the NDVI and provides insights into the retrieval of the FVC in different urban scenes.