Urban surface temperature is a very important variable in the observation and understanding of energy exchange. A comprehensive understanding of the urban thermal environment is of great significance towards the adaptability of urban areas to climate hazards. The heterogeneity of urban space increases the complexity of the urban surface temperature observations and the analyses of the energy exchange. To understand how the urban geometry affects the distribution of surface temperature, we used airborne thermal infrared remotely sensed images at very high spatial resolution (original spatial resolution is 0.2 m × 0.2 m after registration). We did this study in Hong Kong to analyze the effects of various geometric parameters on different facet surface temperatures (roof, road, wall and vegetation) in daytime and nighttime and in different seasons. Results show that the urban geometry has greater impacts on the road temperature than on building temperature, and the impact of the geometric parameters on road surface temperature changes with the time of the day and the season. The building height is a more effective driver of heat dissipation in daytime than nighttime for roof facets. A lower building density improves ground heat dissipation, while a higher building density improves heat dissipation by roof facets. Furthermore, the vegetation only limitedly affects the surface temperatures of facets that are lower than vegetation, but to an extent useful to mitigate urban temperature, which might be a feature relevant in urban design. This research can provide insights useful to city planners and policy makers to better understand the urban thermal environment and help design more livable and healthy cities in the near future.