We propose a concept of hybrid geometry sets for registering cross-source geometric data. Specifically, our method focuses on the coarse registration of geometric data obtained from laser scanning and photogrammetric reconstruction. Due to different characteristics (e.g., variations in noise levels, density, and scales), achieving accurate registration between these data becomes a challenging task. The proposed method uses geometric structures to construct hybrid geometry sets, and the geometric relations between the elements of a hybrid geometry set are encoded in a hybrid feature space. This enables effective and efficient similarity query and correspondence establishment between the hybrid geometry sets. The proposed global registration method works in three steps. Firstly, a set of hybrid geometry sets is constructed using extracted planes and intersection lines. Then the features of the hybrid geometry sets are computed to encode the relative pose and topological relationships between the extracted planes and intersection lines, and their correspondences between the two inputs are established by querying hybrid geometry sets with similar features. Finally, the global registration parameters are calculated using the correspondences, and the registration result is further refined through continuous optimization. The robustness of the method has been evaluated using different real-world cross-source geometric data of urban scenes. Extensive comparisons with state-of-the-art algorithms have also demonstrated its effectiveness.