Deep learning has been widely applied to automated leakage detection and location of natural gas pipe networks. Prevalent deep learning approaches do not consider the spatial dependency of sensors, which limits leakage detection performance. Graph deep learning is a promising alternative to prevailing approaches as it can model spatial dependency. However, the challenge of collecting real-world anomaly data for training limits the accuracy and robustness of currently used graph deep learning approaches. This study proposes a deep probabilistic graph neural network in which attention-based graph neural network is built to model spatial sensor dependency. Variational Bayesian inference is integrated to model the posterior distribution of sensor dependency so that the leakage can be localized. An urban natural gas pipe network experiment is employed to construct the benchmark dataset, in which normal time-series data is applied to develop our proposed model while anomaly leakage data is used for performance comparison between our model and other state-of-the-art models. The results demonstrate that our model exhibits competitive detection accuracy (AUC) = 0.9484, while the additional uncertainty interval provides more comprehensive leakage detection information compared to state-of-the-art deep learning models. In addition, our model's posterior distribution enhances the leakage localization with the accuracy of positioning (PAc) = 0.8, which is higher than that of other state-of-the-art graph deep learning models. This study provides a comprehensive and robust alternative for subsequent decision-making to mitigate natural gas leakage from pipe networks.