Dynamic Security Assessment (DSA) for the future power system is expected to be increasingly complicated with the higher level penetration of renewable energy sources (RES) and the widespread deployment of power electronic devices, which drive new dynamic phenomena. As a result, the increasing complexity and the severe computational bottleneck in real-time operation encourage researchers to exploit machine learning to extract offline security rules for the online assessment. However, traditional machine learning methods lack in providing information on the confidence of their corresponding predictions. A better understanding of confidence of the prediction is of key importance for Transmission System Operators (TSOs) to use and rely on these machine learning methods. Specifically, from the perspective of topological changes, it is often unclear whether the machine learning model can still be used. Hence, being aware of the confidence of the prediction supports the transition to using machine learning in real-time operation. In this paper, we propose a novel Conditional Bayesian Deep Auto-Encoder (CBDAC) based security assessment framework to compute a confidence metric of the prediction. This informs not only the operator to judge whether the prediction can be trusted, but it also allows for judging whether the model needs updating. A case study based on IEEE 68-bus system demonstrates that CBDAC outperforms the state-of-the-art machine learning-based DSA methods and the models that need updating under different topologies can be effectively identified. Furthermore, the case study verifies that effective updating of the models is possible even with very limited data.