The integration of distributed energy resources (DERs) has escalated the challenge of voltage magnitude regulation in distribution networks. Model-based approaches, which rely on complex sequential mathematical formulations, cannot meet the real-time demand. Deep reinforcement learning (DRL) offers an alternative by utilizing offline training with distribution network simulators and then executing online without computation. However, DRL algorithms fail to enforce voltage magnitude constraints during training and testing, potentially leading to serious operational violations. To tackle these challenges, we introduce a novel safe-guaranteed reinforcement learning algorithm, the DistFlow safe reinforcement learning (DF-SRL), designed specifically for real-time voltage magnitude regulation in distribution networks. The DF-SRL algorithm incorporates a DistFlow linearization to construct an expert-knowledge-based safety layer. Subsequently, the DF-SRL algorithm overlays this safety layer on top of the agent policy, recalibrating unsafe actions to safe domains through a quadratic programming formulation. Simulation results show the DF-SRL algorithm consistently ensures voltage magnitude constraints during training and real-time operation (test) phases, achieving faster convergence and higher performance, which differentiates it apart from (safe) DRL benchmark algorithms.

The optimal dispatch of energy storage systems (ESSs) in distribution networks poses significant challenges, primarily due to uncertainties of dynamic pricing, fluctuating demand, and the variability inherent in renewable energy sources. By exploiting the generalization capabilities of deep neural networks (DNNs), the deep reinforcement learning (DRL) algorithms can learn good-quality control models that adapt to the stochastic nature of distribution networks. Nevertheless, the practical deployment of DRL algorithms is often hampered by their limited capacity for satisfying operational constraints in real time, which is a crucial requirement for ensuring the reliability and feasibility of control actions during online operations. This paper introduces an innovative framework, named mixed-integer programming based deep reinforcement learning (MIP-DRL), to overcome these limitations. The proposed MIP-DRL framework can rigorously enforce operational constraints for the optimal dispatch of ESSs during the online execution. This framework involves training a Q-function with DNNs, which is subsequently represented in a mixed-integer programming (MIP) formulation. This unique combination allows for the seamless integration of operational constraints into the decision-making process. The effectiveness of the proposed MIP-DRL framework is validated through numerical simulations, demonstrating its superior capability to enforce all operational constraints and achieve high-quality dispatch decisions and showing its advantage over existing DRL algorithms.

The power generation and consumption of distributed energy resources (DERs) offer significant flexibility potential, which can be utilized to provide services such as peak and frequency regulation. DERs introduce a vast number of variables and constraints, making it complicated to directly integrate them into upper-level dispatch. To address this challenge, virtual power plants (VPPs) emerge, which treat diverse DERs as a collective entity and use aggregated flexibility envelopes to reduce the variable and constraint scale, facilitating upper-level optimization. In VPPs, unified DER modeling and efficient DER aggregation play a crucial role but are challenging. This paper first introduces a novel unified polytope model to represent heterogenous DERs' flexibility region. A coordination transformation is utilized to eliminate redundant variable dimensions and maintain DERs' interface characteristics. A sample-based projection method is then developed, further removing all state variables, resulting in a unified flexibility region. This method is then utilized to calculate the Minkowski sums of individual flexibility polytopes for aggregation. The results of numerical tests demonstrate a considerable reduction in computation time and maintain satisfactory accuracy when the proposed modeling and aggregation approach is adopted.