Reinforcement learning (RL) has become a promising approach for optimizing the dispatch of energy storage systems (ESSs) in distributed energy systems. Utilizing linear methods in the Q-representation of RL often struggles to balance accuracy and efficiency, while neural network (NN) performs well but falls short in terms of explainability and interpretability. To address these challenges, we developed and evaluated a deep-Q-symbolic-network (DQSN) framework, which integrates a symbolic network (SN) into the deep-Q-network (DQN) architecture for optimal dispatch of ESS. We benchmarked the performance of DQSN against DQN using mixed-integer linear programming (MILP) results, focusing on algorithm convergence, training duration, and operational cost accuracy. Our findings indicate that DQSN achieves slightly superior rewards and reduced operational costs with a modest increase in training time. Additionally, while DQN demonstrates superior generalization to unseen scenarios, DQSN excels in accurately fitting training data, enabling DQSN to be a viable alternative to DQN, particularly in applications requiring explainability and interpretability.

Given the uncertainty from renewable production, local loads and battery operating states in microgrid, it is vital to develop an efficient energy management scheme to improve system economics and enhance grid reliability. In this paper, we consider a renewable integrated microgrid scenario including an energy storage system (ESS), bidirectional energy flow from/to conventional power grid and ESS health estimation. We develop a novel demand response-based scheme for microgrid energy management with a long short-term memory (LSTM) network and reinforcement learning (RL), aiming to improve the system operating profit from energy-trading and reduce the battery health cost from energy-scheduling. Specifically, to overcome the uncertainty from future, we utilize LSTM to forecast the unknown demand and electricity price. To obtain the desired ESS control scheme, we apply RL to learn an optimal energy-scheduling strategy. To improve the critical performance of the RL paradigm, we propose a random greedy strategy to encourage exploration. Numerical results show that our proposed algorithm outperforms the baselines by improve the system operating profit by 8.27% and 17.31% while ensuring ESS operating safety. By integrating energy efficiency with sustainable energy management practices, our scheme contributes to long-term environmental and economic resilience.

Deep Reinforcement Learning (DRL) presents a promising avenue for optimizing Energy Storage Systems (ESSs) dispatch in distribution networks. This paper introduces RL-ADN, an innovative open-source library specifically designed for solving the optimal ESSs dispatch in active distribution networks. RL-ADN offers unparalleled flexibility in modeling distribution networks, and ESSs, accommodating a wide range of research goals. A standout feature of RL-ADN is its data augmentation module, based on Gaussian Mixture Model and Copula (GMC) functions, which elevates the performance ceiling of DRL agents, achieving an average performance improvement of 21.43%, 1.08%, 2.76%, by augmenting five-year, one-year and three-month data, respectively. Additionally, RL-ADN incorporates the Tensor Power Flow solver, significantly reducing the computational burden of power flow calculations during training without sacrificing accuracy, maintaining voltage magnitude with an average error not exceeding 0.0001%. The effectiveness of RL-ADN is demonstrated using distribution networks with size varying, showing marked performance improvements in the adaptability of DRL algorithms for ESS dispatch tasks. Furthermore, RL-ADN achieves a tenfold increase in computational efficiency during training, making it highly suitable for large-scale network applications. The library sets a new benchmark in DRL-based ESSs dispatch in distribution networks and it is poised to advance DRL applications in distribution network operations significantly. RL-ADN is available at: https://github.com/ShengrenHou/RL-ADN and https://github.com/distributionnetworksTUDelft/RL-ADN.

Reinforcement Learning (RL) has emerged as a promising solution for defining the optimal dispatch of Energy Storage Systems (ESS) in distributed energy systems. However, a notable gap exists in the literature: a lack of comprehensive and fair comparisons between different RL algorithms, particularly between linear and nonlinear approaches. This study critically evaluates the trade-offs between computational efficiency and operational accuracy among various Linear RL (LRL) strategies and compares them against the nonlinear Deep-Q-Network (DQN) algorithm. Through a comprehensive analysis, this study benchmarks the model-based Mixed-Integer Linear Programming (MILP) results to assess and compare these algorithms' convergence, training efficiency, and optimization accuracy. Results indicate that while LRL approaches the operational cost accuracy of DQN, it faces significant trade-offs in computational efficiency and struggles with generalization across larger and varied datasets. The results illuminate critical areas for further development in LRL methodologies, particularly in enhancing their adaptability and generalization capabilities.