Symbolic Deep Reinforcement Learning for Energy Storage Systems Optimal Dispatch

Abstract

Reinforcement learning (RL) has become a promising approach for optimizing the dispatch of energy storage systems (ESSs) in distributed energy systems. Utilizing linear methods in the Q-representation of RL often struggles to balance accuracy and efficiency, while neural network (NN) performs well but falls short in terms of explainability and interpretability. To address these challenges, we developed and evaluated a deep-Q-symbolic-network (DQSN) framework, which integrates a symbolic network (SN) into the deep-Q-network (DQN) architecture for optimal dispatch of ESS. We benchmarked the performance of DQSN against DQN using mixed-integer linear programming (MILP) results, focusing on algorithm convergence, training duration, and operational cost accuracy. Our findings indicate that DQSN achieves slightly superior rewards and reduced operational costs with a modest increase in training time. Additionally, while DQN demonstrates superior generalization to unseen scenarios, DQSN excels in accurately fitting training data, enabling DQSN to be a viable alternative to DQN, particularly in applications requiring explainability and interpretability.