Learning to control a battery through reinforcement

Abstract

Battery energy storage systems offer control over energy use and enable energy arbitrage (EA) helping to lower energy costs. However, battery owners currently fail to optimally exploit these systems for EA as the battery lifetime decreases, and many EA approaches incorrectly assume constant battery capacity. Battery performance declines over time resulting in reduced capacity that limits the economic benefits. Therefore, considering battery degradation is key to balancing economic profit and lifetime. In response, this work applies reinforcement learning to control a battery providing residential EA services and proposes a semi-supervised learning model to consider degradation. Case studies investigate three scenarios: 1) the approach is trained on a battery with an unrealistic constant maximum capacity to serve as a baseline, 2) the actions from the first scenario are applied to a real-world environment with a battery experiencing capacity decay to acknowledge the effect of neglecting degradation and 3) the approach considers a battery with a real decreasing capacity. Results show not considering degradation when operating a battery (scenario 2), leads to profits 13% lower than the ones obtained in the ideal case (scenario 1). If degradation is considered (scenario 3), the profits are only 4% lower than the profits obtained in the ideal case (scenario 1) and the battery's lifetime is extended by 20% compared to the lifetime achieved when not considering degradation (scenario 2).