Energy storage system control using deep reinforcement learning

Abstract

Energy storage systems (ESS) can function as an energy buffer to resolve the temporal imbalance between energy generation and energy consumption. Users can charge energy to the ESS when the energy price is low or when renewable energy generation is greater than consumption so that the stored energy can be used when the energy demand is high and the energy price is high. Finding the optimal charging and discharging schedule can be cast generally as a stochastic control problem that can be solved using approaches such as Dynamic Programming and Model Predictive Control. Such approaches require explicitly defined analytical models for ESS dynamics and stochastic processes on energy demand, generation, and price

thus, they are difficult to employ if such models are not available. In this thesis, a data-driven control approach was proposed for ESS operation. Specifically, model-free reinforcement learning was employed with a deep neural network function approximator for state-action values and deep neural networks for policy gradient. With these approaches, the control policy is trained with stochastically varying data about the energy demand, wind energy generation, and energy price. Through simulation studies, the proposed method has been shown to optimize multidimensional and continuous ESS control actions, given stochastically varying state observations, to achieve significant reductions in the cost of energy to users.