Learning-based control, in particularReinforcement Learning (RL) reinforcementReinforcement learning, and optimization-based control, in particular model predictive control, each have their advantages and disadvantages for online, real-timeOptimal control optimal controlOptimal control of systems with complex dynamicsDynamic. However, both approaches are highly complementary and therefore there is an increased interest in combining their advantages in an integrated approach. In this chapter, we provide an overview of recent results, challenges, and opportunities on an integrated learning-based and optimization-based control approach. We focus in particular on piecewise affine systems as they are an extension of linear systemsLinear systems that can model or approximate hybridHybrid or nonlinearNonlinearbehaviorBehavior and as they still allow for effective numerical solutionSolution approaches.

Estimating the state thermal storage devices is key to use them efficiently to reduce the uncertainty of renewable sources. Although stratified storage tanks are one of the most efficient and cost-effective storage systems, they lack accurate state estimation methods. In this paper, we propose a general methodology for estimating the state of thermally stratified storage tanks of different topologies and capacity. The method is based on a simple moving horizon estimation technique and a 1-D smooth model that can integrate buoyancy effects into a smooth equation. The novelty of the proposed approach is that it is the first state estimation approach that considers both buoyancy and mixing effects. This distinction is paramount to an adequate estimation of the temperature distribution in the storage tank which can then be used for different aims, namely as a basis for model predictive controls. Besides the novel state estimation approach, the paper has three more contributions: (i) it shows how a model for seasonal storage devices can be further extended to smaller stratified tanks with different topologies; (ii) it modifies such a model so that the model equations can be integrated into a single dynamical equation; (iii) it proposes the most complete case study to date for modeling and estimating temperature distribution inside small stratified storage tanks. The analysis of the proposed approach is done in several stages. First, to validate the applicability of the model to small tanks and multiple topologies, we perform a model identification and parameter estimation for three different stratified tanks. Second, we test the accuracy of the proposed state estimation approach in those three stratified tanks employing the estimated parameters in the first experimental study and the models also previously defined. Finally, to further validate the models, we perform a simulation for each of the three tanks and we compare the accuracy of the simulation against real data. As we show, both the state estimation approach and the model are satisfactorily accurate as they display average mean errors below 2 °C.

State-of-the-art Model Predictive Control (MPC) applications for building heating adopt either a deterministic controller together with a nonlinear model or a linearized model with a stochastic MPC controller. However, deterministic MPC only considers one single realization of the disturbances and its performance strongly depends on the quality of the forecast of the disturbances, which can lead to low performance. In fact, inadequate building energy management can lead to high energy costs and CO₂ emissions. On the other hand, a linearized model can fail to capture some dynamics and behavior of the building under control. In this article, we combine a stochastic scenario-based MPC (SBMPC) controller together with a nonlinear Modelica model that is able to provide a richer building description and to capture the dynamics of the building more accurately than linear models. The adopted SBMPC controller considers multiple realizations of the external disturbances obtained through a statistically accurate model, so as to consider different possible disturbance evolutions and to robustify the control action. To this purpose, we present a scenario generation method for building temperature control that can be applied to several exogenous perturbartions, e.g. solar irradiance, outside temperature, and that satisfies several important stastistical properties, in contrast with simpler and less accurate methods adopted in the literature. We show the benefits of our proposed approach through several simulations in which we compare our method against the standard ones from the literature, for several combinations of a trade-off parameter between comfort and energy cost. We show how our SBMPC controller approach outperforms the standard controllers available in the literature.

In this paper, we propose a data-driven methodology to identify the optimal placement of sensors in a multi-zone building. The proposed methodology is based on statistical tests that study the (in) dependence of measurements from various available sensors. The tests advice on a set of most dissimilar sensors to be retained, as they would convey the maximum information. The method starts with an initial setup that can provide measurements of every building zone to carry out this study; any of these sensors can be removed eventually to decrease costs in normal operation. The method has the advantages of being purely data driven and computationally efficient, as against several methods proposed in the scientific literature, that operate under the premise that detailed building models are available, to evaluate the number/position of the required sensors. This property makes the method scale to different buildings, in an expert free manner. The methodology can help towards better characterization of a building for optimal control and monitoring applications. It is validated against a widely used method – Kalman filtering with Grey-box models, using two different case studies. In both cases, the proposed approach agrees with the results using grey box models, suggesting that the method is reliable, while being quick and efficient.

Electricity balancing is one of the main demanders of short-term flexibility. To improve its integration, the recent regulation of the European Union introduces a common standalone balancing energy market. It allows actors that have not participated or not been awarded in the preceding balancing capacity market to participate as voluntary bidders or ‘second-chance’ bidders. We investigate the effect of these changes on balancing market efficiency and on strategic behavior in particular, using a combination of agent-based modelling and reinforcement learning. This paper is the first to model agents' interdependent bidding strategies in the balancing capacity and energy markets with the help of two collaborative reinforcement learning algorithms. Results reveal considerable efficiency gains in the balancing energy market from the introduction of voluntary bids even in highly concentrated markets while offering a new value stream to providers of short-term flexibility. ‘Second-chance’ bidders further drive competition, reducing balancing energy costs. However, we warn that this design change is likely to shift some of the activation costs to the balancing capacity market where agents are prompted to bid more strategically in the view of lower profits from balancing energy. As it is unlikely that the balancing capacity market can be removed altogether, we recommend integrating European balancing capacity markets on par with balancing energy markets and easing prequalification requirements to ensure sufficient competition.

To assess the impact of implementing energy efficiency and renewable energy measures, urban building energy models are emerging. In these models, due to the lack of data, the natural variability of the existing building stock is often highly underestimated and uncertainty on the simulated energy use arises. Therefore, this work proposes a probabilistic building characterization method to model the variability of the existing residential building stock. The method estimates realistic distributions of five input variables: U-values of the floor, external walls, windows and roof as well as window-to-wall ratio, based on known data (location, geometry and construction year). First, quantile regression has been implemented to generate the uncorrelated distributions based on the Flemish energy performance certificates database. The accuracy of the marginal distributions is good, as the empirical coverage on the 50%, 80%, 90% and 98% prediction interval deviates 0.6% at most. However, it is needed to include the correlations between these variables. Hence, three main methods to build multivariate distributions from marginal distributions and to draw correlated samples are implemented and extensively compared. The Gaussian copula method is put forward as the preferred method. Considering the mean-maximum discrepancy (MMD), this method performs eight times better than the uncorrelated case (MMD of 0.0027 versus 0.0228).

Seasonal thermal energy storage systems (STESSs) can shift the delivery of renewable energy sources and mitigate their uncertainty problems. However, to maximize the operational profit of STESSs and ensure their long-term profitability, control strategies that allow them to trade on wholesale electricity markets are required. While control strategies for STESSs have been proposed before, none of them addressed electricity market interaction and trading. In particular, due to the seasonal nature of STESSs, accounting for the long-term uncertainty in electricity prices has been very challenging. In this article, we develop the first control algorithms to control STESSs when interacting with different wholesale electricity markets. As different control solutions have different merits, we propose solutions based on model predictive control and solutions based on reinforcement learning. We show that this is critical since different markets require different control strategies: MPC strategies are better for day-ahead markets due to the flexibility of MPC, whereas reinforcement learning (RL) strategies are better for real-time markets because of fast computation times and better risk modeling. To study the proposed algorithms in a real-life setup, we consider a real STESS interacting with the day-ahead and imbalance markets in The Netherlands and Belgium. Based on the obtained results, we show that: 1) the developed controllers successfully maximize the profits of STESSs due to market trading and 2) the developed control strategies make STESSs important players in the energy transition: by optimally controlling STESSs and reacting to imbalances, STESSs help to reduce grid imbalances.

To correct grid imbalances and avoid grid failures, the transmission system operator (TSO) deploys balancing reserves and settles these imbalances by penalizing the market actors that caused them. In several countries, it is forbidden to influence the grid imbalances in order to let the TSO retain full control of grid regulation. In this paper, we argue that this approach is not optimal as market actors that trade imbalances under the supervision of the TSO can help balancing the grid more efficiently. For instance, some systems such as solar farms cannot participate in the standard balancing market but do have economic incentives to help regulate the grid by trading with imbalances. Based on this argument, we propose a new market framework where any market actor is allowed to trade with imbalances. We show that, using the new market mechanism, the TSO can keep full control of the grid balance while decreasing the balancing cost. This is of primary importance as: 1) novel approaches to reduce grid imbalances are needed as, while renewable sources are generally not used for grid balancing, the increasing integration of renewable energy sources creates higher imbalances. 2) While long-term storage of energy is key in the energy transition, it needs to become an attractive investment to ensure its widespread use; as we show, the proposed market can guarantee that. Based on a real case study, we show that the new market can provide 10–20% of the total balancing energy needed and reduce the balancing costs.

While the field of electricity price forecasting has benefited from plenty of contributions in the last two decades, it arguably lacks a rigorous approach to evaluating new predictive algorithms. The latter are often compared using unique, not publicly available datasets and across too short and limited to one market test samples. The proposed new methods are rarely benchmarked against well established and well performing simpler models, the accuracy metrics are sometimes inadequate and testing the significance of differences in predictive performance is seldom conducted. Consequently, it is not clear which methods perform well nor what are the best practices when forecasting electricity prices. In this paper, we tackle these issues by comparing state-of-the-art statistical and deep learning methods across multiple years and markets, and by putting forward a set of best practices. In addition, we make available the considered datasets, forecasts of the state-of-the-art models, and a specifically designed python toolbox, so that new algorithms can be rigorously evaluated in future studies.

In this paper, a gradient boosting tree model is proposed to detect, identify and localize single-phase-to-ground and three-phase faults in low voltage (LV) smart distribution grids. The proposed method is based on gradient boosting trees and considers branch-independent input features to be generalizable and applicable to different grid topologies. Particularly, as it is shown, the method can be estimated in a specific grid topology and be employed in a different one. To test the algorithm, the method is evaluated in a simulated real LV distribution grid of Portugal. In this case study, different fault resistances, fault locations and hours of the day are considered. In detail, the algorithm is evaluated at eighteen fault resistance values between 0.1 and 1000 Ω; similarly, nine fault locations are considered within each one of the 32 sectors of the grid and the faults are simulated across different hours of a day. The developed algorithm showed promising results in both out-of-sample branch and fault resistance data especially for fault detection, demonstrating a maximum fault detection error of 0.72%.

Power outages in electrical grids can have very negative economic and societal impacts rendering fault diagnosis paramount to their secure and reliable operation. In this paper, deep neural networks are proposed for fault detection and location in low-voltage smart distribution grids. Due to its key properties, the proposed method solves some of the drawbacks of the existing literature methods, namely a method that: 1) is not limited by the grid topology; 2) is branch-independent; 3) can localize faults even with limited data; 4) is the first to accurately detect and localize high-impedance faults in the low-voltage distribution grid. The generalizability of the method derives from the non-grid specific nature of the inputs that it requires, inputs that can be obtained from any grid. To evaluate the proposed method, a real low-voltage distribution grid in Portugal is considered and the robustness of the method is tested against several disturbances including large fault resistance values (up to 1000 Ω). Based on the case study, it is shown that the proposed methodology outperforms conventional fault diagnosis methods: it detects faults with 100% accuracy, identifies faulty branches with 83.5% accuracy, and estimates the exact fault location with an average error of less than 11.8%. Finally, it is also shown that: 1) even when reducing the available measurements to the bare minimum, the accuracy of the proposed method is only decreased by 4.5%; 2) while deep neural networks usually require large amounts of data, the proposed model is accurate even for small dataset sizes.

Market-based procurement of balancing services in Europe is prone to strategic bidding due to the relatively small market size and a limited number of providers. In the European Union, balancing markets are undergoing substantial regulatory changes driven the efforts to harmonize the market design and better align it with the goals of the energy transition. It is proposed to decouple the balancing energy (real-time) market from the (forward) balancing capacity market and the price of balancing energy will be based on the marginal bid. In this paper, the potential effects of these changes on market participants’ strategies are analyzed using an agent-based model. This model compares the effects of a standalone balancing energy market with different pricing rules on economic efficiency with agents that apply naïve, rule-based and reinforcement-learning strategies. The results indicate that the introduction of a standalone balancing energy market reduces the cost of balancing, even in a concentrated market with strategic bidders. Marginal pricing consistently leads to lower weighted average prices than pay-as-bid pricing, regardless of the level of competition. Nevertheless, in an oligopoly with actors bidding strategically, prices can deviate from the competitive benchmark by a factor of 4–5. This implies that the introduction of a standalone balancing energy market does not entirely solve the issue of strategic bidding, but helps dampen the prices, as compared to the balancing market prior to the design change.

In the context of building heating systems control in office buildings, the current state-of-the-art applies either a deterministic Model Predictive Control (MPC) controller together with a nonlinear model, or a linearized model with a stochastic MPC controller. Deterministic MPC considers only one realization of the external disturbances, which can lead to a low performance solution if the forecasts of the disturbances are not accurate. Similarly, linear models are simplified representations of the building dynamics and might fail to capture some relevant behavior. In this paper, we improve upon the current literature by combining these two approaches, i.e. we adopt a nonlinear model together with a stochastic MPC controller. We consider a scenario-based MPC (SBMPC), where many realizations of the disturbances are considered, so as to include more possible future trajectories for the external disturbances. The adopted scenario generation method provides statistically significant scenarios, whereas so far in the current literature only approximate methods have been applied. Moreover, we use Modelica to obtain the model description, which allows to have a more accurate and nonlinear model. Lastly, we perform simulations comparing standard MPC vs SBMPC vs an optimal control approach with measurements of the external disturbances, and we show how our proposed scenario-based MPC controller can achieve a better performance compared to standard deterministic MPC.

To mitigate the effects of the intermittent generation of renewable energy sources, reliable and efficient energy storage is critical. Since nearly 80% of households energy consumption is destined to water and space heating, thermal energy storage is particularly important. In this context, we propose and validate a new model for one of the most efficient heat storage systems: stratified thermal storage tanks. The novelty of the model is twofold: first, unlike the non-smooth models from the literature, it identifies the mixing and buoyancy dynamics using a smooth and continuous function. This smoothness property is critical to efficiently integrate thermal storage vessels in optimization and control problems. Second, unlike models from literature, it considers two types of buoyancy: slow, linked to naturally occurring buoyancy, and fast, associated with charging/discharging effects. As we show, this distinction is paramount to identify accurate models. To show the relevance of the model, we consider a real tank that can satisfy heat demands up to 100 kW. Using real data from this vessel, we validate the proposed model and show that the estimated parameters correctly identify the physical properties of the vessel. Then, we employ the model in a control problem where the vessel is operated to minimize the cost of providing a given heat demand and we compare the model performance against that of a non-smooth model from literature. We show that: (1)the smooth model obtains the best optimal solutions; (2)its computation costs are 100 times cheaper; (3)it is the best alternative for use in real-time model- based control strategies, e.g. model predictive control.

Due to the increasing integration of renewable sources in the electrical grid, electricity generation is expected to become more uncertain. In this context, seasonal thermal energy storage systems (STESSs) are key to shift the delivery of renewable energy sources and tackle their uncertainty problems. In this paper, we propose an optimal controller for STESSs that, using reinforcement learning, builds bidding functions for the day-ahead market. In detail, considering that there is an uncertain energy demand that the STESS has to satisfy, the controller buys energy in the day-ahead market so that the uncertain demand is satisfied while the profits are maximized. Since prices are low during periods of large renewable energy generation (and vice versa), maximizing the profit of a STESS indirectly shifts the delivery of renewable energy to periods of high energy demand while reducing their uncertainty problems. To evaluate the proposed algorithm, we consider a real STESS providing different yearly-demand levels; then, we compare the performance of the controller to the theoretical upper bound, i.e. the optimal cost of buying energy given perfect knowledge of the demand and prices. The results indicate that the proposed controller performs reasonably well: despite the large uncertainty in prices and demand, the proposed controller obtains 70%-50% of the maximum gains given by the theoretical bound.

Recent research has seen several forecasting methods being applied for heat load forecasting of district heating networks. This paper presents two methods that gain significant improvements compared to the previous works. First, an automated way of handling non-linear dependencies in linear models is presented. In this context, the paper implements a new method for feature selection based on [1], resulting in computationally efficient models with higher accuracies. The three main models used here are linear, ridge, and lasso regression. In the second approach, a deep learning method is presented. Although computationally more intensive, the deep learning model provides higher accuracy than the linear models with automated feature selection. Finally, we compare and contrast the proposed methods with earlier work for day-ahead forecasting of heat load in two different district heating networks.

In this paper, a novel modeling framework for forecasting electricity prices is proposed. While many predictive models have been already proposed to perform this task, the area of deep learning algorithms remains yet unexplored. To fill this scientific gap, we propose four different deep learning models for predicting electricity prices and we show how they lead to improvements in predictive accuracy. In addition, we also consider that, despite the large number of proposed methods for predicting electricity prices, an extensive benchmark is still missing. To tackle that, we compare and analyze the accuracy of 27 common approaches for electricity price forecasting. Based on the benchmark results, we show how the proposed deep learning models outperform the state-of-the-art methods and obtain results that are statistically significant. Finally, using the same results, we also show that: (i) machine learning methods yield, in general, a better accuracy than statistical models; (ii) moving average terms do not improve the predictive accuracy; (iii) hybrid models do not outperform their simpler counterparts.

Motivated by the increasing integration among electricity markets, in this paper we propose two different methods to incorporate market integration in electricity price forecasting and to improve the predictive performance. First, we propose a deep neural network that considers features from connected markets to improve the predictive accuracy in a local market. To measure the importance of these features, we propose a novel feature selection algorithm that, by using Bayesian optimization and functional analysis of variance, evaluates the effect of the features on the algorithm performance. In addition, using market integration, we propose a second model that, by simultaneously predicting prices from two markets, improves the forecasting accuracy even further. As a case study, we consider the electricity market in Belgium and the improvements in forecasting accuracy when using various French electricity features. We show that the two proposed models lead to improvements that are statistically significant. Particularly, due to market integration, the predictive accuracy is improved from 15.7% to 12.5% sMAPE (symmetric mean absolute percentage error). In addition, we show that the proposed feature selection algorithm is able to perform a correct assessment, i.e. to discard the irrelevant features.