This study presents an innovative approach to risk-aware decision-making in water resource management. We focus on a case study in the Netherlands, where risk awareness is key to water system design and policy-making. Recognizing the limitations of deterministic methods in the face of weather, energy system, and market uncertainties, we propose a scalable stochastic Model Predictive Control (MPC) framework that integrates probabilistic forecasting, scenario generation, and stochastic optimal control. We utilize Combined Quantile Regression Deep Neural Networks and Non-parametric Bayesian Networks to generate probabilistic scenarios that capture realistic temporal dependencies. The energy distance metric is applied to optimize scenario selection and generate scenario trees, ensuring computational feasibility without compromising decision quality. A key feature of our approach is the introduction of Exceedance Risk (ER) constraints, inspired by Conditional-Value-at-Risk (CVaR), to enable more nuanced and risk-aware decision-making while maintaining computational efficiency. In this work, we enable the Noordzeekanaal–Amsterdam-Rijnkanaal (NZK-ARK) system to participate in Demand Response (DR) services by dynamically scheduling pumps to align with low hourly electricity prices on the Day Ahead and Intraday markets. Through historical simulations using real water system and electricity price data, we demonstrate that incorporating uncertainty can significantly reduce operational costs—by up to 44 percentage points compared to a deterministic approach—while maintaining safe water levels. The modular nature of the framework also makes it adaptable to a wide range of applications, including hydropower and battery storage systems. ... Read more

This thesis explores risk-aware operational decision-making methods to support the integration of Renewable Energy Sources (RES) into the energy system by enhancing energy flexibility under operational uncertainty. Amidst the urgent global shift towards RES to combat climate change, this work identifies and addresses the challenges posed by the intermittent and uncertain nature of renewable energies, such as wind and solar power, to grid stability and energy reliability.... ... Read more

In this manuscript, we test the operational performance decrease of a probabilistic framework for Demand Response (DR). We use Day Ahead Market (DAM) price scenarios generated by a Combined Quantile Regression Deep Neural Network (CQR-DNN) and a Non-parametric Bayesian Network (NPBN) to maximise profit of a Battery Energy Storage System (BESS) participating on the DAM for energy arbitrage. We apply the generated forecast time series to a stochastic Model Predictive Control (MPC), and compare the performance using a point and perfect forecast. For the probabilistic forecasts, we test two control strategies; 1) minimising the Conditional Value at Risk (CVaR) for making costs, and 2) minimising the expected value of the cost. We apply the MPC in a closed-loop simulation setting and perform a sensitivity analysis of the profit by changing the ratio between battery capacity and the max power, the cluster reduction method, and the number of scenarios used by the MPC. We show that the proposed framework works, but the approach does not increase profit compared to a deterministic point forecast. This can possibly be explained by the deterministic forecast capturing the shape of the price curve with less noise than a probabilistic forecast without enough scenarios. We show that the value of a good forecast becomes smaller as the charging time of the battery becomes larger, due to the battery being unable to exploit small price differences optimally. ... Read more

The Netherlands is a low-lying country situated in the Rhine-Meuse delta. A significant portion of the Netherlands is located below sea level, making the proper management of local and national waterways essential. Polders are used to manage groundwater levels, drain excess rainwater, and store water during times of drought. These polders often have pumping stations that pump water into drainage canals, like the Noordzeekanaal-Amsterdam-Rijnkanaal (NZK-ARK), which receives water from the Rhine river and four local water authorities and connects to the North Sea at IJmuiden through a pumping station and a series of undershot gates.

The operators of the NZK-ARK utilize Model Predictive Control (MPC) to schedule the discharge of water through the gates and pumps. The combination of the pump and gate discharge allows the NZK-ARK to discharge excess water to the North Sea when the sea water level is both higher and lower than the water level in the canal. However, traditional MPC can lead to suboptimal schedules when uncertainty is introduced, resulting from, for example, incoming discharge, fluctuating electricity prices, and the availability of renewable energy. Stochastic MPC allows for the consideration of uncertainty in decision-making, optimizing control actions based on a range of potential scenarios. In the future, the objectives for the control system of the gates and pumps may become more complex and may need to take into account factors like renewable energy availability and electricity prices. Ensuring the effective and efficient management of water in the Netherlands is critical, and the use of polders for water storage and control of groundwater tables, and techniques like MPC and stochastic MPC play important roles in achieving this goal.

In this study, we present a framework that combines probabilistic forecasting, scenario generation and reduction, and stochastic MPC to minimize energy costs associated with pumping at the NZK-ARK. This framework is based on probabilistic forecasts of electricity prices and incoming discharge and is specifically designed for use at the NZK-ARK. By considering the uncertainty present in electricity prices and incoming discharge, our framework allows for the optimization of control actions through the use of stochastic MPC. The ultimate goal of this approach is to reduce energy costs at the NZK-ARK by effectively managing the discharge of water through the pumps and gates while complying with local constraints. ... Read more

Participation in demand response (DR) has been explored for many large energy-using assets based on day ahead electricity markets. In this manuscript, we propose the use of multiple electricity spot markets to enable price-based DR for open canal systems in the Netherlands, where many large pumping stations are used for flood mitigation and control of groundwater levels. In the new strategy for pump-scheduling, we combine the day ahead and intraday electricity markets to be used in a hierarchical receding horizon economic Model Predictive Control (MPC). We formulate the decision problem as a Mixed-Integer Quadratic Problem (MIQP), to be solved to near global optimality. A cost-potential analysis was performed for multiple market strategies and the automatic Frequency Restoration Reserves (aFRRs), using actual market and water system data. We show new insights into the trade-off between CO2 emissions and operating cost, the difference between the German and Dutch markets, and temporal changes in market conditions due to renewable energy integration. We observe that the German energy market is rewarding DR more than the Dutch equivalent, due to the higher renewable energy market penetration. The proposed multi-market strategy leads to a cost decrease of 10 and 16% in the Netherlands in 2017 and 2019, respectively. When applying German market scenarios, we found a cost-saving potential of 56 and 50% in 2017 and 2019, respectively. The cost-saving potential for the aFRR market was found to be up to 12% in the Netherlands and 28% in Germany, through a conservative analysis. The results suggest that the proposed control system, optimising costs over the day ahead, intraday and possibly the aFRR markets, is profitable compared to the current strategy in both the current and future electricity market. ... Read more

In this manuscript we propose a methodology to generate electricity price scenarios from probabilistic forecasts. Using a Combined Quantile Regression Deep Neural Network, we forecast hourly marginal price distribution quantiles for the DAM on which we fit parametric distributions. A Non-parametric Bayesian Network (BN) is applied to sample from these distributions while using the observed rank-correlation in the data to condition the samples. This results in a methodology that can create an unbounded amount of price-scenarios that obey both the forecast hourly marginal price distributions and the observed dependencies between the hourly prices in the data. The BN makes no assumptions on the marginal distribution, allowing us to flexibly change the marginal distributions of hourly forecasts while maintaining the dependency structure. ... Read more

The Netherlands is a low-lying country in the Rhine-Meuse delta. Because a large part of the Netherlands is situated below sea level, proper management of local and national waterways is a necessity. Polders are used to manage groundwater levels, drain excess rainwater and store water for droughts. Typically, pumping stations in local Dutch polders pump water up to a drainage canal (in Dutch: ‘boezem’).

The Noordzeekanaal—Amsterdam-Rijnkanaal (NZK-ARK) is one such drainage canal, receiving discharge from the Rhine and four local water authorities. The canal connects with the North Sea in IJmuiden, through a pumping station and a set of undershot gates. The combination of pump and gate discharge allow the canal to discharge excess water to the North Sea when the sea water level is both higher and lower than the water level in the canal.

Pump and gate discharge is scheduled through Model Predictive Control (MPC), where reliable forecasts are necessary to reliably schedule discharge. The objectives for the control system of the gates and pumps are likely to become more complex in the future. For example, the availability of renewable energy, or electricity prices are to be taken into account when scheduling pump discharge. Research has shown that regular MPC can lead to suboptimal schedules when uncertainty is introduced, for example leading to high energy costs. Stochastic MPC allows for the consideration of uncertainty in decision making, optimising control actions over a set of possible scenario’s.

One way of generating these scenarios is by using a probabilistic forecasts. A Quantile Regression Deep Neural Network (QR-DNN) can be used to forecast quantiles of a forecast variable. When enough quantiles are considered, a Cumulative Distribution Function (CDF) can be constructed. A Bayesian Network (BN) is a graph-structured network that can estimate multi-dimensional Probability Density Functions by conditionalizing random variables according to a user defined structure and observed data. The BN can be applied to sample from the marginal CDF’s generated by the QR-DNN, while respecting autocorrelation or considering exogenous variables that are not yet considered by the QR-DNN.

In this research, we apply probabilistic forecasting methods to generate pump discharge scenarios that can be used in a stochastic MPC for the NZK-ARK. We use actual data from the four local water authorities discharging into the NZK-ARK, and apply a QR-DNN to generate marginal CDF’s of the expected pump discharge into the NZK-ARK. A BN is then applied to generate scenarios by conditionalizing the marginal CDF’s and take multidimensional samples with autocorrelation. ... Read more

In this manuscript we explore European feature importance in Day Ahead Market (DAM) price forecasting models, and show that model performance can deteriorate when too many features are included due to over-fitting. We propose a greedy algorithm to search over candidate countries for European features to be used in a DAM price forecasting model, that can be applied to several regression and machine learning modelling methodologies. We apply the algorithm to build price forecasting models for the Dutch market, using candidate countries selected through an integrated analysis based on open-source European electricity market data. Feature importance is visualised using an Auto Regressive and Random Forest model. We explain these results using cross-border flow and DAM price data. Two types of models (LEAR and the Deep Neural Network) are considered for the DAM price forecasting with and without European features. We show that in the Dutch case, taking European market integration into account improves the Mean Absolute Error (MAE) of the best performing DAM price forecasting model by 3.1%, the relative MAE (rMAE) by 3.85%, and the Symmetrical Mean Absolute Percentage Error (sMAPE) by 0.31 p.p., compared to the best forecasting model without European features. Through statistical testing we show that European features improve the accuracy of the forecasts with statistical significance. ... Read more

In this paper we propose a Quantile Regression Deep Neural Network capable of forecasting multiple quantiles in one model using a combined quantile loss function, and apply it to probabilistically forecast the prices of 8 European Day Ahead Markets. We show that the proposed loss function significantly reduces the quantile crossing problem to (near) 0% in all markets considered, while in some cases simultaneously increasing forecasting performance based on classical point forecast metrics applied to the expected value of the probabilistic forecast. The models are optimized using an automated approach with an elaborate feature- and hyperparameter search space, leading to good model performance in all considered markets. ... Read more

Among the barriers for renewable energy penetration (SDG 7 and SDG 13) are lack of large scale storage and irregularity and unpredictability of supply. Ties van der Heijden and Edo Abraham have a vision on how water infrastructure in the Dutch delta can contribute to the energy transition with model-based optimisation and ‘demand response services’. ... Read more

Participation in demand response (DR) has been explored for many large energy using assets based on day-ahead markets. However, little is known about the use of multiple energy markets or DR for open canal systems. In this article, we propose the use of multiple flexible energy markets to enable DR for open canal systems in the Netherlands, where many large pumping stations are used for flood mitigation. We observed that the Dutch market is not yet rewarding DR, with relatively low-priced fixed-price contracts. However, when applied to the German market scenario, a cost saving of 13% was found. In conclusion, the method of combining two flexible energy markets seems successful. However, more simulations and research are needed to explore the full potential. ... Read more