Increasing shares of renewable generation are leading to more volatile electricity prices, presenting an opportunity for Energy Storage Systems (ESS) participating in short-term electricity markets. Model Predictive Control (MPC) has been shown to be a powerful tool to leverage the latest information at the time of optimization, yet its efficacy depends on the quality of the employed price forecasts. So far, these forecasts have been developed with traditional forecasting methods instead of value-oriented approaches, which consider the downstream decision problem during the forecaster training phase. Existing value-oriented methods, however, often rely on a specific downstream problem structure. This paper addresses these shortcomings by introducing a universally applicable, value-oriented forecasting methodology that employs a generalized loss function designed to account for inter-temporal price variability, using the downstream value (i.e., profit from ESS market participation) as the selection criterion in the hyperparameter tuning step. The proposed methodology is tested on a case study considering different types of ESS participating in the Belgian balancing market through MPC. The method is benchmarked against other forecasting techniques including a neural network trained in traditional, accuracy-oriented fashion. Using real-life data over a test set of two months, we show that the methodology outperforms those traditional techniques in terms of ex-post out-of-sample profit.

Energy Storage Systems (ESS) play a crucial role in managing renewable energy variability. Forecast-informed optimization is typically used to maximize ESS profit in electricity markets. Whereas traditional forecaster training methods use accuracy-based loss functions, Decision-Focused Learning uses a task-aware loss function with the aim of improving ESS profits. This can be achieved by integrating the downstream optimization in the forecaster training procedure. A common task-aware loss function is the Smart Predict-then-Optimize (SPO+) loss. However, its current implementation is prone to overfitting and is limited to linear forecasting models. Here, we extend the SPO+ framework to neural network forecasters with non-linear activation functions while introducing an interior-point training method to mitigate overfitting risks. When applied to an ESS participating in the day-ahead market, our approach outperforms both traditional and other decision-focused benchmarks in terms of obtained ESS profits.

In competitive electricity markets, the optimal bid or offer problem of a strategic agent is commonly formulated as a bi-level program and solved as a mathematical program with equilibrium constraints (MPEC). If the lower-level (LL) part of the problem can be well approximated as a convex problem, this approach leads to a global optimum. However, electricity markets are governed by non-convex (partially known) constraints and reward functions of the participating agents. In this paper, an alternative data-driven paradigm, labeled as a mathematical program with neural network constraint (MPNNC), is developed. The method uses a neural network to represent the mapping between the upper-level (agent) decisions and the lower-level (market) outcomes, i.e., it replaces the lower-level problem with a surrogate model. In the presented case studies, the proposed model is used to find the optimal load shedding strategy of a strategic load-serving entity. First, the MPNNC performance is compared to the MPEC approach, both in convex and non-convex environments, showing that the proposed MPNNC achieves similar performance to an ideal MPEC that has perfect knowledge of the simulated market environment. Then, aggregated supply curves from the Belgian spot exchange are used to assess the potential gains of using the developed model in real-life applications.

Demand Response (DR) programs offer flexibility that is considered to hold significant potential for enhancing power system reliability and promoting the integration of renewable energy sources. Nevertheless, the distributed nature of DR resources presents challenges in developing scalable optimization tools. This paper explores a novel data-driven approach in which DR resources are modeled through their aggregate forecasts using Inverse Optimization. The proposed method utilizes historical price-consumption data to deduce DR price-response behavior via a flexibility curve. The model is assessed within the Belgian single imbalance market context, where a Balance Responsible Party (BRP) employs the inferred flexibility curve to optimize its strategic imbalance positions by managing DR resources through suitable real-time price signals. The accuracy of the estimated flexibility provided by the proposed algorithm is evaluated by comparing it with the XGboost method. The results demonstrate that the model can effectively capture DR behavior and generate profit from providing balancing energy.

This paper formulates an energy community's centralized optimal bidding and scheduling problem as a time-series scenario-driven stochastic optimization model, building on real-life measurement data. In the presented model, a surrogate battery storage system with uncertain state-of-charge (SoC) bounds approximates the portfolio's aggregated flexibility. First, it is emphasized in a stylized analysis that risk-based energy constraints are highly beneficial (compared to chance-constraints) in coordinating distributed assets with unknown costs of constraint violation, as they limit both violation magnitude and probability. The presented research extends state-of-the-art models by implementing a worst-case conditional value at risk (WCVaR) based constraint for the storage SoC bounds. Then, an extensive numerical comparison is conducted to analyze the trade-off between out-of-sample violations and expected objective values, revealing that the proposed WCVaR based constraint shields significantly better against extreme out-of-sample outcomes than the conditional value at risk based equivalent. To bypass the non-trivial task of capturing the underlying time and asset-dependent uncertain processes, real-life measurement data is directly leveraged for both imbalance market uncertainty and load forecast errors. For this purpose, a shape-based clustering method is implemented to capture the input scenarios' temporal characteristics.

To represent the cross-border exchange capacities defined by the flow-based approach in the European resource adequacy assessments, transmission system operators currently employ a data-driven methodology that consists of sequential clustering and correlation steps. This methodology entails assumptions and simplifications within both clustering and correlation analyses that may lead to an erroneous representation of import-export capacities in the subsequent adequacy assessments. While the first stage of this methodology can be improved by leveraging a clustering technique tailored to adequacy assessments, the correlation step presents a poor performance in terms of accuracy and scalability. To address the latter challenges, this paper proposes a supervised learning-based model that can enhance the mapping between several relevant explanatory variables and the pre-clustered flow-based domains, leading to a more accurate representation of the flow-based domains in adequacy assessments. Furthermore, the current paper leverages supervised learning to develop a single-step approach that directly maps the selected explanatory variables to the flow-based domains using the K-Nearest Neighbors algorithm, eliminating the clustering step. This circumvents inaccuracies introduced by the significant intra-cluster discrepancies due to numerous shapes and forms of the flow-based domains and enables an enhanced modeling of the flow-based domains in adequacy assessments. In an extensive case study, we demonstrate that the proposed single-step model can significantly improve the accuracy of adequacy assessments, compared to the best-in-class result obtained by the two-step set-up. Moreover, the proposed single-step model involves no hyper-parameters, eliminates the computational complexity of the two-step set-up, and efficiently upscales to integrate the new zones joining to the flow-based market coupling.

This paper presents new risk-based constraints for the participation of an energy community in day-ahead and real-time energy markets. Forming communities offers indeed an effective way to manage the risk of the overall portfolio by pooling individual resources and associated uncertainties. However, the diversity of flexible resources and the related userspecific comfort constraints make it difficult to properly represent flexibility requirements and to monetize constraint violations. To address these issues, we propose a new risk-aware probabilistic enforcement of flexibility constraints using the conditional-valueat- risk (CVaR). Next, an extended version of the model is introduced to mitigate the distributional ambiguity faced by the community manager when new sites with limited information are embedded in the portfolio. This is achieved by defining the worstcase CVaR based-constraint (WCVaR-BC) that differentiates the CVaR value among different sub-clusters of clients. Both reformulations are linear, thus allowing to tackle large-scale stochastic problems. The proposed risk-based constraints are then trained and evaluated on real data collected from several industrial sites. Our findings indicate that using the WCVaRBC leads to systematically higher out-of-sample reliability, while decreasing the exposure to extreme outcomes.