The decentralisation of energy supply, largely driven by renewable energy sources, has led to increased volatility and imbalances in the power grid. To maintain grid stability, flexible balancing mechanisms are required. Ancillary services offer this flexibility by either injecting power (upward regulation) or absorbing it (downward regulation). Among these services, the Automatic Frequency Restoration Reserve (aFRR) plays a crucial role due to its large regulating capacity and direct impact on imbalance prices through its activation costs. As market volatility increases, short-term trading in intra-day and imbalance markets becomes more important, increasing the need for accurate forecasting. While day-ahead and intra-day market forecasting is well-established in Electricity Price Forecasting (EPF), there is limited research on imbalance markets, especially the Dutch aFRR market, highlighting the novelty of this study. The Dutch market uses a dual pricing system with separate bid ladders for upward and downward regulation.

This research focuses on forecasting aFRR bid ladders in the Dutch electricity market and explores their application in short-term trading strategies. Forecasts are generated three hours ahead of delivery, aligning with decision points for intra-day trading, aFRR participation, or opting for no action. The study includes a detailed market analysis, a review of forecasting methods, the development of a machine learning framework tailored to the aFRR market, evaluation of forecast performance, and the practical use of these forecasts in trading scenarios.

A structured machine learning pipeline is designed, encompassing data pre-processing, transformation, model selection, prediction, and evaluation. During the transformation phase, data is scaled and then reduced in dimensionality using Principal Component Analysis (PCA) to retain key variance. The resulting components are used as inputs for predictive models, including LASSO, XGBoost, and LSTM, which are benchmarked against preliminary bid ladders published three hours before delivery. Forecast accuracy is evaluated using point metrics (sMAPE), interval metrics (PICP and PINAW), and a novel self-developed metric called the Largest Knick Volume (LKV), which captures accuracy at key inflection points in the bid ladders that are most relevant for short-term trading.

The findings are twofold. First, model evaluations both at the PCA level and on the reconstructed bid ladders indicate that all models can track general market trends but do not outperform the benchmark. Diebold-Mariano tests confirm that benchmark performance is superior at the PCA level. After reconstructing the bid ladders, benchmark sMAPE scores are 7% for upward and 8% for downward regulation, outperforming LASSO (7%/10%), XGBoost (8%/11%), and LSTM (7%/11%). Second, the forecasts are integrated into Battery Energy Storage System (BESS) intra-day trading strategies, including one based solely on intra-day prices and three incorporating different aFRR bid ladder positions: gas turbine marginal cost, intra-day price with a premium, and LKV-based positioning. Integrating aFRR forecasts improves trading performance, with some high volume-price strategies boosting revenue by up to 18%.

Although the developed forecasting methods do not outperform the benchmark, the benchmark itself already captures much of the relevant market information available three hours ahead, limiting the scope for additional forecasting improvements. This limitation is attributed to the weak correlation between market fundamentals and bid outcomes, as well as the strong influence of individual actors in the relatively small Dutch aFRR market. Nonetheless, the study shows that using existing market data and participating across multiple markets can improve profitability. The results support the strategic value of data-driven, forecast-informed bidding for enhancing the economic performance of battery storage systems.