The growing share of renewable energy in shortterm European electricity markets has significantly increased congestion management costs and demands. Therefore, current market design is not optional to keep congestion costs low. A proper market would incentivize the integration of flexibilities to boost competition and lower costs, while mitigating risks of manipulation. However, assessing behavioral impacts is challenging due to increasingly interconnected market structures. Studies modeling more than two markets often overlook the strategic opportunities that emerge from these interactions, focusing instead on large-scale dynamics. To capture the detailed impact of bidding strategies, we use reinforcement learning to explore multi-market strategies. By progressively training a Deep Reinforcement Learning (DRL) agent as a market participant - from replicating established behaviors to mastering intricate multimarket interactions - we employ Domain-Informed Curriculum Learning (DomCL), a structured approach that systematically guides learning through staged complexity. We validate our approach against established two-market studies, then evaluate it in two progressively complex four-market case studies spanning a 6-bus network, including historical data. Results show that our DRL-based method improves performance while uncovering challenges that arise as strategic opportunities expand, offering a structured approach for multi-market design analysis.

The design of electricity markets may be facilitated by simulating actors’ behaviors. Recent studies model human decision-makers within markets as agents which learn strategies that maximize expected profits. This work investigates the problem of ‘non-stationarity’ in the context of market simulations, a problem with the learning-algorithms used by such studies which results in agents behaving irrationally, thus limiting the studies’ applicability to real-world strategic behavior. Isolating the source of the problem for a day-ahead electricity market, this paper proposes methods which meliorate this problem in simple test-cases, and proves requirements under which ‘centralized-training, decentralized-execution’ value-learning methods will converge to correct behavior in general. Subsequently, this paper proposes a framework for ‘adversarial market design’ that includes the market-designer as an agent. This allows the optimization of market-designs subject to possibly strategic behavior of participating firms — in turn enabling the automated selection of the optimal market from any set of markets.