Observation and Action Encodings for Reinforcement Learning–Based Qubit Routing
A Controlled Ablation Study in qgym
A. Durmaz (TU Delft - Electrical Engineering, Mathematics and Computer Science)
S. Feld – Mentor (TU Delft - QCD/Feld Group)
A. Kundu – Mentor (TU Delft - QCD/Feld Group)
M.T.J. Spaan – Mentor (TU Delft - Electrical Engineering, Mathematics and Computer Science)
A. Lukina – Graduation committee member (TU Delft - Electrical Engineering, Mathematics and Computer Science)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Running a quantum circuit on hardware with limited qubit connectivity requires inserting SWAP gates, each of which adds depth and exposes qubits to decoherence, so that two qubits that must interact become physically adjacent on the chip. Reinforcement learning (RL) is an increasingly used adaptive alternative to hand-engineered routing heuristics, but how an RL agent’s observation and action encodings should be designed has received little attention, even though environments such as qgym leave both choices to the user.
We study these two encodings one at a time. The first is observation reach, or how many upcoming gates the agent can see; the second is action-space granularity, ranging from single SWAPs through a heuristically pruned set to multi-SWAP macro actions. Holding the reward, algorithm, hyperparameters, and hardware constant, we evaluate both on the coupling graphs of a 7-qubit and a 16-qubit IBM device.
Observation reach has little effect on routing, and the full-circuit view performs poorly on the larger device. Action-space granularity matters much more: macro actions solve substantially more circuits and route them with fewer SWAPs, and this advantage grows with device size. Injecting the same routing heuristic as a soft prior on the action distribution, rather than as a hard mask, preserves completeness while roughly halving SWAP overhead. A disjoint held-out evaluation suggests these results reflect transferable routing skill rather than memorized training circuits.