Reward Engineering for RL-Based Initial Qubit Mapping

Abstract

Quantum compilers are necessary to adapt quantum circuits to the connectivity and noise constraints of Noisy Intermediate-Scale Quantum devices. A key step in the compilation process is initial qubit mapping, where logical qubits are assigned to physical qubits. This assignment affects the overall reliability of the circuit, as mappings that place interacting qubits on non-adjacent physical qubits can require additional SWAP gates during routing. Recently, reinforcement learning techniques have been employed to tackle the initial qubit mapping problem. In this setting, the choice of reward function is particularly important, since it defines the objective the RL agent learns to optimize. This raises a central question for RL-based initial mapping: whether graph-level rewards can serve as useful proxies for downstream compiled-circuit quality. This work addresses this question by engineering reward functions based on hardware-distance and hardware-fidelity signals, including a hybrid reward that combines both, and evaluating how well they predict two downstream metrics: compiled SWAP count and estimated success probability (ESP). The same rewards are used to train action-masked PPO agents with terminal, shaped and (n)-step shaped variants on 5-qubit hardware topologies. Overall, the results show that graph-level rewards provide useful learning signals to guide RL-based initial mapping, but remain limited as proxies for full-compilation performance. This highlights the need for reward design that includes more routing relevant information, such as repeated interactions and gate ordering, to better approximate the effects of routing on final compiled-circuit quality.