Tunneling spectroscopy measurements are often used to probe the energy spectrum of Andreev bound states (ABSs) in semiconductor-superconductor hybrids. Recently, this spectroscopy technique has been incorporated into planar Josephson junctions (JJs) formed in two-dimensional electron gases, a potential platform to engineer phase-controlled topological superconductivity. Here, we perform ABS spectroscopy at the two ends of planar JJs and study the effects of the magnetic vector potential on the ABS spectrum. We show that the local superconducting phase difference arising from the vector potential is equal in magnitude and opposite in sign at the two ends, in agreement with a model that assumes localized ABSs near the tunnel barriers. Complemented with microscopic simulations, our experiments demonstrate that the local phase difference can be used to estimate the relative position of localized ABSs separated by a few hundred nanometers.