Optical antennas enable efficient coupling between propagating light and bonding electromagnetic waves like surface plasmon polaritons (SPPs). Under the illumination of inhomogeneous optical fields, propagating SPPs mediated by multimode antennas could be spatially asymmetric and the asymmetry strongly depends on the position of the antennas relative to the illumination field. Here we develop such asymmetric excitation of SPPs through illuminating a pair of slot antennas with the (1,0) mode Hermite-Gaussian beam. The physical scenario of the interaction between the illumination optical field and the paired slot antennas are elaborated by full-wave electromagnetic simulations. We also carry out experiments to monitor the asymmetric SPPs propagation with a back-focal plane imaging technique. By retrieving the asymmetric intensity ratio of the SPP pattern in the back-focal plane image, lateral displacement of the antennas down to angstrom level is demonstrated.