To study the irreversible changes in the magnetic stray field surrounding a steel structure caused by impact-induced elastic and plastic deformations, a steel cylinder was repeatedly subjected to axial impacts of various magnitudes. Due to impacts that induce elastic deformation, the measured magnetic stray field of the structure converges to a global magnetic equilibrium. However, as soon as plastic deformation develops, a deviation from this trend is observed. From the spatial distribution of the stray field, the location of the plastic deformation is determined. Subsequently, the underlying processes of the measured evolution of the stray field are discussed and successfully incorporated into an elementary model of the structure's magnetisation to simulate the results from the experiment. It is expected that the reported observation is useful for a class of engineering applications in which non-contact and non-collocated measurements can be utilised to identify structural damage under dynamic loading.