Gantry cranes play a pivotal role in various industrial applications, and their reliable operation is paramount. While routine inspections are standard practice, certain defects, particularly in less accessible components, remain challenging to detect early. In this study, first a finite element model is presented, and the damage is introduced using random changes in the stiffness of different parts of the structure. Contrary to the assumption of inherent reliability, undetected defects in crucial structural elements can lead to catastrophic failures. Then, the vibration equations of healthy and damaged models are analyzed to find the displacement, velocity, and acceleration of the different crane parts. The learning vector quantization neural network is used to train and detect the defects. The output is the location of the damage and the damage severity. Noisy data are then used to evaluate the network performance robustness. This research also addresses the limitations of traditional inspection methods, providing early detection and classification of defects in gantry cranes. The study’s relevance lies in the need for a comprehensive and efficient damage detection method, especially for components not easily accessible during routine inspections.