Objective: Recent studies have suggested an association between age-related hearing loss and cognitive decline. Yet, the underlying mechanism explaining this relation remains unknown. In this regard, several studies investigated gray matter (GM) differences in age-related hearing loss but presented inconsistent results regarding the association and regions involved. To our knowledge, a data-driven approach for exploring this association has not been performed. Therefore, we aimed to investigate possible GM differences and regions involved in age-related hearing loss using conventional multivariable linear regression and deep learning. Methods: Within the population-based Rotterdam Study, 2070 participants (mean age: 65.5 years) underwent pure-tone audiometry to quantify hearing thresholds (hearing loss [> 40 dB], n=205; normal-hearing controls [< 20 dB], n=822). Magnetic resonance (MR) imaging was performed to obtain GM volumes of the superior temporal and precentral gyrus, and GM modulated images. Using multivariable linear regression we investigated the associations between age-related hearing loss and GM volume in the superior temporal and precentral gyrus. A convolutional neural network (CNN) was trained to classify hearing loss and normal-hearing controls based on GM modulated images of the whole brain and the region around the superior temporal gyrus. Visualization of relevant features for the classification was performed with gradient-weighted activation mapping (Grad-CAM).Results: We found that age-related hearing loss was significantly associated with smaller GM volumes in the right hemisphere of both the superior temporal gyrus (difference in standardized brain volume per dB increase: -0.006 [95$\%$ CI: -0.010, -0.003]) and precentral gyrus (difference: -0.005 [95$\%$ CI: -0.008, -0.001]). The CNN classification performance ranged between 0.89 and 0.96 area under the receiver-operating characteristic curves. Analysis of relevant features for the classification showed that features were not specific to the superior temporal gyrus or primary auditory cortex, but appeared across the whole brain. Furthermore, we noticed that misclassified subjects were significantly related to age. Conclusion: This study shows that age-related hearing loss is related to both GM volume in the superior temporal and precentral gyrus. Moreover, relevant features for the classification of age-related hearing loss were observed across the whole brain. These results may be explained by a third factor affecting both hearing loss and neurodegeneration. As age likely is the third factor involved, a longitudinal study design or age-matched groups are required in further studies on age-related hearing loss.