We have studied theoretically and experimentally the influence of a dielectric substrate on the frequency-dependent terahertz electric near-field of a small hole in a metal layer. We find that the near-field transmission spectrum and the two-dimensional field distribution of an empty hole in a thin metal layer on a substrate are almost identical to that of a hole which is also filled with the same dielectric material as the substrate. For thicker metal layers, however, the near-field spectra of filled and unfilled holes become very different. In addition, for thick metal layers, the two-dimensional field distributions are more strongly affected by the substrate, especially if we allow for an air gap between the metal and the substrate. Our results validate the -somewhat unusual- two-dimensional field distribution measured beneath a hole in a thick metal foil and highlight the effect that a substrate can have on the measurement of the near-field of an object.