In this study, we describe a method for measuring the spectral reflectance of a paint layer at both the surface and in the volume of the paint layers. We first present a fringes model which illustrates the possibilities for spectral reconstruction using a Short-Time-Fourier-Transform algorithm. We investigate the remaing percentage errors and identified that there is a strong fluctuation along the wavelength range of the spectrometer. Then, we demonstrate the validity of our approach experimentally by measuring the spectral reflectance of a paint layer using a custom-made visible light optical coherence tomography system. There, we reconstruct the spectral reflectance of a paint layer by probing the surface and a depth range below the surface. Finally, we show the importance to include a wavelength sensitive correction in the reconstruction for taking into account the spectral shape of the light in the reference path of the interferometer. This work is part of the Down To The Ground project, in which the results of the OCT inspection will be used directly by a consortium of technical art historians and conservators.