In the time domain, the radio sky in particular along the Galactic plane direction may vary significantly because of various energetic activities associated with stars, stellar, and supermassive black holes. Multi-epoch Very Large Array surveys of the Galactic plane at 5.0 GHz enabled the finding of a catalogue of 39 variable radio sources in the flux density range 1-70 mJy. To probe their radio structures and spectra, we observed 17 sources with the very-long-baseline interferometric (VLBI) imaging technique and collected additional multifrequency data from the literature. We detected all of the sources at 5 GHz with the Westerbork Synthesis Radio Telescope, but only G23.6644-0.0372 with the European VLBI Network (EVN). Together with its decadal variability and multifrequency radio spectrum, we interpret it as an extragalactic peaked-spectrum source with a size of ≲10 pc. The remaining sources were resolved out by the long baselines of the EVN because of either strong scatter broadening at the Galactic latitude < 1° or intrinsically very extended structures on centi-arcsec scales. According to their spectral and structural properties, we find that the sample has a diverse nature. We notice two young H ii regions and spot a radio star and a candidate planetary nebula. The rest of the sources are very likely associated with radio active galactic nuclei (AGNs). Two of them also display arcsec-scale faint jet activity. The sample study indicates that AGNs are common place even among variable radio sources in the Galactic plane.