An effective method for supporting the large volume of information required for future vehicular networks is leveraging caching techniques as well as relying on millimeter-wave (mmWave) frequencies. However, characterizing such a system under mmWave directional beamforming and vehicular mobility is a complex task. In this article, we propose the first stochastic geometry framework for V2X caching in mmWave networks. In addition to common parameters considered in stochastic geometry models, our derivations account for caching as well as the speed and the trajectory of the vehicles. Furthermore, our evaluations provide interesting design insights: (i) higher base station/vehicle densities does not necessarily improve caching performance; (ii) although using a narrower beam leads to a higher SINR, it also reduces the connectivity probability; and (iii) V2X caching can be an inexpensive way of compensating some of the unwanted mmWave channel characteristics.