Unconventional ferroelectricity exhibited by hafnia-based thin films-robust at nanoscale sizes-presents tremendous opportunities in nanoelectronics. However, the exact nature of polarization switching remains controversial. We investigated a La0.67Sr0.33MnO3/Hf0.5Zr0.5O2 capacitor interfaced with various top electrodes while performing in situ electrical biasing using atomic-resolution microscopy with direct oxygen imaging as well as with synchrotron nanobeam diffraction. When the top electrode is oxygen reactive, we observe reversible oxygen vacancy migration with electrodes as the source and sink of oxygen and the dielectric layer acting as a fast conduit at millisecond time scales. With nonreactive top electrodes and at longer time scales (seconds), the dielectric layer also acts as an oxygen source and sink. Our results show that ferroelectricity in hafnia-based thin films is unmistakably intertwined with oxygen voltammetry.