Hafnia-based thin films exhibit unconventional ferroelectricity. These materials also show rich polymorphism, and thus temperature and field-driven phase transitions, as well as oxygen migration. In a bigger context of exploring the synergy between ferroelectricity and diffusion-based structural phenomena, here we study temperature-dependent phase transitions in epitaxial Hf0.5Zr0.5O2(HZO)/La0.67Sr0.33MnO3 (LSMO, bottom electrode) heterostructures. We report topotactic phase transitions and their clear pathways in both LSMO and HZO layers upon heating under vacuum, using in situ scanning transmission electron microscopy (STEM). Specifically, we directly image oxygen and cationic columns using integrated differential phase contrast STEM and follow their evolution with temperature. We also perform in situ high temperature X-ray diffraction in air and show that the LSMO layer undergoes reversible thermal expansion and contraction when heated up to 850 °C, whereas HZO undergoes strain relaxation beyond 750 °C without any reversible phase transition. Our results provide a comprehensive and direct understanding of temperature-dependent structure, defect, and property correlations in these systems.