The reduction of iron ore with carbon-carriers is one of the largest sources of greenhouse gas emissions in the industry, motivating global activities to replace the coke-based blast furnace reduction by hydrogen-based direct reduction (HyDR). Iron oxide reduction with hydrogen has been widely investigated both experimentally and theoretically. The HyDR process includes multiple types of chemical reactions, solid state and defect-mediated diffusion (of oxygen and hydrogen species), several phase transformations, as well as massive volume shrinkage and mechanical stress buildup. However, studies focusing on the chemo-mechanical interplay during the reduction reaction influenced by microstructure are sparse. In this work, a chemo-mechanically coupled phase-field (PF) model has been developed to explore the interplay between phase transformation, chemical reaction, species diffusion, large elasto-plastic deformation and microstructure evolution. Energetic constitutive relations of the model are based on the system free energy which is calibrated with the help of a thermodynamic database. The model has been first applied to the classical core-shell (wüstite-iron) structure. Simulations show that the phase transformation from wüstite to α-iron can result in high stresses and rapidly decelerating reaction kinetics. Mechanical stresses create elastic energy in the system, an effect which can negatively influence the phase transformations, thus causing slow reaction kinetics and low metallization. However, if the elastic stress becomes comparatively high, it can shift the shape of the free energy from a double-well to a single-well case, speed up the transformation and result in a higher reduction degree compared to the low-stress double-well case. The model has been applied to simulate an experimentally characterized iron oxide specimen with its complex microstructure. The observed microstructure evolution during reduction is well predicted by the model. The simulation results also show that isolated pores in the microstructure are filled with water vapor during reduction, which can influence the local reaction atmosphere and dynamics.