This study compares the degradation behavior of the alkali-activated slag/fly ash (AASF) and ordinary Portland cement (OPC) mortars exposed to phosphoric acid with different pH values. The experimental results show that AASF mortars exhibit better resistance than OPC mortars against surface damage, although both systems get white deposits on the surface in phosphoric acid with a relatively high pH level. AASF mortars obtained lower mass loss than OPC mortars in phosphoric acid with pH at 2 and 3. The strength reduction in AASF mortars after immersion in phosphoric acid is more significant than that in OPC mortars. However, total degradation depth of AASF was smaller than that of OPC regardless of the pH of the acid solutions. Based on the experimental data, linear relationships were identified between the slope of degradation depth–mass loss curves and the Al/Si and Ca/Si ratios of the binders. This may indicate a new way to assess the degradation behavior of AASF and OPC based on their chemical compositions.