We identify the presence of monatomic steps at the Si/SiGe or Si/SiO2 interface as a dominant source of variations in the dephasing time of silicon (Si) quantum dot (QD) spin qubits. First, using atomistic tight-binding calculations we show that the g-factors and their Stark shifts undergo variations due to these steps. We compare our theoretical predictions with experiments on QDs at a Si/SiO2 interface, in which we observe significant differences in Stark shifts between QDs in two different samples. We also experimentally observe variations in the g-factors of one-electron and three-electron spin qubits realized in three neighboring QDs on the same sample, at a level consistent with our calculations. The dephasing times of these qubits also vary, most likely due to their varying sensitivity to charge noise, resulting from different interface conditions. More importantly, from our calculations we show that by employing the anisotropic nature of the spin-orbit interaction (SOI) in a Si QD, we can minimize and control these variations. Ultimately, we predict that the dephasing times of the Si QD spin qubits will be anisotropic and can be improved by at least an order of magnitude, by aligning the external dc magnetic field towards specific crystal directions, given other decoherence mechanisms do not dominate over charge noise.