Atomic-resolution cryogenic scanning transmission electron microscopy (cryo-STEM) has provided a path to probing the microscopic nature of select low-temperature phases in quantum materials. Expanding cryo-STEM techniques to broadly tunable temperatures will give access to the rich temperature-dependent phase diagrams of these materials. With existing cryo-holders, however, variations in sample temperature significantly disrupt the thermal equilibrium of the system, resulting in large-scale sample drift. The ability to tune the temperature without negative impact on the overall instrument stability is crucial, particularly for high-resolution experiments. Here, we test a new side-entry continuously variable temperature dual-tilt cryo-holder which integrates liquid nitrogen cooling with a 6-pin micro-electromechanical system (MEMS) sample heater to overcome some of these experimental challenges. We measure consistently low drift rates of 0.3-0.4 Å/s and demonstrate atomic-resolution cryo-STEM imaging across a continuously variable temperature range from ∼100 K to well above room temperature. We conduct additional drift stability measurements across several commercial sample stages and discuss implications for further developments of ultra-stable, flexible cryo-stages.