For the practical diagnosis of inflammatory respiratory diseases, achieving sensitive and rapid NO sensing at the parts per billion level, all at room temperature, is of great significance. Herein, we developed a chemiresistor gas sensor with a sheet-on-sheet structure composed of an amorphous Cu-hemin MOF with reduced graphene oxide (rGO) nanosheets. The SEM images show that the Cu-hemin MOF/rGO composite exhibits a two-dimensional sheet-like structure. Due to its nanosized architecture, the Cu-hemin MOF exhibits a significant number of active sites for efficient NO detection. The Cu-hemin MOF/rGO composite material exhibited excellent NO sensing performance, including high sensitivity (Ra/Rg = 1.06, 50 ppb), reliable repeatability, high selectivity, and fast response/recovery (43 s/367 s, 10 ppm). The mechanism study revealed that the formation of the MOF altered the hemin dimer’s structure, resulting in the release of additional Fe(III)–N4 active sites and improved sensitivity. Moreover, the incorporation of rGO significantly boosted the conductivity of Cu-hemin MOFs. Using this two-dimensional sheet-like material, a mask-type sensor was also prepared and verified to be effective as a flexible and wearable sensing device for parts per billion level exhaled NO detection.