In this work, we investigate how domain adaptation techniques can improve the performance of hand landmark detection models originally trained on RGB images when deployed on infrared (IR) data. Our motivation stems from a medical use case in Nepal, where clinicians require reliable temperature estimation at hand keypoints to detect early signs of leprosy. We evaluate three methods on a small IR dataset (80 labeled images & 5000 unlabeled frames): a shallow adaptation (AdaBN), a deep alignment approach (Deep CORAL), and a test-time subspace alignment method (SSA). Our experiments show that while AdaBN and SSA yield moderate improvements, Deep CORAL achieves stronger gains through targeted training of specific model components. The combination of these methods produces superior results, yielding an 11% improvement in percentage of correct keypoints (PCK@0.05) on our custom annotated IR dataset. These findings demonstrate that combining lightweight and deep domain adaptation approaches can effectively enhance IR hand landmark detection accuracy without requiring large labeled datasets, enabling practical deployment for clinical thermal imaging in resource-limited settings.