Deep learning for thermal-RGB image-to-image translation

Thermal infrared (TIR) and visible (RGB) images offer complementary information, and present different challenges. We report a deep learning approach to translate images from TIR to RGB, and vice-versa, enabling the extraction of information from both modalities with a single camera. We collected a diverse dataset of approximate to 150, , 000 TIR-RGB image pairs, encompassing day, night, dusk, and a variety of scenes. The pix2pix network achieved average structural similarity index (SSIM) of 0.58 for TIR-to-RGB and 0.72 for RGB-to-TIR, outperforming unpaired-data translation. Our contributions include: (1) creating the largest and most diverse publicly available dataset of TIR-RGB image pairs, and (2) demonstrating efficient translation between TIR and RGB image modalities across diverse global priors. This work advances applications including low-light imaging, thermal sensing, and multi-band scene reconstruction with a single camera.