MCINet: Fusing Low-Light Visible-Infrared Image via Max-Merge Complementary Information

Fusing complementary information in the visible-infrared image offers a promising approach to enhance the performance of downstream computer vision tasks (e.g., object detection, segmentation etc) in complicated imaging conditions (e.g., low-illumination). However, due to the robust imaging capacity of the infrared sensor in complicated imaging conditions, most existing methods primarily rely on the salient object intensity information in the infrared modality for fusion, while the visible information (e.g., color, texture etc) is not adequately utilized, and thus limit their generalization capacity in downstream computer vision tasks. In this study, we present a novel image fusion framework, i.e.,MCInet, which attempts toMaximize and merge theComplementaryInformation across visible-infrared modalities for more informative image fusion. To this end, we first introduce the modality-specific processing module into the fusion framework to improve the information representation of each modality image. For visible images, a pre-trained low-light enhance module is adopted to enhance its color and texture information. In addition, for infrared images, a nonlinear mapping module is constructed to suppress the excessive salient object intensity information of infrared modality. Then we establish a reusable MCI block that embeds a cross-image mutual information minimization scheme into an input-aware fusion module. This empowers us to dynamically maximize and merge the complementary information between two input images according to their feature representation. In addition, we introduce a cycle reconstruction loss to self-supervised regularize the fusion results for further enhancement. Experiments on image fusion, object detection, and segmentation tasks demonstrate that the proposed framework can produce more informative fusion results and exhibit better performance in downstream computer vision tasks.