Infrared and visible image fusion with ResNet and zero-phase component analysis

Hui Li, Xiao jun Wu, Tariq S. Durrani

Research output: Contribution to journalArticle

12 Citations (Scopus)

Abstract

In image fusion approaches, feature extraction and processing are key tasks, and the fusion performance is directly affected by the different features and processing methods undertaken. However, most of deep learning-based methods use deep features directly without them. This leads to the fusion performance degradation in some cases. To solve these drawbacks, in our paper, a deep features and zero-phase component analysis (ZCA) based novel fusion framework is proposed. Firstly, the residual network (ResNet) is used to extract deep features from source images. Then ZCA and l1-norm are utilized to normalize the deep features and obtain initial weight maps. The final weight maps are obtained by employing a soft-max operation in association with the initial weight maps. Finally, the fused image is reconstructed using a weighted-averaging strategy. Compared with the existing fusion methods, experimental results demonstrate that the proposed framework achieves better performance in both objective assessment and visual quality. The code of our fusion algorithm is available at https://github.com/hli1221/imagefusion_resnet50.

Original languageEnglish
Article number103039
Number of pages10
JournalInfrared Physics and Technology
Volume102
Early online date12 Sep 2019
DOIs
Publication statusPublished - 30 Nov 2019

Keywords

  • deep learning
  • image fusion
  • infrared image
  • residual network
  • visible image
  • zero-phase component analysis

Fingerprint Dive into the research topics of 'Infrared and visible image fusion with ResNet and zero-phase component analysis'. Together they form a unique fingerprint.

Cite this