Image fusion based on generative adversarial network consistent with perception

Yu Fu, Xiao-Jun Wu, Tariq Durrani

Research output: Contribution to journalArticlepeer-review

Abstract

Deep learning is a rapidly developing approach in the field of infrared and visible image fusion. In this context, the use of dense blocks in deep networks significantly improves the utilization of shallow information, and the combination of the Generative Adversarial Network (GAN) also improves the fusion performance of two source images. We propose a new method based on dense blocks and GANs, and we directly insert the input image-visible light image in each layer of the entire network. We use structural similarity and gradient loss functions that are more consistent with perception instead of mean square error loss. After the adversarial training between the generator and the discriminator, we show that a trained end-to-end fusion network – the generator network – is finally obtained. Our experiments show that the fused images obtained by our approach achieve good score based on multiple evaluation indicators. Further, our fused images have better visual effects in multiple sets of contrasts, which are more satisfying to human visual perception.

Original languageEnglish
Pages (from-to)110-125
Number of pages16
JournalInformation Fusion
Volume72
Early online date27 Feb 2021
DOIs
Publication statusE-pub ahead of print - 27 Feb 2021

Keywords

  • dense block
  • generative adversarial networks
  • image fusion
  • infrared image
  • visible image

Fingerprint Dive into the research topics of 'Image fusion based on generative adversarial network consistent with perception'. Together they form a unique fingerprint.

Cite this