Exploring the capability of text-to-image diffusion models with structural edge guidance for multi-spectral satellite image inpainting

Research output: Contribution to journalLetterpeer-review

32 Downloads (Pure)

Abstract

The letter investigates the utility of text-to-image inpainting models for satellite image data. Two technical challenges of injecting structural guiding signals into the generative process as well as translating the inpainted RGB pixels to a wider set of MSI bands are addressed by introducing a novel inpainting framework based on StableDiffusion and ControlNet as well as a novel method for RGB-to-MSI translation. The results on a wider set of data suggest that the inpainting synthesized via StableDiffusion suffers from undesired artifacts and that a simple alternative of self-supervised internal inpainting achieves a higher quality of synthesis.

Original languageEnglish
Article number5001905
Number of pages5
JournalIEEE Geoscience and Remote Sensing Letters
Volume21
Early online date26 Feb 2024
DOIs
Publication statusE-pub ahead of print - 26 Feb 2024

Keywords

  • image inpainting
  • image completion
  • generative models

Fingerprint

Dive into the research topics of 'Exploring the capability of text-to-image diffusion models with structural edge guidance for multi-spectral satellite image inpainting'. Together they form a unique fingerprint.

Cite this