15 Downloads (Pure)

Abstract

Optimizing images as output of a neural network has been shown to introduce a powerful prior for image inverse tasks, capable of producing solutions of reasonable quality in a fully internal learning context, where no external datasets are involved. Two potential technical approaches involve fitting a coordinate-based Multilayer Perceptron (MLP), or a Convolutional Neural Network to produce the result image as output. The aim of this work is to evaluate the two counterparts, as well as a new framework proposed here, named Neural Knitwork, which maps pixel coordinates to local texture patches rather than singular pixel values. The utility of the proposed technique is demonstrated on the tasks of image inpainting, super-resolution, and denoising. It is shown that the Neural Knitwork can outperform the standard coordinate-based MLP baseline for the tasks of inpainting and denoising, and perform comparably for the super-resolution task.

Original languageEnglish
Article number110378
Number of pages9
JournalPattern Recognition
Volume151
Early online date4 Mar 2024
DOIs
Publication statusE-pub ahead of print - 4 Mar 2024

Keywords

  • generative models
  • image denoising
  • image inpainting
  • image super-resolution
  • image synthesis
  • internal learning
  • zero-shot learning

Fingerprint

Dive into the research topics of 'Neural knitworks: patched neural implicit representation networks'. Together they form a unique fingerprint.

Cite this