Neural knitworks: patched neural implicit representation networks

Mikolaj Czerkawski*, Javier Cardona, Robert Atkinson, Craig Michie, Ivan Andonovic, Carmine Clemente, Christos Tachtatzis

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

44 Downloads (Pure)

Abstract

Optimizing images as output of a neural network has been shown to introduce a powerful prior for image inverse tasks, capable of producing solutions of reasonable quality in a fully internal learning context, where no external datasets are involved. Two potential technical approaches involve fitting a coordinate-based Multilayer Perceptron (MLP), or a Convolutional Neural Network to produce the result image as output. The aim of this work is to evaluate the two counterparts, as well as a new framework proposed here, named Neural Knitwork, which maps pixel coordinates to local texture patches rather than singular pixel values. The utility of the proposed technique is demonstrated on the tasks of image inpainting, super-resolution, and denoising. It is shown that the Neural Knitwork can outperform the standard coordinate-based MLP baseline for the tasks of inpainting and denoising, and perform comparably for the super-resolution task.

Original languageEnglish
Article number110378
Number of pages9
JournalPattern Recognition
Volume151
Early online date4 Mar 2024
DOIs
Publication statusPublished - 31 Jul 2024

Keywords

  • generative models
  • image denoising
  • image inpainting
  • image super-resolution
  • image synthesis
  • internal learning
  • zero-shot learning

Fingerprint

Dive into the research topics of 'Neural knitworks: patched neural implicit representation networks'. Together they form a unique fingerprint.

Cite this