Unsupervised spiking instance segmentation on event data using STDP features

Research output: Contribution to journalArticlepeer-review

10 Downloads (Pure)

Abstract

Spiking Neural Networks (SNN) and the field of Neuromorphic Engineering has brought about a paradigm shift in how to approach Machine Learning (ML) and Computer Vision (CV) problem. This paradigm shift comes from the adaption of event-based sensing and processing. An event-based vision sensor allows for sparse and asynchronous events to be produced that are dynamically related to the scene. Allowing not only the spatial information but a high-fidelity of temporal information to be captured. Meanwhile avoiding the extra overhead and redundancy of conventional high frame rate approaches. However, with this change in paradigm, many techniques from traditional CV and ML are not applicable to these event-based spatial-temporal visual streams. As such a limited number of recognition, detection and segmentation approaches exist. In this paper, we present a novel approach that can perform instance segmentation using just the weights of a Spike Time Dependent Plasticity trained Spiking Convolutional Neural Network that was trained for object recognition. This exploits the spatial and temporal aspects of the SpikeSEG network's internal feature representations adding this new discriminative capability. We highlight the new capability by successfully transforming a single class unsupervised network for face detection into a multi-person face recognition and instance segmentation network.

Original languageEnglish
Pages (from-to)2728-2739
Number of pages12
JournalIEEE Transactions on Computers
Volume71
Issue number11
Early online date19 Jul 2022
DOIs
Publication statusPublished - 1 Nov 2022

Keywords

  • computer vision
  • decoding
  • face recognition
  • feature extraction
  • image segmentation
  • object detection
  • SNN
  • STDP
  • semantics
  • task analysis
  • instance segmenation
  • neuromorphic algorithms
  • neuromorphic engineering
  • unsuperivsed learning

Fingerprint

Dive into the research topics of 'Unsupervised spiking instance segmentation on event data using STDP features'. Together they form a unique fingerprint.

Cite this