A new spiking convolutional recurrent neural network (SCRNN) with applications to event-based hand gesture recognition

Research output: Contribution to journalArticlepeer-review

34 Downloads (Pure)

Abstract

The combination of neuromorphic visual sensors and spiking neural network offers a high efficient bio-inspired solution to real-world applications. However, processing event- based sequences remains challenging because of the nature of their asynchronism and sparsity behavior. In this paper, a novel spiking convolutional recurrent neural network (SCRNN) architecture that takes advantage of both convolution operation and recurrent connectivity to maintain the spatial and temporal relations from event-based sequence data are presented. The use of recurrent architecture enables the network to have a sampling window with an arbitrary length, allowing the network to exploit temporal correlations between event collections. Rather than standard ANN to SNN conversion techniques, the network utilizes a supervised Spike Layer Error Reassignment (SLAYER) training mechanism that allows the network to adapt to neuromorphic (event-based) data directly. The network structure is validated on the DVS gesture dataset and achieves a 10 class gesture recognition accuracy of 96.59% and an 11 class gesture recognition accuracy of 90.28%.

Original languageEnglish
Article number590164
Number of pages14
JournalFrontiers in Neuroscience
Volume14
DOIs
Publication statusPublished - 17 Nov 2020

Keywords

  • spiking neural network (SNN)
  • action recognition
  • neuromorphic
  • event-driven processing

Fingerprint Dive into the research topics of 'A new spiking convolutional recurrent neural network (SCRNN) with applications to event-based hand gesture recognition'. Together they form a unique fingerprint.

Cite this