Movement classification and segmentation using event-based sensing and spiking neural networks

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

1 Citation (Scopus)
52 Downloads (Pure)

Abstract

The development of Spiking Neural Networks (SNN) and the discipline of Neuromorphic Engineering has resulted in a paradigm shift in how Machine Learning (ML) and Computer Vision (CV) problems are approached. At the heart of this shift is the adoption of event-based sensing and processing methods. The production of sparse and asynchronous events that are dynamically connected to the scene is possible with an event-based vision sensor, allowing for the acquisition of not just spatial data but also high-fidelity temporal data. In this work, we describe a novel method for performing instance segmentation of objects, only using their spatio-temporal movement patterns, by utilising the weights of an unsupervised Spiking Convolutional Neural Network that was originally trained for object recognition and extending it to instance segmentation. This takes advantage of the network’s spatial and temporal characteristics encoded within its internal feature representation, to offer this additional discriminative ability. We demonstrate this through a track path identification problem, where 6 identical blobs complete complex movement patterns within the same area at the same time. The network is able to successfully identify all 6 individual movements and segment the movement patterns belonging to each. The work then also explains how these methods map into the more complex Track before Detect problem. A complex track initiation problem, where detection can only be completed after an integration period, due to the low signal, high noise environment. These problem characteristics seem to complement the properties of event-based sensing and processing and initial test results are shown.
Original languageEnglish
Title of host publication2022 Sensor Signal Processing for Defence Conference, SSPD 2022 - Proceedings
Place of PublicationPiscataway, NJ.
PublisherIEEE
ISBN (Electronic)9781665483483
ISBN (Print)9781665483490
DOIs
Publication statusPublished - 23 Sept 2022
Event11th International Conference in Sensor Signal Processing for Defence: from Sensor to Decision - London, United Kingdom
Duration: 13 Sept 202214 Sept 2022
Conference number: 11th
https://sspd.eng.ed.ac.uk/

Publication series

Name2022 Sensor Signal Processing for Defence Conference, SSPD 2022 - Proceedings

Conference

Conference11th International Conference in Sensor Signal Processing for Defence
Abbreviated titleSSPD 2022
Country/TerritoryUnited Kingdom
CityLondon
Period13/09/2214/09/22
Internet address

Keywords

  • spiking neural network (SNN)
  • machine learning
  • computer vision

Fingerprint

Dive into the research topics of 'Movement classification and segmentation using event-based sensing and spiking neural networks'. Together they form a unique fingerprint.

Cite this