Towards on-device learning and reconfigurable hardware implementation for encoded single-photon signal processing

Deep neural networks (DNNs) enhance the accuracy and efficiency of reconstructing key parameters from time- resolved photon arrival signals recorded by single-photon detec- tors. However, the performance of conventional backpropagation- based DNNs is dependent on the optical setup and the biological samples, necessitating frequent retraining of the network, either via transfer learning or from scratch. Newly collected data must be stored and transferred to a high-performance GPU server for retraining, introducing latency and storage overhead. To address these challenges, we propose an online training algorithm based on a one-sided Jacobi rotation-based online sequential extreme learning machine (OSOS-ELM). We fully exploit par- allelism and partition-independent functions to execute OSOS- ELM on a heterogeneous FPGA with integrated ARM cores. Extensive evaluations of OSOS-ELM and OS-ELM demonstrate that both achieve comparable accuracy across different network dimensions (i.e., input, hidden, and output layers), while OSOS- ELM is more hardware-efficient. By leveraging the parallelism of OSOS-ELM, we implement a holistic computing prototype on an Xilinx ZCU104 FPGA. We validate our approach through three typical case studies involving single-photon signal analysis: fog sensing with a commercial single-photon LiDAR, fluorescence lifetime estimation in fluorescence lifetime imaging, and blood flow index reconstruction in diffuse correlation spectroscopy, all of which use one-dimensional data encoded from photonic signals. From a hardware perspective, we optimize the OSOS- ELM workload by employing multi-tasked processing on ARM CPU cores and pipelined execution on the FPGA’s logic fabric. We also implement our OSOS-ELM on the NVIDIA Jetson Xavier NX GPU to comprehensively evaluate its computational performance on another heterogeneous computing platform.