PhD Thesis Defense - Jonathan Courtois – Université Côte d’Azur · CNRS · LEAT : Sparsity in SNNs

Soutenance de Thèse - Jonathan Courtois, Université Côte d'Azur, CNRS, LEAT Supervisors: Benoit Miramond, LEAT, Alain Pegatoquet, LEAT, recorded by EUR DS4H Sparsity Optimization in Spike Neural Networks for Embedded Event-Vision Processing Optimisation de la sparsité dans les réseaux de neurones à impulsion pour le traitement de données événementielles 00:00 Introduction 03:15 Context & Motivation 08:40 Event-Based Vision & SNNs 17:30 Energy Efficiency Metrics 28:10 Satellite Pose Estimation 40:30 Discussion & Perspectives 44:10 Conlusion 46:05 Phd Doctor Serment This PhD thesis investigates Spiking Neural Networks (SNNs) for event-based data processing in embedded systems operating under strict energy constraints, with a particular focus on space applications. While Formal Artificial Neural Networks (FNNs) achieve high performance on complex tasks, their energy consumption often prevents their deployment on embedded or space platforms. Inspired by the human brain — capable of processing complex information with ~20 W — neuromorphic computing offers a promising alternative. SNNs leverage event-driven, spike-based computation, enabling sparse and asynchronous information processing. When combined with event-based cameras, which transmit only meaningful changes in visual scenes, these systems become highly suited for low-power vision applications. This thesis develops a complete methodology for the design and evaluation of embedded SNNs on neuromorphic hardware. The main contribution is to assess the feasibility of complex tasks such as object detection and pose estimation for space applications using neuromorphic, event-based approaches — a research area that remains unexplored at this level of complexity. In light of this novelty, our work progresses methodically through all stages of SNN implementation. We begin by defining a hardware-agnostic metric to evaluate the energy efficiency of SNNs compared to FNNs. The results show that SNNs can achieve a 6- to 8-fold reduction in energy consumption compared to equivalent FNNs while maintaining comparable accuracy, with memory access energy dominating the total consumption. This provides a crucial foundation for designing future solutions. Building on this basis, we contribute to the extension of QUALIA, an open-source framework for designing, training, quantizing and deploying neural networks (FNNs and SNNs) on various platforms (CPU, microcontroller, FPGA). QUALIA bridges the gap between high-level neural network specifications and hardware implementations, supports PyTorch-based SNN development, and integrates SPLEAT, a configurable neuromorphic accelerator on FPGA. The toolchain is completed by QUALIABENCH, an automation solution that democratizes access to neuromorphic technologies by enabling systematic deployment and evaluation of networks on the target hardware without requiring in-depth hardware expertise. Progressing from simple to more complex implementations, we push the limits of SNNs on neuromorphic hardware. The object detection solution deployed on the SPLEAT platform achieves 46% faster inference with a threefold improvement in energy efficiency compared to CPU-only execution, demonstrating the feasibility of implementing complex SNN architectures on reconfigurable platforms. We also investigate the feasibility of two methods for satellite pose estimation: direct event-based monocular 6D pose estimation and keypoint prediction for indirect pose estimation with a U-Net-type architecture. These advanced applications demonstrate that the combination of SNNs and event-based sensors constitutes a viable approach for energy-efficient processing in constrained space environments, with potential applications in autonomous navigation, object detection and environmental monitoring. 🚀 Contributions & Results This work proposes a complete methodology for designing, evaluating and deploying embedded SNNs on neuromorphic hardware: Definition of a hardware-agnostic energy efficiency metric for comparing SNNs and FNNs Demonstration of 6× to 8× energy reduction for SNNs at comparable accuracy Identification of memory access as the dominant energy cost Extension of QUALIA, an open-source framework for FNN/SNN design, training, quantization and deployment Integration with SPLEAT, a configurable FPGA-based neuromorphic accelerator Development of QUALIABENCH, an automated benchmarking and deployment toolchain 🛰 Advanced Applications Object detection on neuromorphic hardware: 46% faster inference, 3× improvement in energy efficiency vs CPU-only execution Satellite pose estimation: Direct monocular 6D pose estimation from events Keypoint-based pose estimation using a U-Net-like architecture Spiking Neural Networks, Event-Based Vision, Neuromorphic Computing, Embedded Systems, Energy Efficiency, FPGA Deployment, Bio-Inspired AI, Space Applications 🎥 Video recorded by: ‪@eurds4h‬