Spike-TBR: a Noise Resilient Neuromorphic Event Representation Journal: Computer Vision and Image Understanding

Gabriele Magrini, Federico Becattini, Luca Cultrera, Lorenzo Berlincioni, Pietro Pala, Alberto Del Bimbo · 2025

Read Spike-TBR: a Noise Resilient Neuromorphic Event Representation Journal: Computer Vision and Image Understanding by Gabriele Magrini, Federico Becattini, Luca Cultrera, Lorenzo Berlincioni, Pietro Pala, Alberto Del Bimbo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-04817.txt.

Introduction

Gabriele Magrini and co-authors harden a way of feeding event camera data to standard vision software. Event cameras report only changes, pixel by pixel, which gives them speed and range that normal cameras lack. Turning that stream into frames is tricky, and noise corrupts the usual encoding, called Temporal Binary Representation.

Their fix, Spike-TBR, routes the encoding through spiking neurons, the brain-like units that fire in pulses and naturally filter noise. They built four variants, each with a different neuron type, and tested them on several data sets. The spiking versions held up better in noisy scenes and even improved results on clean data. The authors present the method as a simple bridge between spike-based sensing and frame-based pipelines, suited to real event-driven vision work.

Catalog record

CREATORGabriele Magrini, Federico Becattini, Luca Cultrera, Lorenzo Berlincioni, Pietro Pala, Alberto Del Bimbo
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-04817

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Spike-TBR: a Noise Resilient Neuromorphic Event Representation Journal: Computer Vision and Image Understanding Gabriele Magrini, Federico Becattini, Luca Cultrera, Lorenzo Berlincioni, Pietro Pala, Alberto Del Bimbo Abstract Event cameras offer significant advantages over traditional frame-based sensors, including higher temporal resolution, lower latency and dynamic range. However, efficiently converting event streams into formats compatible with standard computer vision pipelines remains a challenging problem, particularly in the presence of noise. In this paper, we propose Spike-TBR, a novel event-based encoding strategy based on Temporal Binary Representation (TBR), addressing its vulnerability to noise by integrating spiking neurons. Spike-TBR combines the frame-based advantages of TBR with the noise-filtering capabilities of spiking neural networks, creating a more robust representation of event streams. We evaluate four variants of Spike-TBR, each using different spiking neurons, across multiple datasets, demonstrating superior performance in noise-affected scenarios while improving the results on clean data. Our method bridges the gap between spike-based and frame-based processing, offering a simple noise-resilient solution for event-driven vision applications. MSC 41A05, 41A10, 65D05, 65D17 \KWD key1 , Key2 1 Introduction Traditional RGB cameras are capable of acquiring the content of a scene and output a stream of RGB frames, typically at rates of about 30 frames per second. When some fast-moving object is present in the scene or when the same camera is rapidly moving with respect to the scene, such a limited frame rate may yield a degradation of the sensing quality with motion blur artifacts in the acquired frames. Another typical limitation of traditional cameras relates to their dynamic range, i.e. the ratio between levels of the brightest and darkest parts of an image: the joint presence of very bright and dark regions can compromise the quality of the acquired scene content in some areas. These limitations are addressed by event cameras, also referred to as neuromorphic cameras, through the use of a different image-acquiring paradigm that produces asynchronous events per pixel whenever an illumination change is detected. Frame-based cameras capture entire images at regular time intervals, while event cameras only output information when there is a change in the visual scene. This asynchronous nature of event cameras requires adapting vision algorithms to handle these asynchronous data streams. The simplest approach is to accumulate events over time to generate a frame-like representation. Several works use this method [ 1 , 2 , 3 , 4 ] , which is useful when working with higher-level vision algorithms that expect frames as input. The result of temporal integration can be thought of as an "event-based video", which maintains the advantages of an event camera (e.g., high dynamic range, low power consumption) while presenting the data in a format more amenable to traditional computer vision processing pipelines. Accumulating events into frame-based representations though has its shortcomings. The accumulation time \Delta T controlling the time period within which events are gathered controls the trade-off between the temporal granularity at which the data is analyzed and the number of frames that need to be processed. If, on the one hand, small accumulation times allow computer vision pipelines to better exploit the advantages of neuromorphic sensors by capturing fine-grained movements, on the other hand, the amount of information to be processed might make real-time processing unfeasible. Conversely, a large \Delta T allows fast computation times, but can lead to information loss, as aggregating events often requires some form of quantization. A notable approach that attempts to overcome this trade-off is Temporal Binary Representation (TBR) [ 5 ] , where each pixel is created by interpreting its event history as a binary string (event/no event) and converting it to a single base-10 value. Despite its simplicity and effectiveness, TBR suffers from a high sensitivity to noise: the presence of noise in recent time slices can yield a flip in the most significant bit, largely changing the resulting representation. An orthogonal strategy for processing event data involves the use of Spiking Neural Networks (SNNs) [ 6 , 7 , 8 ] , a type of artificial neural network inspired by the biological functioning of neurons in the brain. In an SNN, neurons communicate with each other via spikes or action potentials instead of continuous signals like in traditional artificial neural networks. This theoretically makes SNNs particularly well-suited for processing event data as spikes can directly represent the events generated by an event camera, yet they do not scale well with typical synchronous hardware. Inspired by the promising properties of SNNs, we propose an enhanced version of TBR that follows a hybrid approach between the spike-based and frame-based paradigms. Our novel encoding strategy, Spike-TBR , leverages an SNN layer to accumulate events on its membrane potential (i.e its current charging state), and we interpret the spikes it fires as the binary digits that make up the TBR representation. The resulting representation is frame-based as TBR, yet it exhibits a highly increased robustness to noise due to the SNN layer, which filters noise by accumulating events into a decaying membrane and fires spikes only when the pixel-wise potential exceeds a threshold. We demonstrate resilience to noise even when training on clean data. The main contributions are the following: i) we propose Spike-TBR, a noise-resistant frame-based event encoding strategy that overcomes the limitations of TBR [ 5 ] ; ii) we analyze four different variants of our approach, studying different SNN neurons; iii) we test on four different datasets, obtaining higher or comparable results to state-of-the-art approaches. Fig. 1 : Spike-TBR pipeline. As the raw event stream is generated by the sensor (left) the SNN membrane accumulates potential V(t) . Once this potential reaches the threshold V_{thr} a spike is released. This stream is constantly sampled with a \Delta t period into binary frames, which every \Delta T are aggregated into a TBR frame. 2 Related Work Several works have addressed the issue of processing event camera data streams [ 9 ] . The interest to adopt traditional computer vision techniques has led to the development of a plethora of strategies to generate event representations, also in the form of event frames [ 10 , 4 , 2 , 1 , 3 , 11 , 5 ] . However, most of these approaches commonly apply a temporal quantization in the form of histograms [ 3 ] or event subsampling [ 12 ] . An alternative approach to processing events is to feed them to specifically tailored architectures, such as Spiking Neural Networks (SNNs). Inspired by the fundamental mechanism of brain computation, SNNs pose themselves as the third generation of neural networks [ 13 ] , as opposed to the McCulloch-Pitts neuron [ 14 ] (first generation) and feedforward and recurrent networks (second generation). The purpose of SNNs is to emulate and reach the computational efficiency of the human brain. Although still far from complete maturity, research on SNNs has seen increasing interest in recent years, driven by the growing need for low-power-consuming neural networks. Thus, they have found applications in various empirical fields, especially when paired with event-based cameras [ 6 , 7 , 8 ] . To this end, various forms of integration between SNNs and neuromorphic cameras have been proposed and implemented; in [ 15 ] , the authors present a sensor fusion

Read the complete text