Fuzzy Encoding-Decoding to Improve Spiking Q-Learning Performance in Autonomous Driving
Aref Ghoreishee, Abhishek Mishra, Lifeng Zhou, John Walsh, Anup Das, and Nagarajan Kandasamy · 2026
Read Fuzzy Encoding-Decoding to Improve Spiking Q-Learning Performance in Autonomous Driving by Aref Ghoreishee, Abhishek Mishra, Lifeng Zhou, John Walsh, Anup Das, and Nagarajan Kandasamy (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-16436.txt.
Introduction
Ghoreishee and colleagues improve spiking neural networks for self-driving cars, where turning dense camera input into sparse spikes loses information and weakens value estimates. Their fuzzy encoder-decoder uses trainable membership functions to build richer spike patterns, then a light decoder rebuilds continuous action values from spiking outputs. On the HighwayEnv benchmark, decision accuracy rose substantially and the gap to non-spiking multi-modal networks narrowed. The design suits fast, energy-lean driving systems that must decide in real time.
Catalog record
| CREATOR | Aref Ghoreishee, Abhishek Mishra, Lifeng Zhou, John Walsh, Anup Das, and Nagarajan Kandasamy |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36789 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.16436
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Fuzzy Encoding-Decoding to Improve Spiking Q-Learning Performance in Autonomous Driving Aref Ghoreishee, Abhishek Mishra, Lifeng Zhou, John Walsh, Anup Das, and Nagarajan Kandasamy Abstract This paper develops an end-to-end fuzzy encoder–decoder architecture for enhancing vision-based multi-modal deep spiking Q-networks in autonomous driving. The method addresses two core limitations of spiking reinforcement learning: information loss stemming from the conversion of dense visual inputs into sparse spike trains, and the limited representational capacity of spike-based value functions, which often yields weakly discriminative Q-value estimates. The encoder introduces trainable fuzzy membership functions to generate expressive, population-based spike representations, and the decoder uses a lightweight neural decoder to reconstruct continuous Q-values from spiking outputs. Experiments on the HighwayEnv benchmark show that the proposed architecture substantially improves decision-making accuracy and closes the performance gap between spiking and non-spiking multi-modal Q-networks. The results highlight the potential of this framework for efficient and real-time autonomous driving with spiking neural networks. I Introduction Reinforcement learning (RL) enables autonomous agents to acquire complex decision-making strategies through interaction with dynamic and uncertain environments. Deep reinforcement learning (DRL) extends this capability by integrating deep neural networks with RL-based policy optimization, allowing agents to process high-dimensional sensory inputs and learn effective control policies directly from raw data. Among DRL methods, Deep Q-Networks (DQNs) have emerged as a foundational framework for sequential decision-making in complex domains, supporting robust, data-driven behavior without the need for handcrafted decision rules. Through mechanisms such as experience replay, target networks, and stabilized value iteration [ 15 ] , DQNs provide the reliability required for real-time autonomous systems, including autonomous driving, robot navigation, and multi-agent coordination [ 28 , 5 , 6 , 26 ] , where decisions must be executed under uncertainty and partial observability. Given the strong capability of deep neural networks to extract high-level patterns from visual data, including spatial structures, geometric relationships, and complex semantic cues, modern autonomous systems often rely heavily on vision-based sensors or convert heterogeneous measurements (e.g., LiDAR, radar, IMU) into image-like representations before downstream processing [ 12 , 10 , 18 ] . When combined with advanced perception modules, such as attention-based fusion layers, DQNs offer a powerful paradigm for integrating diverse sensory modalities into a unified state representation, enabling precise and context-aware decision-making. This integration has demonstrated notable success in autonomous driving and robotic navigation. However, the significant computational and energy demands of conventional DQNs, particularly those employing attention-based fusion mechanisms, remain a major barrier to deployment in resource-limited autonomous platforms [ 7 , 23 , 3 ] . Spiking Neural Networks (SNNs) offer an alternative that is inherently energy-efficient and event-driven. By transmitting information through sparse binary spikes and replacing expensive multiply–accumulate operations with low-cost accumulations, SNNs provide a solution for real-time decision-making on edge-level autonomous platforms. Recent work by Ghoreishee et al. introduced a vision-based multi-modal deep spiking Q-network (MM-DSQN) for high-level autonomous driving decisions, demonstrating a promising direction toward computationally and energy-efficient on-board intelligence [ 4 ] . Despite these advantages, previous studies have reported a notable performance gap between spiking and non-spiking DQNs for vision-based decision-making tasks [ 3 , 13 , 4 ] . We hypothesize that two key limitations of SNN-based DQN pipelines contribute to this gap: 1. Information loss in spike encoding: Converting continuous visual signals into binary spike trains discards fine-grained perceptual information, degrading the quality of the learned state representation and, consequently, the decision-making performance. 2. Sparse and quantized Q-values: The event-driven nature of SNN outputs can cause Q-values to collapse into a narrow and weakly discriminative range, reducing the expressiveness of the Q-function and impairing the subsequent action selection. We propose a computationally efficient population-based fuzzy encoding and decoding approach for vision-based deep spiking Q-networks. Inspired by fuzzification and defuzzification principles, our method learns optimal fuzzy membership functions to minimize information loss during spike encoding and introduces a population-based decoder to reconstruct expressive Q-values from sparse spike outputs. To the best of our knowledge, this is the first work to integrate a fuzzy population-based encoder–decoder mechanism within spiking MM-DQNs for autonomous driving. The main contributions of this work can be summarized as follows. • A new population-based fuzzy encoder that significantly reduces information loss during visual-to-spike conversion by learning compact and computationally efficient fuzzy membership functions. • A population-based fuzzy decoder that reconstructs continuous Q-values from event-driven neural activity, mitigating the sparse Q-value problem in SNNs. Evaluation on the HighwayEnv benchmark [ 11 ] demonstrates that our approach substantially narrows the performance gap between spiking and non-spiking variants while maintaining computational efficiency. Although the original MM-DSQN exhibits a performance drop of approximately 12.5\% compared to the non-spiking variant, the encoder–decoder solution, when integrated within this MM-DSQN, allows the spiking model to match the performance of its non-spiking counterpart. The remainder of the paper is organized as follows. Section II discusses related work. Section III discusses concepts necessary to understand our solution. Section IV develops the encoder–decoder approach, and Section V provides a theoretical analysis of its key properties. Section VI evaluates its performance. We conclude the paper in Section VII . Fig. 1 : The MM-DSQN architecture that uses a spiking cross-attention module to fuse BEV images and LiDAR representations for RL-based decision making (adapted from Ghoreishee et al. [ 4 ] ). II Related Work Previous research on spiking multimodal decision-making covers two main areas: deep spiking reinforcement learning and multimodal reinforcement learning. Both aim to achieve real-time perception, decision-making, and control under computational and energy constraints, with the goal of enabling efficient on-board intelligence for autonomous systems. II-A Deep Spiking Reinforcement Learning The Deep Q-Network (DQN) combines Q-learning with neural networks to enable end-to-end learning from high-dimensional observations [ 16 ] . Techniques such as experience replay and \epsilon -greedy exploration [ 15 ] , together with Double DQN [ 24 ] , Dueling DQN [ 25 ] , and double–dueling variants [ 19 ] , improve value estimation and training stability. To address the high energy cost of DRL, a growing line of work explores deep spiking Q-networks (DSQNs). Early efforts converted pre-trained DQNs into spiking models [ 17 ] , followed by more advanced conversion techniques [ 22 ] , direct training with pseudo-gradients [ 13 ] , and spiking-specific batch normalization [ 21 ] . Despite these advances, many SNN-based RL methods, particularly those in [ 21 , 13 , 22 ] , omit explicit spike encoding and instead rely on the first layer to convert continuous inputs into spikes. Although this approach minimizes information loss, it compromises computational efficiency and weakens the event-driven benefits of SNNs. Population-based spike encoding using radial basis functions has recently been explored for continuous-control RL problems [ 8 , 23 ] . By increasing the representational capacity of SNNs, these methods demonstrate strong performance on low-dimensional proprioceptive and kinematic observations. However, they do not readily extend to vision-based DSQNs—the main bottleneck being that evaluating exponential kernels for every pixel incurs substantial computational overhead, rendering these encoders inefficient and often infeasible for high-dimensional image inputs that require fast and accurate spike conversion. Thus, there remains a significant gap in developing computationally efficient, scalable encoding and decoding mechanisms suited to visual DSQNs. II-B Multi-Modal Reinforcement Learning Multimodal RL combines heterogeneous sensory