UltraLIF: Fully Differentiable Spiking Neural Networks via Ultradiscretization and Max-Plus Algebra Jose Marie Antonio Miñoza Abstract Spiking Neural Networks (SNNs) offer energy-efficient, biologically plausible computation but suffer from non-differentiable spike generation, necessitating reliance on heuristic surrogate gradients. This paper introduces UltraLIF , a principled framework that replaces surrogate gradients with ultradiscretization , a mathematical formalism from tropical geometry providing continuous relaxations of discrete dynamics. The central insight is that the max-plus semiring underlying ultradiscretization naturally models neural threshold dynamics: the log-sum-exp function serves as a differentiable soft-maximum that converges to hard thresholding as a learnable temperature parameter ε → 0 \varepsilon\to 0 . Two neuron models are derived from distinct dynamical systems: UltraLIF from the LIF ordinary differential equation (temporal dynamics) and UltraDLIF from the diffusion equation modeling gap junction coupling across neuronal populations (spatial dynamics). Both yield fully differentiable SNNs trainable via standard backpropagation with no forward-backward mismatch. Theoretical analysis establishes pointwise convergence to classical LIF dynamics with quantitative error bounds and bounded non-vanishing gradients. Experiments on six benchmarks spanning static images, neuromorphic vision, and audio demonstrate improvements over surrogate gradient baselines, with gains most pronounced in single-timestep ( T = 1 T{=}1 ) settings on neuromorphic and temporal datasets. An optional sparsity penalty enables significant energy reduction while maintaining competitive accuracy. 1 Introduction Spiking Neural Networks (SNNs) represent a promising paradigm for energy-efficient machine learning, with significant potential for neuromorphic hardware deployment ( 18 ; 24 ) . In contrast to artificial neural networks (ANNs) communicating via continuous activations, SNNs process information through discrete spike events, emulating biological neural computation. This event-driven nature enables substantial energy savings; Intel’s Loihi chip demonstrates up to 1000\times energy reduction compared to GPUs on certain tasks ( 6 ) . However, SNN training remains challenging due to the non-differentiability of spike generation. The standard Leaky Integrate-and-Fire (LIF) neuron follows the dynamics: \displaystyle v^{(t+1)} \displaystyle=\tau v^{(t)}+I^{(t)} (1) \displaystyle s^{(t+1)} \displaystyle=H(v^{(t+1)}-\theta) (2) where the Heaviside step function H(\cdot) has gradient zero almost everywhere. The dominant approach employs surrogate gradients , replacing the true gradient with a smooth approximation during backpropagation ( 21 ; 35 ) . While empirically effective, surrogate gradients introduce a fundamental mismatch between forward (discrete) and backward (continuous) passes (Figure 1 a), with limited theoretical understanding of convergence properties ( 16 ; 12 ) . Figure 1 : (a) Spike activation functions: Heaviside (hard threshold), surrogate gradient (smooth approximation), and ultradiscretized (principled soft relaxation). (b) Gradients: Heaviside has zero gradient almost everywhere (delta function at threshold); surrogate and ultradiscretized provide smooth gradients, but only ultradiscretized maintains forward-backward consistency. This paper proposes UltraLIF , a theoretically grounded alternative based on ultradiscretization , a limiting procedure from tropical geometry transforming continuous dynamical systems into discrete max-plus systems while preserving structural properties ( 29 ; 11 ) . The key contributions are: 1. Principled differentiability : The \mathrm{LSE} function provides a natural soft relaxation of the max operation underlying spike generation, with explicit convergence bounds as \varepsilon\to 0 (Lemma 3.2 ). 2. Forward-backward consistency : Unlike surrogate methods, UltraLIF employs identical dynamics in forward and backward passes, eliminating gradient mismatch (Remark 5.6 ). 3. Bounded gradients : For any \varepsilon>0 , gradients remain bounded and non-vanishing, enabling stable optimization (Proposition 5.4 ). 4. Consistent low-timestep improvements : On six benchmarks spanning static, neuromorphic, and audio modalities, ultradiscretized models improve over surrogate gradient baselines at T{=}1 , with the largest gains on temporal and event-driven data (+11.22% SHD, +7.96% DVS-Gesture, +3.91% N-MNIST). 2 Related Work Surrogate Gradient Methods. The dominant paradigm for direct SNN training replaces non-differentiable spike gradients with smooth surrogates ( 21 ) . Common choices include piecewise linear ( 1 ) , sigmoid ( 34 ) , and arctangent ( 7 ) functions. Recent work introduces learnable surrogate parameters ( 17 ) and adaptive shapes ( 16 ) . Despite empirical success, the forward-backward mismatch remains theoretically problematic. 12 provide partial justification via stochastic neurons, showing surrogate gradients match escape noise derivatives in expectation. Spike Timing Approaches. SpikeProp ( 2 ) and variants ( 20 ; 14 ) compute exact gradients with respect to spike times. These methods require careful initialization and struggle with silent neurons. Recent work on exact smooth gradients through spike timing ( 10 ) addresses some limitations but remains computationally intensive. ANN-to-SNN Conversion. An alternative approach trains conventional ANNs then converts to SNNs ( 4 ; 25 ; 3 ) . While avoiding direct SNN training, conversion methods typically require many timesteps to achieve ANN accuracy and sacrifice temporal dynamics. Tropical Geometry and Neural Networks. Tropical geometry studies algebraic structures where addition becomes max (or min) and multiplication becomes addition ( 19 ) . 36 establish that ReLU networks compute tropical rational functions. Recent work extends this to graph neural networks ( 23 ) and neural network compression ( 8 ) . The connection to spiking networks via ultradiscretization appears novel. Ultradiscretization. Originating in integrable systems ( 29 ) , ultradiscretization transforms difference equations into cellular automata preserving solution structure. Applications include soliton systems ( 28 ) and limit cycle analysis ( 30 ; 31 ) . Application to neural networks has not been previously explored. 3 Preliminaries 3.1 The Max-Plus Semiring Definition 3.1 (Max-Plus Semiring) . The max-plus semiring is the algebraic structure (\mathbb{R}\cup\{-\infty\},\oplus,\odot) where: • a\oplus b:=\max(a,b) (tropical addition) • a\odot b:=a+b (tropical multiplication) with additive identity -\infty and multiplicative identity 0 . This semiring underlies tropical geometry and provides the limit structure for ultradiscretization. 3.2 Log-Sum-Exp as Soft Maximum The log-sum-exp function with temperature \varepsilon>0 is defined as: \mathrm{LSE}_{\varepsilon}(\mathbf{x}):=\varepsilon\log\left(\sum_{i=1}^{n}e^{x_{i}/\varepsilon}\right) (3) The following lemma establishes its role as a smooth approximation to the maximum. Lemma 3.2 (LSE Convergence) . Let \mathbf{x}=(x_{1},\ldots,x_{n})\in\mathbb{R}^{n} and M:=\max_{i}x_{i} . Then M\leq\mathrm{LSE}_{\varepsilon}(\mathbf{x})\leq M+\varepsilon\log n , hence \lim_{\varepsilon\to 0^{+}}\mathrm{LSE}_{\varepsilon}(\mathbf{x})=M . Moreover, \nabla\mathrm{LSE}_{\varepsilon}(\mathbf{x})=\mathrm{softmax}(\mathbf{x}/\varepsilon)\in(0,1)^{n} . Proof. For the lower bound, the sum includes e^{M/\varepsilon} , so \mathrm{LSE}_{\varepsilon}(\mathbf{x})\geq\varepsilon\log(e^{M/\varepsilon})=M . For the upper bound, since x_{i}\leq M for all i , \mathrm{LSE}_{\varepsilon}(\mathbf{x})\leq\varepsilon\log(n\cdot e^{M/\varepsilon})=M+\varepsilon\log n . The limit follows by the squeeze theorem. The gradient formula follows from direct differentiation: \frac{\partial\mathrm{LSE}_{\varepsilon}}{\partial x_{j}}=\frac{e^{x_{j}/\varepsilon}}{\sum_{i}e^{x_{i}/\varepsilon}}=\mathrm{softmax}(\mathbf{x}/\varepsilon)_{j} . ∎ Remark 3.3 . When the maximum is unique with gap \delta:=M-\max_{i:x_{i}\neq M}x_{i}>0 , the error decays exponentially: \mathrm{LSE}_{\varepsilon}(\mathbf{x})-M=O(\varepsilon e^{-\delta/\varepsilon}) . 4 Method: Ultradiscretized Spiking Neurons The key innovation of this work is applying ultradiscretization , a limiting procedure from tropical geometry, to derive differentiable spiking neurons. This section shows that ultradiscretization can be applied to different neural dynamics, yielding distinct models for temporal and spatial computations. 4.1 Ultradiscretization Framework Ultradiscretization transforms continuous dynamical systems into max-plus (tropical) systems while preserving structural properties ( 29 ) . The procedure operates via the substitution x=e^{X/\varepsilon} followed by the limit \varepsilon\to 0^{+} : \displaystyle x+y=e^{X/\varepsilon}+e^{Y/\varepsilon} \displaystyle\to e^{\max(X,Y)/\varepsilon}\quad\text{(addition $\to$ max)} (4) \displaystyle x\cdot y=e^{X/\varepsilon}\cdot e^{Y/\varepsilon} \displaystyle=e^{(X+Y)/\varepsilon}\quad\text{(multiplication $\to$ addition)} (5) For finite \varepsilon>0 , the log-sum-exp function \mathrm{LSE}_{\varepsilon} (Eq. 3 ) provides a differentiable soft relaxation of the max operation. This is the foundation for all ultradiscretized neurons presented here. 4.1.1 Temporal Dynamics: UltraLIF UltraLIF is derived from the standard single-neuron LIF ordinary differential equation. The membrane potential evolves according to: \tau_{m}\frac{dv}{dt}=-(v-v_{\text{rest}})+R\cdot I(t) (6) where \tau_{m} is the membrane time constant, v_{\text{rest}} the resting potential, R the membrane resistance, and I(t) the input current. Applying forward Euler discretization with timestep \Delta t and setting v_{\text{rest}}=0 yields: v^{(t+1)}=\underbrace{\left(1-\frac{\Delta t}{\tau_{m}}\right)}_{=:\tau_{0}}v^{(t)}+I^{(t)} (7) where the leak factor \tau_{0}\in(0,1) controls temporal decay. The ultradiscretization transform ( 29 ) proceeds by substituting v=e^{V/\varepsilon} , I=e^{J/\varepsilon} , and parameterizing the leak as \tau=e^{T/\varepsilon} where T=\log\tau_{0}<0 : e^{V^{(t+1)}/\varepsilon}=e^{T/\varepsilon}\cdot e^{V^{(t)}/\varepsilon}+e^{J^{(t)}/\varepsilon}=e^{(V^{(t)}+T)/\varepsilon}+e^{J^{(t)}/\varepsilon} (8) Taking \varepsilon\cdot\log of both sides and the limit \varepsilon\to 0^{+} recovers the max-plus dynamics: V^{(t+1)}=\max\left(V^{(t)}+T,\,J^{(t)}\right)=\max\left(V^{(t)}+\log\tau_{0},\,J^{(t)}\right) (9) For differentiable training, the hard maximum is relaxed to the log-sum-exp for finite \varepsilon>0 : V_{\varepsilon}^{(t+1)}=\mathrm{LSE}_{\varepsilon}\left(V^{(t)}+\log\tau_{0},\,I^{(t)}\right) (10) This 2-term LSE captures temporal membrane integration with learnable leak. Note that the \varepsilon -parameterized leak \tau=e^{T/\varepsilon} becomes stronger as \varepsilon\to 0 , yielding sharper temporal dynamics in the tropical limit. 4.1.2 Spatial Dynamics: UltraDLIF An analogous derivation applies ultradiscretization to spatial dynamics, capturing lateral interactions across a neuronal population. Consider a simplified diffusive coupling where membrane potentials spread locally: \frac{\partial v}{\partial t}=D\nabla^{2}v (11) where D>0 is the diffusion coefficient. This models gap junction (electrical synapse) coupling, where ionic currents flow directly between neurons enabling voltage spread ( 5 ; 27 ) . The diffusion equation provides a first-order approximation of such lateral interactions. Discretizing the Laplacian via finite differences \nabla^{2}v\approx(v_{i-1}-2v_{i}+v_{i+1})/\Delta x^{2} and applying forward Euler in time yields: v_{i}^{(t+1)}=v_{i}^{(t)}+\frac{D\Delta t}{\Delta x^{2}}(v_{i-1}^{(t)}-2v_{i}^{(t)}+v_{i+1}^{(t)}) (12) At the balanced diffusion regime where D\Delta t/\Delta x^{2}=1/3 , this simplifies to uniform spatial averaging where each neuron and its neighbors contribute equally: v_{i}^{(t+1)}=\frac{1}{3}v_{i-1}^{(t)}+\frac{1}{3}v_{i}^{(t)}+\frac{1}{3}v_{i+1}^{(t)} (13) This choice of 1/3 lies within the von Neumann stability bound for explicit finite difference schemes applied to the 1D diffusion equation, which requires D\Delta t/\Delta x^{2}\leq 1/2 for numerical stability. The value 1/3 ensures stability while providing symmetric treatment of a neuron and its immediate neighbors, a natural balance for lateral coupling. Remark on subtraction. Standard ultradiscretization cannot handle subtraction directly, as there is no tropical analog of x-y in the max-plus semiring ( 22 ) . This limitation is circumvented by selecting \alpha=1/3 : expanding Eq. ( 12 ) gives coefficients \alpha , (1{-}2\alpha) , \alpha for the three terms, and at \alpha=1/3 all become 1/3>0 , eliminating subtraction entirely (Eq. ( 13 )). For more general diffusion regimes, inversible max-plus algebras extend the framework to handle subtraction via x-y\to\max(X,Y+\eta) , where \eta is an inverse element ( 22 ) . Applying the ultradiscretization transform with v=e^{V/\varepsilon} and taking \varepsilon\to 0^{+} : V_{i}^{(t+1)}=\max\left(V_{i-1}^{(t)},V_{i}^{(t)},V_{i+1}^{(t)}\right) (14) This limit corresponds to morphological dilation , a max-pooling operation over the spatial neighborhood. Relaxing the hard maximum to the log-sum-exp for finite \varepsilon>0 gives: V_{i,\varepsilon}^{(t+1)}=\mathrm{LSE}_{\varepsilon}\left(V_{i-1}^{(t)},V_{i}^{(t)},V_{i+1}^{(t)}\right) (15) This 3-term LSE captures lateral spatial smoothing across neurons. External input I_{i}^{(t)} is added separately (Eq. 19 ), following the standard neural field convention where diffusion handles lateral coupling and an additive term represents external drive. 4.1.3 Comparison of Derivations UltraLIF UltraDLIF Source LIF ODE Diffusion PDE Equation dv/dt=-v/\tau_{m}+I \partial v/\partial t=D\nabla^{2}v LSE terms 2 (temporal) 3 (spatial) Soft form \mathrm{LSE}(V{+}\log\tau_{0},I) \mathrm{LSE}(V_{-1},V_{0},V_{+1}) Models Membrane decay Lateral diffusion Table 1 : Ultradiscretization applied to temporal and spatial dynamics. Both models share the same theoretical foundation (ultradiscretization, LSE soft relaxation) but capture different biological phenomena. UltraLIF models single-neuron temporal dynamics; UltraDLIF models population-level spatial interactions. 4.2 Neuron Models Building on the ultradiscretization framework, complete neuron models are defined incorporating spike generation and reset mechanisms. Both variants share the same spike and reset logic, differing only in membrane dynamics. Definition 4.1 (UltraLIF Neuron (Temporal)) . For temperature \varepsilon>0 , leak factor \tau_{0}\in(0,1) , and threshold \theta>0 : \displaystyle\tilde{V}_{\varepsilon}^{(t+1)} \displaystyle=\mathrm{LSE}_{\varepsilon}\left(V_{\varepsilon}^{(t)}+\log\tau_{0},\,I^{(t)}\right) (16) \displaystyle s_{\varepsilon}^{(t+1)} \displaystyle=\sigma\left(\frac{\tilde{V}_{\varepsilon}^{(t+1)}-\theta}{\varepsilon}\right) (17) \displaystyle V_{\varepsilon}^{(t+1)} \displaystyle=\tilde{V}_{\varepsilon}^{(t+1)}\cdot(1-s_{\varepsilon}^{(t+1)})+V_{\text{reset}}\cdot s_{\varepsilon}^{(t+1)} (18) where \sigma(z)=(1+e^{-z})^{-1} is the logistic sigmoid and V_{\text{reset}}=0 . Definition 4.2 (UltraDLIF Neuron (Spatial)) . For temperature \varepsilon>0 , threshold \theta>0 , and neuron index i : \displaystyle\tilde{V}_{i,\varepsilon}^{(t+1)} \displaystyle=\mathrm{LSE}_{\varepsilon}\left(V_{i-1,\varepsilon}^{(t)},V_{i,\varepsilon}^{(t)},V_{i+1,\varepsilon}^{(t)}\right)+I_{i}^{(t)} (19) \displaystyle s_{i,\varepsilon}^{(t+1)} \displaystyle=\sigma\left(\frac{\tilde{V}_{i,\varepsilon}^{(t+1)}-\theta}{\varepsilon}\right) (20) \displaystyle V_{i,\varepsilon}^{(t+1)} \displaystyle=\tilde{V}_{i,\varepsilon}^{(t+1)}\cdot(1-s_{i,\varepsilon}^{(t+1)})+V_{\text{reset}}\cdot s_{i,\varepsilon}^{(t+1)} (21) The LSE operates over the spatial neighborhood (circular boundary conditions). Soft Spike Mechanism. The soft spike s_{\varepsilon}\in(0,1) interpolates between no-spike ( s_{\varepsilon}\approx 0 ) and spike ( s_{\varepsilon}\approx 1 ). This differs fundamentally from surrogate gradient methods: • Surrogate : Forward uses hard H(V-\theta) ; backward uses smooth g^{\prime}(V) • Ultradiscretized : Both forward and backward use same smooth \sigma((V-\theta)/\varepsilon) On Soft vs. Binary Spikes. A natural concern is that s_{\varepsilon}\in(0,1) does not represent “true” binary spikes. However, this deviation from binary behavior is not problematic in practice. The soft spike is used during training for gradient computation, while at inference one can employ small \varepsilon or hard thresholding ( s=H(V-\theta) ) for neuromorphic deployment, following a standard training-inference separation analogous to dropout or batch normalization. Furthermore, the output layer uses mean spike rate \hat{y}=\frac{1}{T}\sum_{t}s^{(t)} , which is inherently robust to soft versus hard individual spikes since the classification decision depends on aggregate activity rather than precise spike values. The sparsity penalty \lambda\cdot\bar{s} encourages sparse soft activations that correspond to sparse hard spikes in the limit, ensuring energy-efficient inference. Most importantly, Proposition 5.2 guarantees that s_{\varepsilon}\to H(V-\theta) as \varepsilon\to 0 , providing a principled path from soft training dynamics to binary inference behavior. Learnable Parameters. The temperature \varepsilon is made learnable via \varepsilon=\exp(\log\varepsilon_{\text{param}}) , initialized to \varepsilon_{0}=1.0 . For UltraLIF, the leak factor \tau_{0} can also be learned (UltraPLIF variant). For UltraDLIF, similarly UltraDPLIF learns \tau_{0} for an optional temporal component. During training, the network discovers optimal soft-to-hard trade-offs, implementing automatic curriculum learning. 4.3 Network Architecture A feedforward SNN with L layers of UltraLIF neurons is constructed as: \displaystyle I_{l}^{(t)} \displaystyle=W_{l}\cdot s_{l-1}^{(t)}+b_{l} (22) \displaystyle V_{l}^{(t+1)} \displaystyle=\mathrm{LSE}_{\varepsilon}\left(V_{l}^{(t)}+\log\tau_{0},\,I_{l}^{(t)}\right)(1-s_{l}^{(t)})+V_{\text{reset}}\cdot s_{l}^{(t)} (23) \displaystyle s_{l}^{(t+1)} \displaystyle=\sigma\left((V_{l}^{(t+1)}-\theta)/\varepsilon\right) (24) where l\in\{1,\ldots,L\} indexes layers. The output layer employs spike rate coding: \hat{y}=\frac{1}{T}\sum_{t=1}^{T}s_{L}^{(t)} (25) 5 Theoretical Analysis 5.1 Convergence to LIF Dynamics Lemma 5.1 (Sigmoid Convergence) . Let \sigma_{\varepsilon}(x):=\sigma(x/\varepsilon) . For x\neq 0 , \lim_{\varepsilon\to 0^{+}}\sigma_{\varepsilon}(x)=H(x) with exponential convergence rate |\sigma_{\varepsilon}(x)-H(x)|\leq e^{-|x|/\varepsilon} . Proof. For x>0 : \sigma_{\varepsilon}(x)=(1+e^{-x/\varepsilon})^{-1}\to 1 as \varepsilon\to 0^{+} . For x<0 : \sigma_{\varepsilon}(x)=e^{x/\varepsilon}/(e^{x/\varepsilon}+1)\to 0 as \varepsilon\to 0^{+} . The error bound follows from |1-\sigma_{\varepsilon}(x)|=e^{-x/\varepsilon}/(1+e^{-x/\varepsilon})\leq e^{-x/\varepsilon}=e^{-|x|/\varepsilon} for x>0 . For x<0 : |\sigma_{\varepsilon}(x)|=1/(1+e^{-x/\varepsilon})\leq e^{x/\varepsilon}=e^{-|x|/\varepsilon} . ∎ Proposition 5.2 (Convergence to LIF) . Let \{V_{\varepsilon}(t)\} denote the UltraLIF trajectory with temperature \varepsilon>0 , and \{v(t),s(t)\} the standard LIF trajectory with V_{\mathrm{reset}}=0 . Assume (A1) bounded inputs |I(t)|\leq I_{\max} and (A2) threshold margin \delta_{t}:=|v(t)-\theta|>0 . Then \lim_{\varepsilon\to 0^{+}}V_{\varepsilon}(t)=v(t) and \lim_{\varepsilon\to 0^{+}}s_{\varepsilon}(t)=s(t) for each t , with |V_{\varepsilon}(t)-v(t)|\leq t\cdot\varepsilon\log 2 and |s_{\varepsilon}(t)-s(t)|\leq e^{-\delta_{t}/\varepsilon} . The linear error growth follows from the 1-Lipschitz property of \mathrm{LSE}_{\varepsilon} (Lemma C.1 ) and the non-expansiveness of the reset interpolation. The set of inputs violating (A2) has Lebesgue measure zero. Proof. By strong induction on t . Base case: V_{\varepsilon}(0)=v(0) ; spike convergence by Lemma 5.1 . For the inductive step, the update F_{\varepsilon}(V)=\tilde{V}(1-s_{\varepsilon})+V_{\mathrm{reset}}\cdot s_{\varepsilon} with \tilde{V}=\mathrm{LSE}_{\varepsilon}(V+\log\tau_{0},I) satisfies: (i) no spike ( s_{\varepsilon}\to 0 ): F_{\varepsilon}\to\max(v(t)+\log\tau_{0},I(t))=v(t+1) ; (ii) spike ( s_{\varepsilon}\to 1 ): F_{\varepsilon}\to V_{\mathrm{reset}}=0 , matching LIF reset. Each step adds at most \varepsilon\log 2 error (Lemma 3.2 ); the convex reset interpolation does not amplify it. Full details in Appendix C . ∎ Corollary 5.3 (UltraDLIF Convergence) . The analogous result holds for UltraDLIF: as \varepsilon\to 0^{+} , the 3-term \mathrm{LSE}_{\varepsilon}(V_{i-1},V_{i},V_{i+1})\to\max(V_{i-1},V_{i},V_{i+1}) by Lemma 3.2 with n{=}3 , and the full UltraDLIF trajectory converges to the max-plus diffusion dynamics (Eq. 14 ) with error bound |V_{i,\varepsilon}(t)-V_{i}(t)|\leq t\cdot\varepsilon\log 3 . 5.2 Gradient Properties Proposition 5.4 (Bounded Non-Vanishing Gradients) . For any \varepsilon>0 , consider a single-step spike output s_{\varepsilon}=\sigma((\tilde{V}_{\varepsilon}-\theta)/\varepsilon) where \tilde{V}_{\varepsilon} is the pre-reset voltage. The gradient satisfies 0<\frac{\partial s_{\varepsilon}}{\partial\tilde{V}_{\varepsilon}}\leq\frac{1}{4\varepsilon} for all finite \tilde{V}_{\varepsilon} . For weights W with input x at a single layer: \left|\frac{\partial s_{\varepsilon}}{\partial W_{ij}}\right|\leq\frac{\|x\|_{\infty}}{4\varepsilon} . Proof. Let z=(V_{\varepsilon}-\theta)/\varepsilon . Then \frac{\partial s_{\varepsilon}}{\partial V_{\varepsilon}}=\frac{\sigma(z)(1-\sigma(z))}{\varepsilon} . The function \sigma(1-\sigma) achieves maximum 1/4 at \sigma=1/2 , establishing the upper bound. Positivity follows since \sigma(z)\in(0,1) for finite z . The weight gradient bound follows by chain rule: \frac{\partial s_{\varepsilon}}{\partial W_{ij}}=\frac{\partial s_{\varepsilon}}{\partial V_{\varepsilon}}\cdot\frac{\partial V_{\varepsilon}}{\partial I}\cdot x_{j} , with the \mathrm{LSE} gradient being softmax with values in (0,1) . ∎ Corollary 5.5 (Gradient Scaling) . The temperature \varepsilon controls the bias-variance trade-off in gradients: • Larger \varepsilon : smaller gradients, smoother optimization landscape • Smaller \varepsilon : larger gradients near threshold, better LIF approximation 5.3 Forward-Backward Consistency Remark 5.6 (Gradient Consistency) . For any \varepsilon>0 , UltraLIF is a composition of smooth operations ( \mathrm{LSE}_{\varepsilon} , sigmoid, affine maps), so the chain rule applies exactly: the backward pass differentiates the same function computed forward. Surrogate methods use H(V{-}\theta) forward but differentiate a surrogate g(V{-}\theta) backward; 12 show this can be interpreted as differentiating a stochastic forward pass with escape noise. UltraLIF avoids this mismatch: \nabla_{W}\mathcal{L}(f_{\varepsilon}(\mathbf{x};W)) is an exact gradient of the actual forward computation. 5.4 Connection to Tropical Geometry Theorem 5.7 (Tropical Limit) . The map D_{\varepsilon}:(\mathbb{R}_{>0},+,\cdot)\to(\mathbb{R},\oplus_{\varepsilon},+) defined by D_{\varepsilon}(x)=\varepsilon\log x , where a\oplus_{\varepsilon}b=\mathrm{LSE}_{\varepsilon}(a,b) , is a semiring homomorphism. As \varepsilon\to 0^{+} , \oplus_{\varepsilon}\to\oplus=\max (tropical addition), UltraLIF dynamics converge to a piecewise-linear map on \mathbb{R}_{\max} , and decision boundaries approach tropical hypersurfaces. Proof. D_{\varepsilon}(x\cdot y)=\varepsilon\log(xy)=D_{\varepsilon}(x)+D_{\varepsilon}(y) (multiplication \to addition) and D_{\varepsilon}(x+y)=\varepsilon\log(x+y)=\mathrm{LSE}_{\varepsilon}(D_{\varepsilon}(x),D_{\varepsilon}(y)) (addition \to soft-max). Taking \varepsilon\to 0 yields the tropical semiring by Lemma 3.2 . The piecewise-linear limit and tropical hypersurface structure follow from 36 . ∎ 6 Experiments Setup. Evaluation spans six benchmarks: static images (MNIST, Fashion-MNIST, CIFAR-10), neuromorphic vision (N-MNIST, DVS-Gesture), and audio (SHD). A single hidden layer with 64 neurons is used across all experiments, with timesteps T\in\{1,10,30\} . Baselines include LIF, PLIF, AdaLIF, FullPLIF, and DSpike/DSpike+ with surrogate gradients. All four ultradiscretized variants (UltraLIF, UltraPLIF, UltraDLIF, UltraDPLIF) are evaluated. An optional sparsity penalty \mathcal{L}=\mathcal{L}_{\text{CE}}+\lambda\cdot\bar{s} enables explicit accuracy-efficiency trade-offs. Energy is estimated via the relative synaptic operation (SOP) count T\cdot\bar{s} ( 15 ) , which is proportional to computational energy since all models share the same architecture (Appendix D ). Table 2 : Test accuracy (%) on CIFAR-10. UltraPLIF (temporal) achieves best at all timesteps. Model T{=}1 T{=}10 T{=}30 LIF 39.83 44.27 45.69 PLIF 39.83 45.06 46.15 AdaLIF 39.83 44.86 45.83 FullPLIF 39.60 45.43 46.28 DSpike 40.26 44.78 45.34 DSpike+ 40.26 45.42 46.29 Temporal (LIF ODE) UltraLIF 40.72 45.15 45.69 UltraPLIF 43.27 46.19 46.58 Spatial (Diffusion PDE) UltraDLIF 43.11 45.65 45.00 UltraDPLIF 43.11 45.75 45.74 Table 3 : Energy efficiency on CIFAR-10. Energy = relative SOP count ( T\cdot\bar{s} ). Sparsity penalty \lambda reduces spike rate with minimal accuracy loss. At T{=}30 , UltraPLIF with \lambda{=}0.1 achieves best accuracy while reducing energy by 50%. Model T Acc (%) Spike Energy LIF 1 39.83 0.404 0.40 DSpike+ 1 40.26 0.386 0.39 UltraPLIF 1 43.27 0.458 0.46 UltraPLIF ( \lambda{=}0.1 ) 1 43.60 0.240 0.24 PLIF 10 45.06 0.356 3.56 UltraDPLIF 10 45.75 0.469 4.69 UltraDPLIF ( \lambda{=}0.1 ) 10 45.32 0.338 3.38 PLIF 30 46.15 0.377 11.30 UltraPLIF 30 46.58 0.500 15.01 UltraPLIF ( \lambda{=}0.1 ) 30 46.98 0.248 7.44 Table 4 : Test accuracy (%) on MNIST. UltraDLIF (spatial) achieves best at T{=}1 ; UltraPLIF (temporal) at T{=}30 . Model T{=}1 T{=}10 T{=}30 LIF 95.34 97.45 97.45 PLIF 95.34 97.40 97.33 AdaLIF 95.34 97.43 97.45 FullPLIF 95.27 97.33 97.31 DSpike 95.58 97.38 97.48 DSpike+ 95.58 97.39 97.27 Temporal (LIF ODE) UltraLIF 94.37 97.14 97.46 UltraPLIF 95.60 97.30 97.55 Spatial (Diffusion PDE) UltraDLIF 95.67 97.35 97.38 UltraDPLIF 95.67 97.35 97.40 Table 5 : Energy efficiency on MNIST. Energy = relative SOP count ( T\cdot\bar{s} ). UltraDLIF with \lambda{=}0.1 reduces spike rate by 40% (0.446 \to 0.268). At T{=}10 , UltraDPLIF with \lambda{=}0.1 achieves 50% energy reduction. Model T Acc (%) Spike Energy LIF 1 95.34 0.388 0.39 DSpike+ 1 95.58 0.403 0.40 UltraDLIF 1 95.67 0.446 0.45 UltraDLIF ( \lambda{=}0.1 ) 1 95.71 0.268 0.27 DSpike+ 10 97.39 0.415 4.15 UltraDLIF ( \lambda{=}0.01 ) 10 97.56 0.448 4.48 UltraDPLIF ( \lambda{=}0.1 ) 10 97.35 0.237 2.37 Table 6 : Test accuracy (%) on N-MNIST (neuromorphic). UltraDLIF achieves +3.91% over baselines at T{=}1 . Model T{=}1 T{=}10 T{=}30 LIF 88.54 97.48 97.29 PLIF 88.54 97.53 97.61 AdaLIF 88.54 97.38 97.30 FullPLIF 89.00 97.50 97.48 DSpike 90.23 97.39 97.59 DSpike+ 90.23 97.55 97.65 Temporal (LIF ODE) UltraLIF 90.41 96.10 95.87 UltraPLIF 93.11 96.33 95.77 Spatial (Diffusion PDE) UltraDLIF 94.14 97.38 97.46 UltraDPLIF 94.14 97.38 97.68 Table 7 : Test accuracy (%) on DVS-Gesture (neuromorphic). UltraPLIF achieves +7.96% at T{=}1 . Model T{=}1 T{=}10 T{=}30 LIF 52.27 67.05 79.92 PLIF 52.27 68.94 78.79 AdaLIF 47.73 68.94 78.79 FullPLIF 47.73 68.56 77.27 DSpike 51.14 67.42 78.79 DSpike+ 51.14 66.67 78.41 Temporal (LIF ODE) UltraLIF 58.33 69.32 75.00 UltraPLIF 60.23 68.94 75.76 Spatial (Diffusion PDE) UltraDLIF 58.33 69.32 78.41 UltraDPLIF 58.33 68.56 79.92 Table 8 : Test accuracy (%) on SHD (audio). At T{=}1 , UltraDLIF achieves +11.22% over the best baseline (FullPLIF). Baselines lead at T{\geq}10 . Model T{=}1 T{=}10 T{=}30 LIF 27.69 72.66 72.66 PLIF 27.69 71.69 73.19 AdaLIF 27.69 74.03 71.07 FullPLIF 40.02 71.69 72.48 DSpike 38.21 72.04 70.89 DSpike+ 38.21 72.75 73.85 Temporal (LIF ODE) UltraLIF 44.88 58.79 59.14 UltraPLIF 46.91 57.73 59.45 Spatial (Diffusion PDE) UltraDLIF 51.24 67.62 71.60 UltraDPLIF 51.24 68.90 67.84 Table 9 : Summary: T{=}1 accuracy (%) across datasets. Best baseline and best ultradiscretized model shown in parentheses. Full results in Tables 2 – 8 and Appendix E . Dataset Baseline Best Ultra \Delta MNIST 95.58 (DSpike) 95.67 (DLIF) +0.09 Fashion 82.67 (DSpike) 83.02 (PLIF) +0.35 CIFAR-10 40.26 (DSpike) 43.27 (PLIF) +3.01 N-MNIST 90.23 (DSpike) 94.14 (DLIF) +3.91 DVS 52.27 (PLIF) 60.23 (PLIF) +7.96 SHD 40.02 (FullPLIF) 51.24 (DLIF) +11.22 Single-Timestep Advantage. The ultradiscretized models’ advantage is most pronounced at T{=}1 (Table 9 ), where the model must extract maximum information from a single forward pass. Crucially, gains are largest on neuromorphic and temporal datasets : SHD (+11.22%), DVS-Gesture (+7.96%), N-MNIST (+3.91%), and CIFAR-10 (+3.01%). On simpler static datasets, gains are smaller but consistent: Fashion-MNIST (+0.35%), MNIST (+0.09%). Among the two derivations, UltraDLIF (spatial) wins on N-MNIST, SHD, and MNIST, while UltraPLIF (temporal) wins on CIFAR-10, Fashion-MNIST, and DVS-Gesture (Table 15 ), suggesting both variants contribute complementary strengths. Neuromorphic Dataset Performance. Tables 6 , 7 , and 8 show that ultradiscretized models dramatically outperform baselines on neuromorphic benchmarks at T{=}1 . On N-MNIST, UltraDLIF achieves 94.14% versus 90.23% for DSpike (+3.91%). On DVS-Gesture, UltraPLIF (temporal) achieves the best result: 60.23% versus 52.27% for PLIF (+7.96%). These datasets capture asynchronous events from dynamic vision sensors, where the temporal structure is fundamental. The soft max-plus dynamics appear better suited to extract information from sparse, event-driven inputs than hard-thresholding surrogates. Sparsity-Accuracy Trade-off. Without sparsity penalty, ultradiscretized models tend to have higher spike rates than baselines (e.g., CIFAR-10 T{=}1 : UltraPLIF 0.458 vs. LIF 0.404), as the soft spike s_{\varepsilon}\in(0,1) contributes nonzero activity even below threshold. The sparsity penalty \lambda addresses this and enables flexible energy-accuracy trade-offs (Tables 3 , 5 ). On CIFAR-10, \lambda{=}0.1 reduces spike rates by 48% (0.458 \to 0.240) while actually improving accuracy (43.27% \to 43.60%). On MNIST, \lambda{=}0.1 reduces spike rates by 40% at T{=}1 (0.446 \to 0.268) while maintaining or slightly improving accuracy. Remarkably, at T{=}30 on CIFAR-10, UltraPLIF with \lambda{=}0.1 achieves both best accuracy (46.98%) and 50% energy reduction compared to the no-penalty baseline. Timestep Scaling. Performance gaps narrow at higher T , and baselines often overtake: on CIFAR-10, the T{=}1 advantage (+3.01% for UltraPLIF) persists but shrinks at T{=}10 and T{=}30 . On SHD, ultradiscretized models lead dramatically at T{=}1 (+11.22%) but baselines surpass them at T{\geq}10 . This pattern is consistent: on N-MNIST, UltraDLIF leads by +3.91% at T{=}1 but baselines lead at T{=}10 . The explanation is that surrogate gradients suffer from forward-backward mismatch, but with sufficient T , the averaging effect of spike rate coding masks individual spike errors. At low T , each spike carries more information, making the consistency of ultradiscretization more valuable. Learnable Temperature. The temperature \varepsilon implements automatic curriculum learning, starting soft (large \varepsilon ) for easy optimization then sharpening (small \varepsilon ) to approximate discrete spikes. Unlike DSpike’s heuristic sharpness parameter, \varepsilon has principled convergence guarantees (Proposition 5.2 ). Ablation studies (Appendix E.1 ) confirm that learned \varepsilon consistently outperforms fixed values, converging to the range 0.66–1.08. Computational Cost. The \mathrm{LSE} operation adds minor overhead ( \sim 5% wall-clock time) compared to standard LIF, but is fully parallelizable on GPU/TPU. 7 Discussion Temporal and Spatial Instantiations. UltraLIF and UltraDLIF arise from applying ultradiscretization to different source equations (LIF ODE vs. diffusion PDE), yielding models with distinct computational properties. UltraLIF (2-term LSE) captures temporal membrane dynamics; UltraDLIF (3-term LSE) models lateral spatial diffusion. Both share the same theoretical guarantees from Lemmas and Theorems in Section 5 . Relation to Surrogate Gradients. The soft spike ( 17 ) superficially resembles sigmoid surrogates. The key distinction is consistency : ultradiscretized neurons employ identical soft dynamics in both forward and backward passes, whereas surrogate methods use hard spikes forward with soft gradients backward. This consistency eliminates the gradient mismatch analyzed in 12 . Biological Interpretation. The temperature \varepsilon admits interpretation as neural noise or stochasticity. Biological cortical neurons exhibit highly irregular firing patterns ( 26 ) ; the ultradiscretization framework provides a principled model where \varepsilon quantifies this variability. Tropical Geometry Perspective. Theorem 5.7 connects SNN dynamics to tropical algebraic geometry. This opens avenues for applying tropical techniques such as Newton polytopes, Bezout bounds, and tropical Nullstellensatz to analyze SNN expressivity and decision boundaries. Consequence: A New Activation Family. A direct consequence of the ultradiscretization framework is that UltraLIF subsumes classical activation functions: \mathrm{LSE}_{\varepsilon}(0,x) recovers the softplus (and ReLU as \varepsilon\to 0 ), while s_{\varepsilon} is a scaled sigmoid. The learnable \varepsilon thus interpolates between hard spiking and smooth regimes. Details and connections to morphological neural networks are in Appendix B . Limitations. UltraDLIF and UltraDPLIF produce identical results in our experiments, indicating that the learnable leak \tau does not differentiate the spatial variant; the 3-term LSE dominates the dynamics. Tasks requiring precise spike timing may benefit differently from temporal (UltraLIF) versus spatial (UltraDLIF) variants. The soft spike s_{\varepsilon}\in(0,1) during training deviates from binary spikes; however, as discussed in Section 4.2 , this is addressed by: (1) using hard thresholding at inference for neuromorphic deployment, (2) the convergence guarantee s_{\varepsilon}\to H(V-\theta) as \varepsilon\to 0 , and (3) the learned \varepsilon converging to moderate values (0.66–1.08) that balance differentiability and spike sharpness (Figure 3 ). 8 Conclusion This paper introduced UltraLIF, a principled framework for differentiable spiking neural networks grounded in ultradiscretization from tropical geometry. Two neuron models, UltraLIF (temporal, from the LIF ODE capturing membrane decay) and UltraDLIF (spatial, from the diffusion PDE modeling gap junction coupling across neuronal populations), use log-sum-exp as a soft relaxation of max-plus dynamics, yielding fully differentiable SNNs without surrogate gradients. Theoretical analysis establishes convergence to classical LIF dynamics, bounded gradients, and forward-backward consistency. Experiments on six benchmarks demonstrate improvements over surrogate gradient baselines, with the largest gains at T{=}1 on neuromorphic and audio data (SHD +11.22%, DVS +7.96%, N-MNIST +3.91%). An optional sparsity penalty enables significant energy reduction while maintaining accuracy. The connection to tropical geometry opens new directions for principled analysis of spiking computation. Impact Statement This paper presents theoretical work whose goal is to advance the field of Machine Learning, specifically in the domain of energy-efficient spiking neural networks. The ultradiscretization framework provides mathematical foundations that could accelerate deployment of neuromorphic computing systems, potentially reducing the energy footprint of machine learning applications. All experiments are validated on existing public benchmark datasets. There are many potential societal consequences of this work, none of which must be specifically highlighted here. Appendix A Spike Mechanism Comparison Figure 2 : Comparison of spike mechanisms. (a) Traditional LIF uses Heaviside H(V-\theta) in the forward pass but a smooth surrogate \sigma^{\prime} for gradients, creating forward-backward mismatch (shaded region). (b) Ultradiscretized LIF (temporal, 2-term LSE from LIF ODE) and (c) Ultradiscretized DLIF (spatial, 3-term LSE from diffusion PDE) use identical smooth functions in both passes, ensuring gradient consistency. The membrane potential equations below each panel show the distinct derivations. Appendix B Additional Discussion UltraLIF as a New Activation Family. The ultradiscretization framework yields a novel activation function family with deep connections to existing architectures. Observe that \mathrm{LSE}_{\varepsilon}(0,x)=\varepsilon\log(1+\exp(x/\varepsilon)) is precisely the softplus function, which converges to \mathrm{ReLU}(x)=\max(0,x) as \varepsilon\to 0 . Thus, the membrane dynamics in UltraLIF generalize ReLU-family activations to the spiking domain. Meanwhile, the spike function s_{\varepsilon}=\sigma((V-\theta)/\varepsilon) is a shifted, scaled sigmoid, i.e., a soft step function. Together, UltraLIF combines soft-max aggregation (ReLU family) with soft-step thresholding (sigmoid family), both controlled by the learnable temperature \varepsilon . This provides a principled spectrum of activations: • \varepsilon\to 0 : Hard max + hard step (classical LIF, non-differentiable) • \varepsilon\to\infty : Linear + constant (no nonlinearity) • \varepsilon learned: Optimal sharpness (UltraLIF, fully differentiable) Unlike heuristic surrogate gradients, this activation family has rigorous theoretical grounding in tropical geometry and maintains forward-backward consistency by design. Connection to Morphological Neural Networks. The spatial max operation in UltraDLIF (Eq. 14 ) corresponds to morphological dilation , a fundamental operation in mathematical morphology. Deep morphological networks ( 9 ) show that max pooling is equivalent to dilation with a flat structuring element. This connection suggests that UltraDLIF performs learned morphological operations on neural activation patterns, providing an alternative interpretation grounded in image processing theory. Appendix C Proofs C.1 Proof of Proposition 5.2 The proof proceeds by strong induction on t , tracking errors explicitly at each step. At the base case t=0 , the voltage error is zero by initialization ( V_{\varepsilon}(0)=v_{0}=v(0) ), and the spike error |s_{\varepsilon}(0)-s(0)|\leq e^{-\delta_{0}/\varepsilon}\to 0 follows from Lemma 5.1 and the threshold margin assumption (A2). For the inductive step, assume |V_{\varepsilon}(k)-v(k)|\leq k\cdot\varepsilon\log 2 holds for all k\leq t , and consider the UltraLIF update F_{\varepsilon}(V)=\tilde{V}(1-s_{\varepsilon})+V_{\mathrm{reset}}\cdot s_{\varepsilon} with \tilde{V}=\mathrm{LSE}_{\varepsilon}(V+\log\tau_{0},I) at step t+1 . When no spike occurs at t , the inductive hypothesis and Lemma 5.1 imply s_{\varepsilon}(t)\to 0 , so the reset interpolation reduces to V_{\varepsilon}(t+1)\approx\tilde{V}_{\varepsilon}(t+1)=\mathrm{LSE}_{\varepsilon}(V_{\varepsilon}(t)+\log\tau_{0},I(t)) . Since \mathrm{LSE}_{\varepsilon} is 1-Lipschitz (Lemma C.1 ), the accumulated error propagates with factor at most 1, and the intrinsic LSE approximation contributes at most \varepsilon\log 2 (Lemma 3.2 with n=2 ), giving |V_{\varepsilon}(t{+}1)-v(t{+}1)|\leq|V_{\varepsilon}(t)-v(t)|+\varepsilon\log 2\leq(t{+}1)\varepsilon\log 2 . When a spike does occur, s_{\varepsilon}(t)\to 1 drives the reset interpolation toward V_{\mathrm{reset}}=0 , matching the standard LIF reset exactly; the error thus resets to O(\varepsilon) , dominated by (t{+}1)\varepsilon\log 2 . Crucially, the reset interpolation \tilde{V}(1-s)+V_{\mathrm{reset}}\cdot s is a convex combination for s\in(0,1) , making it non-expansive: perturbations in \tilde{V} are damped by the factor (1-s)\leq 1 . Combined with the 1-Lipschitz property of \mathrm{LSE}_{\varepsilon} , the one-step map never amplifies errors, yielding the linear growth C_{t}=t . Spike convergence at t+1 then follows from Lemma 5.1 : |s_{\varepsilon}(t{+}1)-s(t{+}1)|\leq e^{-\delta_{t+1}/\varepsilon} under Assumption (A2). ∎ C.2 Lipschitz Properties Lemma C.1 . For fixed \varepsilon>0 , \mathrm{LSE}_{\varepsilon}:\mathbb{R}^{n}\to\mathbb{R} is 1-Lipschitz in \|\cdot\|_{\infty} , and \sigma_{\varepsilon}:\mathbb{R}\to(0,1) is (4\varepsilon)^{-1} -Lipschitz. Proof. By the mean value theorem, |\mathrm{LSE}_{\varepsilon}(\mathbf{x})-\mathrm{LSE}_{\varepsilon}(\mathbf{y})|\leq\sup_{\mathbf{z}}|\langle\nabla\mathrm{LSE}_{\varepsilon}(\mathbf{z}),\mathbf{x}-\mathbf{y}\rangle| . Since \nabla\mathrm{LSE}_{\varepsilon}=\mathrm{softmax}(\cdot/\varepsilon) has \|\nabla\mathrm{LSE}_{\varepsilon}\|_{1}=1 (Lemma 3.2 ), Hölder’s inequality gives |\langle\nabla\mathrm{LSE}_{\varepsilon},\mathbf{x}-\mathbf{y}\rangle|\leq\|\nabla\mathrm{LSE}_{\varepsilon}\|_{1}\cdot\|\mathbf{x}-\mathbf{y}\|_{\infty}=\|\mathbf{x}-\mathbf{y}\|_{\infty} . The sigmoid bound follows from |\sigma_{\varepsilon}^{\prime}(x)|\leq 1/(4\varepsilon) (Proposition 5.4 ). ∎ C.3 Tropical Geometry Analysis Theorem 5.7 establishes that UltraLIF dynamics converge to piecewise-linear maps on the max-plus semiring, with decision boundaries approaching tropical hypersurfaces. Three concrete extensions exploiting this connection are developed below, providing explicit expressivity bounds, temporal dynamics analysis, and capacity results. C.3.1 Tropical Characterization of Decision Boundaries An explicit geometric characterization of UltraLIF decision boundaries in the tropical limit is provided. Setup. Consider a single-hidden-layer UltraLIF network with h neurons, n inputs, and C output classes at T{=}1 in the tropical limit ( \varepsilon\to 0^{+} ). Starting from V^{(0)}=\mathbf{0} , each hidden neuron j\in[h] computes: V_{j}^{(1)}=\max(\log\tau_{0},\;\mathbf{w}_{j}^{\top}\mathbf{x}) (26) where \mathbf{w}_{j}\in\mathbb{R}^{n} is the weight vector for neuron j . Definition C.2 (Hyperplane Arrangement) . Each neuron j defines a hyperplane \mathcal{H}_{j}:=\{\mathbf{x}\in\mathbb{R}^{n}:\mathbf{w}_{j}^{\top}\mathbf{x}=\log\tau_{0}\} . The collection \mathcal{A}=\{\mathcal{H}_{1},\ldots,\mathcal{H}_{h}\} partitions \mathbb{R}^{n} into connected regions where the spike pattern \mathbf{s}\in\{0,1\}^{h} is constant. Proposition C.3 (Hyperplane Arrangement Bound) . A single-hidden-layer UltraLIF partitions \mathbb{R}^{n} into at most R(h,n):=\sum_{k=0}^{\min(n,h)}\binom{h}{k} linear regions. Within each region, the network output is constant. Proof. Each neuron defines a half-space \mathcal{H}_{j}^{+}=\{\mathbf{x}:\mathbf{w}_{j}^{\top}\mathbf{x}>\log\tau_{0}\} where s_{j}=1 . The spike pattern is determined by membership in intersections of these half-spaces. The number of regions created by h hyperplanes in \mathbb{R}^{n} in general position (Theorem C.6 ) is ( 33 ) : r_{n}(\mathcal{A})=\sum_{k=0}^{n}\binom{h}{k} When hh vanish since \binom{h}{k}=0 for k>h , yielding the equivalent formula \sum_{k=0}^{\min(n,h)}\binom{h}{k} . Within each region, \mathbf{s} is constant, so the output \hat{y}_{c}=\sum_{j:s_{j}=1}W^{\mathrm{out}}_{cj} is constant. ∎ Tropical Hypersurface Structure. The decision boundary \mathcal{B}_{ij}=\{\mathbf{x}:\hat{y}_{i}(\mathbf{x})=\hat{y}_{j}(\mathbf{x})\} is a union of (n{-}1) -faces of the arrangement where class scores are equal. This realizes Theorem 5.7 ’s connection: \mathcal{B}_{ij} is a tropical hypersurface in the sense that it is the locus where two piecewise-linear functions achieve equality. C.3.2 Temporal Expressivity Amplification Proposition C.4 (Exponential Growth) . A single-hidden-layer UltraLIF unrolled for T timesteps partitions \mathbb{R}^{n} into at most R(h,n)^{T} regions. Proof. Let \phi^{(t)}:\mathbb{R}^{n}\to\mathbb{R}^{n} denote the map from input to hidden layer membrane potentials at timestep t . Each \phi^{(t)} is piecewise-linear with at most R(h,n) regions by Proposition C.3 . The T -step computation is the composition \phi^{(T)}\circ\cdots\circ\phi^{(1)} . By induction on T , the composition has at most R(h,n)^{T} linear regions. Base case ( T=1 ): immediate from Proposition C.3 . Inductive step: assume the claim holds for T-1 . Then \phi^{(T-1)}\circ\cdots\circ\phi^{(1)} has at most R(h,n)^{T-1} regions. Composing with \phi^{(T)} (which has R(h,n) regions) yields at most R(h,n)^{T-1}\cdot R(h,n)=R(h,n)^{T} regions, since each region of the (T-1) -step map can be subdivided by the R(h,n) hyperplanes of \phi^{(T)} . ∎ T=1 vs T \geq 10 Analysis. At T{=}1 , expressivity is R(h,n)\approx 10^{19} for typical settings, far exceeding dataset size. The advantage comes from gradient quality: UltraLIF’s forward-backward consistency avoids spurious local minima induced by surrogate gradient mismatch. At T{\geq}10 , expressivity grows to R(h,n)^{10} . Output averaging \hat{y}=\frac{1}{T}\sum_{t}W^{\mathrm{out}}\mathbf{s}^{(t)} smooths individual spike errors. Gradient mismatch becomes less critical as errors cancel across timesteps, explaining why baselines recover (SHD: UltraLIF +11.22% at T{=}1 but -2.16\% at T{=}30 ). C.3.3 Zonotope Volume and Expressivity Definition C.5 (Zonotope) . The zonotope generated by \mathbf{w}_{1},\ldots,\mathbf{w}_{h}\in\mathbb{R}^{n} is \mathcal{Z}(\mathbf{w}_{1},\ldots,\mathbf{w}_{h}):=\{\sum_{i=1}^{h}\lambda_{i}\mathbf{w}_{i}:\lambda_{i}\in[0,1]\} . The volume \mathrm{vol}_{n}(\mathcal{Z}) quantifies geometric diversity of weight directions. When \mathrm{vol}_{n}(\mathcal{Z})=0 , weights are linearly dependent and the arrangement degenerates. Theorem C.6 (Volume-Expressivity Connection) . Let W=[\mathbf{w}_{1},\ldots,\mathbf{w}_{h}]^{\top}\in\mathbb{R}^{h\times n} be the weight matrix. The hyperplane arrangement achieves the maximal region count R(h,n)=\sum_{k}\binom{h}{k} if and only if the hyperplanes are in general position. General position holds if and only if: 1. For any n{+}1 weight vectors \mathbf{w}_{i_{1}},\ldots,\mathbf{w}_{i_{n+1}} , no point \mathbf{x} satisfies all n{+}1 hyperplane equations simultaneously. 2. Any subset of n weight vectors \{\mathbf{w}_{i_{1}},\ldots,\mathbf{w}_{i_{n}}\} with n\leq\min(h,\dim(\mathbb{R}^{n})) has \det(\mathbf{w}_{i_{1}},\ldots,\mathbf{w}_{i_{n}})\neq 0 when n=\dim(\mathbb{R}^{n}) . Moreover, if h\geq n and the weight matrix W has rank n , then \mathrm{vol}_{n}(\mathcal{Z}(\mathbf{w}_{1},\ldots,\mathbf{w}_{h}))>0 implies the arrangement is non-degenerate with at least 2^{n} regions. Proof. The first statement follows from Zaslavsky’s characterization of general position ( 33 ) : the arrangement achieves the maximal count when no n{+}1 hyperplanes meet at a point and all intersections are transverse. Condition (1) ensures no common intersection of n{+}1 hyperplanes. Condition (2) ensures transversality: any n hyperplanes intersect at a unique point (when n weight vectors are linearly independent) rather than a higher-dimensional face. For the volume statement, suppose h\geq n and \mathrm{rank}(W)=n . Then there exist n linearly independent weight vectors, say \mathbf{w}_{i_{1}},\ldots,\mathbf{w}_{i_{n}} . The zonotope contains the parallelepiped spanned by these vectors: \mathcal{P}=\{\sum_{j=1}^{n}\lambda_{j}\mathbf{w}_{i_{j}}:\lambda_{j}\in[0,1]\} The volume satisfies \mathrm{vol}_{n}(\mathcal{Z})\geq\mathrm{vol}_{n}(\mathcal{P})=|\det(\mathbf{w}_{i_{1}},\ldots,\mathbf{w}_{i_{n}})|>0 by linear independence. These n linearly independent hyperplanes partition \mathbb{R}^{n} into at least 2^{n} regions (the 2^{n} orthants when hyperplanes pass through the origin, or their translated analogues). Thus R\geq 2^{n} when \mathrm{vol}_{n}(\mathcal{Z})>0 . Conversely, if \mathrm{vol}_{n}(\mathcal{Z})=0 , the weight vectors lie in a proper subspace of dimension 0 Implications. Initialization. Standard random initialization (Kaiming, Xavier) samples from isotropic Gaussians, yielding near-orthogonal weights with high probability in high dimensions. This ensures \mathrm{vol}_{n}(\mathcal{Z})>0 and non-degenerate arrangements. Explicit orthogonalization (QR decomposition) maximizes volume for fixed norms. Regularization. A volume-regularized loss \mathcal{L}=\mathcal{L}_{\text{CE}}-\alpha\log\mathrm{vol}_{n}(\mathcal{Z}) encourages diverse weight directions, preventing rank collapse. Pruning. If \mathbf{w}_{j}\approx c\cdot\mathbf{w}_{k} , removing neuron j reduces \mathrm{vol}_{n}(\mathcal{Z}) negligibly, providing a principled pruning criterion based on geometric redundancy. Appendix D Experimental Setup D.1 Datasets and Preprocessing Evaluation is conducted on six benchmarks spanning static images, neuromorphic vision, and audio: Static Image Datasets. • MNIST : 28{\times}28 grayscale handwritten digits, 10 classes. 60,000 train / 10,000 test samples. Normalization: mean 0.1307, std 0.3081. • Fashion-MNIST : 28{\times}28 grayscale fashion items, 10 classes. 60,000 train / 10,000 test samples. Normalization: mean 0.2860, std 0.3530. • CIFAR-10 : 32{\times}32 RGB natural images, 10 classes. 50,000 train / 10,000 test samples. Normalization per channel: mean (0.4914, 0.4822, 0.4465), std (0.247, 0.243, 0.262). Training augmentation: random crop (32 \times 32 with 4-pixel padding), random horizontal flip. Neuromorphic Datasets. • N-MNIST : Neuromorphic MNIST captured with DVS camera. 60,000 train / 10,000 test samples. Input dimension: 2{\times}34{\times}34 (ON/OFF polarity channels). • DVS-Gesture : 11 hand gesture classes recorded with DVS128 camera. 1,176 train / 288 test samples. Input dimension: 2{\times}128{\times}128 . Audio Dataset. • SHD : Spiking Heidelberg Digits, 20 spoken digit classes. 8,156 train / 2,264 test samples. Input dimension: 700 frequency channels. Temporal Encoding. For static datasets, rate coding converts pixel intensities to spike trains: P(\text{spike at }t)=\text{gain}\cdot x_{\text{pixel}},\quad\text{gain}=0.5 (27) where x_{\text{pixel}}\in[0,1] is the normalized pixel value. For neuromorphic datasets, events are binned into T temporal frames using the Tonic library’s ToFrame transform with n_time_bins {}=T . D.2 Model Hyperparameters Table 10 : Hyperparameters for all neuron models. Parameter Value Description Common parameters \theta (threshold) 0.5 Spike threshold \tau_{0} (leak) 0.9 Membrane time constant V_{\text{reset}} 0.0 Reset potential Surrogate gradient (baselines) \beta (sharpness) 10.0 Sigmoid surrogate steepness AdaLIF specific \beta_{\text{adapt}} 0.1 Threshold adaptation strength \tau_{\text{adapt}} 0.9 Adaptation decay constant DSpike specific b_{0} (init) 4.0 Initial sharpness parameter Ultradiscretized models \varepsilon_{0} (init) 1.0 Initial temperature \varepsilon range [0.1, 20.0] Clamped during training D.3 Architecture Single hidden layer with 64 neurons. Input-to-hidden and hidden-to-output are fully connected layers. Timesteps T\in\{1,10,30\} to evaluate performance across temporal regimes. Output is computed as mean spike rate over time: \hat{y}=\frac{1}{T}\sum_{t=1}^{T}W_{\text{out}}\cdot s^{(t)} (28) D.4 Baseline Model Definitions All baselines use surrogate gradients: hard spike s=H(v-\theta) in the forward pass, smooth gradient \partial s/\partial v=\sigma^{\prime}((v-\theta)\beta) in the backward pass. LIF (Leaky Integrate-and-Fire): \displaystyle v^{(t+1)} \displaystyle=\tau v^{(t)}+I^{(t)} (29) \displaystyle s^{(t+1)} \displaystyle=H(v^{(t+1)}-\theta),\quad v\leftarrow v(1-s) (30) PLIF (Parametric LIF, 7 ): \displaystyle v^{(t+1)} \displaystyle=\tau v^{(t)}+I^{(t)},\quad\tau=\sigma(\tau_{\text{param}})\text{ learnable} (31) AdaLIF (Adaptive LIF, 1 ): \displaystyle v^{(t+1)} \displaystyle=\tau v^{(t)}+I^{(t)} (32) \displaystyle\theta^{(t+1)} \displaystyle=\theta_{0}+\beta_{\text{adapt}}\cdot b^{(t)} (33) \displaystyle b^{(t+1)} \displaystyle=\tau_{\text{adapt}}\cdot b^{(t)}+(1-\tau_{\text{adapt}})\cdot s^{(t)} (34) where b is the adaptation variable that increases after spikes. FullPLIF (Fully Parametric LIF): \displaystyle v^{(t+1)} \displaystyle=\tau v^{(t)}+I^{(t)} (35) \displaystyle\tau \displaystyle=\sigma(\tau_{\text{param}}),\quad\theta=\sigma(\theta_{\text{param}}) (36) Both \tau and \theta are learnable, constrained to (0,1) via sigmoid. DSpike ( 16 ): \displaystyle v^{(t+1)} \displaystyle=\tau v^{(t)}+I^{(t)} (37) \displaystyle s^{(t+1)} \displaystyle=\frac{\tanh(k(v_{\text{norm}}-0.5))+\tanh(k/2)}{2\tanh(k/2)} (38) where v_{\text{norm}}=v/(2\theta) normalizes membrane potential to [0,1] , and k is a learnable sharpness parameter (initialized to 4.0). DSpike+ : DSpike with learnable \tau=\sigma(\tau_{\text{param}}) . D.5 Proposed Methods Ultradiscretized variants use soft spike in both forward and backward passes (no surrogate gradients): UltraLIF (Temporal, 2-term LSE from LIF ODE): \displaystyle V^{(t+1)} \displaystyle=\mathrm{LSE}_{\varepsilon}(V^{(t)}+\log\tau_{0},\,I^{(t)}) (39) \displaystyle s_{\varepsilon}^{(t+1)} \displaystyle=\sigma((V^{(t+1)}-\theta)/\varepsilon) (40) UltraDLIF (Spatial, 3-term LSE from diffusion PDE): \displaystyle V_{i}^{(t+1)} \displaystyle=\mathrm{LSE}_{\varepsilon}(V_{i-1}^{(t)},V_{i}^{(t)},V_{i+1}^{(t)})+I_{i}^{(t)} (41) \displaystyle s_{i,\varepsilon}^{(t+1)} \displaystyle=\sigma((V_{i}^{(t+1)}-\theta)/\varepsilon) (42) UltraPLIF and UltraDPLIF add learnable \tau=\sigma(\tau_{\text{param}}) . D.6 Sparsity Penalty To encourage energy efficiency, ultradiscretized variants support an optional sparsity penalty: \mathcal{L}=\mathcal{L}_{\text{CE}}+\lambda\cdot\bar{s} (43) where \bar{s} is the mean spike rate and \lambda\in\{0,0.01,0.1\} . This enables explicit control over the accuracy-efficiency trade-off. D.7 Training Details Table 11 : Training configuration. Parameter Value Optimizer Adam Learning rate 10^{-3} Batch size 128 Epochs 100 LR scheduler Cosine annealing Weight init PyTorch default (Kaiming) Seed 42 Loss Function. Cross-entropy on mean spike rates: \mathcal{L}_{\text{CE}}=-\sum_{c}y_{c}\log(\text{softmax}(\hat{y})_{c}) (44) Energy Estimation. SNN energy consumption is estimated using the standard synaptic operation (SOP) framework ( 15 ) . In SNNs, spike-driven computation replaces multiply-accumulate (MAC) operations with accumulate-only (AC) operations, since pre-synaptic activations are binary: \displaystyle E_{\text{SNN}} \displaystyle=T\cdot\bar{s}\cdot N_{\text{syn}}\cdot E_{\text{AC}} (45) \displaystyle E_{\text{ANN}} \displaystyle=N_{\text{syn}}\cdot E_{\text{MAC}} (46) where T is the number of timesteps, \bar{s} is the mean spike rate, N_{\text{syn}} is the number of synaptic connections, E_{\text{AC}}\approx 0.9\,\text{pJ} , and E_{\text{MAC}}\approx 4.6\,\text{pJ} at 45nm technology ( 13 ) . Since all models in this work share the same architecture (and thus the same N_{\text{syn}} ), the energy column in results tables reports the relative SOP count T\cdot\bar{s} , which is proportional to E_{\text{SNN}} up to a constant factor. This enables direct energy comparison across models. Note that this estimate focuses on computational energy and does not account for memory access or data movement overhead, which can be significant in practice ( 32 ) . D.8 Hardware and Compute All experiments were conducted on NVIDIA T4 GPUs (16GB VRAM) using PyTorch 2.0+ with torch.compile for acceleration. Training time per model: • MNIST/Fashion: \sim 3 minutes (100 epochs) • CIFAR-10: \sim 8 minutes (100 epochs) • N-MNIST: \sim 15 minutes (100 epochs) • DVS-Gesture: \sim 20 minutes (100 epochs) • SHD: \sim 10 minutes (100 epochs) Total compute for all experiments: approximately 50 GPU-hours across three T4 VMs. Note on neuromorphic hardware. All experiments in this work were conducted on conventional GPUs. Deployment on dedicated neuromorphic hardware (e.g., Intel Loihi 2, IBM TrueNorth, SpiNNaker 2, BrainScaleS-2) has not yet been evaluated. Since the ultradiscretized spike function converges to hard thresholding as \varepsilon\to 0 (Proposition 5.2 ), the trained models can in principle be mapped to neuromorphic substrates by quantizing the soft spike to binary at inference time. Benchmarking latency, energy consumption, and accuracy on neuromorphic platforms is an important direction for future work. Appendix E Additional Experiments Figure 3 : Epsilon ablation on MNIST ( T{=}1 , 100 epochs). (a) Learned \varepsilon exhibits a characteristic U-shaped trajectory: initial drop from 1.0 to \sim 0.42 (sharpening phase), followed by recovery to model-specific optima (0.66–1.08). This suggests the network first learns sharp discrimination, then softens for generalization. (b) Learned \varepsilon (dashed lines) consistently matches or exceeds all fixed values across models, validating the benefit of learnable temperature. E.1 Ablation Study: Learnable Temperature \varepsilon A key design choice in ultradiscretized neurons is whether to fix the temperature parameter \varepsilon or learn it during training. This ablation study on MNIST at T{=}1 compares fixed values \varepsilon\in\{0.5,1.0,2.0\} against learned \varepsilon (initialized to 1.0) across all four ultradiscretized variants. Table 12 : Ablation: Effect of learnable \varepsilon on accuracy (%). Learned \varepsilon consistently achieves best or tied-best accuracy across all models. Model \varepsilon{=}0.5 \varepsilon{=}1.0 \varepsilon{=}2.0 Learned Temporal (LIF ODE) UltraLIF 96.56 96.26 95.71 96.79 UltraPLIF 96.61 96.48 96.26 96.63 Spatial (Diffusion PDE) UltraDLIF 96.70 96.54 96.39 96.71 UltraDPLIF 96.70 96.54 96.39 96.71 Table 13 : Ablation: Effect of learnable \varepsilon on spike rate. Learned \varepsilon achieves lowest spike rates for spatial models. Model \varepsilon{=}0.5 \varepsilon{=}1.0 \varepsilon{=}2.0 Learned Temporal (LIF ODE) UltraLIF 0.488 0.579 0.650 0.500 UltraPLIF 0.427 0.459 0.528 0.443 Spatial (Diffusion PDE) UltraDLIF 0.417 0.412 0.423 0.393 UltraDPLIF 0.417 0.412 0.423 0.393 Key findings: (1) Learned \varepsilon provides consistent accuracy gains across all models, though margins are small on MNIST. (2) Smaller fixed \varepsilon (0.5) outperforms larger values (2.0), consistent with Lemma 3.2 , as tighter approximation to the max yields better LIF emulation. (3) Learned \varepsilon converges to the range 0.66–1.08 (Table 14 ), automatically finding optimal soft-to-hard trade-offs. (4) Spatial models achieve lowest spike rates with learned \varepsilon , suggesting the network learns to balance accuracy and efficiency. Table 14 : Final learned \varepsilon values after training (initialized at 1.0). Model Final \varepsilon UltraLIF (temporal) 0.661 UltraPLIF (temporal) 0.900 UltraDLIF (spatial) 1.084 UltraDPLIF (spatial) 1.084 Table 15 reports full results on Fashion-MNIST. UltraPLIF (temporal) achieves the best accuracy at T{=}1 , while baselines lead at higher timesteps. Table 15 : Test accuracy (%) on Fashion-MNIST. UltraPLIF (temporal) achieves best at T{=}1 . Baselines lead at T{\geq}10 . Model T{=}1 T{=}10 T{=}30 LIF 82.45 86.06 86.70 PLIF 82.45 85.99 86.48 AdaLIF 82.45 86.12 86.59 FullPLIF 82.18 86.26 86.33 DSpike 82.67 86.24 86.42 DSpike+ 82.67 86.03 86.76 Temporal (LIF ODE) UltraLIF 81.79 85.76 86.65 UltraPLIF 83.02 86.03 86.59 Spatial (Diffusion PDE) UltraDLIF 82.79 85.88 86.01 UltraDPLIF 82.79 85.69 85.92 E.2 Full Sparsity Results Tables 16 – 21 present full sparsity results for all ultradiscretized models across sparsity penalty values \lambda\in\{0,0.01,0.1\} and timesteps T\in\{1,10,30\} . Table 16 : Sparsity results on MNIST. Accuracy (%), spike rate, and relative SOP count ( T\cdot\bar{s} ). Model \lambda Acc Spike Energy T{=}1 UltraLIF 0 94.37 0.620 0.62 UltraLIF 0.01 94.55 0.608 0.61 UltraLIF 0.1 94.37 0.537 0.54 UltraPLIF 0 95.60 0.468 0.47 UltraPLIF 0.01 95.50 0.453 0.45 UltraPLIF 0.1 95.81 0.289 0.29 UltraDLIF 0 95.67 0.446 0.45 UltraDLIF 0.01 95.62 0.425 0.43 UltraDLIF 0.1 95.71 0.268 0.27 UltraDPLIF 0 95.67 0.446 0.45 UltraDPLIF 0.01 95.62 0.425 0.43 UltraDPLIF 0.1 95.71 0.268 0.27 T{=}10 UltraLIF 0 97.14 0.519 5.19 UltraLIF 0.01 97.15 0.498 4.98 UltraLIF 0.1 97.23 0.328 3.28 UltraPLIF 0 97.30 0.492 4.92 UltraPLIF 0.01 97.28 0.466 4.66 UltraPLIF 0.1 97.37 0.242 2.42 UltraDLIF 0 97.35 0.479 4.79 UltraDLIF 0.01 97.56 0.448 4.48 UltraDLIF 0.1 97.19 0.254 2.54 UltraDPLIF 0 97.35 0.476 4.76 UltraDPLIF 0.01 97.37 0.444 4.44 UltraDPLIF 0.1 97.35 0.237 2.37 T{=}30 UltraLIF 0 97.46 0.502 15.07 UltraLIF 0.01 97.41 0.475 14.26 UltraLIF 0.1 97.51 0.270 8.09 UltraPLIF 0 97.55 0.492 14.76 UltraPLIF 0.01 97.52 0.464 13.91 UltraPLIF 0.1 97.53 0.229 6.87 UltraDLIF 0 97.38 0.469 14.07 UltraDLIF 0.01 97.52 0.427 12.81 UltraDLIF 0.1 97.11 0.208 6.24 UltraDPLIF 0 97.40 0.481 14.43 UltraDPLIF 0.01 97.34 0.439 13.18 UltraDPLIF 0.1 96.99 0.209 6.27 Table 17 : Sparsity results on Fashion-MNIST. Accuracy (%), spike rate, and relative SOP count ( T\cdot\bar{s} ). Model \lambda Acc Spike Energy T{=}1 UltraLIF 0 81.79 0.652 0.65 UltraLIF 0.01 82.06 0.629 0.63 UltraLIF 0.1 81.62 0.530 0.53 UltraPLIF 0 83.02 0.472 0.47 UltraPLIF 0.01 82.81 0.451 0.45 UltraPLIF 0.1 83.26 0.279 0.28 UltraDLIF 0 82.79 0.429 0.43 UltraDLIF 0.01 83.01 0.411 0.41 UltraDLIF 0.1 83.05 0.267 0.27 UltraDPLIF 0 82.79 0.429 0.43 UltraDPLIF 0.01 83.01 0.411 0.41 UltraDPLIF 0.1 83.05 0.267 0.27 T{=}10 UltraLIF 0 85.76 0.522 5.22 UltraLIF 0.01 86.07 0.510 5.10 UltraLIF 0.1 85.98 0.334 3.34 UltraPLIF 0 86.03 0.493 4.93 UltraPLIF 0.01 85.93 0.457 4.57 UltraPLIF 0.1 86.11 0.273 2.73 UltraDLIF 0 85.88 0.456 4.56 UltraDLIF 0.01 85.63 0.442 4.42 UltraDLIF 0.1 85.74 0.292 2.92 UltraDPLIF 0 85.69 0.456 4.56 UltraDPLIF 0.01 85.86 0.427 4.27 UltraDPLIF 0.1 85.74 0.278 2.78 T{=}30 UltraLIF 0 86.65 0.507 15.22 UltraLIF 0.01 86.46 0.485 14.56 UltraLIF 0.1 86.36 0.287 8.60 UltraPLIF 0 86.59 0.480 14.41 UltraPLIF 0.01 86.49 0.453 13.59 UltraPLIF 0.1 86.72 0.271 8.13 UltraDLIF 0 86.01 0.467 14.01 UltraDLIF 0.01 85.81 0.429 12.88 UltraDLIF 0.1 85.94 0.274 8.23 UltraDPLIF 0 85.92 0.463 13.88 UltraDPLIF 0.01 85.80 0.458 13.75 UltraDPLIF 0.1 85.84 0.276 8.27 Table 18 : Sparsity results on CIFAR-10. Accuracy (%), spike rate, and relative SOP count ( T\cdot\bar{s} ). Model \lambda Acc Spike Energy T{=}1 UltraLIF 0 40.72 0.706 0.71 UltraLIF 0.01 40.58 0.662 0.66 UltraLIF 0.1 39.81 0.459 0.46 UltraPLIF 0 43.27 0.458 0.46 UltraPLIF 0.01 43.22 0.444 0.44 UltraPLIF 0.1 43.60 0.240 0.24 UltraDLIF 0 43.11 0.481 0.48 UltraDLIF 0.01 43.13 0.465 0.47 UltraDLIF 0.1 43.04 0.337 0.34 UltraDPLIF 0 43.11 0.481 0.48 UltraDPLIF 0.01 43.13 0.465 0.47 UltraDPLIF 0.1 43.04 0.337 0.34 T{=}10 UltraLIF 0 45.15 0.504 5.04 UltraLIF 0.01 44.72 0.494 4.94 UltraLIF 0.1 45.18 0.311 3.11 UltraPLIF 0 46.19 0.494 4.94 UltraPLIF 0.01 46.06 0.457 4.57 UltraPLIF 0.1 46.13 0.241 2.41 UltraDLIF 0 45.65 0.471 4.71 UltraDLIF 0.01 45.58 0.452 4.52 UltraDLIF 0.1 45.39 0.334 3.34 UltraDPLIF 0 45.75 0.469 4.69 UltraDPLIF 0.01 46.26 0.452 4.52 UltraDPLIF 0.1 45.32 0.338 3.38 T{=}30 UltraLIF 0 45.69 0.480 14.39 UltraLIF 0.01 45.84 0.466 13.99 UltraLIF 0.1 45.92 0.285 8.54 UltraPLIF 0 46.58 0.500 15.01 UltraPLIF 0.01 46.31 0.485 14.56 UltraPLIF 0.1 46.98 0.248 7.44 UltraDLIF 0 45.00 0.491 14.73 UltraDLIF 0.01 45.32 0.446 13.38 UltraDLIF 0.1 45.41 0.322 9.65 UltraDPLIF 0 45.74 0.496 14.89 UltraDPLIF 0.01 46.09 0.450 13.49 UltraDPLIF 0.1 45.79 0.331 9.94 Table 19 : Sparsity results on N-MNIST. Accuracy (%), spike rate, and relative SOP count ( T\cdot\bar{s} ). Model \lambda Acc Spike Energy T{=}1 UltraLIF 0 90.41 0.579 0.58 UltraLIF 0.01 90.37 0.597 0.60 UltraLIF 0.1 90.74 0.546 0.55 UltraPLIF 0 93.11 0.396 0.40 UltraPLIF 0.01 92.73 0.409 0.41 UltraPLIF 0.1 93.28 0.318 0.32 UltraDLIF 0 94.14 0.506 0.51 UltraDLIF 0.01 93.97 0.492 0.49 UltraDLIF 0.1 93.52 0.291 0.29 UltraDPLIF 0 94.14 0.506 0.51 UltraDPLIF 0.01 93.97 0.492 0.49 UltraDPLIF 0.1 93.52 0.291 0.29 T{=}10 UltraLIF 0 96.10 0.447 4.47 UltraLIF 0.01 96.22 0.415 4.15 UltraLIF 0.1 96.39 0.222 2.22 UltraPLIF 0 96.33 0.445 4.45 UltraPLIF 0.01 96.34 0.358 3.58 UltraPLIF 0.1 96.60 0.127 1.27 UltraDLIF 0 97.38 0.460 4.60 UltraDLIF 0.01 97.37 0.335 3.35 UltraDLIF 0.1 97.00 0.155 1.55 UltraDPLIF 0 97.38 0.463 4.63 UltraDPLIF 0.01 97.40 0.306 3.06 UltraDPLIF 0.1 96.93 0.144 1.44 T{=}30 UltraLIF 0 95.87 0.404 12.13 UltraLIF 0.01 95.77 0.391 11.72 UltraLIF 0.1 96.30 0.267 8.02 UltraPLIF 0 95.77 0.463 13.90 UltraPLIF 0.01 95.91 0.412 12.36 UltraPLIF 0.1 96.48 0.240 7.21 UltraDLIF 0 97.46 0.429 12.86 UltraDLIF 0.01 97.34 0.301 9.04 UltraDLIF 0.1 97.28 0.140 4.20 UltraDPLIF 0 97.68 0.430 12.89 UltraDPLIF 0.01 97.52 0.275 8.26 UltraDPLIF 0.1 97.33 0.125 3.74 Table 20 : Sparsity results on DVS-Gesture. Accuracy (%), spike rate, and relative SOP count ( T\cdot\bar{s} ). Model \lambda Acc Spike Energy T{=}1 UltraLIF 0 58.33 0.726 0.73 UltraLIF 0.01 57.58 0.746 0.75 UltraLIF 0.1 57.95 0.690 0.69 UltraPLIF 0 60.23 0.619 0.62 UltraPLIF 0.01 57.20 0.610 0.61 UltraPLIF 0.1 55.68 0.601 0.60 UltraDLIF 0 58.33 0.774 0.77 UltraDLIF 0.01 56.44 0.833 0.83 UltraDLIF 0.1 58.71 0.790 0.79 UltraDPLIF 0 58.33 0.774 0.77 UltraDPLIF 0.01 56.44 0.833 0.83 UltraDPLIF 0.1 58.71 0.790 0.79 T{=}10 UltraLIF 0 69.32 0.707 7.07 UltraLIF 0.01 68.56 0.709 7.09 UltraLIF 0.1 67.80 0.702 7.02 UltraPLIF 0 68.94 0.559 5.59 UltraPLIF 0.01 69.70 0.567 5.67 UltraPLIF 0.1 70.83 0.543 5.43 UltraDLIF 0 69.32 0.611 6.11 UltraDLIF 0.01 71.97 0.601 6.01 UltraDLIF 0.1 70.45 0.543 5.43 UltraDPLIF 0 68.56 0.615 6.15 UltraDPLIF 0.01 71.97 0.614 6.14 UltraDPLIF 0.1 73.11 0.578 5.78 T{=}30 UltraLIF 0 75.00 0.719 21.58 UltraLIF 0.01 75.38 0.719 21.56 UltraLIF 0.1 75.00 0.707 21.20 UltraPLIF 0 75.76 0.593 17.79 UltraPLIF 0.01 75.76 0.588 17.63 UltraPLIF 0.1 76.14 0.559 16.77 UltraDLIF 0 78.41 0.560 16.79 UltraDLIF 0.01 78.41 0.552 16.56 UltraDLIF 0.1 81.06 0.501 15.04 UltraDPLIF 0 79.92 0.570 17.09 UltraDPLIF 0.01 77.65 0.556 16.68 UltraDPLIF 0.1 79.55 0.521 15.62 Table 21 : Sparsity results on SHD. Accuracy (%), spike rate, and relative SOP count ( T\cdot\bar{s} ). Model \lambda Acc Spike Energy T{=}1 UltraLIF 0 44.88 0.551 0.55 UltraLIF 0.01 45.76 0.542 0.54 UltraLIF 0.1 46.86 0.507 0.51 UltraPLIF 0 46.91 0.390 0.39 UltraPLIF 0.01 47.61 0.394 0.39 UltraPLIF 0.1 48.32 0.344 0.34 UltraDLIF 0 51.24 0.686 0.69 UltraDLIF 0.01 50.62 0.629 0.63 UltraDLIF 0.1 51.33 0.565 0.56 UltraDPLIF 0 51.24 0.686 0.69 UltraDPLIF 0.01 50.62 0.629 0.63 UltraDPLIF 0.1 51.33 0.565 0.56 T{=}10 UltraLIF 0 58.79 0.472 4.72 UltraLIF 0.01 57.99 0.462 4.62 UltraLIF 0.1 60.51 0.397 3.97 UltraPLIF 0 57.73 0.420 4.20 UltraPLIF 0.01 58.48 0.411 4.11 UltraPLIF 0.1 58.79 0.348 3.48 UltraDLIF 0 67.62 0.416 4.16 UltraDLIF 0.01 69.52 0.442 4.42 UltraDLIF 0.1 69.92 0.367 3.67 UltraDPLIF 0 68.90 0.461 4.61 UltraDPLIF 0.01 65.77 0.496 4.96 UltraDPLIF 0.1 70.27 0.399 3.99 T{=}30 UltraLIF 0 59.14 0.458 13.73 UltraLIF 0.01 59.41 0.451 13.52 UltraLIF 0.1 61.00 0.404 12.11 UltraPLIF 0 59.45 0.449 13.48 UltraPLIF 0.01 60.16 0.445 13.35 UltraPLIF 0.1 60.60 0.385 11.54 UltraDLIF 0 71.60 0.459 13.76 UltraDLIF 0.01 70.23 0.458 13.73 UltraDLIF 0.1 73.19 0.386 11.57 UltraDPLIF 0 67.84 0.454 13.62 UltraDPLIF 0.01 67.71 0.449 13.48 UltraDPLIF 0.1 69.88 0.404 12.13 Key observations across datasets: (1) Moderate sparsity ( \lambda{=}0.1 ) consistently reduces spike rates by 40–50% with minimal accuracy loss, and in several cases (MNIST T{=}1 , Fashion T{=}1 , CIFAR-10 T{=}30 ) actually improves accuracy, suggesting that sparsity acts as a regularizer. (2) The sparsity-accuracy trade-off is most favorable on temporal models (UltraPLIF), which achieve the lowest energy at competitive accuracy.