Looped SSMs: Depth-Recurrence and Input Reshaping for Time Series Classification
Mónika Farsang, Ramin Hasani, Daniela Rus, Radu Grosu · 2026
Read Looped SSMs: Depth-Recurrence and Input Reshaping for Time Series Classification by Mónika Farsang, Ramin Hasani, Daniela Rus, Radu Grosu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-16048.txt.
Introduction
Farsang and three colleagues shrink state space models, compact recurrent rivals of large language models on edge devices. Looping a small model through depth recurrence matches a much larger standard model with no performance loss. Presenting inputs at a fixed, consistent time granularity also lifts performance. Both gains held across four architectures: LRU, S5, LinOSS, and LrcSSM.
Catalog record
| CREATOR | Mónika Farsang, Ramin Hasani, Daniela Rus, Radu Grosu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 60315 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.16048
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Looped SSMs: Depth-Recurrence and Input Reshaping for Time Series Classification Mónika Farsang, Ramin Hasani, Daniela Rus, Radu Grosu Abstract State Space Models (SSMs) are inherently recurrent along the sequence dimension, yet depth-recurrence - reusing the same block repeatedly across layers, as recently applied in looped transformers - has not been explored in this model family. We show that a looped SSM with k k parameters iterated L L times consistently closely matches or outperforms a standard SSM with k ⋅ L k\cdot L independent parameters across four architectures (LRU, S5, LinOSS, LrcSSM) and six time series classification benchmarks, despite operating within a strictly smaller hypothesis space, as we formally establish. Since the larger model contains the looped model as a special case, this dominance cannot be explained by expressivity and instead points to parameter sharing across depth as a beneficial inductive bias that simplifies optimization. These results demonstrate that depth-recurrence is orthogonal to sequence-recurrence and independently beneficial. We further show that input reshaping is an equally neglected design axis: concatenating timesteps for low-dimensional inputs, or flattening and rechunking the joint feature-time dimension for high-dimensional ones, yields accuracy gains of 1-6% across all models, confirmed over 5 random seeds. Both techniques provide standalone improvements that compound when combined, suggesting that depth and input reshaping are two independent and underexplored design axes for SSMs on time series. † † footnotetext: 1 TU Wien † † footnotetext: 2 MIT CSAIL † † footnotetext: 3 Liquid AI † † footnotetext: Corr. author: [email protected] 1 Introduction State Space Models (SSMs) have emerged as a compelling alternative to attention-based transformers for tasks requiring efficient processing of long sequences ( Gu et al. 2022 ; Hasani et al. 2023 ; Orvieto et al. 2023 ) . By maintaining a compact recurrent state updated across the sequence dimension, SSMs achieve linear-time learning and constant-time inference complexity, and strong empirical performance, across a wide range of domains, including language, audio, and time series ( Gu and Dao 2024 ; Rusch and Rus 2025 ; Farsang and Grosu 2026 ) . This native sequence-level recurrence is central to their identity - yet it has also implicitly constrained how researchers think about scaling them up. The general design pattern is simple, by usually stacking L independent SSM blocks, each with its own parameters, and letting depth provide representational diversity. A parallel line of work on transformers has recently challenged the analogous assumption made for the transformers architecture. Looped transformers ( Dehghani et al. 2019 ; Geiping et al. 2026 ; Zhu et al. 2025 ) reuse the same block repeatedly across depth, showing that a model with k parameters iterated L times can match or outperform a standard model with k\cdot L independent parameters. This finding has driven a reappraisal of what depth actually contributes to transformer computation, shifting the narrative from more parameters to more iterative refinement . However, despite this growing body of work, this question has never been explored in the context of SSMs. This omission is understandable: SSMs are already recurrent, so adding another axis of recurrence may seem redundant. We argue the opposite: depth-recurrence is precisely orthogonal to sequence-recurrence, and this orthogonality makes depth-recurrence an independent and unexplored source of inductive bias for SSMs. Beyond depth recurrence, a second underexplored architectural dimension in SSMs is input reshaping. The way raw time series are shaped into the input embeddings - how many time steps are grouped together, and whether the feature and temporal dimensions are kept separate - directly determines the information density and sequence length the model sees. However, this topic has received no attention compared to other architectural choices. In vision, the analogous question of how to tokenize images has been studied extensively: ViT ( Dosovitskiy et al. 2021 ) introduced fixed-size patch embeddings that deliberately mix spatial positions within a token, and subsequent work has shown that a varying patch size, overlap, and ordering can significantly affect performance ( Liu et al. 2021 ; Beyer et al. 2023 ) . Here, we introduce input reshaping , a simple pre-processing step adjusting the information density of input data, and study its effects on both low- and high-dimensional input data. The interaction between input dimensionality, sequence length, and model capacity for SSMs on time series remains poorly understood. This work addresses both gaps simultaneously. We apply depth-recurrence to four representative SSM architectures: LRU ( Orvieto et al. 2023 ) , S5 ( Smith et al. 2023 ) , LinOSS ( Rusch and Rus 2025 ) , and LrcSSM ( Farsang and Grosu 2026 ) , across six time series classification benchmarks spanning input dimensionalities of 2-63 and sequence lengths of 400-18,000. We also show how input reshaping positively affects the performance of these models and how, in combination with depth recurrence, it further improves SSM accuracy. Our central empirical finding is that a looped SSM with k parameters iterated L times consistently outperforms a standard SSM with k\cdot L independent parameters. We formally establish that the latter contains the former as a special case, meaning that the larger model has strictly more expressive power; this dominance points to parameter sharing across depth as a beneficial inductive bias. Contributions. Our main contributions in this paper are as follows: • We introduce depth-recurrence to SSMs , showing for the first time that reusing the same block repeatedly across depth is complementary to their native sequence-level recurrence. We systematically evaluate three parameter-sharing strategies under two supervision schemes (final layer and block-wise loss) across four architectures and six benchmarks. • We show empirically that a looped SSM with k parameters iterated L times consistently matches or outperforms a standard SSM with k\cdot L independent parameters, despite having as little as 1/L of its parameters, and formally establish that this advantage cannot be attributed to greater expressivity, identifying parameter sharing as a beneficial inductive bias. • We introduce input reshaping , a parameter-free preprocessing strategy that adapts the information density of the input to the model: concatenating consecutive timesteps for low-dimensional inputs, and flattening and rechunking the joint feature-time dimension for high-dimensional ones, yielding accuracy gains of 1-6\% across all four architectures. 2 Related Work State Space Models for Sequence Modeling. Structured State Space Models (SSMs) emerged as a scalable alternative to attention-based transformers for long-range sequence modeling. S4 ( Gu et al. 2022 ) introduced a parameterization of linear SSMs via the HiPPO framework, achieving strong performance on the Long Range Arena benchmark. S5 ( Smith et al. 2023 ) streamlined this design by replacing the bank of independent SISO systems with a single MIMO SSM solved via parallel scan. The Linear Recurrent Unit (LRU) ( Orvieto et al. 2023 ) demonstrated that careful initialization and normalization of a simple diagonal linear RNN is sufficient to match deep SSMs without any continuous-time motivation. LinOSS ( Rusch and Rus 2025 ) introduced forced second-order linear ODE dynamics, drawing inspiration from cortical oscillations and achieving strong performance on time series tasks. Most recently, LrcSSM ( Farsang and Grosu 2026 ) extended this line to nonlinear recurrent dynamics by forcing a diagonal Jacobian structure, enabling parallel training while retaining nonlinear expressivity. All of these works focus on the design of individual layers stacked