Sym Seq Bench : a unified framework for the generation and analysis of rule-based symbolic sequences and datasets
Zajzon, Barna, Bouhadjar, Younes, Fabre, Maxime, Schmidt, Felix, Ostendorf, Noah, Neftci, Emre, Morrison, Abigail, Duarte, Renato · 2025
Read Sym Seq Bench : a unified framework for the generation and analysis of rule-based symbolic sequences and datasets by Zajzon, Barna, Bouhadjar, Younes, Fabre, Maxime, Schmidt, Felix, Ostendorf, Noah, Neftci, Emre, Morrison, Abigail, Duarte, Renato (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-24977.txt.
Introduction
Barna Zajzon, Younes Bouhadjar, Maxime Fabre, and others introduce SymSeqBench, a unified approach for creating and studying rule-based symbolic series. Sequential structure supports language, motion, and choice-making, and AI systems must learn it too. Their tools, including SymSeq, strictly create ordered symbolic data sets with known rules.
The approach is domain neutral while linking tasks to formal theories of computation and computability. Cognitive science and AI readers get benchmarks where the true rule behind a sequence is known, so learning can be judged exactly. Controlled symbolic series help split genuine rule learning from pattern rote recall.
Catalog record
| CREATOR | Zajzon, Barna, Bouhadjar, Younes, Fabre, Maxime, Schmidt, Felix, Ostendorf, Noah, Neftci, Emre, Morrison, Abigail, Duarte, Renato |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 116 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-24977
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Sym Seq Bench : a unified framework for the generation and analysis of rule-based symbolic sequences and datasets Zajzon, Barna, Bouhadjar, Younes, Fabre, Maxime, Schmidt, Felix, Ostendorf, Noah, Neftci, Emre, Morrison, Abigail, Duarte, Renato Abstract Sequential structure is a key feature of multiple domains of natural cognition and behavior, such as language, movement and decision-making. Likewise, it is also a central property of tasks to which we would like to apply artificial intelligence. It is therefore of great importance to develop frameworks that allow us to evaluate sequence learning and processing in a domain agnostic fashion, whilst simultaneously providing a link to formal theories of computation and computability. To address this need, we introduce two complementary software tools: SymSeq , designed to rigorously generate and analyze structured symbolic sequences, and SeqBench , a comprehensive benchmark suite of rule-based sequence processing tasks to evaluate the performance of artificial learning systems in cognitively relevant domains. In combination, SymSeqBench offers versatility in investigating sequential structure across diverse knowledge domains, including experimental psycholinguistics, cognitive psychology, behavioral analysis, neuromorphic computing and artificial intelligence. Due to its basis in Formal Language Theory (FLT), SymSeqBench provides researchers in multiple domains with a convenient and practical way to apply the concepts of FLT to conceptualize and standardize their experiments, thus advancing our understanding of cognition and behavior through shared computational frameworks and formalisms. The tool is modular, openly available and accessible to the research community. 1 Introduction Symbols and symbol systems are central constructs in both cognitive sciences and theoretical computer science, providing a unique formal link between cognition and computation [ 64 ] . This relation, manifested as a branch of the mathematical theory of computation known as Formal Language Theory (FLT), grounds the investigation of complex cognitive, psychological and behavioral processes onto a systematic terminology and set of conventions for describing generative rule systems and the structures they generate [ 74 , 49 ] . From simple pattern perception to language and sequential decision-making, such formal systems allow the algorithmic specification of complex cognition and can be used to understand, model and analyze cognition and behavior [ 114 , 119 , 120 ] . Evaluating sequence learning and processing has long been central to advancing our understanding of both artificial and natural intelligence, from Lashley’s seminal work on serial order and the syntax of action [ 85 ] to contemporary large-scale neural sequence models. Accounting for the full scope of sequence processing is nevertheless challenging: system-specific constraints and the many dimensions along which task characteristics and model demands vary mean that most tasks probe only a limited portion of the problem space. Canonical experimental paradigms and principled benchmarks make it possible to address this challenge by providing shared tasks and evaluation protocols that allow systematic comparison and reveal key limitations in learning and generalization [ 70 , 134 , 55 , 124 ] . The rise of linguistically-competent and computationally proficient artificial intelligence [ 145 ] has led to renewed interest in tasks and metrics that can evaluate systems’ capacity to learn and generalize complex structured sequences [ 147 , 52 , 138 ] . Ideally, and given that human intelligence remains the ultimate reference point, such tasks should be grounded in cognitive theory and capture relevant aspects of human reasoning, behavior and cognition [ 84 , 66 , 83 , 113 ] . We argue that moving beyond monolithic approaches or discrete language-theoretic classes toward a more fluid understanding of temporal processing requires tools that capture its multifaceted nature and complexity at multiple scales, enabling the principled design of tasks with controllable sequence complexity. These allow researchers to scrutinize the properties of sequential structure, generative capacity and the requirements it imposes on the computational machinery. Strongly inspired by decades of human psycholinguistic research, the tools we present here address these needs by providing a comprehensive framework for the specification, generation, manipulation, and analysis of symbolic sequences and corresponding embeddings, as well as a benchmark suite of datasets and tasks that can be used to evaluate the performance of artificial cognitive systems. With applications spanning experimental psychology, behavioral analysis, neuromorphic computing, and artificial intelligence, among others, our framework aims to facilitate interdisciplinary research and the adoption of formal constructs to multiple scientific domains that investigate structured sequential data [ 140 , 112 ] . We illustrate the scope and applicability with concrete use-cases in areas such as dataset generation for artificial grammar learning experiments, the analysis and comparison of behavioral ethograms in different experimental conditions, the evaluation of neuromorphic, artificial neural network architectures, and the mechanistic dissection of neural sequence models. While other tools and approaches have been proposed over the years that address specific aspects of symbolic sequence processing, cognitive modeling and computational benchmarking, they largely do so with a narrow focus. For example, specialized software exists to automatically generate and select string sets for artificial grammar learning (AGL) experiments [ 5 ] , to evaluate large language models’ meta-learning and generalization from minimal exposure [ 1 ] , to systematically deploy standardized neuroscience experiments and tasks [ 105 ] or to investigate neuromorphic embodied agents’ performance in standardized environments and tasks [ 166 , 76 ] , among others. An integrated, broadly applicable tool requires a higher level of abstraction and generalization. The adoption of FLT concepts, as we propose here, has proven insightful for predicting the generalization limits of artificial neural networks [ 40 ] , proposing architectural augmentations that could increase their expressiveness [ 40 , 141 ] or to expose computational similarities and differences between biological and artificial intelligence [ 50 ] . These studies provide powerful tools and valuable theoretical insights into the algorithmic and computational properties and limitations of neural architectures designed to cope with sequential structural regularities. They demonstrate the importance of appropriate computational formalisms and benchmark tasks, but are not easily generalizable across research domains. The ubiquity of temporal structure in animal cognition and behavior, as well as the power of adopting formal, systematic constraints and notations across scientific domains, urges a more substantive integration and the development of tools that can homogenize experimental paradigms and metrics, providing a shared conceptual framework to interpret and analyze different aspects of biological and artificial intelligence. Unlike existing specialized software, SymSeqBench offers a comprehensive, multi-disciplinary approach to analyze human, animal, and artificial intelligence, from designing psycholinguistic experiments and evaluating neuromorphic architectures to dissecting the syntactic structure of behavioral and biological sequences. Below, we explain the theoretical bases and formalisms linking the approaches proposed throughout the paper, we proceed to explain how the tool is organized and implemented, followed by a set of concrete use-cases demonstrating its scope and applicability across a wide range of problem domains. 1.1 Theoretical Foundations We consider sequences as elements of (potentially infinite) formal languages, defined over a finite alphabet and generated by a formal grammar. A formal language \mathcal{L} comprises the set of all words or strings \mathcal{S} that can be formed by following a generative grammar \mathcal{G} , i.e. a set of production rules. Each string S_{i} comprises a finite sequence of symbols \sigma_{i} , drawn from a finite alphabet \mathcal{A} . The generative grammar \mathcal{G} defines the syntax of the language, specifying how