What Really Controls Temporal Reasoning in Large Language Models: Tokenisation or Representation of Time?

Gagan Bhatia Ahmad Muhammad Isa Maxime Peyrard Wei Zhao, [0.4em] University of Aberdeen Université Grenoble Alpes & CNRS [email protected] · 2026

Read What Really Controls Temporal Reasoning in Large Language Models: Tokenisation or Representation of Time? by Gagan Bhatia Ahmad Muhammad Isa Maxime Peyrard Wei Zhao, [0.4em] University of Aberdeen Université Grenoble Alpes & CNRS [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-19017.txt.

Introduction

Bhatia and co-authors test what limits time reasoning in large language models. Their bench holds 15,000 items across five languages and three calendar systems. Tasks cover date math, time zone shifts, and time links in text. They measure how dates get split into tokens and probe inner time signals. In low-resource languages, bad splitting breaks year, month, and day parts, and scores collapse. In rich settings, a straight inner time line predicts success best. They release the bench code in public.

Catalog record

CREATORGagan Bhatia Ahmad Muhammad Isa Maxime Peyrard Wei Zhao, [0.4em] University of Aberdeen Université Grenoble Alpes & CNRS [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56252
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2603.19017

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

What Really Controls Temporal Reasoning in Large Language Models: Tokenisation or Representation of Time? Gagan Bhatia Ahmad Muhammad Isa Maxime Peyrard Wei Zhao, [0.4em] University of Aberdeen Université Grenoble Alpes & CNRS [email protected] Abstract We present MultiTempBench , a multilingual temporal reasoning benchmark spanning three tasks, date arithmetic, time zone conversion, and temporal relation extraction across five languages (English, German, Chinese, Arabic, and Hausa) and multiple calendar conventions (Gregorian, Hijri, and Chinese Lunar). MultiTempBench contains 15,000 15{,}000 examples built by translating 750 750 curated English questions and expanding each into controlled date-format variants. We evaluate 20 LLMs and introduce the multilingual Date Fragmentation Ratio (mDFR), calibrated with human severity ratings, together with geometric-probing analyses of internal temporal representations . We find tokenisation quality of temporal artefacts is a resource-dependent bottleneck: in low-resource languages and rarer calendar formats, fragmentation disrupts Year/Month/Day separation and accuracy collapses, while high-resource settings are often robust to digit-level splitting. Beyond tokenisation, crossed mixed-effects regression shows that temporal linearity is the strongest predictor of temporal reasoning in high-resource languages, whereas fragmentation is the stronger predictor in low-resource languages. 1 1 1 https://github.com/gagan3012/mtb \setcode utf8 1 Introduction Time is a universal substrate for human reasoning, but temporal expressions are deeply language- and culture- specific. Real-world systems—calendar assistants, travel planners, clinical and legal timeline reconstruction, historical question answering, and forecasting—must interpret and manipulate dates, times, and temporal relations expressed in heterogeneous surface forms (e.g., 2024-05-01 vs. “1 May 2024”) and under distinct calendrical conventions (e.g., Gregorian vs. Hijri vs. Lunar). These requirements are inherently multilingual: users routinely mix scripts, localized month lexemes, and calendar markers, and many high-stakes workflows depend on correct temporal normalisation and arithmetic across languages and regions. Recent temporal benchmarks have advanced our understanding of LLMs’ abilities in date arithmetic, temporal ordering, and time-sensitive QA, but they overwhelmingly focus on English and Gregorian representations ( 40 ; 8 ; 45 ; 5 ; 13 ; 43 ; 19 ; 33 ; 32 ) . In parallel, emerging work on cross-calendar reasoning highlights that LLMs remain inadequate for inter-calendar conversion and that non-Gregorian temporal structure is still underexplored despite its global relevance ( 11 ; 34 ; 24 ; 41 ; 12 ) . Figure 1: Mechanistic understanding of multilingual temporal reasoning in MultiTempBench . An orthogonal challenge concerns how dates are presented to the model . Temporal strings are structured symbolic objects, yet subword tokenisers (e.g., BPE and byte-level tokenisation) can fragment them into opaque substrings, potentially erasing semantic boundaries such as year/month/day separators and calendar markers ( 36 ; 3 ) . More broadly, multilingual tokenisation is known to induce systematic inequities: low-resource languages often incur heavier fragmentation (a “token tax”), which increases effective sequence length and can degrade downstream performance ( 1 ; 31 ; 20 ; 14 ) . For numeracy specifically, the choice between digit-level tokenisation and larger numeric chunks yields distinct arithmetic failure modes: inconsistent segmentation forces the model to infer place value and grouping from unstable boundaries, complicating the learning of multi-digit operations and carry-like mechanisms ( 35 ; 16 ) . These observations motivate a natural hypothesis: multilingual temporal failures may be primarily a tokenisation problem . However, two gaps prevent a mechanistic account. First, existing tokenisation analyses typically target general text or broad downstream tasks, rather than controlled temporal expressions that combine digits, delimiters, month lexemes, and calendar markers across scripts and calendar systems. Even temporally focused studies largely remain monolingual or calendar-specific ( 3 ; 11 ; 24 ) . Second, even when behavioural gaps are documented (e.g., accuracy differences and systematic error patterns across languages and formats), we lack clarity on where the failure arises in the processing pipeline: does temporal information degrade at the input layer, fail to form an abstract representation suitable for computation, or break during reasoning and decoding? Prior mechanistic work suggests that LLMs can encode ordered scalar attributes in ways that are approximately linearly decodable from hidden states, and can exhibit stable latent directions corresponding to monotonic temporal progression ( 10 ; 7 ) , but these findings have not been connected to multilingual, multi-calendar temporal reasoning in a controlled setting. As a result, it remains unclear whether multilingual temporal competence requires (i) better surface segmentation, (ii) a shared internal “calendar geometry” that supports computation, or (iii) both. Our contributions are summarized as follows: (i) A controlled multilingual, multi-calendar benchmark. We release MultiTempBench , comprising 15,000 examples across 5 languages, 3 temporal tasks, and multiple date-format complexity levels, including Gregorian, Hijri, and Lunar calendar systems. We evaluate a broad set of LLMs (open-weight and proprietary) in a zero-shot setting, quantifying how language resource level, format complexity, and calendar system affect temporal reasoning. (ii) A multilingual fragmentation metric. We propose mDFR , a multilingual extension of Date Fragmentation Ratio ( 3 ) that penalises semantically destructive segmentations (e.g., digit splitting and boundary loss) with weights learned from human severity ratings. (iii) A mechanistic account of multilingual temporal performance gaps. We show that bottlenecks shift by language: date fragmentation is most predictive of failure in low-resource regimes (where it disrupts access to compositional components), whereas temporal linearity (probe R^{2} ) is the strongest predictor of temporal task performance in high-resource languages once calendar components (e.g., Year/Month/Day) are frequently accessible in training data . This supports a two-stage view in which tokenisation controls an LLM’s surface-level access to calendar components while temporal linearity controls internal temporal representations. To test this directly, we complement descriptive analyses with a crossed mixed-effects regression over all model predictions, allowing us to compare the contribution of date fragmentation and temporal linearity across resource levels. 2 Related Works Tokenisation bias in multilingual models. Tokenisation remains a critical source of disparity in multilingual LLMs. Recent studies confirm that low-resource languages and dialects suffer a token tax, inflated sequence lengths that degrade performance and increase compute costs ( 20 ; 14 ) . This disparity is particularly acute for Indian languages, where standard vocabularies often fragment morphological units ( 15 ) . Such fragmentation may critically impact numeric and temporal reasoning. 3 identify date fragmentation as a hidden bottleneck, showing that Byte-Pair Encoding (BPE) often splits dates into opaque substrings that hinder temporal arithmetic. Similarly, 35 demonstrate that standard tokenisation degrades arithmetic performance compared to single-token number embeddings. Our work extends this line of inquiry by introducing MultiTempBench to systematically isolate how tokenisation quality of dates , measured via our proposed mDFR metric, affect reasoning capabilities in a multilingual setting. Mechanisms of time: memorisation vs. reasoning. Evaluating temporal understanding requires distinguishing between pattern matching and robust reasoning. Indeed, recent studies reveal that while LLMs maintain stable performance on memorisation-based temporal tasks, their accuracy sharply declines on reasoning-intensive tasks, especially when navigating temporal shifts or integrating new knowledge ( 23 ; 17 ; 18 ) . Benchmarks such as ChronoSense ( 13 ) , SPAN 25 , DateLogicQA ( 4 ) and TimeBench ( 5 ) reveal that LLMs struggle with symbolic constraints and temporal commonsense. Mechanistically, 10 find that models possess linear subspaces representing space and time, suggesting relevant information is encoded but not always exploited. However, 22 observe that computation is often aggregated only at the final token, creating a fragile

Read the complete text