Speaking Numbers to LLMs: Multi-Wavelet Number Embeddings for Time Series Forecasting
Defu Cao, Zijie Lei, Muyan Weng, Jiao Sun, Yan Liu · 2026
Read Speaking Numbers to LLMs: Multi-Wavelet Number Embeddings for Time Series Forecasting by Defu Cao, Zijie Lei, Muyan Weng, Jiao Sun, Yan Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-26487.txt.
Introduction
Defu Cao, Zijie Lei, Muyan Weng, Jiao Sun, and Yan Liu address a mismatch between language models and numbers. Language models read text as discrete tokens. Time series data are continuous numbers, such as prices or temperatures over time. Standard token handling blurs number order and hurts forecasting. Their system, TempoWave, replaces how each number enters the model.
TempoWave turns each value into digit-wise embeddings built from wavelets. An embedding is the list of numbers a model uses to represent an input. Wavelets describe a signal at several scales at once, from fine wiggles to broad trends. The design keeps exact digit identity and fine local detail visible to the model. It also survives common rescaling of the data. Across five forecasting benchmarks with added text context, TempoWave beat standard number handling and rival interfaces. The authors argue the number interface itself is a key bottleneck.
Catalog record
| CREATOR | Defu Cao, Zijie Lei, Muyan Weng, Jiao Sun, Yan Liu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.26487
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Speaking Numbers to LLMs: Multi-Wavelet Number Embeddings for Time Series Forecasting Defu Cao, Zijie Lei, Muyan Weng, Jiao Sun, Yan Liu Abstract Large language models (LLMs) are attractive for context-aware time series forecasting because they can integrate heterogeneous textual signals, yet their discrete, language-oriented tokenization and embedding interfaces are misaligned with continuous numerical values, often harming numerical ordering and forecasting reliability. We propose TempoWave, a plug-and-play temporal wavelet digit interface that maps each scalar observation into digit-wise embeddings constructed from multi-wavelet, multi-scale coefficients. By directly overriding standard token representations, TempoWave seamlessly exposes both fine-grained local fluctuations and macro global structures in a transformer-compatible form, ensuring that precise numerical formatting, distinct digit identity, and robustness to common normalization operations are maintained throughout the LLM pipeline. Experiments across five context-enriched forecasting benchmarks demonstrate that TempoWave consistently improves LLM-based forecasters over standard numeric tokenization and alternative embedding interfaces, achieving a new state-of-the-art. These results highlight the numeric interface as a key bottleneck and suggest that principled multi-resolution embeddings can better couple LLMs’ contextual reasoning with precise forecasting. Our code is available at DC-research/TempoWAVE and our model can be accessed at Melady/TempoWAVE . 1 Introduction Time series analysis, the study of data points ordered chronologically, is indispensable across sectors such as finance, healthcare, and climate science 3 ; 27 . Accurate forecasting supports resource allocation, risk management, and early warning systems, yet the underlying data generating processes are often complex and evolving 2 ; 39 . Practical time series typically exhibit non-stationarity, mixed periodicities, regime shifts, long- and short-range temporal dependencies, and substantial noise 18 ; 6 ; 5 ; 9 . These properties make it difficult to learn models that simultaneously capture fine-grained local fluctuations and long-horizon global structure, while remaining robust under distribution shifts and limited supervision. In parallel, Large Language Models (LLMs) 22 have become strong general-purpose sequence learners. They can exploit long contexts, perform in-context pattern induction, and naturally integrate textual information 14 ; 43 ; 40 . This is particularly appealing for time series intelligence because many exogenous drivers that affect temporal dynamics are expressed in language, such as policy changes, market news, clinical narratives, and operational logs. Moreover, the few-shot and zero-shot generalization behavior of LLMs suggests a promising pathway for domains where labeled time series data is scarce and task distribution varies across entities, locations, or time periods. Despite this promise, directly adapting LLMs to time series forecasting remains challenging. LLMs are optimized for discrete token prediction, whereas time series forecasting fundamentally requires precise modeling of continuous values. This mismatch can lead to unreliable numerical behavior even when the model captures high-level temporal patterns 20 ; 36 . More critically, language-oriented tokenization fragments numbers into sub-tokens in ways that are not tied to magnitude, for example “2026” \rightarrow “20” and “26”. Such fragmentation breaks ordinal relations and obscures the continuity intrinsic to temporal processes. As a result, two numerically close values may be mapped to very different token sequences, while numerically distant values can share sub-tokens, introducing spurious similarity. In LLM-based forecasting pipelines, this translation layer between real-valued sequences and discrete tokens becomes a principal bottleneck. Recent research has pursued several avenues to bridge the gap between LLMs and time series analysis. One direction develops specialized foundation models tailored to time series 7 ; 4 . Another uses agentic or multimodal systems that couple LLMs with dedicated forecasting tools 37 ; 17 . A third direction focuses on input adaptations, including patching 21 , quantization 24 , or converting time series into symbolic or textual representations 15 . While these approaches can improve usability and efficiency, they often trade away numerical faithfulness, blur fine-grained variations, or rely on external components that reduce end-to-end differentiability and complicate analysis. Consequently, there remains a persistent gap in representing continuous numerical values inside the standard transformer input space in a principled and information-preserving manner. In this work, we focus on the representation bottleneck and ask whether LLMs can be equipped with numerically grounded embeddings that preserve quantitative relations and multi-scale temporal structure while remaining compatible with standard transformer inputs. An effective forecasting representation should support reasoning across resolutions. Local changes are crucial for short-term dynamics and anomaly-sensitive regimes, while trends and seasonal components dominate long-horizon behavior. Motivated by wavelet analysis, which provides a natural multi-resolution decomposition, we propose Multi-Wavelet Number Embedding (TempoWave) , an input embedding interface that maps each scalar observation into a dense vector encoding multi-scale structure. TempoWave is designed to be injected into LLM backbones without requiring language tokenization of numbers, thereby reducing the disconnect between numeric magnitude and the model’s discrete interface. Beyond forecasting accuracy, we also aim to understand how the embedding interface shapes numerical structure inside the model. To this end, we introduce diagnostic analyses that probe whether local neighborhoods in representation space respect numeric ordering, a property we refer to as monotonic neighborhood consistency. These analyses help explain when TempoWave improves forecasting and provide guidance for designing numerically grounded interfaces for LLMs. Extensive experiments on diverse forecasting benchmarks show that TempoWave consistently improves LLM-based forecasters compared to standard tokenization and common adaptation methods, and it is competitive with strong time-series-specific models in settings requiring precise numerical forecasting. Contributions. Our contributions are as follows: • A wavelet-based numeric interface for LLM forecasting. We propose Multi-Wavelet Number Embedding (TempoWave) , an input embedding interface that maps each real-valued numerical observation into a dense vector with multi-resolution structure, enabling direct use of standard LLM backbones for numerical forecasting without relying on language tokenization of numbers. • Structural Faithful and Stable Numeric Representation. We analyze TempoWave from a multi-scale signal processing perspective and establish properties related to numerical faithfulness and stability, including improved separability across values and robustness to common normalization operations used in transformers. • Comprehensive evaluation with diagnostic evidence. We conduct extensive experiments on diverse forecasting benchmarks and show consistent gains over LLM baselines using tokenization and common input adaptations. We further provide diagnostic analyses that probe monotonic neighborhood consistency, offering evidence for how TempoWave reshapes numerical neighborhoods and helping explain its forecasting improvements. 2 Related Work Research on applying Large Language Models (LLMs) to time series analysis has grown rapidly, motivated by LLMs’ strong sequence modeling and their ability to integrate contextual information. Existing efforts can be grouped into three directions: time series foundation models trained on large-scale temporal corpora, LLM-centered agentic or multimodal systems, and input adaptation strategies that re-design how numerical values are represented for transformer backbones. Time series foundation models. A major direction is to pre-train foundation models directly on large and diverse time series collections to learn transferable temporal representations. Representative examples include Chronos 1 , TimesFM 10 , and other large-scale temporal pre-training efforts 30 ; 8 ; 33 . These models demonstrate the benefits of scaling and pre-training for forecasting, but they often rely on patching, discretization, or quantization to interface with transformers, which can blur fine-grained numerical differences and introduce information loss in precision-sensitive regimes. LLM agents and multimodal time series systems. Another line of work uses general-purpose LLMs as reasoning engines within larger pipelines, where forecasting or numerical computation is delegated to specialized tools and the LLM performs orchestration and interpretation 34 ; 29 ; 35 .