Manifold-Aware Temporal Domain Generalization for Large Language Models
Yiheng Yao, Zekun Cai, Xinyuan Song, Hiroki Hill Kobayashi, Xuan Song, Ryosuke Shibasaki, Liang Zhao · 2026
Read Manifold-Aware Temporal Domain Generalization for Large Language Models by Yiheng Yao, Zekun Cai, Xinyuan Song, Hiroki Hill Kobayashi, Xuan Song, Ryosuke Shibasaki, Liang Zhao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-11965.txt.
Description
Learning from past domains to predict future ones grows costly as backbones scale, since dynamics models must encode every adapted parameter. Zekun Cai and five co-authors propose LoTiC, a method that splits temporal updates into shared low-rank bases and a compact core. The bases define a stable update subspace while only the core changes over time, so change is learned in a small core space, not the full adapter.
The split is justified through the geometry of weight increments and supports continuous, recurrent, and time-conditioned core models. Across six controlled and real-world benchmarks, three data types, and both classification and generation, LoTiC ranks first in 14 of 15 settings. That includes near-perfect future-domain accuracy with an 8-billion-weight backbone. It adds only a tiny fraction of trainable parameters beyond LoRA and trains about two to seven times faster than dynamics baselines.
Catalog record
| CREATOR | Yiheng Yao, Zekun Cai, Xinyuan Song, Hiroki Hill Kobayashi, Xuan Song, Ryosuke Shibasaki, Liang Zhao |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-11965