Manifold-Aware Temporal Domain Generalization for Large Language Models

Yiheng Yao, Zekun Cai, Xinyuan Song, Hiroki Hill Kobayashi, Xuan Song, Ryosuke Shibasaki, Liang Zhao · 2026

Read Manifold-Aware Temporal Domain Generalization for Large Language Models by Yiheng Yao, Zekun Cai, Xinyuan Song, Hiroki Hill Kobayashi, Xuan Song, Ryosuke Shibasaki, Liang Zhao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-11965.txt.

Description

Learning from past domains to predict future ones grows costly as backbones scale, since dynamics models must encode every adapted parameter. Zekun Cai and five co-authors propose LoTiC, a method that splits temporal updates into shared low-rank bases and a compact core. The bases define a stable update subspace while only the core changes over time, so change is learned in a small core space, not the full adapter.

The split is justified through the geometry of weight increments and supports continuous, recurrent, and time-conditioned core models. Across six controlled and real-world benchmarks, three data types, and both classification and generation, LoTiC ranks first in 14 of 15 settings. That includes near-perfect future-domain accuracy with an 8-billion-weight backbone. It adds only a tiny fraction of trainable parameters beyond LoRA and trains about two to seven times faster than dynamics baselines.

Catalog record

CREATORYiheng Yao, Zekun Cai, Xinyuan Song, Hiroki Hill Kobayashi, Xuan Song, Ryosuke Shibasaki, Liang Zhao
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-11965

Browse the full library · About the Tower of Babel