Can Large Language Models Keep Up? Benchmarking Online Adaptation to Continual Knowledge Streams
Jiyeon Kim, Hyunji Lee, Dylan Zhou, Sue Hyun Park, Seunghyun Yoon, Trung Bui, Franck Dernoncourt, Sungmin Cha, Minjoon Seo · 2026
Read Can Large Language Models Keep Up? Benchmarking Online Adaptation to Continual Knowledge Streams by Jiyeon Kim, Hyunji Lee, Dylan Zhou, Sue Hyun Park, Seunghyun Yoon, Trung Bui, Franck Dernoncourt, Sungmin Cha, Minjoon Seo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-07392.txt.
Description
Kim and colleagues benchmark whether language models can adapt online as facts change over time. Their OAKS suite streams fine-grained context chunks in which individual facts evolve repeatedly, with dense labels tracking whether models follow each change. Across fourteen models and varied inference styles, even top models and agentic memory systems adapt poorly. State tracking lags and streaming distractions trip the models up, showing online adaptation remains an open problem.
Catalog record
| CREATOR | Jiyeon Kim, Hyunji Lee, Dylan Zhou, Sue Hyun Park, Seunghyun Yoon, Trung Bui, Franck Dernoncourt, Sungmin Cha, Minjoon Seo |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 93731 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.07392