Can Large Language Models Keep Up? Benchmarking Online Adaptation to Continual Knowledge Streams

Jiyeon Kim, Hyunji Lee, Dylan Zhou, Sue Hyun Park, Seunghyun Yoon, Trung Bui, Franck Dernoncourt, Sungmin Cha, Minjoon Seo · 2026

Read Can Large Language Models Keep Up? Benchmarking Online Adaptation to Continual Knowledge Streams by Jiyeon Kim, Hyunji Lee, Dylan Zhou, Sue Hyun Park, Seunghyun Yoon, Trung Bui, Franck Dernoncourt, Sungmin Cha, Minjoon Seo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-07392.txt.

Description

Kim and colleagues benchmark whether language models can adapt online as facts change over time. Their OAKS suite streams fine-grained context chunks in which individual facts evolve repeatedly, with dense labels tracking whether models follow each change. Across fourteen models and varied inference styles, even top models and agentic memory systems adapt poorly. State tracking lags and streaming distractions trip the models up, showing online adaptation remains an open problem.

Catalog record

CREATORJiyeon Kim, Hyunji Lee, Dylan Zhou, Sue Hyun Park, Seunghyun Yoon, Trung Bui, Franck Dernoncourt, Sungmin Cha, Minjoon Seo
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE93731
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.07392

Browse the full library · About the Tower of Babel