Can Structural Cues Save LLMs? Evaluating Language Models in Massive Document Streams DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Information systems → Web searching and information discovery; Data stream mining
Yukyung Lee, Yebin Lim, Woojun Jung, Wonjun Choi, Susik Yoon · 2026
Read Can Structural Cues Save LLMs? Evaluating Language Models in Massive Document Streams DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Information systems → Web searching and information discovery; Data stream mining by Yukyung Lee, Yebin Lim, Woojun Jung, Wonjun Choi, Susik Yoon (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-19250.txt.
Introduction
Lee and co-authors test language models on huge streams of news documents where many events mix. Their StreamBench draws on top stories from 2016 and 2025. It holds 605 events and over 15,000 documents in three tasks. Models must cluster topics, answer time-based questions, and write summaries. Adding cues that sort key facts by event lifts scores on two tasks. Clustering gains reach 4.37 percent and time questions 9.63 percent. Time reasoning itself stays a weak point.
Catalog record
| CREATOR | Yukyung Lee, Yebin Lim, Woojun Jung, Wonjun Choi, Susik Yoon |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 61479 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.19250
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Can Structural Cues Save LLMs? Evaluating Language Models in Massive Document Streams DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Information systems → Web searching and information discovery; Data stream mining Yukyung Lee, Yebin Lim, Woojun Jung, Wonjun Choi, Susik Yoon Abstract . Evaluating language models in streaming environments is critical, yet underexplored. Existing benchmarks either focus on single complex events or provide curated inputs for each query, and do not evaluate models under the conflicts that arise when multiple concurrent events are mixed within the same document stream. We introduce StreamBench, a benchmark built from major news stories in 2016 and 2025, comprising 605 events and 15,354 documents across three tasks: Topic Clustering, Temporal Question Answering, and Summarization. To diagnose how models fail, we compare performance with and without structural cues, which organize key facts by event. We find that structural cues improve performance on clustering (up to +4.37%) and temporal QA (up to +9.63%), helping models locate relevant information and separate distinct events. While temporal reasoning remains an open challenge inherent to current LLMs, consistent gains across tasks show that structural cues are a promising direction for future work in massive document streams. 1. Introduction Figure 1 . Two challenges in streaming environments. Intra-topic conflict: within Dixie Fire, the most recent event (Event 3) has fewer documents than the outdated event (Event 1), making it harder to find the latest information. Inter-topic conflict: when asked about Dixie Fire (Topic 1), the LLM may confuse “8 firefighters” (Event 2) from Bootleg Fire (Topic 2). Information streams evolve in real time, particularly in domains such as news, where documents continuously arrive from diverse sources and facts are updated over time. A single news story often spans multiple topics and associated events, where the corresponding documents emerge intertwined within the same stream. This document stream setting gives rise to a range of downstream tasks, including topic discovery ( 2 ; 34 ; 11 ; 21 ) , question answering (QA) ( 35 ; 20 ) , and summarization ( 33 ; 26 ) . Traditionally, each of these tasks has been addressed by task-specific models developed in an ad-hoc manner ( 10 ; 9 ) . With the rise of foundation models, there is growing interest in applying large language models (LLMs) directly to document streams, leveraging their broad capabilities to handle multiple tasks with a single model ( 29 ; 5 ) . However, LLMs operate within a fixed context window. As new documents arrive, old information and new updates share the same window, making it increasingly difficult to identify what is relevant and up to date ( 6 ) . Such inherently dynamic and unbounded streaming settings pose nontrivial challenges for LLMs, beyond what a typical static, long-context setting presents. Specifically, we identify two key conflicts. First, intra-topic conflict: within a single topic, documents accumulate over time as new events occur, biasing models toward older information and making it harder to identify the most recent state. Second, inter-topic conflict: when documents from multiple related topics overlap in the same context window, models struggle to distinguish which facts belong to which topic. These conflicts place a significant cognitive burden on models, as they must organize scattered information by topic and time while reasoning over them ( 17 ; 18 ) . In Figure 1 , for example, a news story about California Wildfire contains multiple topics, such as Dixie Fire and Bootleg Fire, where fresh news articles from associated events continuously feed into a document stream. When an LLM is asked to reason in the latest context of the stream (e.g., "How many people were injured in the Dixie Fire?"), it faces difficulty identifying exact information in Event 3, which is more recent but has been reported by fewer articles than Event 1 (i.e., intra-topic conflict). The confusion intensifies when the model needs to distinguish relevant information from concurrent topics, such as "8 firefighters" from Bootleg Fire (i.e., inter-topic conflict). However, existing benchmarks fall short in evaluating LLM capabilities in such streaming settings. While several time-sensitive datasets have been proposed ( 35 ; 20 ; 14 ; 29 ) , they primarily evaluate temporally grounded questions over static snapshots, rather than continuously expanding contexts with accumulating information. Other recent benchmarks address complex event understanding ( 37 ) , but focus on single events, failing to capture scenarios where multiple events develop concurrently with varying quantities. Moreover, existing evaluations report only end-to-end performance without diagnosing why models fail, and focus on identifying the problem rather than exploring directions to address it. In this work, we present a systematic and comprehensive evaluation of LLM capabilities in streaming environments, featuring the following contributions: • First, we construct StreamBench , a benchmark built from real-world news streams spanning 2016 and 2025, comprising 605 events and 15,354 documents. We evaluate seven LLMs of varying scales (1B–123B) across three tasks: Topic clustering, Temporal QA, and Summarization. These tasks capture distinct cognitive demands of streaming environments: organizing scattered information, answering specific questions, and compressing accumulated texts over evolving stories. StreamBench features dynamic concurrent events with varying text quantity, capturing the core characteristics of streaming environments. • Second, to diagnose why models struggle under these settings, we introduce structural cues as a diagnostic probe. The core difficulty in real-world streaming environments is that information from multiple topics arrives mixed together over time, leading to intra- and inter-topic conflicts. Prior work in a static setting has shown that structured knowledge representations help models handle complex information ( 23 ; 30 ) . If failures primarily stem from difficulties in organizing scattered information, providing auxiliary organizational support should alleviate these issues. Based on this intuition, we design the simplest form of structural cues as key facts and entities organized by topic, serving as supplementary signals for LLMs. By comparing performance with and without these cues, we identify what aspects become easier with organization, and what difficulties remain. Our evaluation shows that current LLMs struggle in streaming environments across all three tasks. Comparing performance with and without structural cues, we observe that they provide partial improvements, helping models find relevant information. Structural cues improve event separation in clustering (up to 4.37% in B 3 F1) and temporal ordering in QA (up to 9.63% in accuracy). However, reasoning over the located information remains challenging. In clustering, precise boundary detection remains difficult despite reduced over-clustering. In temporal QA, models still struggle with tracking the current state of entities even when information is well-organized. In summarization, structural cues show smaller gains (up to 0.87% in ROUGE-L and 3.40% in METEOR). Overall, structural cues consistently help models find and organize information, while reasoning over temporal dynamics remains an open challenge inherent to current LLMs. We believe our findings and benchmark can motivate further exploration into how LLMs handle conflicts in streaming document environments. 2. Related Work 2.1. Temporal and Streaming Evaluation Recent benchmarks evaluate how LLMs handle time-varying knowledge. Streaming QA ( 20 ) , RealTime QA ( 14 ) , and FreshQA ( 29 ) test whether models rely on up-to-date information rather than outdated parametric knowledge. Daily