From Static Benchmarks to Dynamic Protocol: Agent-Centric Text Anomaly Detection for Evaluating LLM Reasoning
Seungdong Yoa, Sanghyu Yoon, Suhee Yoon, Dongmin Kim, Ye Seul Sim, Junhyun Lee , Woohyung Lim · 2026
Read From Static Benchmarks to Dynamic Protocol: Agent-Centric Text Anomaly Detection for Evaluating LLM Reasoning by Seungdong Yoa, Sanghyu Yoon, Suhee Yoon, Dongmin Kim, Ye Seul Sim, Junhyun Lee , Woohyung Lim (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-23729.txt.
Description
Seungdong Yoa and six co-authors move from static benchmarks to a dynamic plan for agent-centric anomaly spotting. LLM testing has relied on static data sets, giving limited scaling and failing to capture shifting thinking powers.
The new agent-centric model adds a dynamic plan in which self-run agents make, check, and solve problems. A teacher agent makes trial problems, a checker agent checks soundness and guards against attacks, and a student agent attempts checked problems. An invalid problem is revised by the teacher until it passes checks. If the student solves right, the checker prompts the teacher to make harder variants. The benchmark scales in hardness on its own as capable agents are swapped into any role, giving stepwise testing without hand-made sets. Using text anomaly spotting as the format, needing cross-sentence thinking and blocking pattern-matching shortcuts, the plan exposes corner-case errors.
Catalog record
| CREATOR | Seungdong Yoa, Sanghyu Yoon, Suhee Yoon, Dongmin Kim, Ye Seul Sim, Junhyun Lee , Woohyung Lim |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 93 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-23729