From Static Benchmarks to Dynamic Protocol: Agent-Centric Text Anomaly Detection for Evaluating LLM Reasoning

Seungdong Yoa, Sanghyu Yoon, Suhee Yoon, Dongmin Kim, Ye Seul Sim, Junhyun Lee , Woohyung Lim · 2026

Read From Static Benchmarks to Dynamic Protocol: Agent-Centric Text Anomaly Detection for Evaluating LLM Reasoning by Seungdong Yoa, Sanghyu Yoon, Suhee Yoon, Dongmin Kim, Ye Seul Sim, Junhyun Lee , Woohyung Lim (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-23729.txt.

Description

Seungdong Yoa and six co-authors move from static benchmarks to a dynamic plan for agent-centric anomaly spotting. LLM testing has relied on static data sets, giving limited scaling and failing to capture shifting thinking powers.

The new agent-centric model adds a dynamic plan in which self-run agents make, check, and solve problems. A teacher agent makes trial problems, a checker agent checks soundness and guards against attacks, and a student agent attempts checked problems. An invalid problem is revised by the teacher until it passes checks. If the student solves right, the checker prompts the teacher to make harder variants. The benchmark scales in hardness on its own as capable agents are swapped into any role, giving stepwise testing without hand-made sets. Using text anomaly spotting as the format, needing cross-sentence thinking and blocking pattern-matching shortcuts, the plan exposes corner-case errors.

Catalog record

CREATORSeungdong Yoa, Sanghyu Yoon, Suhee Yoon, Dongmin Kim, Ye Seul Sim, Junhyun Lee , Woohyung Lim
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE93 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-23729

Browse the full library · About the Tower of Babel