CDR-Bench: Evaluating Faithful Execution of Compositional, Order-Sensitive Data Refinement Recipes
Yuchen Huang, Xiang Li, Zhenqing Ling, Sijia Li, Qianli Shen, Daoyuan Chen Yi R. (May) Fung, Yaliang Li, [0.5em] HKUST · 2026
Read CDR-Bench: Evaluating Faithful Execution of Compositional, Order-Sensitive Data Refinement Recipes by Yuchen Huang, Xiang Li, Zhenqing Ling, Sijia Li, Qianli Shen, Daoyuan Chen Yi R. (May) Fung, Yaliang Li, [0.5em] HKUST (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-31435.txt.
Description
Huang and seven colleagues introduce CDR-Bench, a benchmark for whether language models follow multi-step data refinement recipes exactly. Refining text means applying operators in sequence, and both the mix and the order decide the result. Older benchmarks test editing alone or tangle it with code and tools, leaving direct recipe following unmeasured.
CDR-Bench holds 3,462 tasks across four real data domains and 29 operators, in atomic, order-free, and order-sensitive settings, with exact reference outputs. Across more than ten leading models, scores drop sharply once steps compose, and success on order-sensitive recipes collapses. The gap shows current models lack the procedural faithfulness that reliable data work demands, however fluent their single edits look.
Catalog record
| CREATOR | Yuchen Huang, Xiang Li, Zhenqing Ling, Sijia Li, Qianli Shen, Daoyuan Chen Yi R. (May) Fung, Yaliang Li, [0.5em] HKUST |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 89 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.31435