CDR-Bench: Evaluating Faithful Execution of Compositional, Order-Sensitive Data Refinement Recipes

Yuchen Huang, Xiang Li, Zhenqing Ling, Sijia Li, Qianli Shen, Daoyuan Chen Yi R. (May) Fung, Yaliang Li, [0.5em] HKUST · 2026

Read CDR-Bench: Evaluating Faithful Execution of Compositional, Order-Sensitive Data Refinement Recipes by Yuchen Huang, Xiang Li, Zhenqing Ling, Sijia Li, Qianli Shen, Daoyuan Chen Yi R. (May) Fung, Yaliang Li, [0.5em] HKUST (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-31435.txt.

Description

Huang and seven colleagues introduce CDR-Bench, a benchmark for whether language models follow multi-step data refinement recipes exactly. Refining text means applying operators in sequence, and both the mix and the order decide the result. Older benchmarks test editing alone or tangle it with code and tools, leaving direct recipe following unmeasured.

CDR-Bench holds 3,462 tasks across four real data domains and 29 operators, in atomic, order-free, and order-sensitive settings, with exact reference outputs. Across more than ten leading models, scores drop sharply once steps compose, and success on order-sensitive recipes collapses. The gap shows current models lack the procedural faithfulness that reliable data work demands, however fluent their single edits look.

Catalog record

CREATORYuchen Huang, Xiang Li, Zhenqing Ling, Sijia Li, Qianli Shen, Daoyuan Chen Yi R. (May) Fung, Yaliang Li, [0.5em] HKUST
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE89 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.31435

Browse the full library · About the Tower of Babel