From 0-Order Selection to 2-Order Judgment: Combinatorial Hardening Exposes Compositional Failures in Frontier LLMs

Hanmeng Liu, Shichao Weng, Xiulai Li, Zhicai Zhang, Anli Yan, Xiaozhang Liu, [2pt] Hainan University, Haikou, China · 2026

Read From 0-Order Selection to 2-Order Judgment: Combinatorial Hardening Exposes Compositional Failures in Frontier LLMs by Hanmeng Liu, Shichao Weng, Xiulai Li, Zhicai Zhang, Anli Yan, Xiaozhang Liu, [2pt] Hainan University, Haikou, China (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-07268.txt.

Description

Hanmeng Liu and five co-authors harden multiple-choice reasoning tests in a formal, validity-preserving way. Benchmarks saturate quickly and suffer contamination, while ad hoc tricks like paraphrasing add surface difficulty without logical rigor. LogiHard transforms simple selection questions into higher-order logical judgments, demanding more reasoning steps from the test taker.

An adaptive testing layer controls difficulty precisely with fewer questions. Across twelve leading models, hardened questions cut accuracy sharply, and transfer to a general benchmark showed a similar drop. The authors blame a completeness-checking gap in training rather than missing knowledge, a failure pattern human test takers did not share.

Catalog record

CREATORHanmeng Liu, Shichao Weng, Xiulai Li, Zhicai Zhang, Anli Yan, Xiaozhang Liu, [2pt] Hainan University, Haikou, China
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.07268

Browse the full library · About the Tower of Babel