Large Language and Reasoning Models are Shallow Disjunctive Reasoners

Irtaza Khalid, Amir Masoud Nourollah, Steven Schockaert · 2025

Read Large Language and Reasoning Models are Shallow Disjunctive Reasoners by Irtaza Khalid, Amir Masoud Nourollah, Steven Schockaert (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-23487.txt.

Description

Irtaza Khalid, Amir Masoud Nourollah, and Steven Schockaert test how deeply reasoning models really reason in a 2025 paper. Language models often rely on shortcuts and fail on fresh examples. Newer reasoning models, trained with reinforcement learning and long chains of thought, promise more, but most proof comes from math and coding tasks.

The team used spatial and temporal reasoning tasks where difficulty can be dialed precisely. Reasoning models beat plain models when a task needed one path of deduction. Both kinds failed when a task needed juggling several paths at once, even after fine-tuning. The authors call this shallow disjunctive reasoning: models try one branch at a time rather than holding alternatives together.

Catalog record

CREATORIrtaza Khalid, Amir Masoud Nourollah, Steven Schockaert
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE69 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-23487

Browse the full library · About the Tower of Babel