Reasoning Depth and Environment Complexity: A Controlled Study of RLVR Data Allocation across Logical Reasoning Tasks

Yihua Zhu, Qianying Liu, Fei Cheng, Jiaxin Wang, Akiko Aizawa, Sadao Kurohashi, Hidetoshi Shimodaira · 2026

Read Reasoning Depth and Environment Complexity: A Controlled Study of RLVR Data Allocation across Logical Reasoning Tasks by Yihua Zhu, Qianying Liu, Fei Cheng, Jiaxin Wang, Akiko Aizawa, Sadao Kurohashi, Hidetoshi Shimodaira (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-26934.txt.

Introduction

Zhu and six co-authors study how reward-driven learning from checkable scores should split data across depth of thought and complexity of the setting. Beyond depth alone, they vary distractor load and linked parts, and test four thought forms: deduction, abduction, induction, and analogy. Three findings emerge: joint depth-complexity coverage beats single-axis recipes, thought families differ, and uniform mixing beats staged curricula. Off-the-shelf models show the same deductive-over-abductive asymmetry.

Catalog record

CREATORYihua Zhu, Qianying Liu, Fei Cheng, Jiaxin Wang, Akiko Aizawa, Sadao Kurohashi, Hidetoshi Shimodaira
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE112161
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.26934

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Reasoning Depth and Environment Complexity: A Controlled Study of RLVR Data Allocation across Logical Reasoning Tasks Yihua Zhu, Qianying Liu, Fei Cheng, Jiaxin Wang, Akiko Aizawa, Sadao Kurohashi, Hidetoshi Shimodaira Abstract Reinforcement learning with verifiable rewards (RLVR) has become central to post-training reasoning models, yet a key limitation of existing studies is their narrow view of the reasoning space: difficulty is treated as reasoning depth alone, and reward is concentrated on forward deductive state tracking. We instead characterize the reasoning space along two dimensions. Difficulty. Beyond reasoning depth, we study environment complexity, where models must identify the correct path amid distractors and interacting structures. Rewarded reasoning form. We consider four abilities core to real-world reasoning: deductive state tracking, abductive recovery of hidden events or facts, inductive rule induction, and analogical transfer. To disentangle these factors, we construct a synthetic knowledge-graph environment with controlled pre- and post-training distributions, where each instance varies along depth, complexity, and task family. Three findings emerge: joint depth–complexity coverage outperforms single-axis recipes; reasoning families respond non-uniformly, with abductive reasoning degrading outside the RL-covered region and task correlations clustering into deductive–abductive and inductive–analogy pairs; and uniform mixing outperforms staged curricula under a fixed budget. We also find that recent off-the-shelf models exhibit the same deductive-over-abductive asymmetry, suggesting that this gap is not merely an artifact of our controlled setup. 1 Introduction Figure 1: Two-axis data distribution and three main findings. (a) A knowledge-graph schematic shows reasoning depth and environment complexity. (b,c) Finding 1: joint depth–complexity coverage outperforms single-axis recipes. (d) Finding 2: abductive reasoning generalizes poorly outside the RL-covered data distribution. (e) Finding 3: uniform RL outperforms multi-stage curriculum under a fixed budget. Reinforcement learning with verifiable rewards has become a central post-training paradigm for improving the reasoning ability of language models. By rewarding outputs that can be automatically verified, RLVR has enabled substantial progress on mathematical reasoning, code generation, STEM problem solving, and long-horizon reasoning ( 8 ; 35 ; 3 ; 33 ) . However, the success of RLVR also exposes a more fundamental design question: How should RL training examples be distributed across the reasoning space during post-training? This distribution determines which reasoning behaviors are reinforced and generalized, yet how to shape it across difficulty levels and forms of reasoning remains poorly understood. A natural dimension of this distribution is reasoning difficulty. Existing post-training studies have examined difficulty through a depth-oriented lens, emphasizing longer reasoning chains, harder compositional structures, longer reasoning horizons, or training near the model’s competence boundary ( 60 ; 23 ; 56 ; 16 ; 54 ) . However, as illustrated in Figure 1 (a), a model can fail along two distinct axes: the target reasoning path may be long, requiring many sequential inferences, or the path could be buried in a complex environment of distractors and interacting structures. Difficulty in RLVR data thus has two distinct axes: reasoning depth , which stresses multi-step commitment, and environment complexity , which stresses relevant-structure selection. This motivates our first research question RQ1: How should RL training examples be allocated across reasoning depth and environment complexity during RLVR post-training? In particular, under a fixed post-training budget, should training data concentrate on one difficulty axis, jointly cover both axes, or follow a staged curriculum over the depth–complexity space? Apart from difficulty, a second dimension is the form of reasoning that receives reward. Existing reasoning-oriented post-training mainly targets forward, deductive tasks such as mathematics and code ( 8 ; 1 ; 51 ) . However, logical reasoning is broader: abductive inference recovers hidden causes, inductive reasoning extracts rules from instances, and analogy transfers relational patterns across cases. This motivates our second research question RQ2: Do different logical reasoning families behave uniformly under RLVR post-training? We ask whether reasoning forms differ in how they use context, generalize beyond training coverage, and respond to the same post-training distribution. Answering these questions with off-the-shelf models is difficult: their pre-training distributions are not observable, so post-training gains cannot be cleanly attributed to a specific RLVR data distribution. A controlled experimental setting is therefore necessary to isolate how that distribution shapes reasoning improvement. We construct a synthetic knowledge-graph reasoning environment in which both pre-training and post-training distributions are explicitly specified. Each data instance is grounded in a knowledge graph that combines a dynamic event-state component with a static kinship component. As shown in Figure 1 (a), we vary two orthogonal dimensions of difficulty: reasoning depth D , which controls the length of the target reasoning chain, and environment complexity T , which controls the amount of distractor structure around that chain, including depth-focused, complexity-focused, jointly covered, or curriculum-based data recipes. From the same underlying world, we generate instances of four reasoning families: deductive state tracking, abductive recovery of missing events or facts, inductive rule induction, and analogy-based relation transfer. We first pre-train a model on shallow reasoning depth, low-complexity regions, and then apply Group Relative Policy Optimization (GRPO) 49 post-training under controlled data recipes, that vary coverage over the D\times T grid and reasoning families. This controlled setting yields three main findings. Joint depth–complexity coverage produces broader RLVR gains. (Figure 1 (b,c)) Jointly covered recipes produce more robust gains across the grid space over only reasoning depth or only environment complexity recipes. Reasoning families exhibit distinct post-training behavior. The four task families are not interchangeable RL targets: The task-pair correlations cluster into deductive–abductive and inductive–analogy pairs. Abductive reasoning is distribution sensitive and can even degrade outside the relevant training region (Figure 1 (d)). Moreover, evaluation of off-the-shelf LLMs reproduces such deductive-vs.-abductive asymmetry, performing poorly on abductive reasoning instances. Uniform mixing is stronger than staged curriculum under a fixed budget. (Figure 1 (e)) Easy-to-easy-to-hard schedules help harder regions but lose global coverage. Together, these findings show that RLVR data distribution shapes not only how much reasoning improves, but also where improvement occurs, how it transfers, and which reasoning forms benefit. Figure 2: Controlled KGRL framework. The figure summarizes the synthetic KG world, four reasoning task families, and pre-training/RL post-training data distributions used to study depth–complexity coverage. 2 Methodology We define a controlled KG reasoning framework. Appendices A.2 and A.3 give full data-generation and training details. 2.1 Controllable KG Reasoning World Graph Definition Each reasoning instance is grounded in a knowledge graph \mathcal{G}=(\mathcal{P},\mathcal{O},\mathcal{E}_{\mathrm{dyn}},\mathcal{E}_{\mathrm{stat}},s_{0}), where \mathcal{P} is a set of persons, \mathcal{O} is a set of objects, \mathcal{E}_{\mathrm{dyn}} is a time-ordered sequence of dynamic events, \mathcal{E}_{\mathrm{stat}} is a set of static kinship relations over persons, and s_{0} is the initial world state. Dynamic and Static Worlds The generator separates each story into dynamic and static components. The dynamic event-state component updates object-centered facts such as ownership, possession, and integrity, while the static kinship component stores fixed relations among people. Figure 2 (a) shows a toy instance: the red-ball chain tracks events from Alice’s initial ownership to Dave’s final possession, and the black kinship edges provide fixed relational facts in the same story. 2.2 Task Setup Logical Reasoning Task We instantiate four logical task families from the same controlled graph world. Deductive and abductive tasks use the dynamic component: deductive tasks track observed events forward to a queried state value, while abductive tasks recover a missing event or initial fact consistent with the observed states.

Read the complete text