From Implicit to Explicit: Token-Efficient Logical Supervision for Mathematical Reasoning in LLMs

Shaojie Wang, Liang Zhang {shaojiewang, liangzhang}@hkustgz.edu.cn · 2026

Read From Implicit to Explicit: Token-Efficient Logical Supervision for Mathematical Reasoning in LLMs by Shaojie Wang, Liang Zhang {shaojiewang, liangzhang}@hkustgz.edu.cn (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-03682.txt.

Introduction

Shaojie Wang and Liang Zhang argue that language models fail at math less from weak calculation than from weak logic. Their analysis finds errors about logical relationships, which variables connect and which operation applies, behind over ninety percent of wrong predictions. Standard step-by-step fine-tuning barely reduced those errors, because the logic stays implicit in long worked solutions.

Their framework, First-Step Logical Reasoning, trains models on the first planning step alone: choosing the variables and the operation from the problem statement. Isolating that step gives explicit supervision for logical understanding while staying light and token-efficient. The idea reframes math training: before teaching a model to write full solutions, teach it to read a problem's structure correctly, since every later step inherits that first decision.

Catalog record

CREATORShaojie Wang, Liang Zhang {shaojiewang, liangzhang}@hkustgz.edu.cn
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-03682

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

From Implicit to Explicit: Token-Efficient Logical Supervision for Mathematical Reasoning in LLMs Shaojie Wang, Liang Zhang {shaojiewang, liangzhang}@hkustgz.edu.cn Abstract Recent studies reveal that large language models (LLMs) exhibit limited logical reasoning abilities in mathematical problem-solving, instead often relying on pattern-matching and memorization. We systematically analyze this limitation, focusing on logical relationship understanding, which is a core capability underlying genuine logical reasoning, and reveal that errors related to this capability account for over 90% of incorrect predictions, with Chain-of-Thought Supervised Fine-Tuning (CoT-SFT) failing to substantially reduce these errors. To address this bottleneck, we propose F irst- S tep L ogical R easoning ( FSLR ), a lightweight training framework targeting logical relationship understanding. Our key insight is that the first planning step-identifying which variables to use and which operation to apply-encourages the model to derive logical relationships directly from the problem statement. By training models on this isolated step, FSLR provides explicit supervision for logical relationship understanding, unlike CoT-SFT which implicitly embeds such relationships within complete solution trajectories. Extensive experiments across multiple models and datasets demonstrate that FSLR consistently outperforms CoT-SFT under both in-distribution and out-of-distribution settings, with average improvements of 3.2% and 4.6%, respectively. Moreover, FSLR achieves 4-6× faster training and reduces training token consumption by over 80%. 1 Introduction Mathematical reasoning, as a crucial cognitive skill that supports problem-solving in numerous scientific and practical applications, has attracted particular attention in LLM research 1 ; 27 ; 5 . Although LLMs have achieved near-human accuracy on mathematical benchmarks such as GSM8K 4 , recent studies 20 ; 9 ; 16 reveal that LLMs exhibit limited logical reasoning abilities—the capacity to understand "novel" problems and derive appropriate solution steps, rather than relying on pattern-matching and memorization. As a fundamental capability for mathematical problem-solving, genuine logical reasoning enables models to understand the underlying rationale of problems and transfer knowledge to new situations 22 . Recently, Supervised Fine-Tuning (SFT) with chain-of-thought (CoT) supervision has received increasing attention and has been widely adopted to improve reasoning abilities of LLMs by training models on complete solution trajectories 25 . However, it remains unclear whether CoT-SFT can elicit genuine logical reasoning. Figure 1: Error analysis on Qwen2.5-7B comparing Base, CoT-SFT, and FSLR(Ours) models. Each bar shows the breakdown of correct predictions, logical relationship understanding errors, and other errors. To investigate this question, we conduct a systematic analysis comparing pretrained LLMs with their fine-tuned counterparts on mathematical reasoning tasks. Following the above discussion, genuine logical reasoning requires understanding a given problem before deriving solution steps. For mathematical problems, such understanding involves grasping how variables in the problem relate to each other and which operations connect them—what we term logical relationships. We refer to the ability to grasp these relationships as logical relationship understanding, and adopt it as our diagnostic metric for genuine logical reasoning. Specifically, we use GPT-4o 10 to analyze all incorrect predictions and identify whether failures arise from misunderstanding these logical relationships (as shown in Figure 1 ). Using Qwen2.5-7B as an example, we find that logical relationship understanding errors account for over 90% of incorrect predictions—a pattern that holds consistently across multiple LLMs (Appendix A ). Since CoT-SFT fails to substantially reduce these errors, they become the primary bottleneck limiting further improvements in overall accuracy. These findings raise a natural research question: how can we more directly improve LLMs’ understanding of logical relationships, thereby enhancing their overall performance? To answer this question, we examine why CoT-SFT falls short. CoT-SFT trains models to imitate complete solution trajectories, while logical relationships are implicitly embedded in these trajectories, never directly targeted by the training objective. We hypothesize that this implicit supervision leads to insufficient learning of logical relationships and consequently results in the phenomenon illustrated in Figure 1 . To address this limitation, we propose F irst- S tep L ogical R easoning ( FSLR ), a lightweight training framework that provides explicit supervision for enhancing logical relationship understanding. Specifically, we design a first-planning-step prompt schema that asks models to identify only what needs to be calculated first—which variables to use and which operation to apply, without solving the full problem. Then, FSLR isolates this first planning step as a standalone training task, thereby providing explicit supervision for logical relationship understanding. This design offers several key advantages: (1) More focused supervision : FSLR provides a more direct training signal for logical relationship understanding by isolating the first planning step explicitly, alleviating the core limitation we identified earlier; (2) Reduced training cost : predicting only the initial planning step requires significantly fewer tokens than complete trajectories; (3) Improved generalization : since logical relationship understanding forms the foundation of the entire reasoning chain, strengthening this ability is expected to benefit the entire problem-solving process. As shown in Figure 1 , FSLR achieves a substantial reduction in both logical relationship understanding errors and overall errors compared to CoT-SFT. Our main contributions are as follows: • We systematically analyze the reasoning capabilities of LLMs and identify logical relationship understanding as a critical bottleneck: such errors account for over 90% of failures, and CoT-SFT fails to substantially mitigate this issue. • We propose FSLR, a lightweight training framework that provides a more focused training task for logical relationship understanding by training models to identify the initial planning step obtained under a specially designed prompting scheme. • Extensive experiments across multiple models and datasets demonstrate that FSLR substantially outperforms CoT-SFT in both in-distribution and out-of-distribution settings, while requiring significantly fewer training tokens. 2 Related Work 2.1 Mathematical Reasoning in LLMs Mathematical reasoning has emerged as a critical testbed for evaluating whether large language models possess genuine reasoning abilities. Driven by this goal, the community has introduced diverse benchmarks 4 ; 21 ; 19 ; 13 ; 18 ; 6 , on which recent LLMs have achieved remarkable success. However, recent studies question whether these successes reflect genuine logical reasoning. GSM-Symbolic 20 and GSM-Plus 16 reveal that LLMs exhibit noticeable performance variance across problem instantiations and are far from robust. Other work shows that LLMs are easily distracted by irrelevant information 24 and blindly apply learned skills without assessing their applicability to modified contexts 9 . While these works clearly diagnose that current LLMs often rely on pattern matching rather than true reasoning, they do not propose concrete training strategies to improve this capability. 2.2 Chain-of-Thought Fine-Tuning Chain-of-Thought Supervised Fine-Tuning (CoT-SFT) has emerged as a widely adopted approach for enhancing LLM reasoning by exposing models to step-by-step solution trajectories 11 ; 15 ; 30 ; 14 . However, recent studies reveal several limitations: CoT-SFT decreases the faithfulness of reasoning 17 , leads models to memorize task-specific templates rather than acquiring transferable abilities 3 , and introduces spurious features that cause hallucinations 2 . While these studies reveal important limitations of CoT-SFT, none directly targets the logical relationship understanding bottleneck we identified. 3 Methodology 3.1 Task Formulation We formalize CoT-SFT and FSLR to clarify how they differ in supervising logical relationship understanding. CoT-SFT. Given a mathematical problem p and its complete solution trajectory s=(s_{1},s_{2},...,s_{n}) , CoT-SFT trains models to generate the full reasoning chain by optimizing: \mathcal{L}_{\text{CoT}}=-\sum_{i=1}^{n}\log P(s_{i}|p,s_{1},...,s_{i-1};\theta) While logical relationships are embedded within these steps, they are never explicitly targeted by the training objective.

Read the complete text