Dual-Track CoT: Budget-Aware Stepwise Guidance for Small LMs

Atharva Patil, Sricharan Ramesh, Sagnik Chatterjee · 2026

Read Dual-Track CoT: Budget-Aware Stepwise Guidance for Small LMs by Atharva Patil, Sricharan Ramesh, Sagnik Chatterjee (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-25039.txt.

Introduction

Chatterjee, Patil and Ramesh propose Dual-Track CoT, budget-aware stepwise guidance for small language models. Large models reason via chain-of-thought, but smaller models of about 7 to 8 billion parameters struggle under tight budgets. Existing methods like self-consistency and Tree-of-Thoughts improve performance but cost many tokens and lack step control. The project asks whether small models can reason reliably with the same or fewer tokens. Process supervision and simple controls, such as rejecting redundant steps and token budgets, might substitute for model scale.

Catalog record

CREATORAtharva Patil, Sricharan Ramesh, Sagnik Chatterjee
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41578
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2604.25039

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Dual-Track CoT: Budget-Aware Stepwise Guidance for Small LMs Atharva Patil, Sricharan Ramesh, Sagnik Chatterjee Abstract 1 Problem statement Large Language Models (LLMs) solve many reasoning tasks via chain-of-thought (CoT) prompting, but smaller models (about 7–8B parameters) still struggle with multi-step reasoning under tight compute and token budgets. Existing test-time reasoning methods such as self-consistency (sampling multiple rationales and voting), Tree-of-Thoughts (search over intermediate thoughts), and critique–revise loops improve performance, but often at high token cost and without fine-grained step-level control. This project 1 1 1 Code and experiments are available at https://github.com/atharvadpatil/DualTrack-COT . aims to address that gap: can Small Language Models (SLMs) reason reliably using the same or fewer tokens? This question is both scientific and practical. Scientifically, it probes whether process supervision and simple test-time controls (such as token budgets and rejection of redundant steps) can substitute for model scale or large sampling counts. Practically, many deployments (on-device, low-latency, or cost-constrained settings) cannot afford huge models or dozens of sampled rationales per query. A method that improves SLM reasoning at fixed cost would therefore be directly useful. 2 What we proposed vs. what we accomplished • Build a Dual CoT pipeline with a Decomposer SLM and an Evaluator SLM working together. • Use few-shot prompting to improve accuracy of both models. • Modify the GSM8K training dataset to create step-level supervision that matches the framework. • Fine-tune the Decomposer on the modified GSM8K dataset. • Fine-tune the Evaluator on PRM-800K for step-level scoring. • Planned but not pursued: scoring on separate criteria such as logical validity, relevance, consistency, complexity, efficiency, and token cost. We dropped this after early experiments showed the evaluator struggled to produce stable multi-criterion scores. • Planned but not kept: beam search in the final architecture. In practice, beams often collapsed to identical steps generated in each beam through greedy token generation which led to wastage of tokens without improving accuracy. • Evaluate the framework on 50 samples from the GSM8K test set. • Perform manual error analysis on all test samples. • Study the effect of token constraints on accuracy. • Introduce a rejection cache to save evaluator tokens on redundant steps and check its impact on accuracy. 3 Related work Recent advances in reasoning with large language models (LLMs) have moved beyond linear, single-path generation toward structured or self-corrective paradigms. A prominent branch of work explores search-based reasoning, where models branch and evaluate multiple lines of thought. The Tree of Thoughts (ToT) framework 6 , represents problem solving as a tree of reasoning units, allowing the model to explore, evaluate, and backtrack across paths. This deliberate structure improves performance over standard Chain-of-Thought (CoT) prompting but incurs high computational cost. Expanding and scoring many branches is expensive, limiting its practicality for smaller or resource-constrained models. In a complementary approach, Critic-CoT 7 trains LLMs to critique and iteratively refine their own chain-of-thought, evaluating each intermediate step for correctness before revision. However, Critic-CoT mainly operates along a single trajectory, so early mistakes may persist if the model fails to fully correct them; extensions to multi-path variants would likely reintroduce ToT’s computational burden. Parallel to these efforts, Self-Consistency 4 introduces an unsupervised decoding strategy that samples diverse reasoning paths and aggregates final answers by majority vote. It improves accuracy without retraining but suffers from high inference cost and lacks intra-chain correction. The mistakes made early in a chain propagate to the end since errors are never revised mid-way. More recently, researchers have turned toward process-supervised or verifier-guided reasoning to detect and correct errors at the step level. The STEPCO 5 framework integrates a process-supervised verifier (PSV) which scores each reasoning step and triggers local revisions when errors are detected. Although effective, STEPCO generates a full candidate path before verification, meaning tokens might be wasted on later steps following undetected mistakes, and its application in the original work is more focused on numeric or structured tasks. Two closely related works — Math-Shepherd 3 and AutoPSV 2 — further advance process-supervised reasoning. Math-Shepherd introduces a process reward model (PRM) that assigns rewards to reasoning steps without human labels, using Monte Carlo continuations from each prefix to estimate step quality. This mechanism allows reranking of reasoning traces and reinforcement learning updates via PPO, boosting performance especially on math benchmarks like GSM8K and SVAMP. However, its reliance on large-scale sampling makes it computationally expensive and primarily confined to mathematical reasoning. In contrast, AutoPSV infers process-level supervision via relative changes in verifier confidence. It trains a verifier on final-answer correctness, then uses fluctuations in confidence at each step to infer which steps may be erroneous. AutoPSV thus offers annotation-free process supervision at low cost, though its current form is more oriented toward error detection or reranking rather than stepwise correction. Despite these advances, most existing systems intervene only after full reasoning chains have been generated. They lack real-time, during-generation guidance and often consume excessive tokens exploring redundant or unpromising reasoning paths. 4 Datasets GSM8K (Decomposer fine-tuning and evaluation). We use GSM8K (Grade School Math 8K) 2 2 2 https://huggingface.co/datasets/openai/gsm8k as our primary benchmark for grade-school math word problems. The dataset contains 8.5K problems paired with worked solutions that include intermediate reasoning and end with an explicit final answer marker. Each example has a question field (natural language problem statement) and an answer field (step-by-step solution in free text). We fine-tune our Decomposer on the GSM8K training split and use a small held-out portion as a validation set, keeping the official test split untouched for final evaluation. Example (GSM8K format). "question": "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?" "answer": "Natalia sold <<48/2 = 24>> clips in May. Natalia sold <<48 + 24 = 72>> clips altogether. #### 72" PRM-800K (Evaluator fine-tuning). We fine-tune our Evaluator on PRM-800K 3 3 3 https://huggingface.co/datasets/tasksource/PRM800K . , a process reward modeling dataset that provides step-level quality labels for mathematical reasoning. Each completion includes a text field (the candidate step) and a discrete rating label indicating step quality. We train the Evaluator to map a problem and candidate step to this rating, which provides a step-level quality signal. At inference time, we use this capability to accept high quality steps and reject steps that are incorrect, vague, or unhelpful. Example (PRM-800K completion format). "text": "7.8 minutes is the same as 7 minutes and 0.8 minutes.", "rating": 1 "text": "Right. 7.8 minutes is the same as 7 minutes and 0.8 minutes.", "rating": 0 Why these datasets are challenging. GSM8K trains the Decomposer only on correct trajectories: the gold solutions contain successful reasoning steps but no explicit negatives. The model sees what correct next steps look like, but not what wrong steps look like or how to recover from them. As a result, even after fine-tuning, the Decomposer can still make arithmetic mistakes or adopt incorrect decompositions, and there is no explicit training signal for error recovery. Multi-step error propagation remains a central challenge: an early conceptual or numeric mistake can corrupt the history, so later steps remain locally plausible but lead to the wrong final answer. For the Evaluator, PRM-800K provides ratings for step quality

Read the complete text