DiSCTT: Consensus-Guided Self-Curriculum for Efficient Test-Time Adaptation in Reasoning Mohammad Mahdi Moradi, Sudhir Mudur Abstract Test-time adaptation offers a promising avenue for improving reasoning performance in large language models without additional supervision, but existing approaches often apply a uniform optimization objective across all inputs, leading to inefficient or unstable adaptation on heterogeneous reasoning problems. We propose DiSCTT, a difficulty-aware, consensus-guided self-curriculum framework that dynamically allocates test-time optimization strategies based on instance-level epistemic uncertainty estimated from agreement among sampled reasoning trajectories. Inputs with high consensus are consolidated via supervised fine-tuning using majority-agreed solutions as pseudo-labels, while low-consensus inputs are optimized via reinforcement learning with a consensus-regularized objective that encourages diversity under relevance constraints. Across a broad suite of mathematical and general reasoning benchmarks, DiSCTT consistently outperforms strong test-time adaptation baselines, achieving higher accuracy with reduced variance and substantially lower computation and wall-clock training times. These results demonstrate that explicitly accounting for instance difficulty and uncertainty enables more stable, efficient, and effective test-time adaptation for reasoning models. 1 Introduction Large language models (LLMs) have achieved strong performance on mathematical and general reasoning tasks through supervised fine-tuning, chain-of-thought prompting, and reinforcement learning ( 10 ) . Despite these advances, inference-time behavior remains largely static: once deployed, a model applies a fixed policy to all inputs, regardless of input difficulty or the model’s own uncertainty. This rigidity has motivated growing interest in test-time adaptation , where models update their behavior during inference without access to ground-truth labels ( 3 ; 12 ) . Reasoning tasks are a particularly natural setting for such adaptation, as they exhibit wide variation in difficulty and admit multiple valid solution paths. A key challenge in test-time adaptation is that reasoning problems are inherently heterogeneous. Easier instances often benefit from consolidation, where reinforcing high-confidence solutions stabilizes correct behavior. Harder instances, by contrast, require structured exploration to uncover alternative reasoning strategies. Most existing test-time training methods apply a single optimization objective uniformly across all inputs (Figures 1 , 2 ). Supervised self-training can saturate quickly and offers limited benefit on difficult problems. On the other hand, uniform reinforcement learning introduces unnecessary variance on already-solved instances, often leading to unstable or inefficient learning dynamics. These limitations point to a broader principle: effective test-time adaptation for reasoning requires allocating different learning objectives to different inputs based on model uncertainty. In particular, test-time learners must (i) estimate instance difficulty online, (ii) consolidate high-confidence solutions efficiently, and (iii) focus exploration on uncertain instances while stabilizing reinforcement learning in the absence of external supervision. Applying a single objective uniformly at inference time inevitably trades off over-optimizing easy instances against under-exploring genuinely hard ones. Reasoning problems also pose challenges for uncertainty estimation. Token-level confidence scores or entropy are poorly aligned with multi-step reasoning, where errors often emerge only at the trajectory level ( 2 ) . In contrast, agreement among independently sampled reasoning trajectories can provide a useful proxy for epistemic uncertainty: high consensus indicates stable reasoning modes, while disagreement reflects uncertainty over inference strategies. This trajectory-level consensus therefore serves as an effective, label-free estimate of relative instance difficulty at test time. We operationalize this principle with DiSCTT (Difficulty-aware Consensus-Guided Self-Curriculum Test-Time Adaptation). DiSCTT uses consensus among sampled reasoning completions to estimate epistemic uncertainty online and dynamically routes inputs between two complementary learning objectives. Inputs with high consensus are optimized via supervised fine-tuning using majority-agreed solutions as pseudo-labels, consolidating correct behavior with low variance. Inputs with low consensus are optimized via reinforcement learning, enabling structured exploration of alternative reasoning paths. This routing is periodically recomputed, yielding a self-evolving curriculum that adapts to the model’s changing competence rather than relying on a fixed easy–hard decomposition. We summarize our contributions as follows: • Consensus-based difficulty estimation at test time. We formalize agreement among independently sampled reasoning trajectories as an online, instance-level estimator of epistemic uncertainty, enabling difficulty-aware adaptation without access to ground-truth labels. • Difficulty-aware self-curriculum for test-time adaptation. We introduce a dynamic routing mechanism that allocates supervised consolidation to high-consensus inputs and reinforcement learning to low-consensus inputs, yielding a self-evolving curriculum that adapts to the model’s changing competence during inference. • Stabilized label-free reinforcement learning for reasoning. We propose a correctness-gated, population-relative reward augmented with relevance-aware semantic gating, which encourages controlled exploration over reasoning trajectories while improving the stability of policy updates in the absence of external supervision. • Extensive empirical evaluation across models and benchmarks. We show that DiSCTT consistently improves accuracy, stability, and computational efficiency across diverse mathematical and general reasoning benchmarks, model scales, and test-time budgets. Figure 1 : Accuracy on MATH-500 over test-time training updates for different adaptation strategies. Uniform test-time training baselines show early saturation or unstable gains, whereas DiSCTT achieves more stable and sustained improvements through difficulty-aware routing. Figure 2 : AMC accuracy for different adaptation strategies. Difficulty-aware routing and correctness-gated exploration improve convergence stability and prevent performance collapse observed under uniform RL. 2 Related Work Modern reasoning performance in large language models is closely tied to eliciting intermediate reasoning steps and aggregating multiple sampled solutions. Chain-of-thought prompting improves multi-step reasoning by inducing explicit intermediate steps ( 11 ) . Beyond single-sample decoding, self-consistency demonstrates that sampling many reasoning paths and selecting the most frequent final answer via majority voting yields substantial gains across reasoning tasks ( 9 ) . This use of agreement among model generations as a weak supervision signal underpins a growing class of label-free or weakly supervised approaches for improving reasoning without access to ground-truth labels at inference time. However, in most prior works, consensus is used as a static selection mechanism rather than as a dynamic signal to guide how learning objectives should be allocated. Test-Time Training (TTT) formalizes adaptation at inference by updating model parameters on unlabeled test inputs using self-supervised objectives, primarily to address distribution shift ( 8 ) . Verifier-Driven Selective Test-Time Training (VDS-TTT) extends this paradigm by conditioning supervised fine-tuning updates using an external verifier ( 5 ) . While verifier-based selection can improve stability, performance becomes strongly dependent on verifier quality and availability, and such methods do not model population-level difficulty or allocate exploration across reasoning instances. In reasoning-specific settings, Test-Time Reinforcement Learning (TTRL) extends TTT by applying reinforcement learning updates at inference time using majority-based signals derived from multiple generations ( 14 ) . TTRL establishes that label-free self-improvement is feasible for complex reasoning tasks, but also exposes two key challenges. First, updates are applied uniformly, remaining blind to epistemic uncertainty and problem difficulty. Second, repeated self-updates under uniform reinforcement pressure can introduce instability, noisy learning dynamics, or partial collapse. To mitigate exploration collapse in label-free reinforcement learning, EVOL-RL proposes coupling majority-based selection with novelty-promoting variation, implemented via embedding-space diversity and GRPO-based optimization ( 13 ) . EVOL-RL applies a uniform optimization objective across all inputs and while effective, does not explicitly account for heterogeneity in instance difficulty. Consequently, novelty-driven exploration is encouraged even for high-confidence, already-solved instances, introducing unnecessary variance and potentially destabilizing learning. In addition, novelty is defined globally over the response population, without conditioning on instance-specific reasoning structure or semantic relevance, which can promote spurious or off-task deviations. Orthogonal to how test-time learning signals are constructed, reinforcement learning has become a standard optimization backbone for improving reasoning and alignment in language models, including RLHF-style pipelines ( 1 ; 6 ) and reinforcement learning from verifiable rewards in mathematical reasoning. Group Relative Policy Optimization (GRPO), introduced in DeepSeekMath, has emerged as a practical optimizer for reasoning-centric reinforcement learning due to its stability and efficiency under group-based sampling ( 7 ) . In this work, GRPO is adopted purely as an optimization backbone, while our contributions focus on when reinforcement learning should be applied and how exploration should be stabilized under fully label-free conditions. Prior work establishes three complementary directions: label-free test-time adaptation (TTRL), selective or verifier-driven updates (VDS-TTT), and diversity-promoting reinforcement learning without external supervision (EVOL-RL). DiSCTT unifies and extends these paradigms through a difficulty-aware allocation of learning objectives, rather than applying reinforcement learning uniformly or modulating rewards within a single objective. Specifically, DiSCTT addresses difficulty blindness via consensus-based epistemic routing, stabilizes optimization through correctness-gated reinforcement learning with relevance-aware semantic gating, and replaces global semantic novelty with population-aware distributional divergence over majority-correct reasoning trajectories. Together, these choices enable stable, scalable, and fully label-free test-time self-improvement for reasoning-centric language models. More broadly, DiSCTT is also related to prior work on adaptive computation, curriculum learning, and uncertainty-aware optimization, which allocate computation or learning effort based on estimates of confidence or difficulty. Figure 3 : Overview of DiSCTT. For each input, multiple reasoning completions are sampled and evaluated for consensus. High-consensus inputs are routed to supervised fine-tuning using majority-agreed solutions as pseudo-labels, while low-consensus inputs are optimized via reinforcement learning. This difficulty-aware bifurcation is periodically recomputed, yielding a self-evolving test-time curriculum. 3 Method We present DiSCTT, a test-time adaptation framework that allocates learning objectives based on instance difficulty. For each input, the model samples multiple reasoning completions and uses their agreement to estimate epistemic uncertainty, which governs routing between supervised fine-tuning and reinforcement learning. High-consensus inputs are consolidated via supervised fine-tuning, while low-consensus inputs are optimized via reinforcement learning, avoiding unnecessary high-variance updates on already-solved instances. Figure 3 provides an overview of the DiSCTT framework, illustrating consensus-based difficulty estimation, dynamic routing between learning objectives, and the resulting self-evolving curriculum. Reinforcement learning is applied only to a dynamically selected subset of difficult inputs, while high-confidence instances are consolidated through supervised fine-tuning, enabling stable and efficient test-time reasoning improvement. We next describe how consensus induces a difficulty-aware partition of test-time inputs, followed by the supervised and reinforcement learning updates applied to each partition. 3.1 Problem Setting Let \mathcal{D}=\{x_{j}\}_{j=1}^{N_{d}} denote a dataset of reasoning problems. Given an input x_{j} , a policy \pi_{\theta} samples M independent reasoning completions: \mathcal{Y}_{j}=\{y_{j,1},\dots,y_{j,M}\},\quad y_{j,i}\sim\pi_{\theta}(\cdot\mid x_{j}), where each completion y_{j,i}=(r_{j,i},a_{j,i} ) consists of a reasoning trace r_{j,i} and a final answer a_{j,i} . 3.2 Consensus-Based Difficulty Estimation For each completion, y_{j,i} , we extract its final answer a_{j,i} . The empirical agreement ratio for input x_{j} is defined as: c_{j}=\frac{1}{M}\max_{a}\sum_{i=1}^{M}\mathbf{1}[a_{j,i}=a] (1) It measures the degree to which independently sampled reasoning trajectories converge to the same solution. High agreement is considered evidence of low epistemic uncertainty and low agreement as an indicator of unresolved ambiguity or reasoning instability. Using a fixed threshold \rho , we partition the dataset as: \mathbb{D}_{\text{easy}}=\{x_{j}\mid c_{j}\geq\rho\},\quad\mathbb{D}_{\text{hard}}=\{x_{j}\mid c_{j}<\rho\} (2) Crucially, this partition is temporary and policy-dependent. Difficulty is recomputed periodically during training, allowing problems to migrate between subsets as the model’s competence evolves. 3.3 Dynamic Self-Curriculum Training Training proceeds in alternating phases of SFT and RL. During SFT, the model is optimized on \mathbb{D}_{easy} using majority-agreed solutions as pseudo-labels. During RL, optimization is restricted to \mathbb{D}_{hard} , where the model is encouraged to explore correct yet informative reasoning trajectories under a structured reward. These two phases are interleaved over training, but difficulty reassessment is not performed at every optimization step. Specifically, after a fixed number K of training steps, the current policy \pi_{\theta} is used to regenerate a fresh set of completions for all inputs, agreement ratios are recomputed, and the dataset is re-partitioned into \mathbb{D}_{easy} and \mathbb{D}_{hard} accordingly. Formally, difficulty reassignment is triggered whenever the training step index t satisfies t\bmod K=0 . This periodic regeneration mechanism induces a closed-loop self-curriculum in which the difficulty of each problem is re-evaluated under the evolving policy. As training progresses, problems that initially exhibit low agreement may transition into the high-consensus regime, while previously easy problems may remain stable. By decoupling difficulty estimation from individual gradient steps and updating it only at fixed intervals, the curriculum remains responsive to policy improvements while avoiding excessive non-stationarity. 3.4 Supervised Objective For inputs x_{j}\in\mathbb{D}_{easy} , we perform supervised fine-tuning using the majority-agreed completion y_{j}^{\star} . \mathcal{L}_{\text{SFT}}(\theta)=-\mathbb{E}_{(x_{j},y_{j}^{\star})\sim D_{\text{easy}}}\left[\log\pi_{\theta}(y_{j}^{\star}\mid x_{j})\right] (3) This phase reinforces reasoning trajectories that are both consensus-consistent and low-variance, consolidating stable inference patterns. 3.5 Reinforcement Learning Objective For each input assigned to the hard subset \mathbb{D}_{hard} we optimize the policy using Group-Relative Policy Optimization (GRPO) over its N sampled completions. Each completion y_{i}=(r_{i},a_{i}) recieves a reward defined as: \displaystyle R(y_{i}) \displaystyle=\mathbf{1}[a_{i}=a_{\text{maj}}(x)]\cdot\big(\alpha+\beta\,\mathrm{JSD}_{\mathrm{nov}}(r_{i})\big) (4) \displaystyle\cdot\big(\varepsilon+(1-\varepsilon)\,g_{\mathrm{rel}}(r_{i})\big) which decomposes into correctness gate, population-relative novelty, and relevance-aware gating. This multiplicative structure enforces a strict logical ordering: only only majority-agreed solutions are reinforced; among them, novel reasoning paths are encouraged; and novelty is rewarded only insofar as it remains semantically aligned with the input. Correctness gating. Let \{a_{i}\}_{i=1}^{N} be the set of final answers produced for input x . We define the majority answer a_{\text{maj}}(x)=\mathop{\mathrm{arg\,max}}_{a}\sum_{i=1}^{N}\mathbf{1}[a_{i}=a] (5) and treat it as a pseudo-label induced by self-consistency. The correctness indicator is: \mathbf{1}[a_{i}=a_{\text{maj}}(x)] (6) implements a consensus-based supervision signal without access to external ground-truth labels. This mechanism provides a conservative yet internally consistent correctness criterion, ensuring that policy updates reinforce only those reasoning trajectories that align with the dominant solution mode under the current policy. An empirical analysis of majority voting is provided in Appendix C.1 . Population-relative novelty. To encourage informative exploration in reasoning space, we reward deviation from established population-level solution modes rather than absolute novelty. Let T_{i} denote the set of reasoning token positions in r_{i} , excluding prompt tokens and the final answer. For each prefix r_{i,