CoRe : Collaborative Reasoning via Cross Teaching

Kshitij Mishra, Mirat Aubakirov, Martin Takac, Nils Lukas, Salem Lahlou · 2026

Read CoRe : Collaborative Reasoning via Cross Teaching by Kshitij Mishra, Mirat Aubakirov, Martin Takac, Nils Lukas, Salem Lahlou (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-21600.txt.

Introduction

Kshitij Mishra, Mirat Aubakirov, Martin Takac, Nils Lukas, and Salem Lahlou let language models teach each other during training. Models make different reasoning mistakes: on one problem, one model's split of the task works while another's fails. That spread is usually wasted.

Their framework, CoRe, turns a peer's success into a learning signal through cross teaching. When one model solves a case, its path becomes a lesson for the others. Training together this way spreads strengths across the group. The method aims at model teams that end up individually stronger, not just collectively lucky.

Catalog record

CREATORKshitij Mishra, Mirat Aubakirov, Martin Takac, Nils Lukas, Salem Lahlou
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE58 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-21600

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

CoRe : Collaborative Reasoning via Cross Teaching Kshitij Mishra, Mirat Aubakirov, Martin Takac, Nils Lukas, Salem Lahlou Abstract Large language models exhibit complementary reasoning errors: on the same instance, one model may succeed with a particular decomposition while another fails. We propose Collaborative Reasoning ( CoRe ), a training-time collaboration framework that converts peer success into a learning signal via a cross-teaching protocol. Each problem is solved in two stages: a cold round of independent sampling, followed by a contexted rescue round in which models that failed receive hint extracted from a successful peer. CoRe optimizes a combined reward that balances (i) correctness, (ii) a lightweight DPP-inspired diversity term to reduce error overlap, and (iii) an explicit rescue bonus for successful recovery. We evaluate CoRe across four standard reasoning datasets GSM8K, MATH, AIME, and GPQA. With only 1,000 training examples, a pair of small open source models ( 3B + 4B ) reaches Pass@2 of 99.54% on GSM8K and 92.08% on MATH, compared to 82.50% and 74.82% for single-model training. On harder datasets, the 3B + 4B pair reaches Pass@2 of 77.34% on GPQA (trained on 348 examples) and 79.65% on AIME (trained on 792 examples), using a training-time budget of at most 1536 context tokens and 3072 generated tokens. Overall, these results show that training-time collaboration can reliably convert model complementarity into large gains without scaling model size. 1 1 1 Code: https://github.com/Mishrakshitij/CoRe.git 1 Introduction Large language models (LLMs) can solve many multi-step reasoning problems when prompted to produce intermediate steps (chain-of-thought; 34 ). However, even strong models exhibit systematic and complementary failures: different architectures, pretraining mixes, and post-training pipelines lead to distinct error modes on the same dataset. In practice, ensembles and multi-agent prompting exploit this diversity at inference time (e.g., self-consistency 33 , ranking-and-fusion 14 , and debate-style interactions 13 ). A natural next question is whether we can push this complementarity into training so that models become better collaborators . Figure 1 : CoRe : Collaborative Reasoning via Cross Teaching Training. Most recent gains in LLM reasoning come from inference-time scaffolding (more samples, search, or multi-agent prompting), which increases compute but does not change the underlying policy ( 33 ; 38 ; 5 ) . These methods often break under correlated failure: if a model misses a key insight, additional samples and even additional agents can converge to the same wrong plan; more generally, ensemble gains are limited when member errors are correlated ( 3 ; 16 ) . Training-time collaboration is challenging due to sparse correctness supervision (motivating process-level feedback), multi-model credit assignment, and the risk of diversity collapse through mutual imitation or agreement-style training ( 20 ; 6 ; 10 ; 41 ) . This motivates a reinforcement learning (RL) view: we must jointly optimize (i) exploitation for correctness, (ii) exploration for complementary traces that reduce overlap, and (iii) rescue-specific reward on peer-disagreement cases. We propose Collaborative Reasoning ( CoRe ) (Figure 1 ), an algorithm for training multiple LLMs to collaborate on reasoning tasks. CoRe is built around a simple micro-round protocol: (1) in a cold round, each model samples several independent solution traces; (2) if at least one model succeeds, we extract out a successful trace as hint; and (3) in a contexted round, each model resamples with hint, and models that previously failed are eligible for an explicit rescue bonus when they recover. The key intuition is that ”peer success” provides a targeted supervision signal precisely on the instances where one model needs help. Our goal is not only to improve single-model accuracy but also to reduce error overlap between collaborators, thereby increasing the probability that at least one model solves a problem. We operationalize this with an explore–exploit reward design. Exploitation rewards correctness (with optional partial credit), while exploration rewards encourage diverse reasoning traces using a greedy Determinantal Point Process (DPP) approximation. Our key contributions are three-fold: 1. We propose Collaborative Reasoning ( CoRe ), a novel algorithm to jointly train N LLM policies with a two-round protocol–a cold round of independent sampling followed by a contexted rescue round where failed models learn from hints extracted from successful peers, with hint dropout to improve robustness. 2. We propose a combined reward that simultaneously optimizes (i) correctness, (ii) exploration-exploitation of complementary traces (reducing correlated failures), and (iii) an explicit rescue bonus to incentivize successful recovery on peer-disagreement. 3. Extensive empirical analysis that show across GSM8K, MATH, AIME, GPQA, and MedMCQA, CoRe consistently improves collaborative performance, achieving 99.54% Oracle Team Pass@2 on GSM8K and strong gains on harder benchmarks (e.g., 92.08% on MATH, 79.65% on AIME, and 77.34% on GPQA), while training on no more than 1,000 examples. 2 Related Work Reasoning via prompting, search, and inference-time ensembling. Prompting methods elicit intermediate reasoning traces, most notably chain-of-thought ( 34 ) , and can be strengthened with decompositional prompting such as least-to-most ( 43 ) . Sampling-based inference improvements like self-consistency ( 33 ) reduce variance by aggregating multiple traces, while search-based frameworks explicitly explore reasoning branches (e.g., tree-of-thought ( 38 ) ). Other lines interleave reasoning with actions or external tools (e.g., ReAct ( 39 ) ). Model combination methods further improve accuracy by selecting or fusing candidate generations (e.g., ranking-and-fusion in LLM-Blender ( 14 ) ). In contrast, CoRe moves collaboration into training : peer successes become an on-policy learning signal (via hints + rescue), rather than only an inference-time aggregation trick. Reinforcement learning and preference optimization for LLM post-training. Alignment and post-training commonly rely on RLHF-style objectives ( 24 ) or preference optimization such as DPO ( 25 ) . For reasoning-focused post-training, group-based policy optimization (e.g., GRPO) has shown strong empirical performance ( 29 ) , and recent work proposes more stable variants such as sequence-level clipping (GSPO) ( 42 ) and smooth, token-adaptive trust-region control (SAPO) ( 8 ) . Prior work has also applied preference optimization directly to chain-of-thought traces to improve mathematical reasoning ( 18 ) . CoRe is largely orthogonal to these optimizer choices: we contribute a multi-model data collection protocol at training time itself and reward shaping aimed at reducing correlated failures. Diversity, exploration, and reducing correlated failures. Diversity is a classic lever for improving ensemble performance and reducing error correlation ( 3 ; 17 ) . In sequence generation, diverse decoding procedures such as Diverse Beam Search encourage exploration of multiple modes ( 31 ) . Determinantal point processes (DPPs) provide a principled repulsive model for selecting diverse subsets ( 15 ) . Recent SD-E 2 ( 22 ) explores diverse reasoning traces within a single small model ( 23 ) . Building on these ideas, we introduce a lightweight greedy approximation ( DPP-lite ) over hybrid semantic/structural distances between reasoning traces, and use it as an explicit exploration reward to reduce overlap between collaborators rather than merely diversifying outputs at inference time. Figure 2 : CoRe training. In Round A , each model samples K independent traces. If any trace succeeds, the best one is used as teacher context for Round B , where models resample with explore–exploit rewards and a rescue bonus for successful recoveries. Multi-agent and multi-model collaboration. Multi-agent prompting and debate-style protocols can improve factuality and reasoning at inference

Read the complete text