Regime-Aware Peer Specialization for Robust RAG under Heterogeneous Knowledge Conflicts
Bo Wang, Heyan Huang, Yaolin Li, Yanghao Zhou, Ziyi Yang, Ge Shi, Chong Feng · 2026
Read Regime-Aware Peer Specialization for Robust RAG under Heterogeneous Knowledge Conflicts by Bo Wang, Heyan Huang, Yaolin Li, Yanghao Zhou, Ziyi Yang, Ge Shi, Chong Feng (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-30518.txt.
Introduction
Bo Wang, Heyan Huang, and five colleagues train retrieval-augmented language models to handle sources that disagree with the model. Retrieved passages can be solid, mixed, or outright adversarial. One training signal for all cases muddles incompatible lessons. Their framework, RAPS-DA, sorts conflicts into three regimes first. Grounding, Arbitration, and Resistance name the three regimes. A specialist peer model is trained for each.
Every training sample routes to its matching specialist peer. Supervision flows back as on-policy reverse-divergence training. A token-level selector adds a second layer of control. It weighs teacher disagreement, student divergence, and student uncertainty. Uninformative or unstable tokens get filtered out. Confidently wrong tokens get upweighted. Supervision concentrates on the highest-conflict tokens. The specialists exist only during training, keeping deployment simple. Five conflict scenarios and two distribution-shift benchmarks test it. RAPS-DA beats prompting, decoding, fine-tuning, and reinforcement baselines.
Catalog record
| CREATOR | Bo Wang, Heyan Huang, Yaolin Li, Yanghao Zhou, Ziyi Yang, Ge Shi, Chong Feng |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 57 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.30518
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Regime-Aware Peer Specialization for Robust RAG under Heterogeneous Knowledge Conflicts Bo Wang, Heyan Huang, Yaolin Li, Yanghao Zhou, Ziyi Yang, Ge Shi, Chong Feng Abstract Retrieval-augmented generation (RAG) improves language models by grounding generation in external context. However, it can be fragile when the retrieved context conflicts with the model’s parametric knowledge. Such conflicts span a reliability spectrum, ranging from reliable and partially reliable evidence to adversarial context. Existing remedies often handle such heterogeneous conflicts with regime-agnostic supervision, which can conflate incompatible learning signals across reliability regimes. To disentangle these signals, we propose RAPS-DA , a regime-aware peer specialization framework that addresses conflict at two complementary granularities. At the sample level , conflicts are divided into three regimes, including Grounding , Arbitration , and Resistance , with one same-scale peer specialist trained per regime from a shared base model. Each sample is then hard-routed to its regime-matched peer for on-policy reverse-KL supervision. At the token level , a dual-layer selector uses inter-teacher disagreement, student–teacher divergence, and student entropy to filter uninformative or unstable tokens, upweight confidently misaligned ones, and gradually focus supervision on high-conflict tokens as the student matures. Gains stem from specialization at a fixed model scale, not from a stronger teacher, and the peer specialists exist only during training, so the deployed student requires no regime labels or peer access. Experiments on five conflict scenarios and two out-of-distribution benchmarks show RAPS-DA surpasses all prompting, decoding, fine-tuning, RL, and single-teacher baselines. 1 Introduction Retrieval-augmented generation (RAG) grounds large language models in external evidence, yet its success rests on the assumption that retrieved context is reliable [ 19 , 13 ] . When this assumption breaks, RAG systems face a trust–resistance dilemma: they may over-trust incorrect passages [ 28 , 35 ] , while resistance-oriented training can cause them to reject genuinely informative evidence [ 6 , 32 ] . Generally, context reliability is a spectrum in which reliable evidence should be integrated, adversarial evidence should be rejected, and mixed-quality evidence requires selective trust. This heterogeneity creates a training challenge: How can a single model be trained when different reliability regimes demand conflicting behaviors? A growing body of work has sought to address this challenge. Prompting and decoding methods [ 8 , 27 , 7 ] improve inference-time behavior but generalize poorly. Supervised and RL methods [ 5 , 9 , 22 ] apply a single training signal across mixed-reliability data, so conflicting regimes impose contradictory gradient updates. These methods either lack a trainable adaptation mechanism or provide supervision that is too coarse to resolve token-level follow-or-reject decisions. By contrast, on-policy distillation (OPD) [ 1 , 11 ] offers a more suitable foundation: it provides dense token-level supervision on the student’s own rollouts, directly shaping pivotal tokens where the model decides whether to follow or reject a retrieved claim [ 10 , 4 ] . Yet naive OPD still uses one teacher for all regimes, inheriting the same cross-regime interference. Despite these differences, existing approaches share a common limitation, namely regime-agnostic supervision. Adapting OPD to knowledge conflict further exposes two coupled difficulties. Sample-level regime heterogeneity . We distinguish three reliability regimes in RAG (Figure 1 a), namely Grounding (reliable context to integrate), Arbitration (mixed-quality context requiring selective trust), and Resistance (adversarial context to reject), each demanding markedly different behaviors. Empirically, models trained per-regime outperform a jointly-trained model on their corresponding subsets, revealing a clear regime-wise specialization pattern. As a result, a regime-agnostic teacher receives conflicting optimization signals from different regimes and systematically underfits them all. Token-level supervision noise . In conflict RAG, pivotal tokens often correspond to answer entities, source-attribution markers, rejection phrases, or option tokens where a small change determines whether the model follows CK or falls back to PK. The challenge is amplified in a multi-teacher setting, where disagreement among teachers introduces additional supervision heterogeneity [ 10 , 4 , 18 ] . Consequently, naively distilling from multiple specialized teachers does not reliably transfer their expertise to the student (Figure 1 b). Figure 1: Motivation for regime-aware peer specialization. (a) Contextual-knowledge reliability is modeled as a continuous spectrum and partitioned into three regimes—Resistance ( R , low reliability), Arbitration ( A , medium), and Grounding ( G , high)—each requiring a distinct behavior: PK retention, selective trust, or evidence integration. (b) Per-regime accuracy of representative methods (bars) against the oracle peer specialist upper bound (dashed lines). RAG and Mixed SFT collapse on Resistance; single-teacher OPD (OPCD) improves over them but still falls well short of the oracle across all regimes. RAPS-DA substantially narrows this gap without requiring regime labels at inference. To address these challenges, we propose RAPS-DA ( R egime- A ware P eer S pecialization with D ifficulty A nnealing), a training framework that handles heterogeneous knowledge conflicts through regime-aware peer supervision and difficulty-aware token learning. All peer teachers are fine-tuned from a shared base model on regime-specific data, so the improvements come from specialization at a fixed model scale rather than from using a stronger or larger teacher. The framework has three modules: regime-routed peer supervision, conflict-aware token selection, and difficulty annealing. The first two modules target sample-level regime heterogeneity and token-level supervision noise, respectively, while the third stabilizes their interaction during training. (i) Regime-routed peer supervision ( addresses sample-level regime heterogeneity ). We train regime-specialized peer teachers and hard-route each sample to its matched peer for on-policy reverse-KL supervision, with the base model serving as an implicit KL anchor. Hard routing eliminates the contradictory gradient updates that a single jointly-trained teacher would impose across regimes. (ii) Conflict-aware token selection ( addresses token-level supervision noise ). Three diagnostic signals (inter-teacher conflict, student–teacher gap, and student entropy) drive a hard mask that filters uninformative or unstable tokens, together with a soft weight that emphasizes tokens where the student is confidently misaligned with the regime-matched teacher. (iii) Difficulty annealing (stabilizes the combination of (i) and (ii)). Routing concentrates supervision on regime-matched teachers, while token selection further amplifies high-conflict positions. Applying both mechanisms from the beginning can prematurely focus training on a small set of difficult tokens. We therefore start with broad token coverage and gradually concentrate supervision on harder positions as training progresses. ALL specialization and routing occur only during training. At inference time, the deployed system is a single student model that requires neither regime labels nor access to peer teachers. Our key observation is that sample-level routing and token-level selection address complementary challenges. Routing resolves policy mismatch across reliability regimes, while token selection improves the quality of supervision within each regime. Together, however, they can over-concentrate training on difficult tokens before the student has learned stable regime-specific behaviors. Difficulty annealing mitigates this effect by gradually shifting supervision from broad coverage to high-conflict tokens. Empirically, each module helps on its own, but only their combination yields the largest gains in our experiments. Empirically, the three components are complementary, and their combination provide the most substantial benefits. Our main contributions are as follows. • We characterize knowledge conflict in RAG as a spectrum of reliability regimes and show that regime-agnostic supervision induces conflicting optimization signals, motivating regime-specialized supervision. • We propose RAPS-DA, a regime-aware peer specialization framework that combines regime-routed peer supervision, conflict-aware token selection, and difficulty