ReM-MoA: Reasoning Memory Sustains Mixture-of-Agents Scaling

Heng Ping, Arijit Bhattacharjee, Peiyu Zhang, Shixuan Li, Wei Yang, Ali Jannesari, Nesreen Ahmed, Paul Bogdan · 2026

Read ReM-MoA: Reasoning Memory Sustains Mixture-of-Agents Scaling by Heng Ping, Arijit Bhattacharjee, Peiyu Zhang, Shixuan Li, Wei Yang, Ali Jannesari, Nesreen Ahmed, Paul Bogdan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-24437.txt.

Introduction

Ping and colleagues give teams of language model agents a memory, so getting deeper stops making them worse. Mixture-of-Agents designs layer many agents into reasoning pipelines, and should improve with scale. Existing variants degrade, plateau early, or saturate as layers are added, though. Their 2026 framework, ReM-MoA, sustains scaling through structured reasoning memory shared across layers.

A Ranked Reasoning Memory stores reasoning traces from every layer. A Reviewer Agent ranks them comparatively. Curated Diversified Memory Routing then exposes different agents to distinct mixes of successful and failed traces. Exploration diversity is preserved, while high quality reasoning propagates. Across five reasoning benchmarks, spanning maths, formal logic, code, knowledge, and commonsense, ReM-MoA beats earlier mixture variants in both depth and width scaling. Its advantage widens with depth. Ping's team names cross layer memory the missing mechanism for scalable multi agent inference.

Catalog record

CREATORHeng Ping, Arijit Bhattacharjee, Peiyu Zhang, Shixuan Li, Wei Yang, Ali Jannesari, Nesreen Ahmed, Paul Bogdan
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.24437

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ReM-MoA: Reasoning Memory Sustains Mixture-of-Agents Scaling Heng Ping, Arijit Bhattacharjee, Peiyu Zhang, Shixuan Li, Wei Yang, Ali Jannesari, Nesreen Ahmed, Paul Bogdan Abstract Mixture-of-Agents (MoA) architectures improve inference-time scaling by organizing multiple LLM agents into layered reasoning pipelines. However, existing MoA variants fail to sustain gains as depth increases, exhibiting degradation, early plateauing, or saturation. We propose ReM-MoA, a memory-augmented MoA framework that sustains scaling through two mechanisms: (1) a Ranked Reasoning Memory that persistently stores and ranks reasoning traces from all layers using a comparative Reviewer Agent, and (2) a Curated Diversified Memory Routing scheme that exposes different agents to distinct combinations of successful and failed traces, preserving exploration diversity while propagating high-quality reasoning. We further introduce an optional multi-domain Reviewer distillation pipeline that improves ranking quality through frontier-model supervision. Across five reasoning benchmarks spanning math, formal logic, code, knowledge, and commonsense, ReM-MoA consistently outperforms prior MoA variants across both depth and width scaling, and its advantage widens with depth, establishing structured cross-layer reasoning memory as a key missing mechanism for scalable multi-agent inference. 1 Introduction Inference-time computation has emerged as a complementary scaling axis to parameter scaling for large language models ( 24 ) . Among inference-time strategies, multi-LLM collaboration leverages collective reasoning by aggregating outputs from multiple agents, with paradigms ranging from majority voting ( 26 ) to multi-agent debate ( 3 ; 16 ) . A more recent paradigm, Mixture-of-Agents (MoA) ( 25 ) , organizes N proposer agents into L stacked layers followed by a final aggregator, enabling iterative refinement across depth. In principle, increasing depth and width should provide a natural path toward scalable collective reasoning. In practice, however, existing MoA systems fail to sustain gains as architectures deepen, often degrading, plateauing, or saturating after only a few layers. Existing MoA variants attempt to sustain scaling through three broad strategies. The first strategy, information consolidation , curates the inputs of each layer to reduce redundancy, exemplified by RMoA ( 28 ) , which selects a diverse subset of prior outputs and forwards only the inter-layer residuals between adjacent layers. The second, inter-agent interaction , exemplified by AttentionMoA ( 27 ) , enables agents to refine each other within a layer through natural-language attention. The third, quality-guided selection , exemplified by VeriMoA ( 19 ) , maintains a global cache of the highest-scoring solutions but requires domain-specific verifiers such as hardware description language simulators, limiting its applicability to specialized tasks. While these methods differ architecturally, none sustains performance as MoA depth increases. Three mechanisms drive this scaling collapse ( 28 ; 27 ) . First, raw inter-layer concatenation propagates both useful reasoning and accumulated errors, allowing mistakes to compound across depth. Second, architectures with only local inter-layer communication fail to preserve strong reasoning trajectories generated earlier in the hierarchy. Third, as agents consume increasingly similar accumulated context, exploration diversity collapses and agents converge toward redundant solution paths, undermining the core benefit of multi-agent reasoning ( 32 ; 15 ) . More importantly, retaining memory alone is insufficient: exposing all agents to identical accumulated reasoning can itself accelerate convergence toward repetitive trajectories. Sustained MoA scaling therefore requires simultaneously preserving high-quality reasoning across layers while maintaining diversity among agents within each layer. To address these scaling failures, we propose ReM-MoA , a Mixture-of-Agents framework centered on a Ranked Reasoning Memory that stores reasoning traces produced across the layers. Unlike episodic or retrieval-based memory systems that accumulate experience across tasks, ReM-MoA maintains a structured cross-layer reasoning memory within a single problem instance, enabling deeper layers to revisit prior reasoning paths. At each layer, proposer agents generate reasoning traces, and a Reviewer Agent ( 33 ) comparatively evaluates these traces by assigning scores together with targeted rationales. The ranked traces and rationale are then stored in memory ( 18 ) . Subsequent layers consult this memory through a Curated Diversified Memory Routing scheme that exposes different agents to distinct combinations of high-quality and low-quality reasoning trajectories, providing contrastive reasoning signals while preserving exploration diversity. ReM-MoA therefore treats MoA scaling as a joint quality-diversity preservation problem rather than solely an aggregation or interaction problem. As an optional enhancement, the Reviewer Agent can be further improved through frontier-model distillation ( 9 ) , where a strong teacher model supervises a smaller reviewer fine-tuned using parameter-efficient LoRA ( 10 ) . Multi-domain training produces a general-purpose Reviewer that transfers across reasoning tasks. Our contributions are as follows: • We empirically characterize three scaling failure modes in existing MoA systems: degradation under raw concatenation, early plateauing under local residual propagation, and diversity collapse under intra-layer interaction mechanisms. These observations motivate structured cross-layer reasoning memory as a critical mechanism for sustained MoA scaling. • We propose ReM-MoA , which sustains MoA scaling through a Ranked Reasoning Memory and a Curated Diversified Memory Routing scheme. At each layer, a comparative Reviewer Agent ranks reasoning traces with rationales, and subsequent agents receive distinct combinations of successful and failed traces, jointly preserving quality and diversity. An optional frontier-model distillation pipeline further upgrades the Reviewer Agent through multi-domain training. • Across five reasoning benchmarks (math, formal logic, code, knowledge, and commonsense), ReM-MoA consistently outperforms prior MoA variants across all tested depth and width configurations, with the performance gap widening rather than diminishing as the architecture scales. 2 Related Work Mixture-of-Agents variants. Building on the original MoA framework ( 25 ) , several variants extend the three paradigms from Section 1 . Within information consolidation , beyond RMoA ( 28 ) , SMoA ( 14 ) sparsifies agent interactions via dedicated judge and moderator agents, while Self-MoA ( 15 ) instead argues quality dominates diversity, aggregating samples from a single strong model. Within inter-agent interaction , beyond AttentionMoA ( 27 ) , DEI ( 32 ) ensembles specialist agents with complementary expertise. The quality-guided paradigm is exemplified by VeriMoA ( 19 ) and related agentic frameworks for hardware description language generation ( 20 ; 21 ; 22 ) , which rely on domain-specific feedback such as hardware simulators. No prior MoA variant maintains a structured cross-layer memory of reasoning traces accessible to subsequent layers. Memory in LLM agents. Explicit memory has been used for LLM agent self-improvement: Reflexion ( 23 ) stores verbal self-reflection from past failures, ReasoningBank ( 18 ) distills reasoning strategies from successful and failed trajectories across tasks, and AgentAuditor ( 17 ) augments LLM evaluators with retrieval-augmented examples for safety judgment. Our Ranked Reasoning Memory instead operates within a single problem across MoA depth, holding ranked traces from each layer for subsequent layers to consult. LLM-as-Judge and judge distillation. LLM-as-Judge ( 33 ) has emerged as a scalable proxy for human preference, with pairwise comparison generally more reliable than independent scoring ( 30 ) . JudgeLM ( 34 ) distills frontier-model judgments into open-source judges via knowledge distillation ( 9 ) and parameter-efficient fine-tuning ( 10 ) . Our Reviewer Agent extends this with a layer-wise comparative ranking that feeds a downstream reasoning memory. Figure 1: Scaling behavior of three existing MoA variants on three benchmarks (MATH, MMLU-redux, Formal Logic) along two scaling axes. All variants share the same heterogeneous proposer pool: Qwen-2.5-7B, Qwen-2.5-Coder-7B, and Llama-3.1-8B. Top row : layer depth scaling ( L from

Read the complete text