Forage V2: Knowledge Evolution and Transfer in Autonomous Agent Organizations

Huaqing Xie · 2026

Read Forage V2: Knowledge Evolution and Transfer in Autonomous Agent Organizations by Huaqing Xie (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-19837.txt.

Introduction

Xie presents Forage V2, a system for autonomous agents in open-world tasks where the finish line is not given in advance. Such agents suffer denominator blindness, systematically underestimating the scope of the target space. Version 1 answered this with co-evolving evaluation, where an independent Evaluator discovers what complete means, and method isolation, where neither side sees the other's code. Version 2 grows from a single run into a learning organization, with experience accumulating across runs, transferring across model capabilities, and guards against knowledge degradation.

Catalog record

CREATORHuaqing Xie
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE116235
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.19837

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Forage V2: Knowledge Evolution and Transfer in Autonomous Agent Organizations Huaqing Xie Abstract Autonomous agents operating in open-world tasks—where the completion boundary is not given in advance—face a structural challenge we term denominator blindness : they systematically underestimate the scope of the target space, reporting high coverage against a fraction of what exists. Forage V1 addressed this for individual runs through co-evolving evaluation (an independent Evaluator discovers and refines what “complete” means) and method isolation (the Evaluator and Planner cannot see each other’s code). V2 extends the architecture from a single expedition to a learning organization : experience accumulates across runs, transfers across model capabilities, and institutional safeguards prevent the degradation of accumulated knowledge. We demonstrate two claims across three task types (web scraping, API queries, mathematical reasoning). Knowledge accumulation : over six runs, knowledge entries grow from 0 to 54 (NVIDIA task), and the Evaluator’s denominator estimates stabilize as domain understanding deepens. Knowledge transfer : a weaker agent (Sonnet) seeded with a stronger agent’s (Opus) accumulated knowledge narrows a 6.6 percentage-point coverage gap to 1.1pp, halves cost ($9.40 → \to $5.13), converges in half the rounds (mean 4.5 vs. 7.0), and three independent seeded runs arrive at exactly the same denominator estimate (266), suggesting that organizational knowledge calibrates evaluation itself. V2’s contribution is architectural, not algorithmic: rather than making one model stronger through fine-tuning or reinforcement learning, it designs institutions—audit separation, contract protocols, organizational memory—that make any agent more reliable upon entry. The accumulated experience is organizational, model-agnostic, and transferable: stored as readable documents where each run enriches the institution, and every future agent—regardless of provider or capability level—inherits that enrichment. The deeper claim is that a minimal set of structural constraints enables credible autonomous judgment in spaces where no ground truth is given. 1 Introduction 1.1 The Harness Era The development of autonomous AI agents has followed a three-stage progression. Pre-training builds foundational capabilities through large-scale learning on text corpora. Post-training —reinforcement learning from human feedback, test-time compute scaling, and related techniques—refines these capabilities toward human preferences and reasoning depth. The third stage—variously called the harness layer , scaffolding, or orchestration layer—equips agents with the external structure needed to operate in the real world: retrieval-augmented generation, persistent memory, tool use, skill composition, and workflow orchestration. The harness layer is where much of today’s applied AI engineering happens—Claude Code ( Anthropic 2025a ) , Cursor, and similar systems represent practical instantiations. Yet a critical dimension remains underserved: evaluation . Who judges whether the agent did enough? When should it stop? Current approaches fall into two categories. The first relies on human-authored evaluation criteria : a person defines what “complete” means before the agent begins, and the agent is measured against this fixed standard ( Anthropic 2025b ) . This works for benchmarks with known answers but breaks down in open-world tasks where the target space is itself unknown. In practice, the gap is filled by humans in the loop : users repeatedly correct agents that declare premature completion, redirect exploration that missed obvious sources, and manually judge when the work is “good enough.” This pattern—the human as implicit evaluator—is the default in current agent systems, and it does not scale. The second approach lets the agent evaluate itself —and this is where a systematic failure mode emerges. 1.2 V1: Denominator Blindness and Co-Evolving Evaluation In prior work ( Xie 2026 ) , we identified and named denominator blindness : the systematic tendency of autonomous agents to report high task completion against an underestimated total. An agent tasked with exploring an open-ended space—collecting all articles from a publication archive, or gathering all products in a category—may find a fraction of the target items, declare its job done, and report 100% coverage. The agent’s numerator may be accurate, but it never discovered the denominator. Forage V1 addressed this for individual runs through two mechanisms. First, co-evolving evaluation : rather than fixing evaluation criteria before execution, V1 employs two LLM agents—an Evaluator that discovers and refines the definition of “complete,” and a Planner that designs and executes strategies. Both evolve simultaneously: as the Planner discovers new information, the Evaluator revises its assessment of the total scope; as the Evaluator identifies gaps, the Planner targets them. A non-LLM Executor runs the Planner’s scripts deterministically. Second, method isolation : the Evaluator and Planner cannot see each other’s code. The Evaluator writes eval.py (how to measure completeness); the Planner writes action.py (how to execute). Neither can read the other’s script. This prevents self-serving evaluation—the structural tendency for an agent grading its own work to conflate “I tried hard” with “I succeeded.” V1 demonstrated that this architecture works across two benchmarks—government announcement archives and a historical publication collection. Without independent evaluation, a single agent self-reported 100% coverage at 15.9% actual recall; with full Forage, agents achieved 98.8% actual recall with calibrated self-assessment. The architecture transferred across two task types without code changes. 1.3 From Expedition to Organization V1 operates as a single expedition: a team ventures into unknown territory, draws a map, and returns. The map reflects the team’s best understanding—credible but not necessarily complete, shaped by the sources discovered and the definitions adopted along the way. The next team, however, starts from scratch. Hard-won discoveries about the territory—which paths lead nowhere, which sources are reliable, what the boundaries look like—are lost between expeditions. V2 extends Forage from a single expedition to a learning organization . After each run, both agents independently extract transferable lessons through a post-mortem process. These lessons accumulate in a persistent knowledge base, and subsequent runs begin with this accumulated context. The organization remembers. This is not merely a technical improvement—it reflects a philosophical shift. V1 asked: how can two agents establish credible completion judgments? V2 asks: how can an agent organization accumulate and transfer experience while maintaining the integrity guarantees that make its judgments trustworthy? The answer draws from institutional design rather than agent enhancement: method isolation as audit separation, format contracts as public agreements, and knowledge as organizational memory that newcomers consult but are not bound by (§ 6.2 ). Forage does not make individual agents stronger; it designs institutions that make ordinary agents reliable. 1.4 Contributions This report presents two claims, supported by experiments across three task types: 1. Knowledge Accumulation. The organization learns across runs. Knowledge entries grow from 0 to 54 over six runs (NVIDIA task), and the Evaluator’s denominator estimates stabilize as accumulated domain knowledge narrows the search space. 2. Knowledge Transfer. A stronger agent’s (Opus) accumulated knowledge enables a weaker agent (Sonnet) to approach comparable performance. Without knowledge, Sonnet achieves 90.7–96.5% coverage in 6–8 rounds (mean 93.1%). With Opus’s accumulated knowledge, Sonnet achieves 92.5–100% coverage in 3–7 rounds (mean 98.6%, with 5 of 6 runs at 99.7–100%) at half the cost—and its denominator estimates converge to the same range as Opus’s. Both claims are demonstrated on NVIDIA desktop GPU data collection (web scraping, 265–411 candidate products across conditions) and supported by UniProt protein collection (API queries, 28–30 candidates) and Q10 mathematical reasoning (First Proof benchmark). Cross-task knowledge transfer—whether lessons from one task domain benefit a different domain—is not claimed; this is V3 future work. (We note that some knowledge entries are obviously universal—e.g., “this source

Read the complete text