PolitNuggets: Benchmarking Agentic Discovery of Long-Tail Political Facts

Yifei Zhu · 2026

Read PolitNuggets: Benchmarking Agentic Discovery of Long-Tail Political Facts by Yifei Zhu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-14002.txt.

Introduction

Zhu offers PolitNuggets, a multilingual test for agent-led hunt of rare political facts from far-flung sources. It builds life tales for 400 world elites, with more than ten thousand facts. Scoring runs through FactNet, a proof-tied rule set that rates finds, fine detail and speed. Trials show today's systems trip on fine detail and swing widely in speed.

Catalog record

CREATORYifei Zhu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE81380
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.14002

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

PolitNuggets: Benchmarking Agentic Discovery of Long-Tail Political Facts Yifei Zhu Abstract Large Reasoning Models (LRMs) embedded in agentic frameworks have transformed information retrieval from static, long-context question answering into open-ended exploration. Yet real-world use requires models to discover and synthesize “long-tail” facts from dispersed sources, a capability that remains under-evaluated. We introduce PolitNuggets , a multilingual benchmark for agentic information synthesis via constructing political biographies for 400 global elites, covering over 10,000 political facts. We standardize evaluation with an optimized Supervisor–Searcher multi-agent system and propose FactNet , an evidence-conditional protocol that scores discovery, fine-grained accuracy, and efficiency. Across models and settings, we find that current systems often struggle with fine-grained details, and vary substantially in efficiency. Finally, using benchmark diagnostics, we relate agent performance to underlying model capabilities, highlighting the importance of short-context extraction, multilingual robustness, and reliable tool use. 1 Introduction Reasoning and synthesizing information within a given context is the defining capability of modern Large Reasoning Models (LRMs). The key framework can be called Reasoning in Context , where a model is passively provided a finite set of evidence and must extract or synthesize answers from it ( 6 ; 5 ) . The rapid growth of context windows has enabled strong performance on long-document tasks ( 16 ; 1 ; 25 ; 2 ; 18 ; 22 ; 24 ) . However, a new paradigm is emerging. By integrating LRMs into agentic frameworks equipped with retrieval tools, models can now actively explore, filter, and construct their own context from open-ended sources like the webpages and codebases ( 9 ; 15 ; 26 ) . This unlocks a different layer of complexity: Reasoning through Context . Unlike the passive in-context setting, here the agent must navigate a potentially unbounded information space, making sequential decisions on what to read, when to stop, and how to synthesize fragmented evidence into a coherent whole ( 19 ) . Figure 1: Agent performance heatmap on an example biography (Erik Solheim), illustrating the “head” vs. “long-tail” synthesis gap. While production systems like OpenAI Deep Research ( 11 ) and Perplexity Deep Research ( 14 ) demonstrate the promise of this agentic paradigm, there remains a lack of rigorous benchmarks for Reasoning through Context under longitudinal synthesis demands. Many existing agentic evaluations emphasize short-horizon interactions or isolated fact retrieval ( 23 ; 8 ; 19 ) , and therefore under-measure the professional workflow of reconstructing a coherent narrative from scattered, disconnected, and sometimes contradictory sources. Further, few have linked the performance of a model’s reasoning through context ability with the performance of a model’s reasoning in context. To bridge this gap, we introduce PolitNuggets, a benchmark grounded in a high-impact and realistic task: the construction of political biographies. Wikipedia, while a triumph of collaborative human curation, exhibits systematic coverage gaps—particularly for non-US officials—and often lacks the fine-grained precision required for professional domains like, academic research or political consulting. PolitNuggets tests models’ reasoning-through-context abilities by discovering the long-tail biography “nuggets” from the open web. This evaluation demands long-horizon reasoning, multi-language understanding, and reliable tool use. Our benchmark also characterizes a static corpus to evaluate models’ reasoning-in-context ability. Our evaluation of models within an agentic framework reveals that, although agents maintain high precision, they consistently struggle with recall in open-ended settings. We also observe a substantial performance degradation for Non-US entities (up to \sim 40\% relative drop in F1 in some settings), highlighting a pronounced International Evidence Gap and demonstrating that multilingual robustness is a prerequisite for realistic use. We also connect the reasoning through context ability with the reasoning in context ability. Interestingly, the evaluation results reveal a Long-Context Paradox : strong long-context reading (Reasoning in Context) does not reliably predict end-to-end agent performance (Reasoning through Context); rather, success is driven by short-context reading precision, reliable tool use, and multi-language understanding. 1.1 Traversing a latent fact network We conceptualize political biography reconstruction not as single-shot retrieval, but as traversing a latent fact network. Let a target biography induce a directed graph G=(V,E) , where nodes V are atomic “political nuggets” (e.g., Minister of Defense, 2012–2015 ) and edges E are latent temporal/causal links expressed in unstructured text (e.g., “After resigning in 2015, she joined the World Bank” ). The agent starts from a seed (entity name and minimal metadata) and must recover the relevant subset of V by expanding along implicit edges discovered in documents. This induces an optimization trilemma over correctness, coverage, and cost. Agents must maintain high precision (avoid unsupported events), high coverage (high recall over missing events in the long tail), and low efficiency cost (search steps/tokens). This framing explains why naive RAG is insufficient: the missing long-tail nodes may be weakly connected, requiring multi-hop query reformulation and evidence accumulation. PolitNuggets evaluates whether agents can approximate the full latent fact network while retaining the efficiency of strategic traversal. Strategic traversal jumps between salient nodes (low cost, but vulnerable to missing weakly connected phases if reasoning fails). Figure 2: Language composition of retrieved evidence across countries. Bars show the share of retrieved tokens that are English vs. non-English; the right-side labels show the number of evaluated cases per country in our benchmark. 2 Benchmark & Task The PolitNuggets benchmark evaluates agents on their ability to construct accurate, time-resolved career histories for 400 political elites sourced from global government directories. 2.1 Multilingual evidence in the wild The evidence required to reconstruct political careers is inherently multilingual. Traversing a global biography is not merely a search problem: an agent must reason through multilingual context to decide what to read next, how to reformulate queries, and when a claim is sufficiently supported. To characterize the language composition of the documents an agent must consume, we compute (for each country) the fraction of retrieved evidence tokens that are in English versus non-English, based on the full set of pages and passages collected during the agentic experiments (Figure 2 ). Our benchmark instances are drawn from the WhoGov dataset with a US and Non-US sampling design. We randomly sample 200 Non-US cabinet politicians from WhoGov (which provides names and basic metadata for over 58,000 global cabinet members from 1966 to 2023), and we also randomly sample 200 US legislators and senators. After preprocessing and filtering (e.g., ID matching), this yields the 400-entity evaluation set reflected in Figure 2 . 2.2 Evaluation Levels: Event-Level vs. Attribute-Level To disentangle an agent’s ability to find relevant evidence from its ability to extract fine-grained details , we compute F1 at two levels of granularity, following standard slot-filling terminology. 1. Event-Level F1 (Discovery): Measures whether the agent correctly identifies the existence of a biographical event. A prediction is a true positive if the Role and Organization match the ground truth and the Year (Start/End) is correct. This primarily tests discovery (did the agent find the right nugget?). 2. Attribute-Level F1 (Granularity): Measures whether the agent can fill fine-grained attributes for an event (slot filling). A prediction matches only if the event-level criteria are met and the Start Month, End Month (within a 1-month tolerance), and Exact Official Title are correct. This primarily tests reading comprehension and schema compliance (did the agent read details correctly?). The above slot structure (Role/Organization/Date/Title) applies to career and

Read the complete text