HyperEyes: Dual-Grained Efficiency-Aware Reinforcement Learning for Parallel Multimodal Search Agents
Guankai Li, Jiabin Chen, Xichen Zhang, Yuan Lu · 2026
Read HyperEyes: Dual-Grained Efficiency-Aware Reinforcement Learning for Parallel Multimodal Search Agents by Guankai Li, Jiabin Chen, Xichen Zhang, Yuan Lu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-07177.txt.
Introduction
Guankai Li and four co-authors make multimodal search agents work in parallel instead of one entity at a time. Sequential searching wastes rounds when a query splits into independent parts. HyperEyes fuses visual grounding and retrieval into a single action, letting one round search several entities at once, with efficiency treated as a training goal.
Training combines curated parallel examples with a two-level reinforcement method that penalizes needless tool calls and corrects failed attempts in detail. A new benchmark of three hundred cases grades both accuracy and cost together. Across six benchmarks, HyperEyes beat the strongest comparable open agent on accuracy while using far fewer tool-call rounds.
Catalog record
| CREATOR | Guankai Li, Jiabin Chen, Xichen Zhang, Yuan Lu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 94 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.07177
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
HyperEyes: Dual-Grained Efficiency-Aware Reinforcement Learning for Parallel Multimodal Search Agents Guankai Li, Jiabin Chen, Xichen Zhang, Yuan Lu Abstract Existing multimodal search agents process target entities sequentially, issuing one tool call per entity and accumulating redundant interaction rounds whenever a query naturally decomposes into independent sub-retrievals. For such decomposable queries, we argue that effective multimodal agents should search wider rather than longer : dispatching multiple grounded queries concurrently within a round, rather than sequentially. To this end, we present HyperEyes, a parallel multimodal search agent that fuses visual grounding and retrieval into a single atomic action, enabling concurrent search across multiple entities while treating inference efficiency as a first-class training objective. HyperEyes is trained in two stages: for cold-start supervision, we develop a Parallel-Amenable Data Synthesis Pipeline covering visual multi-entity and textual multi constraint queries, and curate efficiency-oriented trajectories via Progressive Rejection Sampling. Building on this foundation, our central contribution, a Dual-Grained Efficiency-Aware Reinforcement Learning framework, operates at two complementary levels. At the macro level, we propose TRACE (Tool-use Reference-Adaptive Cost Efficiency), a trajectory-level reward whose reference is monotonically tightened during training to suppress superfluous tool calls without over-restricting genuine multi-hop search. At the micro level, we adapt On-Policy Distillation (OPD) to the multimodal agentic search setting, injecting dense token-level corrective signals from an external teacher on failed rollouts to mitigate the credit-assignment deficiency of sparse outcome rewards. Since most existing multimodal search benchmarks evaluate accuracy as the sole metric, omitting inference cost and parallel-search capability, we further introduce IMEB, a human-curated benchmark that jointly evaluates multimodal search capability and efficiency, comprising 300 multi-entity visual instances. Across six benchmarks, HyperEyes-30B surpasses the strongest open-source multimodal search agent of comparable scale by 9.9% in accuracy with 5.3 × \times fewer tool-call rounds on average. Code & Data are publicly available at https://github.com/DeepExperience/HyperEyes . 0 0 footnotetext: † Equal Contributor. ‡ Project Lead. ∗ Corresponding author: Yuan Lu ( [email protected] ) 1 Introduction Figure 1 : Comparison between conventional multimodal search agents and HyperEyes. While conventional agents suffer from redundant interaction rounds to process multiple entities, HyperEyes achieves high efficiency by grounding and searching multiple entities concurrently in a single turn. The parametric knowledge of Large Language Models (LLMs) [ 4 , 24 , 1 ] and Multimodal Large Language Models (MLLMs) [ 27 , 12 ] is structurally constrained by their training data cutoff. This limitation drives the development of search agents [ 40 , 14 ] , which actively invoke external retrieval tools to ground responses in real-time, verifiable information. However, the prevailing paradigm of multimodal search agents relies heavily on sequential tool invocations to deepen the reasoning chain [ 10 , 38 , 8 , 5 ] . While effective for multi-hop reasoning tasks, this sequential approach incurs severe interaction redundancy when queries can be decomposed into independent sub-retrievals. Although parallel tool invocation has emerged in text-based agents [ 43 , 18 , 15 ] and recent visual models [ 11 ] to address this bottleneck, possessing parallel capability does not guarantee efficient search behavior. As existing models [ 8 , 10 , 38 ] are optimized primarily through pure accuracy rewards, they lack the incentive to prefer a compact parallel trajectory over a verbose one. Consequently, without explicit efficiency objectives, parallel capability often degrades into brute-force over-searching, forcing models to undergo numerous unnecessary interaction rounds to recover accuracy. To overcome this fundamental inefficiency, we propose HyperEyes , a parallel multimodal search agent designed around the principle of “search wider, not longer.” As illustrated in Figure 1 , whereas conventional agents suffer from redundant interaction rounds to process multiple entities, HyperEyes achieves high efficiency by grounding and searching multiple entities concurrently in a single turn. It operates on a Unified Grounded Search (UGS) action space that fuses visual grounding and retrieval into a single atomic action, extending text-level parallelism to the visual modality. To ensure the learned policy is parallel and strictly non-redundant, we pair this architecture with a Dual-Grained Efficiency-Aware reinforcement learning (RL) framework that treats efficiency as a primary optimization objective. At the macro level, it features TRACE, a trajectory-level reference that dynamically tightens during training to guide the policy toward optimal efficiency. At the micro level, it introduces On-Policy Distillation (OPD) [ 9 ] , which resolves ambiguous credit assignment by providing dense per-token supervision from an expert teacher on failed rollouts. Furthermore, we support this training paradigm with a Parallel-Amenable Data Synthesis Pipeline, which utilizes Progressive Rejection Sampling to curate high-quality, efficiency-oriented cold-start trajectories. Standard evaluations [ 13 , 31 , 7 , 8 ] , however, primarily assess final answer accuracy, masking the inefficiencies of verbose search trajectories. To quantify the efficiency gains achieved by parallel search, we introduce the Image Multi-Entity Benchmark ( IMEB ), a human-curated dataset that pioneers the joint evaluation of multimodal search agents on both accuracy and search efficiency. Each instance features a multi-entity image paired with a question that strictly requires concurrent localization and retrieval across multiple entities. Under this comprehensive evaluation, we demonstrate that parallel search breadth acts as the primary bottleneck in multi-entity visual search. In summary, our main contributions are as follows: • Parallel multimodal search agent. We propose HyperEyes, an efficient agent operating on a Unified Grounded Search action space. We optimize it via a Parallel-Amenable Data Synthesis pipeline and a Dual-Grained Efficiency-Aware RL framework, combining dynamic trajectory-level efficiency constraints with token-level On-Policy Distillation. • Efficiency-aware benchmark. We introduce IMEB, the first human-curated benchmark to jointly evaluate answer accuracy and search efficiency, establishing operational efficiency as a first-class metric in multi-entity visual scenarios. • Strong empirical performance. Across six benchmarks, HyperEyes-30B establishes state-of-the-art results. It Pareto-dominates existing models, surpassing the strongest open-source agent by 9.9% in accuracy while requiring 5.3 \times fewer tool-call rounds on average. 2 Related Work 2.1 Text-based Search Agents To overcome the inherent limitations of static, single-hop Retrieval-Augmented Generation (RAG) [ 17 ] in resolving complex, multi-hop queries, information seeking has fundamentally shifted toward Agentic Deep Research. While early frameworks bridged this gap via iterative prompting (e.g., ReAct [ 40 ] , Self-Ask [ 26 ] ) and supervised fine-tuning, the current frontier focuses intensely on long-horizon search and robust multi-turn tool calling to tackle sophisticated, open-domain challenges. Search-R1 [ 14 ] treats web navigation as a sequential decision-making process optimized via Reinforcement Learning (RL). Advanced frameworks such as DeepDive [ 19 ] and MiroThinker [ 33 ] push the boundaries of complex multi-step planning, enabling models to track dynamic states, execute iterative tool invocations, and maintain goal consistency over extended reasoning cycles. Concurrently, the rise of Test-Time Scaling (TTS) [ 41 ] has catalyzed deep investigations into the optimal allocation of inference computation. Recent paradigms exploring mechanisms like "wide search" and the "search more, think less" strategy systematically evaluate the trade-offs between expansive external knowledge gathering and deep internal reasoning, demonstrating that scaling exploratory search steps can effectively alleviate the cognitive burden on the LLM’s reasoning engine. However, despite these algorithmic leaps in long-horizon planning, pure text search agents remain fundamentally bottlenecked by their unimodal nature. When navigating the real-world web, they inevitably suffer from critical semantic loss upon encountering visually rich evidence—such as data charts, spatial UI layouts,