Agentic Search in the Wild: Intents and Trajectory Dynamics from 14M+ Real Search Requests DOI: XXXXXXX.XXXXXXX ISBN: 978-1-4503-XXXX-X/26/07 Conference: The 49th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 20–24, 2026; Melbourne | Naarm, Australia

Jingjie Ning, João Coelho, Yibo Kong, Yunfan Long, Bruno Martins, João Magalhães, Jamie Callan, Chenyan Xiong · 2026

Read Agentic Search in the Wild: Intents and Trajectory Dynamics from 14M+ Real Search Requests DOI: XXXXXXX.XXXXXXX ISBN: 978-1-4503-XXXX-X/26/07 Conference: The 49th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 20–24, 2026; Melbourne | Naarm, Australia by Jingjie Ning, João Coelho, Yibo Kong, Yunfan Long, Bruno Martins, João Magalhães, Jamie Callan, Chenyan Xiong (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-17617.txt.

Introduction

Jingjie Ning and co-authors analyze agentic search at large scale. LLM search agents now handle multi-step information tasks. The field lacks proof of how those sessions unfold. The team studies 14.44 million search requests. They span 3.97 million sessions from DeepResearchGym. Logs show intents and trajectory dynamics across real use.

Such scale turns anecdote into pattern. The study tracks how retrieved evidence is used, step by step. Designers can see where agents loop, stall, or succeed. That guides better agent planners and search tools. The work gives the IR community a shared fact base. Agent claims can now be checked against logs.

Catalog record

CREATORJingjie Ning, João Coelho, Yibo Kong, Yunfan Long, Bruno Martins, João Magalhães, Jamie Callan, Chenyan Xiong
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-17617

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Agentic Search in the Wild: Intents and Trajectory Dynamics from 14M+ Real Search Requests DOI: XXXXXXX.XXXXXXX ISBN: 978-1-4503-XXXX-X/26/07 Conference: The 49th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 20–24, 2026; Melbourne | Naarm, Australia Jingjie Ning, João Coelho, Yibo Kong, Yunfan Long, Bruno Martins, João Magalhães, Jamie Callan, Chenyan Xiong Abstract . LLM-powered search agents are increasingly being used for multi-step information seeking tasks, yet the IR community lacks empirical understanding of how agentic search sessions unfold and how retrieved evidence is used. This paper presents a large-scale log analysis of agentic search based on 14.44M search requests (3.97M sessions) collected from DeepResearchGym, i.e. an open-source search API accessed by external agentic clients. We sessionize the logs, assign session-level intents and step-wise query-reformulation labels using LLM-based annotation, and propose Context-driven Term Adoption Rate (CTAR) to quantify whether newly introduced query terms are traceable to previously retrieved evidence. Our analyses reveal distinctive behavioral patterns. First, over 90% of multi-turn sessions contain at most ten steps, and 89% of inter-step intervals fall under one minute. Second, behavior varies by intent. Fact-seeking sessions exhibit high repetition that increases over time, while sessions requiring reasoning sustain broader exploration. Third, agents reuse evidence across steps. On average, 54% of newly introduced query terms appear in the accumulated evidence context, with contributions from earlier steps beyond the most recent retrieval. The findings suggest that agentic search may benefit from repetition-aware early stopping, intent-adaptive retrieval budgets, and explicit cross-step context tracking. We have released the anonymized logs to support future research and reproducibility, available at a public Hugging Face repository . 1. Introduction Information retrieval is shifting from human-initiated search into agentic search 2 ; 30 ; 40 ; 51 , where LLM-powered agents plan and execute multi-step information seeking with retrieval tools. Instead of issuing a single query and consuming a ranked list, an agent iteratively reformulates queries, retrieves evidence, and updates subsequent queries based on retrieved context. While agent capabilities are increasingly demonstrated on controlled benchmarks 19 ; 29 ; 49 , the benchmark scores alone do not reveal how agents’ queries evolve across steps, or how retrieved context shapes subsequent queries. The aforementioned questions matter for practical system design. Agents may waste retrieval budget through repetitive or overly narrow reformulations, fail to explore alternative facets, or under-use evidence accumulated across steps. Understanding session structure can inform query-policy control, and quantifying evidence reuse can guide budget allocation and evaluation design. As agents consume results programmatically, leaving no observable trace of what they found useful, the logs lack implicit feedback signals (e.g., direct clicks) that anchor traditional behavioral inference. This creates a measurement gap. We can observe sequences of queries and retrieved evidence, but it remains unclear how sessions unfold, how behavior differs by intent, what reformulation moves dominate, and whether agents incorporate evidence from earlier steps. Figure 1 . Intent–trajectory structure of agentic search logs. To address this gap, we analyze agentic search at two complementary levels: what the agent is trying to accomplish in a session (session-level intent), and how the agent pursues that goal through step-wise search actions (trajectory-level query reformulation). Figure 1 illustrates this structure on a short example session. To operationalize these levels, we develop a measurement framework with three components: (1) LLM-based annotation pipelines that assign interpretable intent and trajectory labels to sessions and step-pairs following standard taxonomies; (2) offline replay of logged queries to reconstruct the retrieved evidence available at each step; and (3) Context-driven Term Adoption Rate (CTAR), a metric we introduce to quantify whether newly introduced query terms can be lexically traced to retrieved evidence, including contributions from steps beyond the most recent retrieval. We apply this framework to logs collected from DeepResearchGym (DRGym) 7 , i.e. a research-oriented reproducible search API accessed by external agentic clients. With permission from the DRGym organizers, we study 14.44M logged search requests spanning six months, which we sessionize into 3.97M sessions. This provides an at-scale view of autonomous agents operating in the wild under a shared retrieval backend. Our results answer the practical questions raised above about budget waste, reformulation behavior, and evidence reuse in multi-step agentic search. We find that over 90% of multi-turn sessions contain at most ten steps, and 89% of inter-step intervals fall under one minute. Retrieval depth (i.e. the number of documents requested per query) is largely static, suggesting agents treat it as a fixed parameter rather than adapting within sessions. Behavior also varies by intent. Fact-seeking sessions exhibit the highest repetition, which increases over time, indicating that agents can enter near-duplicate loops when retrieval is unproductive, while sessions requiring reasoning sustain broader exploration throughout. We also find that agents frequently incorporate terms from previously retrieved evidence, with measurable contributions from earlier steps beyond the most recent retrieval. Our contributions can be summarized as follows: • We provide a large-scale behavioral characterization of agentic search from a reproducible search infrastructure (14.44M search requests, 3.97M sessions), offering an at-scale view of autonomous agents operating in the wild. • We introduce CTAR, a metric for quantifying evidence-conditioned query evolution, and use it to demonstrate cross-step evidence reuse beyond the most recent retrieval. • We provide practical design takeaways from the logs, including repetition-aware stopping, intent-adaptive retrieval budgeting, and cross-step context tracking. We have released the anonymized logs to support future research and reproducibility, available at this public Hugging Face repository . 2. Related Work Human Search Behavior and Log Analysis: Large-scale query logs have long been used to study search behavior in the wild 8 ; 16 ; 41 , offering scalable, behavior-grounded signals for characterizing session dynamics and query reformulation beyond what offline benchmarks capture. A core theme is within-session learning. Eickhoff et al. 10 trace how newly introduced terms relate to evidence observed before reformulation (e.g., SERP snippets and visited pages), alongside complementary work on interpreting implicit feedback such as clicks and dwell time 1 ; 12 ; 20 . Exploratory search and navigation studies further document differences in branching and interaction patterns across users and information needs 28 ; 43 ; 48 , while sessionization analyses examine how sessions begin and end in practice 21 . We adopt this evidence-traceability perspective for autonomous agents and operationalize it using retrieved evidence text, enabling systematic comparisons of agent behaviors across intents and guiding design choices such as retrieval budgeting and cross-step context management. While a small line of work compares humans and agents directly 46 ; 47 ; 56 , these comparisons are often task-specific or simulation-based, motivating complementary large-scale log analyses of how autonomous agents search across sessions in the wild. LLM Interaction Platforms and Usage Logs: Recent efforts analyze large-scale interaction data from LLM systems and evaluation platforms. Chatbot Arena (LMSYS LLM Arena) aggregates pairwise preference votes 6 , and LMSYS-Chat-1M releases one million multi-model conversations collected in the wild 54 . OpenAI reports how people use ChatGPT at scale 33 , and Anthropic presents privacy-preserving analyses of millions of Claude conversations to characterize economic task usage 14 . SciArena extends the Arena protocol to scientific literature-grounded tasks and provides a corresponding benchmark 53 . These works capture usage and preference signals, but typically do not expose tool-level retrieval

Read the complete text