LineRetriever: Planning-Aware Observation Reduction for Web Agents
Imene Kerboua, Sahar Omidi Shayegan, Megh Thakkar, Xing Han Lù, Massimo Caccia, Véronique Eglin, Alexandre Aussem, Jérémy Espinas, Alexandre Lacoste · 2025
Read LineRetriever: Planning-Aware Observation Reduction for Web Agents by Imene Kerboua, Sahar Omidi Shayegan, Megh Thakkar, Xing Han Lù, Massimo Caccia, Véronique Eglin, Alexandre Aussem, Jérémy Espinas, Alexandre Lacoste (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-00210.txt.
Introduction
Imene Kerboua and eight co-authors present LineRetriever, a tool that trims web pages down to what a software agent needs, in a 2025 paper. Agents that browse the web for users are fed the full page code, which is often too long for the model to hold.
Simple fixes cut lines from the end or pick lines that look similar to the task, and both can drop facts the agent needs later. LineRetriever instead uses a language model to pick the lines most useful for the agent's next steps. Tests show the page view shrinks a great deal at each step while task success stays steady. The work targets agents that plan ahead, not just react.
Catalog record
| CREATOR | Imene Kerboua, Sahar Omidi Shayegan, Megh Thakkar, Xing Han Lù, Massimo Caccia, Véronique Eglin, Alexandre Aussem, Jérémy Espinas, Alexandre Lacoste |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 23 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-00210
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LineRetriever: Planning-Aware Observation Reduction for Web Agents Imene Kerboua, Sahar Omidi Shayegan, Megh Thakkar, Xing Han Lù, Massimo Caccia, Véronique Eglin, Alexandre Aussem, Jérémy Espinas, Alexandre Lacoste Abstract While large language models have demonstrated impressive capabilities in web navigation tasks, the extensive context of web pages, often represented as DOM or Accessibility Tree (AxTree) structures, frequently exceeds model context limits. Current approaches like bottom-up truncation or embedding-based retrieval lose critical information about page state and action history. This is particularly problematic for adaptive planning in web agents, where understanding the current state is essential for determining future actions. We hypothesize that embedding models lack sufficient capacity to capture plan-relevant information, especially when retrieving content that supports future action prediction. This raises a fundamental question: how can retrieval methods be optimized for adaptive planning in web navigation tasks? In response, we introduce LineRetriever , a novel approach that leverages a language model to identify and retrieve observation lines most relevant to future navigation steps. Unlike traditional retrieval methods that focus solely on semantic similarity, LineRetriever explicitly considers the planning horizon, prioritizing elements that contribute to action prediction. Our experiments demonstrate that LineRetriever can reduce the size of the observation at each step for the web agent while maintaining consistent performance within the context limitations. 1 Introduction Web agents powered by large language models (LLMs) face a critical challenge when processing modern websites: Web pages representations are often very long and exceed the context window limitations of even advanced LLMs. This constraint undermines web agents’ effectiveness when crucial navigational information becomes unavailable during decision-making. Information retrieval is an established field in Natural Language Processing and has become increasingly important in the context of LLMs due to their limited context windows. In tasks involving long or complex observations retrieval mechanisms help reduce input length while preserving task-relevant information. This focused context allows LLMs to reason more effectively, reducing errors caused by irrelevant or noisy inputs and enabling the model to concentrate on the most salient elements for decision-making. For example, retrieval-augmented generation (RAG) has been shown to improve factual accuracy by injecting relevant documents parts into the generation process Lewis et al. 2021 . As such, retrieval not only supports scalability but also enhances the accuracy and efficiency of LLM-driven agents. In the web agents domain, prior research has employed retrieval mechanisms as a strategy for context reduction in observations. For example, Deng et al. 2023 uses a reranking embedding models, that given chunks of the DOM, ranks them from top-relevant to less relevant according to the current state fo the page and the task goal. Lù et al. 2024 uses a similar approach, only this time replacing the reranker which is computation-heavy with a lighter approach, a retrieval embedding model, this model is trained and expected to return the top-k relevant chunks from the DOM. However, these approaches present limitations in a zero-shot setting. Relying on semantic similarity solely does not always provide all information needed by a working agent, because the observation stream in such environments not only contains information about the current state but also encapsulates the effects of previous actions on the interface, which helps define future actions. Alternatively, some researchers have implemented a simple approach that truncates observations from the bottom to accommodate context-length constraints Drouin et al. 2024 ; Zhou et al. 2023 . Despite adequate performance on established benchmarks, there is no empirical evidence establishing a causal link between information loss from truncation and subsequent task failures. In this work, we present LineRetriever , a simple method that utilizes a smaller language model to select and extract observation lines with the highest relevance to subsequent navigation decisions. In contrast to conventional retrieval approaches that emphasize semantic relevance exclusively, LineRetriever is asked to indirectly incorporate planning considerations. Additionally, we investigate whether smaller LLMs can effectively extract the most crucial information from the web page observation so that it can be used by a larger LLM serving as the action model. Specifically, we aim to use a small LLM to retrieve the subset of lines from the AxTree that are most relevant for achieving the task goal, given the current observation, the goal specification, and the history of actions taken by the agent. Our experimental results demonstrate that LineRetriever effectively minimizes observation size at each interaction step while sustaining comparable performance levels within established context boundaries. We list our contributions as follows: • We introduce a simple yet novel method that reduces the observation size, creating more efficient web agents. • We provide extensive experimental validation demonstrating LineRetriever ’s effectiveness across various web navigation tasks. Figure 1: Overview of the LineRetrieverAgent pipeline. 2 Related Work 2.1 Observation Processing in Web Agents The field of web agents has evolved rapidly in recent years, particularly with the integration of large language models (LLMs) for understanding and interacting with complex web interfaces Nakano et al. 2022 ; Zhou et al. 2023 ; Drouin et al. 2024 . In general, approaches rely 3 types of observation: (1) AxTrees Zhou et al. 2023 ; Drouin et al. 2024 ; Sodhi et al. 2024 ; Yang et al. 2024 , (2) DOM Lù et al. 2024 ; Deng et al. 2023 or (3) screenshots Liu et al. 2023 ; Furuta et al. 2023 ; Yang et al. 2023 ; Koh et al. 2024 , each having their limitations. DOM-based approaches employ retrieval Lù et al. 2024 or reranking Deng et al. 2023 embedding models on DOM chunks, enabling agents to process only the most relevant information for task completion while filtering out noisy, irrelevant content that degrades performance. In contrast, AxTree-based methods have traditionally relied less on retrieval since AxTrees are typically more concise and contain fewer technical keywords than DOM representations, allowing them to fit within model context limits Drouin et al. 2024 . However, as web applications become more complex and AxTrees grow larger, context length limitations and increasing costs due to longer pages processing are more frequent, necessitating intelligent filtering approaches. Retrieval applied on the observation requires understanding the interactive elements and their relationships to user goals, making traditional embedding approaches less effective for navigation tasks where planning considerations and goal alignment are crucial. Our approach addresses these limitations by introducing an LLM-based retriever that explicitly incorporates planning context and user goal when filtering observations, enabling more effective selection of navigation-relevant content. 2.2 Retrieval Methods for LLMs Early RAG approaches combined dense vector retrievers with generators to answer knowledge-intensive questions using retrieved evidence chunks Lewis et al. 2021 , with extensions like REALM integrating retrieval during both pretraining and downstream tasks Guu et al. 2020 . However, these embedding-based methods may not capture some context dependencies critical for planning in web navigation tasks. Recent work demonstrates that LLMs can serve as intelligent retrievers by directly scoring or ranking documents. RankGPT showed that prompting GPT-3.5/4 in pairwise format enables high-quality reranking without fine-tuning Sun et al. 2024 , leading to efficient open-source variants like RankVicuna and RankZephyr distilled from GPT outputs Pradeep et al. 2023 , and RankLLaMA adapted through contrastive fine-tuning Ma et al. 2023 . These approaches better capture nuanced semantic relevance than traditional embedding-based retrievers, particularly valuable for ranking