ATLAS: Actor-Critic Task-Completion with Look-ahead Action Simulation
Jiali Cheng, Anjishnu Kumar, Roshan Lal, Rishi Rajasekaran, Hani Ramezani, Omar Zia Khan, Oleg Rokhlenko, Sunny Chiu-Webster, Gang Hua, Hadi Amiri · 2025
Read ATLAS: Actor-Critic Task-Completion with Look-ahead Action Simulation by Jiali Cheng, Anjishnu Kumar, Roshan Lal, Rishi Rajasekaran, Hani Ramezani, Omar Zia Khan, Oleg Rokhlenko, Sunny Chiu-Webster, Gang Hua, Hadi Amiri (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-22732.txt.
Introduction
Web agents based on large language models often fail on new sites. They also take long, wasteful action paths. Jiali Cheng leads a team with nine co-authors. In 2025 they present WebATLAS, an agent that learns from past web tasks.
The agent builds a map of a site by exploring with curiosity. It stores past outcomes as memory. Before it acts, it tests options in a mental rollout. A planner, a simulator, and a critic judge each option. This helps it avoid old mistakes. On a web task test, it reached a success rate well above the prior best mark.
Catalog record
| CREATOR | Jiali Cheng, Anjishnu Kumar, Roshan Lal, Rishi Rajasekaran, Hani Ramezani, Omar Zia Khan, Oleg Rokhlenko, Sunny Chiu-Webster, Gang Hua, Hadi Amiri |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-22732
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ATLAS: Actor-Critic Task-Completion with Look-ahead Action Simulation Jiali Cheng, Anjishnu Kumar, Roshan Lal, Rishi Rajasekaran, Hani Ramezani, Omar Zia Khan, Oleg Rokhlenko, Sunny Chiu-Webster, Gang Hua, Hadi Amiri Abstract We observe that current state-of-the-art web-agents are unable to effectively adapt to new environments without neural network fine-tuning, without which they produce inefficient execution plans due to a lack of awareness of the structure and dynamics of the new environment. To address this limitation, we introduce ATLAS ( A ctor-Critic T ask-completion with L ook-ahead A ction S imulation), a memory-augmented agent that is able to make plans grounded in a model of the environment by simulating the consequences of those actions in cognitive space . Our agent starts by building a "cognitive map" by performing a lightweight curiosity driven exploration of the environment. The planner proposes candidate actions; the simulator predicts their consequences in cognitive space; a critic analyzes the options to select the best roll-out and update the original plan; and a browser executor performs the chosen action. On the WebArena-Lite Benchmark, we achieve a 63% success rate compared to 53.9% success rate for the previously published state-of-the-art. Unlike previous systems, our modular architecture requires no website-specific LLM fine-tuning. Ablations show sizable drops without the world-model, hierarchical planner, and look-ahead-based replanner confirming their complementary roles within the design of our system. \workshoptitle LAW 2025: Bridging Language, Agent, and World Models for Reasoning and Planning 1 Introduction Autonomous agents that can navigate and act on the web have the potential to perform complex tasks like information gathering, transactions, and site configuration on behalf of users ( Yao et al. 2022 ; Sodhi 2023 ) . However, current web-based agents fall far short of human-level reliability on long-horizon tasks ( Koh et al. 2024 ) . The difficulty stems from partial observability, vast action spaces, the need for multi-step planning and memory in a web environment. For example, tasks in WebArena ( Zhou et al. 2024 ) span diverse websites and require reasoning over multi-page navigation and content. An agent might be asked to “Tell me how many fulfilled orders I have over the past three days, and the total amount spent” on an e-commerce site, or “Set the homepage URL on my GitLab profile to https://egg.tart.com.” Solving these tasks demands understanding site structure (e.g. admin dashboards, forms), remembering relevant information (like login states or filters applied), and avoiding irreversible mistakes (like deleting data or purchasing an item unintentionally). Large language models (LLMs) have markedly improved semantic understanding and generation, suggesting they can enrich web navigation. Yet reliable long-horizon control remains elusive because LLM agents are typically reactive and lack structured memory and explicit planning. Current state-of-the-art web agents such as Plan-and-Act by Erdogan et al. 2025 have significant gaps - the agent’s planner module is not grounded to the structure of the website and requires website-specific model fine-tuning of both planner and executor modules in order to enable new use-cases. We address these gaps with ATLAS—an inference-time, actor–critic web agent that plans before acting via look-ahead simulation and retrieves structured memories to remain goal-directed over extended interactions. ATLAS has a modular architecture with four components: (1) a Planner that decomposes the task into subgoals; (2) an Actor that proposes diverse next-step candidates; (3) a Critic that forecasts each candidate’s outcome by simulating state transitions and selects the safest, goal-advancing action; and (4) a multi-layer memory that is updated online and queried on demand. Together, these modules perform a simulated look-ahead tree search in conceptual space, enabling adaptive environment-grounded planning, and efficient action selection in realistic, partially observable websites, including those with unexpected environmental hazards . Our agent uses a hierarchical memory structure: (i) Working Memory for recent context; (ii) a Cognitive Map encoding state transitions and expected action outcomes; and (iii) Semantic Memory that captures environment-specific constraints (e.g. formats, hazards) in the form of world knowledge. The map is constructed via curiosity-driven exploration (MCTS-style trajectory mining) and agentic summarization that records action-to-outcome deltas in natural language, avoiding HTML bloat. During inference, the Actor conditions on the plan, retrieved memories, and in-context trajectories; the Critic filters out risky or myopic moves by simulating their consequences. We summarize our contributions as the following: • An actor-critic planner with LLM-based look-ahead that evaluates actions via simulated outcomes. • A multi-layer memory with a cognitive map built through exploration and agentic summarization, used online for retrieval and replanning. • A practical modular architecture that integrates planning , memory , and simulation to transform high-level instructions into safe, executable action sequences for long-horizon web tasks. Unlike previous systems, our system does not require website-specific LLM fine-tuning to ground to new websites and can thus easily be ported to new websites and new underlying LLMs. 2 Related Work LLM-Based Autonomous Agents The integration of large language models into autonomous agents has revolutionized web navigation capabilities. Yao et al. 2023 pioneered the ReAct framework, demonstrating how LLMs can effectively combine reasoning and acting in interactive environments. This approach has been extended by Shinn et al. 2024 with Reflexion, which incorporates self-reflection mechanisms for improved decision-making over extended horizons. Web Navigation Agents Early web navigation systems relied predominantly on rule-based approaches and predefined scripts Liu et al. 2018 , which, while interpretable, lacked the adaptability required for dynamic web environments. Recent advances have shifted toward learning-based methods, with several notable developments in autonomous web agents. Zhou et al. 2024 introduced WebArena, a comprehensive benchmark for evaluating web agents across realistic multi-step tasks, establishing a foundation for systematic evaluation in this domain. Building on this, Liu et al. 2024a introduced WebArena-Lite, a curated subset addressing quality and scalability concerns in the original benchmark. Recent work has focused on enhancing LLM agents with structured approaches to planning and memory and demonstrated significant progress: Qi et al. 2024 applied reinforcement learning principles to web navigation, achieving notable improvements through policy optimization. Zhang et al. 2025 developed WebPilot, focusing on multimodal understanding of web content, while Erdogan et al. 2025 introduced Plan-and-Act, emphasizing hierarchical task decomposition. Wang et al. 2024b introduced Agent Workflow Memory (AWM), demonstrating the importance of persistent memory in multi-step web tasks. Yang et al. 2024 developed AgentOccam and demonstrated the effectiveness of simplifying the action space to natural language. Memory-Augmented Agents Memory mechanisms have emerged as crucial components for long-horizon task completion. Weston et al. 2014 established foundational work on memory networks, which has been adapted for sequential decision-making contexts. More recently, Zhong et al. 2023 proposed MemoryBank, a comprehensive framework for managing episodic and semantic memory in LLM-based agents. The concept of cognitive maps, originally from cognitive science Tolman 1948 , has been adapted for artificial agents. Wayne et al. 2018 demonstrated neural implementations of cognitive mapping in reinforcement learning contexts, while Park et al. 2023 showed how LLMs can maintain and utilize spatial-temporal memory representations for complex behavioral simulation. Planning and Simulation in AI Agents Tree search and simulation-based planning have long been central to AI agent design. While classical approaches like Monte Carlo Tree Search Browne et al. 2012 have proven effective in discrete domains, recent work has extended these concepts to natural language environments. Yao et al. 2024 introduced Tree of Thoughts, enabling LLMs