Permission Manifests for Web Agents Lightweight Agent Standards Working Group (LAS-WG)

Samuele Marro, Alan Chan, Xinxing Ren, Lewis Hammond, Jesse Wright, Gurjyot Wanga, Tiziano Piccardi, Nuno Campos, Tobin South, Jialin Yu, Sunando Sengupta, Eric Sommerlade, Alex Pentland, Philip Torr, Jiaxin Pei, [0.5em] University of Oxford Institute for Decentralized AI Centre for the Governance o · 2026

Read Permission Manifests for Web Agents Lightweight Agent Standards Working Group (LAS-WG) by Samuele Marro, Alan Chan, Xinxing Ren, Lewis Hammond, Jesse Wright, Gurjyot Wanga, Tiziano Piccardi, Nuno Campos, Tobin South, Jialin Yu, Sunando Sengupta, Eric Sommerlade, Alex Pentland, Philip Torr, Jiaxin Pei, [0.5em] University of Oxford Institute for Decentralized AI Centre for the Governance o (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-02371.txt.

Introduction

Samuele Marro, Alan Chan, and many co authors in the Lightweight Agent Standards Working Group propose a simple way for websites to tell automated agents what they may and may not do. Language model agents now navigate pages, extract structured data, and complete tasks end to end, far beyond what the old robots dot txt convention was designed for. Without a clear permission channel, sites resort to blanket blocking and CAPTCHAs.

Their answer is a manifest file, agent-permissions dot json, written in the same spirit as robots dot txt. Owners list allowed interactions in plain JSON and can point to APIs where those exist. Agents can parse the file automatically and obey it. The low effort on both sides is meant to protect useful automation, commerce, and accessibility tools.

Catalog record

CREATORSamuele Marro, Alan Chan, Xinxing Ren, Lewis Hammond, Jesse Wright, Gurjyot Wanga, Tiziano Piccardi, Nuno Campos, Tobin South, Jialin Yu, Sunando Sengupta, Eric Sommerlade, Alex Pentland, Philip Torr, Jiaxin Pei, [0.5em] University of Oxford Institute for Decentralized AI Centre for the Governance o
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-02371

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Permission Manifests for Web Agents Lightweight Agent Standards Working Group (LAS-WG) Samuele Marro, Alan Chan, Xinxing Ren, Lewis Hammond, Jesse Wright, Gurjyot Wanga, Tiziano Piccardi, Nuno Campos, Tobin South, Jialin Yu, Sunando Sengupta, Eric Sommerlade, Alex Pentland, Philip Torr, Jiaxin Pei, [0.5em] University of Oxford Institute for Decentralized AI Centre for the Governance of AI Coral Protocol Cooperative AI Foundation Webair Johns Hopkins University Witan Labs Stanford University Microsoft UT Austin Abstract The rise of Large Language Model (LLM)-based web agents represents a significant shift in automated interactions with the web. Unlike traditional crawlers that follow simple conventions, such as robots.txt , modern agents engage with websites in sophisticated ways: navigating complex interfaces, extracting structured information, and completing end-to-end tasks. Existing governance mechanisms were not designed for these capabilities. Without a way to specify what interactions are and are not allowed, website owners increasingly rely on blanket blocking and CAPTCHAs, which undermine beneficial applications such as efficient automation, convenient use of e-commerce services, and accessibility tools. We introduce agent-permissions.json , a robots.txt -style lightweight manifest where websites specify allowed interactions, complemented by API references where available. This framework provides a low-friction coordination mechanism: website owners only need to write a simple JSON file, while agents can easily parse and automatically implement the manifest’s provisions. Website owners can then focus on blocking non-compliant agents, rather than agents as a whole. By extending the spirit of robots.txt to the era of LLM-mediated interaction, and complementing data use initiatives such as AIPref, the manifest establishes a compliance framework that enables beneficial agent interactions while respecting site owners’ preferences. (a) Before and after. (b) Examples of rules. Figure 1 : agent-permissions.json is a minimalist, straightforward permission manifest for agents interacting with web pages. 1 Introduction The proliferation of Large Language Model (LLM)-based web agents has fundamentally changed how automated systems interact with web content. Unlike traditional web crawlers that primarily index content, modern LLM-powered agents perform sophisticated interactions: conducting searches to find specific information ( 29 ) , submitting forms as part of automated workflows ( 13 ; 7 ) , and executing end-to-end tasks ( 41 ; 14 ) . These capabilities enable powerful applications, such as intelligent research assistants ( 4 ) and automated accessibility tools ( 24 ) , as well as new opportunities for e-commerce ( 44 ) and recommendation systems ( 26 ) . However, even beneficial tools can overwhelm websites or violate usage policies when deployed at scale ( 8 , see, e.g.,) . Unable to distinguish between beneficial AI agents and potentially harmful automated traffic ( 12 ; 36 ) , many site operators have adopted a defensive approach, implementing blanket 403 (“Forbidden”) responses, CAPTCHA challenges, and aggressive rate limiting against any client exhibiting suspected automated behavior ( 15 ; 28 ; 22 ; 6 ) . While understandable from a security perspective, this approach creates significant barriers to legitimate AI-driven services ( 20 ) , depriving both users and websites of the value of service automation. A key bottleneck is the lack of a standardized and lightweight mechanism for website owners to express nuanced policies: which automated interactions are welcome, which are prohibited, and under what conditions different behavior are allowed. For example, a travel-booking assistant might be allowed to check seat availability or flight status, but should not confirm or pay for bookings without human approval. Similarly, an accessibility auditor might be allowed to activate dropdown menus and form fields to verify usability, but might be forbidden from submitting or modifying data. Such policies cannot currently be expressed in a standardized fashion, which means that website owners face a binary choice: either block all automated access, or implement complex, burdensome authentication systems requiring API keys, OAuth flows, and extensive integration efforts. For the majority of the web, where ambiguity (rather than malice) is a major problem, a simple coordination mechanism could help substantially. To address this problem, this paper proposes agent-permissions.json , a robots.txt -style permission manifest 1 1 1 https://github.com/las-wg/agent-permissions.json . Drawing inspiration from the simplicity and widespread adoption of the robots.txt standard, we present a lightweight framework that allows website owners to declaratively specify interaction policies for AI agents ( Figure 1(b) ). This approach requires no credential provisioning, complex authentication infrastructure, or ongoing maintenance overhead: it is merely a simple manifest file that agents can easily discover and follow. Our thesis is that a permission manifest provides a familiar, low-friction entry point for signaling permissible interactions with a website. Rather than treating all agents as a threat, we argue that the community should make a distinction: some agents are compliant , i.e., open to following website policies when performing actions on behalf of the user, while others are rule-breaking . We suspect that many agents would be compliant if given the opportunity, just as many major search crawlers comply with robots.txt . agent-permissions.json is meant to coordinate compliant agents so that they can both respect website policies and provide useful services for the user. Rule-breaking agents will remain a problem, but websites can focus their detection and blocking efforts on them while benefiting from the traffic of compliant agents. Not all websites will want to make this trade-off, especially those whose funding depends upon human attention (e.g., from ads). Our primary target is sites whose value comes from delivering information and services, such as e-commerce platforms, documentation and research repositories, and scheduling and booking interfaces. Such websites will likely find agent-permissions.json attractive. Our proposed manifest, coupled with data use standards (e.g., AIPref 2 2 2 https://datatracker.ietf.org/wg/aipref/about/ ), aims to preserve the autonomy of website owners while creating predictable interaction patterns that benefit developers, end users, and the broader ecosystem of AI-powered web services. 2 Background and Positioning To understand the need for a permission manifest system for AI agents, we must first examine the historical context of web automation governance and the limitations of current approaches. We thus provide a quick summary of existing standards and how they fit different use cases in Table 1 . Use Case Solution Permissions for crawling robots.txt Guiding agentic crawlers llms.txt Permissions for data use AIPref, ai.txt Offering structured access REST, MCP, A2A Blocking malicious agents WAF, CAPTCHA Monetizing agentic interactions Pay per Crawl, x402, AP2 Permissions for UI interactions agent-permissions.json ( Ours ) Table 1 : Existing and proposed web governance mechanisms for agents, and where agent-permissions.json fits. robots.txt. The robots.txt standard ( 25 ) , introduced in 1994, represents one of the web’s most successful examples of lightweight, declarative access control. Site owners communicate crawling preferences to automated agents through a standardized text file placed at the root of their website. The protocol’s widespread adoption stems from two key characteristics that make it an ideal model for modern web automation governance. First, robots.txt requires minimal technical infrastructure. Website operators only need to create a plain text file with straightforward syntax: no database configurations, authentication servers, or complex middleware deployments. This simplicity has made robots.txt the de-facto standard for crawl management on the Web ( 38 ) . Second, the standard places minimal burden on agents. Web crawlers have universally implemented robots.txt parsing, making compliance a solved problem rather than a recurring engineering challenge. The

Read the complete text