On the Regulatory Potential of User Interfaces for AI Agent Governance
K. J. Kevin Feng, Tae Soo Kim, Rock Yuren Pang, Faria Huq, Tal August, Amy X. Zhang · 2025
Read On the Regulatory Potential of User Interfaces for AI Agent Governance by K. J. Kevin Feng, Tae Soo Kim, Rock Yuren Pang, Faria Huq, Tal August, Amy X. Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-00742.txt.
Introduction
K Kevin Feng, Tae Soo Kim, Rock Yuren Pang, Faria Huq, Tal August, and Amy Zhang wrote a 2025 paper on governing AI agents. Their focus is the user interface. Agents that act on their own over long stretches can cause real harm if they go wrong. Most guardrail plans target the system under the hood, such as prompt filters or agent ID cards.
Feng and coauthors look instead at the interface, the screens and controls a person actually sees and touches. A well built interface could let users set limits, check actions, and step in early. Their work maps what interface design can add to agent safety.
Catalog record
| CREATOR | K. J. Kevin Feng, Tae Soo Kim, Rock Yuren Pang, Faria Huq, Tal August, Amy X. Zhang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 21 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-00742
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
On the Regulatory Potential of User Interfaces for AI Agent Governance K. J. Kevin Feng, Tae Soo Kim, Rock Yuren Pang, Faria Huq, Tal August, Amy X. Zhang Abstract AI agents that take actions in their environment autonomously over extended time horizons require robust governance interventions to curb their potentially consequential risks. Prior proposals for governing AI agents primarily target system-level safeguards (e.g., prompt injection monitors) or agent infrastructure (e.g., agent IDs). In this work, we explore a complementary approach: regulating user interfaces of AI agents as a way of enforcing transparency and behavioral requirements that then demand changes at the system and/or infrastructure levels. Specifically, we analyze 22 existing agentic systems to identify UI elements that play key roles in human-agent interaction and communication. We then synthesize those elements into six high-level interaction design patterns that hold regulatory potential (e.g., requiring agent memory to be editable). We conclude with policy recommendations based on our analysis. Our work exposes a new surface for regulatory action that supplements previous proposals for practical AI agent governance. 1 Introduction AI agents—compound AI systems that take actions in their environment on behalf of a user under limited direct supervision—are increasingly being built and deployed into the real world [ 7 , 45 , 29 , 18 , 31 , 34 , 20 , 6 ] . Modern agents have demonstrated proficiency in autonomously pursuing complex, multi-step goals in economically-valuable domains including software engineering [ 51 , 28 ] , online shopping [ 52 ] , and machine learning research [ 8 , 48 ] . Their proficiency in autonomous task completion is rapidly improving [ 36 ] , a source of both excitement and concern. Agents may unlock new levels of productivity and economic growth, but their deployment is accompanied by heightened risks [ 45 , 3 ] . These risks come from the inherent difficulty in anticipating these risks [ 6 ] , increased attack surfaces for malicious actors [ 15 , 33 , 42 ] , accelerating the gradual disempowerment of humans [ 32 ] , and a general loss of human control over AI systems [ 2 ] . Prior work in agent governance proposed ways to mitigate risks, primarily through two avenues: system-level safeguards and agent infrastructure [ 7 , 45 , 5 , 4 , 47 , 41 , 40 ] . System-level safeguards fortify the AI model or agent scaffolding; it includes techniques such as training a model to seek user confirmation for sensitive actions or proactively refuse harmful requests, and building monitors for prompt injection attacks [ 41 , 40 ] . Agent infrastructure is external to the system but still mediates and influences agents’ interactions with their environments; these include agent IDs [ 5 , 37 ] , isolated channels for agent activities [ 7 ] , and frameworks for authenticated delegation of authority [ 47 ] . While promising, these approaches operate “behind the scenes”—in model training or system architecture, limiting end users’ visibility into and control over agent behavior during actual deployment. In this work, we address this challenge by proposing a new target for regulation in user-facing agentic systems: the user interface (UI). Regulation of UIs and UI elements is a well-established practice outside of AI for ensuring consumer safety. In the US, UK, EU, and Canada, marketing emails are legally required to contain a button for one-click unsubscribe [ 16 , 22 , 23 ] . The Americans with Disabilities Act (ADA) and the EU Accessibility Act require UIs to be perceivable and operable by people with visual and motor impairments [ 1 , 14 ] . The General Data Protection Regulation (GDPR) in the EU and the California Consumer Privacy Act (CCPA) both mandate the presentation of UIs for privacy consent management and opt-out controls [ 21 , 49 ] . Because UI elements map to underlying system functionality and act as a key layer for user-system communication and control, regulating UIs of agents can jumpstart the implementation of certain system- and infrastructure-level interventions. An illustration of the conceptual differences between system-, infrastructure-, and UI-level interventions can be found in Appendix A.1 . We begin our investigation of the regulatory potential by analyzing 22 deployed agentic systems to identify UI elements in human-agent communication. We synthesize our analysis into six high-level design patterns that can serve as targets for regulation, mapping each pattern to previously proposed system- and infrastructure-level governance for agents. We conclude with policy recommendations and fruitful avenues of collaboration for the technical, policy, and design communities. 2 Method We collected and analyzed 22 agentic systems from academic papers in human-AI interaction, product releases, and open-source projects. To be considered “agentic,” the system needed to satisfy the following 4 inclusion criteria at the time of analysis: 1) Is publicly available; 2) Is an interactive software system; 3) Operates using multi-step workflows; and 4) Calls tools and/or executes actions. These criteria were inspired by previous literature (e.g., [ 6 , 3 , 18 ] ), and partially for practical reasons (e.g., we cannot analyze a system in a rigorous, reproducible way if its details are not publicly available). Full definitions for our inclusion criteria can be found in Appendix A.3 . Three authors divided up the analysis using visual thematic analysis, a common method in HCI for identifying design patterns in UIs [ 50 , 17 ] . Further details about our analysis process are in Appendix A.4 . Examples from our analysis and the full list of agentic systems are also in the Appendix. 3 Six Design Patterns with Regulatory Potential Our analysis yielded six design patterns (see visual examples in Appendix A.2 ) that can serve as levers for regulatory action. These are not exhaustive but a starting point—new patterns may emerge as more agentic systems are deployed and UIs evolve. Fortunately, strong incentives already exist for adopting these patterns due to usability benefits. However, we still consider it important for these patterns to be regulated to prevent developers from removing them out of convenience, market pressures, or A/B testing. 3.1 Visible thoughts, plans, and actions Description. The agent’s reasoning, planning, and actions are represented as a step-by-step sequence that users can trace in real time or retrospectively. This trace may appear inline with the chat, alongside the cursor, or alongside components on which the agent is acting. By surfacing both what the agent is doing and why, these interfaces make the decision-making process explicit. Regulatory promises and challenges. Revealing the agent’s reasoning and actions enhances transparency, accountability, and user oversight. It enables users to identify unsafe behavior, intervene, and calibrate trust in the system. However, these benefits depend on the faithfulness of the expressed reasoning to the agent’s actual internal processes [ 12 , 10 ] . Another challenge is managing granularity: too little detail undermines oversight, while too much risks overwhelming users and reducing usability. Connections with prior governance proposals. Regulating this design pattern advances prior calls for agent oversight. Visible thought and action traces serve as an oversight layer in the agent infrastructure, as proposed by Chan et al. [ 7 ] , and operationalizes Shavit et al.’s [ 45 ] proposed practice of legibility