Terminal Is All You Need: Design Properties for Human-AI Agent Collaboration
Alexandre De Masi · 2026
Read Terminal Is All You Need: Design Properties for Human-AI Agent Collaboration by Alexandre De Masi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-10664.txt.
Introduction
Alexandre De Masi argues that the terminal, not the graphical interface, is the design model for human-AI agent collaboration. Agent research chases GUI control, yet the tools agents use most, and use best, are terminal-based. He traces that to three properties: a shared representation between agent and interface, actions that stay transparent in the medium, and a low barrier for humans to join. Each is grounded in HCI theory, and terminals meet all three by default. Other modalities must be engineered to match them.
Catalog record
| CREATOR | Alexandre De Masi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 17386 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.10664
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Terminal Is All You Need: Design Properties for Human-AI Agent Collaboration Alexandre De Masi Abstract . While research on AI agents focuses on enabling them to operate graphical user interfaces, the most effective and widely adopted agent tools in practice are terminal-based. We argue that this convergence is not coincidental. It reflects three design properties central to effective human-AI-UI collaboration: representational compatibility between agent and interface, transparency of agent actions within the interaction medium, and low barriers to entry for human participants. We ground each property in established HCI theory, show how terminal-based tools satisfy them by default, and argue that any modality, including graphical and spatial interfaces, must be deliberately engineered to achieve them. Rather than a legacy artifact, the terminal serves as a design exemplar whose properties any agent-facing modality must replicate. Accepted as poster at the CHI 2026 Workshop on Human-AI-UI Interactions Across Modalities (CUCHI’26), April 14, 2026, Barcelona, Spain. $ Fix the login bug Agent: Reading auth.py... Agent: Found missing None check on line 42. Agent: Proposed fix: --- app/auth.py + if user is None: + return None Apply? [y/n] y ✓ File saved. Running tests... ✓ All 14 tests passed. Figure 1. Terminal-based agent interaction: natural language request, code diff, approval gate, and test execution in one text stream. 1. Introduction AI agents that perceive, interpret, and operate user interfaces must serve a dual audience: the interface must remain legible and controllable by the human while being interpretable and operable by the agent. Yet current approaches to this dual-audience problem diverge sharply in their effectiveness. GUI agents attempt to bridge this gap by perceiving screenshots or accessibility trees and generating clicks and keystrokes ( 26 ; 7 ) . The Agent-Computer Interface (ACI) framework treats this as a dedicated design problem, distinct from human-facing interface design ( 28 ) . Yet the approach faces persistent difficulties: the best model achieves 12.24% task success on OSWorld versus 72.36% for humans ( 26 ) , and frontier web agents show up to 59% lower success rates on realistic tasks than on simpler benchmarks ( 27 ) . A different pattern is emerging among multi-step agentic tools, which autonomously read files, plan changes, execute commands, and run tests. Unlike single-action tools like GitHub Copilot that suggest inline completions within graphical editors, agentic tools are converging on text-based sequential interaction (Figure 1 ). By late 2024, AI-generated code accounted for approximately 30% of new Python functions on GitHub ( 6 ) . A study of over 129,000 GitHub projects found coding agent adoption rates between 15% and 23%, with rapid growth since early 2025 and traces of over 50 distinct agentic tools ( 19 ) . Claude Code, OpenAI Codex, and Cursor’s agent mode all center on a text-based sequential pattern, even though these tools are built by competing organizations. Cursor itself is a graphical IDE, yet its agentic capabilities operate through a terminal-like text channel embedded within the editor. Single-action code completion tools face their own adoption challenges: experienced open-source developers using AI assistants were measured as 19% slower than without AI, despite perceiving themselves as faster ( 2 ) . This perception-reality gap directly motivates the transparency property we identify below. We do not take this as evidence that command-line interfaces (CLIs) are inherently superior. We take it as a signal that demands analysis. We identify three design properties that account for this convergence, ground them in established HCI theory, and argue that they constitute core design requirements for effective human-AI-UI collaboration. Modalities that fail to address them risk encountering the same difficulties that GUI agents face today. 2. Background: The Dual-Audience Interface Human Agent UI P2: Transparency P1: Representational Compatibility P3: Low Barriers Figure 2. The Human-Agent-UI triangle with the three design properties mapped to its edges. Classical HCI optimized interfaces for a single audience: the human user. Direct manipulation reduces the cognitive burden of command recall and supports rapid visual feedback ( 21 ) . The Gulfs of Execution and Evaluation formalize these advantages: good interfaces minimize the distance between user intention and available action, and between system state and user perception ( 15 ) . AI agents change this picture. In the Human-Agent-UI triangle (Figure 2 ), the interface must support two kinds of interaction simultaneously. The human must observe, direct, and oversee the agent’s work. The agent must perceive the interface state, select actions, and execute them reliably. These requirements can conflict. A GUI optimized for human visual perception may present the agent with a difficult grounding problem ( 7 ; 29 ) . Conversely, an interface optimized for agent operation (such as a raw API) may achieve representational compatibility with the agent while remaining opaque to human oversight. The design challenge is to identify interface properties that serve both audiences. Terminal-based agent tools offer a compelling case study: they have emerged organically from practice and demonstrably satisfy both requirements. Three properties account for this effectiveness. 3. Property 1: Representational Compatibility The first property concerns the alignment between the representational format native to the agent and the format of the interface. When these match, the translation overhead between agent reasoning and interface action is minimized. # Agent output IS terminal input: $ refactor auth module > mkdir app/auth > mv app/auth.py app/auth/core.py > cat app/auth/validate.py def validate(token: str) -> bool: payload = jwt.decode(token) return payload is not None # Same medium, no translation. Figure 3. Agent text output maps directly to shell commands and Python code with no perception layer needed. Large language models process and generate text. Their mechanisms for interacting with external systems, from function calling to code generation to shell commands, are all text-based. Terminal interfaces likewise consume and produce text. The structural mapping is direct: tool invocations correspond to shell commands, parameters to arguments, and output arrives through standard text streams (Figure 3 ). No additional perception or translation layer is required. A carefully designed text-based ACI outperformed agents using the default Linux shell by 10.7 percentage points on SWE-bench ( 28 ) . Using executable Python code as the agent action space yields up to 20% higher success rates than JSON-based function calling, because code is a more expressive text-based representation ( 25 ) . GUI agents, by contrast, must solve a perception problem before they can act: interpreting pixel-level screenshots, computing spatial coordinates, and mapping these to motor actions ( 7 ) . This translation layer is the primary bottleneck for GUI agent performance ( 29 ) . Implication for other modalities. Representational compatibility alone is insufficient. Raw APIs are text-based but opaque to human oversight and inaccessible to non-expert users. The terminal succeeds because it achieves compatibility within a medium the human can also read . For GUI agents, providing semantic interface representations (accessibility trees, DOM structures, UI element metadata) rather than raw pixels would reduce translation overhead while preserving human legibility. 4. Property 2: Transparency of the Interaction Medium The second property concerns whether the interaction medium makes agent actions, reasoning, and history visible and inspectable by the human collaborator. In terminal-based agent tools, the text stream simultaneously serves as communication channel, explanation surface, chronological record, and approval mechanism. Every proposed action is expressed as