How Do AI Agents Do Human Work? Comparing AI and Human Workflows Across Diverse Occupations

Zora Zhiruo Wang Yijia Shao Omar Shaikh Daniel Fried Graham Neubig Diyi Yang Carnegie Mellon University Stanford University [email protected] · 2025

Read How Do AI Agents Do Human Work? Comparing AI and Human Workflows Across Diverse Occupations by Zora Zhiruo Wang Yijia Shao Omar Shaikh Daniel Fried Graham Neubig Diyi Yang Carnegie Mellon University Stanford University [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-22780.txt.

Introduction

How do AI agents actually do human work, step by step? Zora Zhiruo Wang leads a team with five co-authors. In 2025 they give the first direct comparison of human and agent workflows. Skills covered include data analysis, engineering, and writing. Design and computation are also covered.

The team built a toolkit to extract workflows. It turns computer use logs into clear, structured steps. Agents often match human workflow shape. Yet agents lean hard on programmatic methods. Humans lean more on user interface actions. That gap holds even for open, visual tasks like design. Results also hint where agents fit best in mixed teams.

Catalog record

CREATORZora Zhiruo Wang Yijia Shao Omar Shaikh Daniel Fried Graham Neubig Diyi Yang Carnegie Mellon University Stanford University [email protected]
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE83 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-22780

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

How Do AI Agents Do Human Work? Comparing AI and Human Workflows Across Diverse Occupations Zora Zhiruo Wang Yijia Shao Omar Shaikh Daniel Fried Graham Neubig Diyi Yang Carnegie Mellon University Stanford University [email protected] Abstract AI agents are continually optimized for tasks related to human work, such as software engineering and professional writing, signaling a pressing trend with significant impacts on the human workforce. However, these agent developments have often not been grounded in a clear understanding of how humans execute work, to reveal what expertise agents possess and the roles they can play in diverse workflows. In this work, we study how agents do human work by presenting the first direct comparison of human and agent workers across multiple essential work-related skills: data analysis, engineering, computation, writing, and design. To better understand and compare heterogeneous computer-use activities of workers, we introduce a scalable toolkit to induce interpretable, structured workflows from either human or agent computer-use activities. 1 1 1 text Using such induced workflows, we compare how humans and agents perform the same tasks and find that: (1) While agents exhibit promise in their alignment to human workflows, they take an overwhelmingly programmatic approach across all work domains, even for open-ended, visually dependent tasks like design, creating a contrast with the UI-centric methods typically used by humans. (2) Agents produce work of inferior quality, yet often mask their deficiencies via data fabrication and misuse of advanced tools. (3) Nonetheless, agents deliver results 88.3% faster and cost 90.4–96.2% less than humans, highlighting the potential for enabling efficient collaboration by delegating easily programmable tasks to agents. 1 Introduction AI agents are increasingly developed to perform tasks traditionally carried out by human workers ( Brynjolfsson et al. 2018 ; Eloundou et al. 2023 ; Bick et al. 2024 ) , as reflected in the growing competence of computer-use agents in work-related tasks such as software engineering ( Jimenez et al. 2024 ; Wang et al. 2025a ) and writing ( Clark et al. 2018 ) . Nonetheless, they still face challenges in many scenarios such as basic administrative or open-ended design tasks ( Xu et al. 2024 ) , sometimes creating a gap between expectations and reality in agent capabilities to perform real-world work ( Zhang et al. 2021 ; Ryoo et al. 2025 ) . To further improve agents’ utility at such tasks, we argue that it is necessary to look beyond their end-task outcome evaluation as measured in existing studies ( Patwardhan et al. 2025 ) , and investigate how agents currently perform human work — understanding their underlying workflows to gain deeper insights into their work process, especially how it aligns or diverges from human workers, to reveal the distinct strengths and limitations between them. Therefore, such an analysis should not benchmark agents in isolation ( Xu et al. 2024 ; Patwardhan et al. 2025 ) , but rather be grounded in comparative studies of human and agent workflows. More broadly speaking, to foster a collaborative future where humans and AI work effectively together, it is crucial to understand how both operate in practice, and to anticipate the organizational and behavioral changes required as agents assume greater roles ( Masters et al. 2025 ) . To this end, we first propose to perform a direct comparison of how human and agent workers do the same tasks. Beyond specific domains ( Jimenez et al. 2024 ; Choi and Schwarcz 2024 ) , we aim to provide a holistic view of how agents might reshape the broader human workforce, by systematically studying human and agent workflows across occupations via essential shared skills — data analysis, engineering, computation, writing, and design — instantiated with 16 realistic, long-horizon tasks. We estimate these tasks to be representative of 287 computer-using U.S. occupations and 71.9% of their daily work activities (§ 2 ). Comparing human and agent activities is not trivial: we lack a unified representation and a scalable channel for analysis. We first collect human activities in a similar way to agent activities with actions and screen states. However, activities in their raw format, low-level mouse and keyboard actions, are largely context-independent (i.e., a single key press may not map to a meaningful user-intended task) ( Shaw et al. 2023 ) thus unintuitive to interpret for both human and agent readers ( Wang et al. 2024b ) . Inspired by humans who naturally reason about work in terms of higher-level workflows : structured sequences of actions performed together to achieve a goal ( Deka et al. 2016 ; Liu et al. 2018 ; Wang et al. 2025c ) . We develop a workflow induction procedure that transforms heterogeneous computer-use activities into hierarchical, interpretable workflows. These workflows provide a shared representation that enables effective comparative analysis in our study (§ 3 ). Using this method, we compare 48 human workers and 4 representative LLM agent frameworks on 16 curated work-related tasks. We find that: \bullet Agents take an overwhelmingly programmatic approach. While agents show initial promise in exhibiting human-aligned workflows, they write programs to solve essentially all tasks, even when equipped with and trained for UI interactions. This prevails across work domains — not only programmable tasks such as data analysis, but heavily among open-ended, visual tasks such as design; forming a stark contrast to the visual-oriented human workflows ( Figure 4 ). \bullet Human workflows are substantially altered by AI automation, but not by AI augmentation. One quarter of human activities we studied involve AI tools, with most used for augmentation purposes: integrating AI into existing workflows with minimal disruption, while improving efficiency by 24.3%. In contrast, AI automation markedly reshapes workflows and slows human work by 17.7%, largely due to additional time spent on verification and debugging ( Figure 5 ). \bullet Agents produce lower-quality work, concerningly by fabrication and tool misuse. Agents produce work of inferior quality, most notably by fabricating data to deliver plausible outcomes, or misuse advanced tools (e.g., deep research) to mask their limitations (e.g., retrieve alternative files when unable to read user-provided ones); among issues in intent misinterpretation, vision inability, and trouble transforming between program- and UI-friendly data formats ( Figure 6 ). \bullet Teaming humans and agents based on their accuracy and efficiency advantages. Despite the gap in quality, agents exhibit clear advantages in efficiency. Compared to an average human worker, agents deliver work 88.3–96.6% faster and at 90.4–96.2% lower costs. Our induced workflows naturally suggest a division of labor: readily programmable steps can be delegated to agents for efficiency, while humans handle the steps where agents fall short. Together, this forms a human-agent teaming jointly optimized for work quality and efficiency ( Figure 7 ). Figure 1: Overview of our study: a direct human-agent worker comparison (a) supported by our workflow induction toolkit (b), and four major analysis findings (c). 2 Studying Humans and Agents At Work We propose the first study to directly compare agent workflows with respect to human workers. \bullet First, this comparison between human and agent workers goes beyond measuring performance — it reveals how each completes tasks, uncovering their underlying skill profiles and behavioral differences. Existing evaluations often reduce agent performance to numerical metrics. To build a collaborative future where humans

Read the complete text