How Do Developers Interact with AI? An Exploratory Study on Modeling Developer Programming Behavior DOI: 10.1145/3808120 Journal: PACMSE Volume: 3 FSE FSE113 7 fse26mainb-p272-p CCS: Software and its engineering Collaboration in software development CCS: Human-centered computing Human computer interaction (HCI) CCS: Computing methodologies Artificial intelligence
Yinan Wu, Ze Shi Li, Kathryn Thomasset Stolee, Bowen Xu · 2026
Read How Do Developers Interact with AI? An Exploratory Study on Modeling Developer Programming Behavior DOI: 10.1145/3808120 Journal: PACMSE Volume: 3 FSE FSE113 7 fse26mainb-p272-p CCS: Software and its engineering Collaboration in software development CCS: Human-centered computing Human computer interaction (HCI) CCS: Computing methodologies Artificial intelligence by Yinan Wu, Ze Shi Li, Kathryn Thomasset Stolee, Bowen Xu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-16393.txt.
Introduction
Wu, Li, Stolee and Xu studied how programmers work with AI help, looking beyond prompts and edits to intentions and feelings. In a study, 76 developers split into AI-assisted and unaided groups completed programming tasks and labelled their intentions, actions and emotions from screen recordings. The team distilled the results into S-IASE, a model spanning intention, action, supporting tool and emotion. AI-assisted developers stayed more focused on creating and checking code, and steadier emotionally, though some reported guilt or self-doubt about relying on help.
Catalog record
| CREATOR | Yinan Wu, Ze Shi Li, Kathryn Thomasset Stolee, Bowen Xu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 75939 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.16393
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
How Do Developers Interact with AI? An Exploratory Study on Modeling Developer Programming Behavior DOI: 10.1145/3808120 Journal: PACMSE Volume: 3 FSE FSE113 7 fse26mainb-p272-p CCS: Software and its engineering Collaboration in software development CCS: Human-centered computing Human computer interaction (HCI) CCS: Computing methodologies Artificial intelligence Yinan Wu, Ze Shi Li, Kathryn Thomasset Stolee, Bowen Xu Abstract . Artificial Intelligence (AI) is reshaping how developers adopt software engineering practices, yet the multi-dimensional nature of developer-AI interaction remains under-explored. Prior studies have primarily examined dimensions observable from developer activities such as “Prompt Crafting” and “Code Editing,” overlooking how hidden intentions and emotional dimensions intertwine with concrete actions during AI-assisted programming. Understanding the interplay is essential for improving developer experience and future AI assistant designs. To understand this phenomenon, we conducted a mixed-methods study with 76 developers. We first split developers into AI-assisted and non-AI groups. Each developer performed a programming task (either Python with API management or Java with SQL). Developers retrospectively labeled their self-reported intentions, tool-supported actions, and emotions (on a 7-point valence scale) from screen recordings, supplemented by participant surveys and interviews. Our user study resulted in a novel model, named S-IASE , with four dimensions to describe programming behavior for a given development state : intention , action , supporting tool , and emotion . Our analysis reveals several aggregated and sequential behavioral patterns. For example, for aggregated patterns, using AI assistants often led developers to focus more on actively “creating” code, evaluating, and verifying the generated results; for sequential patterns, AI-assisted participants showed emotionally stable development flows, as opposed to non-AI-assisted participants who experienced more fluctuating emotions. Interviews revealed further nuance: some developers reported impostor-like feelings, expressing guilt or self-doubt about relying on AI for programming. The uncovered patterns indicate that our model can provide actionable insights for improving AI assistants’ responsiveness, training developers in AI collaboration, and designing developer-centric AI studies. Our work bridges an important gap in understanding the complexities of developer-AI interaction in the programming context and sheds light on future developer-centric research directions. † † cc-license: by 1. Introduction Software engineering is undergoing a significant shift due to the rise of AI assistants ( 33 ; 72 ; 13 ) . AI assistants such as GitHub Copilot ( 54 ) and ChatGPT ( 58 ) have gained widespread attention and quickly attracted millions of subscribers from students to professional developers alike. GitHub Copilot, in particular, promises to be an “ AI [code] editor for everyone ” ( 22 ) . Prior research has explored these AI assistants in various capacities to support software developers throughout the software development lifecycle, ranging from requirements ( 62 ; 56 ) , design ( 57 ) , development ( 63 ; 12 ; 70 ; 44 ) , testing ( 3 ) , to even maintenance ( 42 ; 9 ) . Due to the wide variety of areas in which a developer can leverage AI, recent large-scale developer surveys found that 76% of developers in 2024 ( 2 ) and 84% of developers in 2025 are already using or plan to use AI in their work ( 1 ) . Yet, as AI assistants become embedded in everyday software programming, a central question arises: how do developers interact with these assistants? Without sufficient understanding, we cannot design AI assistants that better align with developer programming behavior and ultimately boost developers’ productivity. Most prior research has focused on ostensible benefits (e.g., reduced repetitive coding ( 46 ) ) or on narrow contexts (e.g., novice learning ( 5 ; 34 ) ). Even when prior studies have looked at interaction, they have often done so from the researchers’ observations, identifying a limited set of modes (e.g., “acceleration” vs. “exploration” modes ( 7 ) ). However, few studies have focused on developer-AI interaction from the developer’s perspective. We have limited perspectives on what developers think and feel when they interact with AI assistants as well as how those perceptions differ from those without AI. Mozannar et al. ( 43 ) are among the first to explore developer-AI interaction from the developer’s perspective. They investigated how developers interact with GitHub Copilot during time-restricted coding tasks and developed a taxonomy called CUPS to categorize developer states when interacting with AI assistants. While their work represents a significant step toward understanding developer-AI interaction, the CUPS taxonomy treated a “state” as a single coding activity and focused on transitions between activities (e.g., from “Prompt Crafting” to “Code Editing” ). However, such activity-based labels failed to capture the subtleties and multidimensionality of developer-AI interaction in the context of programming. By “ dimension ,” we refer to a distinct, analyzable component of developer-AI interaction (e.g., intentions, actions). Each dimension provides a lens to study and compare interactions. Google’s SIA (State–Intention–Action) model ( 55 ) further inspired us to view a programming state not as a single coding activity, but as a snapshot of development context, which includes the specific tools, and gives rise to intentions and actions. These insights led to our understanding that developer-AI programming behavior involves more than one dimension. This motivated the main question guiding our research: How do developers interact with AI assistants in the broader programming context? Our exploratory work aims to understand more deeply how AI assistants shape the different dimensions of developer programming behavior and how these dimensions could be characterized to the sequences of development states. We conducted a mixed-methods user study with a group of 76 participants with varied backgrounds in terms of education, programming experience, and familiarity with the topic of the tasks. The participants were assigned to two groups: AI and non-AI. Participants from the AI group could freely use any AI assistants in their preferred way during task execution. In contrast, those from the non-AI group, who had little to no experience with using AI for programming in the past, performed the task without using AI. Within each group, participants were assigned to one of two programming tasks: a Java task with MySQL Database Operations or a Python task with GitHub REST API. From this study, we found that developer programming behavior when interacting with AI involves four distinct dimensions. We formalize this in our proposed S-IASE model, which characterizes a developer’s S tate when they interact with AI assistants, using their I ntention , pairs of < A ction , S upporting tool >, and E motion . state n-1 → state n [intention, <action, supporting tool>, emotion] → state n+1 Our model details how intention (e.g., To Implement Code \to To Resolve Errors ), actions supported with tools (e.g., <Reading and Comprehension, No Tool> \to <Crafting Query/Prompt, AI Tool> ), and emotions (e.g., Neutral \to Positive ) manifest throughout developers’ states. Our proposed model unlocks the new opportunity to perform analysis and reveal behavioral patterns. Specifically, on top of S-IASE , we performed a mixed-methods analysis and uncovered two complementary types of patterns. First, aggregated patterns derived from descriptive statistics. For instance, our results show that using AI assistants often makes the programming experience more focused on actively “creating” code, evaluating, and verifying the generated results. Second, sequential patterns identified through automated sequential pattern mining approaches (e.g., CloSpan ( 69 ) ).