Usage, Effects and Requirements for AI Coding Assistants in the Enterprise: An Empirical Study Conference: ; ;
Maja Vukovic, Rangeet Pan, Tin Kam Ho, Rahul Krishna, Raju Pavuluri, Michele Merler · 2026
Read Usage, Effects and Requirements for AI Coding Assistants in the Enterprise: An Empirical Study Conference: ; ; by Maja Vukovic, Rangeet Pan, Tin Kam Ho, Rahul Krishna, Raju Pavuluri, Michele Merler (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-20112.txt.
Introduction
Maja Vukovic and five colleagues study how AI coding assistants are used inside large companies. Their 2026 study looks at usage, effects, and what firms require from these tools. Large language models now help with many coding tasks, from reading old code to writing tests and fixing bugs.
The authors gather evidence from the enterprise setting, where rules on safety, privacy, and control shape what tools can do. Their study asks how workers actually use the assistants day to day, and what gains or problems show up. Results of this kind help firms decide how to roll out coding assistants with care.
Catalog record
| CREATOR | Maja Vukovic, Rangeet Pan, Tin Kam Ho, Rahul Krishna, Raju Pavuluri, Michele Merler |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 64 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-20112
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Usage, Effects and Requirements for AI Coding Assistants in the Enterprise: An Empirical Study Conference: ; ; Maja Vukovic, Rangeet Pan, Tin Kam Ho, Rahul Krishna, Raju Pavuluri, Michele Merler Abstract . The rise of large language models (LLMs) has accelerated the development of automated techniques and tools for supporting various software engineering tasks, e.g., program understanding, code generation, software testing, and program repair. As CodeLLMs are being employed toward automating these tasks, one question that arises, especially in enterprise settings, is whether these coding assistants and the code LLMs that power them are ready for real-world projects and enterprise use cases, and how do they impact the existing software engineering process and user experience. In this paper we survey 57 developers from different domains and with varying software engineering skill about their experience with AI coding assistants and CodeLLMs. We also reviewed 35 user surveys on the usage, experience and expectations of professionals and students using AI coding assistants and CodeLLMs. Based on our study findings and analysis of existing surveys, we discuss the requirements for AI-powered coding assistants. intedjava]java frame=lines, framesep=1mm, baselinestretch=1, fontsize= , breaklines, breakanywhere, linenos, numbersep=4pt, style=xcode 1. Introduction The evolution of modern software engineering has consistently leaned toward automation so as to increase efficiency and reduce repetitive and low-level development tasks. Over the past decade, various techniques ranging from the evolution of scripting utilities to more sophisticated development frameworks have attempted to simplify common software development tasks ( 14 ) . The advent of Code Large Language Models (CodeLLM) powered AI-coding assistants represents yet another promising step in this direction, allowing developers to automate code generation, suggest com- pletions, produce tests, and even modernize legacy codebases. The use of large language models (LLMs) allows these AI assistants to enable contextually relevant guidance, streamline code creation, and offer assistance in learning new programming languages. More recently, the rise of agentic systems has enabled further automation of more complex software engineering tasks such as bug fixing ( 18 ) . In this paper we survey 57 industry developers about their experience with AI coding assistants. The surveyed group came from several functional domains (e.g. software, consulting, research, etc.), with varying levels of software engineering experience. The intent of this survey was to understand the following: • What are the different utilization of coding assistants and desired use cases across the functional verticals • How does the development experience influence the expectations of an AI coding assistant. • What are the differences in software engineering processes introduced by AI coding assistants. • What is the overall perceived effectiveness of the assistants • What are the requirements for the future AI coding assistants. Stemming from the user study, we look for requirements that will shape the research road map and features for future iterations of coding assistants. The contribution of our work is a survey that provides a comparative study of usage of AI coding assistants across a diverse set of users, understanding the impact of that usage in terms of organizational structure, productivity and quality. 2. Related Work Research on AI coding assistants has grown rapidly, with GitHub Copilot, ChatGPT, Tabnine, and enterprise specific tools such as Amazon Q and IBM Watsonx Code Assistant among the most studied ( 37 ) . Large surveys ( 57 ; 59 ; 46 ; 48 ) and randomized controlled trials ( 29 ; 10 ; 7 ; 39 ) consistently show perceived productivity gains ( 56 ) , often in the range of 12–25% speedups and up to almost one third of developers code being written with help from the AI tool ( 27 ) . Industry case studies further validate these benefits in real-world settings ( 3 ; 50 ) . Major categories with satisfactory outcomes include code generation and refactoring, testing, code exaplanation and addressing data analytics issues ( 12 ) . While productivity advantages are clear, the extent of improvement varies across tasks type and complexity ( 54 ) , developer expertise ( 16 ) , and organizational contexts. Less experienced and causal developers report higher adoption rates of AI coding tools and greater productivity gains ( 11 ) , using them to reduce opportunity costs. In open source, evidence has been found that this enhanced development activity translated even into increased short to medium-term career benefits for developers ( 25 ) . On the other hand recent studies ( 5 ) have found discrepancies between perceived and measured productivity for experienced developers. Beyond efficiency, concerns about quality and security remain salient. Empirical audits show that Copilot-generated code often contains vulnerabilities, particularly in security-critical domains ( 41 ; 22 ; 15 ; 52 ) . Perceptions are similarly mixed: developers report faster delivery but raise doubts about maintainability, integration and correctness ( 26 ; 31 ; 53 ) . Studies on readability and naming practices find no consistent degradation but highlight variability depending on context ( 50 ; 24 ; 58 ) . Overall, the literature underscores that productivity gains may come at the cost of quality assurance and over-reliance is identified as a risk factor ( 1 ) . Adoption and trust ( 43 ; 36 ) have emerged as central challenges. Legal and ethical questions compound these issues, as many practitioners remain unclear about copyright and licensing implications ( 48 ; 4 ) . Recent studies ( 33 ) found that widespread organizational expectations in industries for rapid productivity gains are limited by uneven adoption and familiarity with the tools from different team members, thus advocating for needed investment in learning. Studies of learning and collaboration reveal similarly dual effects ( 28 ) . Novices benefit from scaffolding and improved comprehension of large codebases ( 47 ; 42 ; 30 ; 6 ) , yet risk shallow learning and over-reliance ( 2 ; 1 ) . Professionals likewise report efficiency gains but worry about deskilling in core competencies, even as new skills around prompting and verification emerge ( 9 ; 49 ; 40 ) . The socio-economical consequences of the wide adoption of AI based coding tools are also being researched, with focus not only on workflow efficiency but also its potential for developer replacement ( 23 ) and displacement ( 13 ) . The use of AI coding assistants beyond software engineers has been largely understudied, with few exceptions discovering for example that scientists often use code generating models as an information retrieval tool for navigating unfamiliar programming languages and libraries ( 38 ) . Despite these advances, our analysis of prior studies reveals significant gaps. First, most of the studies focus on immediate productivity or single specific tasks ( 35 ; 8 ; 34 ) rather than long-term maintainability, software quality, or organizational outcomes. Second, despite the recent proliferation of AI tools (Cursor, Windsurf, Replit, Zencoder, Cody, BlackboxAI, abacus AI, Qodo, Bolt etc.) studies have predominantly focused only on very few, namely ChatGPT and Github Copilot. Third, participants are often drawn from homogeneous cohorts, restricting generalizability across industries, regions, and developer demographics. These omissions highlight the need for more diverse, longitudinal, and governance-focused studies to inform both research and practice. 3. Survey on AI coding assistants use in enterprise and academia In this Section