Do AI Agents Really Improve Code Readability? Conference: 23rd International Conference on Mining Software Repositories; April 13–14, 2026; Rio de Janeiro, Brazil 23rd International Conference on Mining Software Repositories (MSR ’26), April 13–14, 2026, Rio de Janeiro, Brazil DOI: 10.1145/3793302.3793590 ISBN: 979-8-4007-2474-9/2026/04 CCS: Software and its engineering Integrated and visual development environments CCS: Software and its engineering Automatic programming CCS: Software and its engineering Software evolution CCS: Software and its engineering Maintaining software
Kyogo Horikawa † \dagger , Kosei Horikawa § \lx@sectionsign , Yutaro Kashiwa § \lx@sectionsign , Hidetake Uwano † \dagger , Hajimu Iida § \lx@sectionsign · 2026
Read Do AI Agents Really Improve Code Readability? Conference: 23rd International Conference on Mining Software Repositories; April 13–14, 2026; Rio de Janeiro, Brazil 23rd International Conference on Mining Software Repositories (MSR ’26), April 13–14, 2026, Rio de Janeiro, Brazil DOI: 10.1145/3793302.3793590 ISBN: 979-8-4007-2474-9/2026/04 CCS: Software and its engineering Integrated and visual development environments CCS: Software and its engineering Automatic programming CCS: Software and its engineering Software evolution CCS: Software and its engineering Maintaining software by Kyogo Horikawa † \dagger , Kosei Horikawa § \lx@sectionsign , Yutaro Kashiwa § \lx@sectionsign , Hidetake Uwano † \dagger , Hajimu Iida § \lx@sectionsign (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-13723.txt.
Introduction
Kyogo Horikawa and co-authors ask whether AI agents really improve code readability. Readability is key to software quality, and poor readability lengthens development and raises bug risk. Prior studies mix all kinds of refactoring, so the readability effect alone was unclear. The team pulled readability-related commits from the AIDev dataset and tracked metric changes in 403 commits. Agents mostly targeted logic complexity and documentation, not naming or formatting. Many readability commits made classic metrics worse, with the Maintainability Index down in most cases.
Catalog record
| CREATOR | Kyogo Horikawa † \dagger , Kosei Horikawa § \lx@sectionsign , Yutaro Kashiwa § \lx@sectionsign , Hidetake Uwano † \dagger , Hajimu Iida § \lx@sectionsign |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 22881 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.13723
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Do AI Agents Really Improve Code Readability? Conference: 23rd International Conference on Mining Software Repositories; April 13–14, 2026; Rio de Janeiro, Brazil 23rd International Conference on Mining Software Repositories (MSR ’26), April 13–14, 2026, Rio de Janeiro, Brazil DOI: 10.1145/3793302.3793590 ISBN: 979-8-4007-2474-9/2026/04 CCS: Software and its engineering Integrated and visual development environments CCS: Software and its engineering Automatic programming CCS: Software and its engineering Software evolution CCS: Software and its engineering Maintaining software Kyogo Horikawa † \dagger , Kosei Horikawa § \lx@sectionsign , Yutaro Kashiwa § \lx@sectionsign , Hidetake Uwano † \dagger , Hajimu Iida § \lx@sectionsign Abstract . Code readability is fundamental to software quality and maintainability. Poor readability extends development time, increases bug-inducing risks, and contributes to technical debt. With the rapid advancement of Large Language Models, AI agent-based approaches have emerged as a promising paradigm for automated refactoring, capable of decomposing complex tasks through autonomous planning and execution. While prior studies have examined refactoring by AI agents, these analyses cover all forms of refactoring, including performance optimization and structural improvement. As a result, the extent to which AI agent-based refactoring specifically improves code readability remains unclear. This study investigates the impact of AI agent-based refactoring on code readability. We extracted commits containing readability-related keywords from the AIDev dataset and analyzed changes in readability metrics before and after each commit, covering 403 commits evaluated using multiple quantitative metrics. Our results indicate that AI agents primarily target logic complexity (42.4%) and documentation improvements (24.2%) rather than surface-level aspects like naming conventions or formatting. However, contrary to expectations, readability-focused commits often degraded traditional quality metrics: the Maintainability Index decreased in 56.1% of commits, while Cyclomatic Complexity increased in 42.7%. † † cc-license: by 1. Introduction Code readability is a critical factor influencing software maintainability and sustainability ( 23 ) . Poor readability leads to technical debt ( 10 ; 5 ) and increased communication costs ( 4 ; 26 ; 29 ) . To address this, software engineering research has proposed various approaches, including automated refactoring tools ( 13 ; 32 ; 20 ) , quantitative metrics ( 3 ; 30 ) , and machine learning-based prediction models ( 31 ; 28 ) . While these traditional approaches have reduced developer burden, the recent emergence of Large Language Models (LLMs) has elevated automated refactoring to a new level of practicality. Particularly since the emergence of conversational LLMs such as ChatGPT, numerous practical applications have been reported ( 37 ; 12 ) . In this workflow, developers provide refactoring instructions through prompts, and LLMs immediately present code improvement suggestions, achieving significant efficiency gains compared to conventional methods. Cases of LLM-assisted refactoring are actively shared in Stack Overflow and GitHub discussions ( 7 ; 36 ) , indicating that this technology is becoming increasingly adopted in development practice. More recently, there has been a shift from single prompt-based approaches to AI agent-based approaches. These approaches enable the decomposition of complex tasks into multiple subtasks, allowing for more sophisticated refactoring through autonomous cycles of planning, execution, and verification ( 38 ) . Existing research has analyzed what types of refactoring AI agents perform and to what extent ( 14 ) . However, these studies examine refactoring activities in general, which include not only readability-focused refactoring but also refactoring aimed at performance optimization ( 8 ) , structural improvement ( 2 ) , and other objectives ( 16 ) . As a result, the specific impact of AI agent-based refactoring on code readability has not been sufficiently clarified. This represents an important research gap, as understanding the effectiveness of agent-based approaches for readability improvement is essential for developers who seek to leverage these tools for this particular purpose. Therefore, this study aims to clarify how much AI agent-based refactoring contributes to code readability. Specifically, we extracted commits containing keywords related to readability from the AIDev dataset ( 19 ) and investigated the impact of these commits on readability. We identified 403 commits and calculated three readability metrics for the code before and after each commit, allowing a thorough analysis of the effects of agent-based refactoring. Our findings show that readability-oriented commits do not necessarily lead to improvements in structural code quality and are often accompanied by quality degradation. Figure 1 . Overview of the study design A diagram showing the overview of the study design. 2. Related Work Code Readability and Assessment. Code readability is central to maintainability, prompting extensive research into its quantification. Buse and Weimer ( 4 ) and Scalabrino et al. ( 29 ) proposed models to predict readability based on code features. Furthermore, empirical studies by Dantas et al. ( 9 ) and Oliveira et al. ( 23 ) classified how human developers improve readability in practice, identifying categories such as identifier renaming and comment additions. More recently, Takerngsaksiri et al. ( 34 ) compared LLM-generated code with human-written code, finding comparable readability in enterprise settings. However, these studies focused on either human-driven improvements or code generation from scratch; whether AI agents can effectively improve the readability of existing code through refactoring has not been examined. Refactoring by LLMs and Agents. The emergence of LLMs has advanced automated refactoring. While Cordeiro et al. ( 7 ) examined the refactoring capabilities of LLMs, their focus was on prompt-based approaches. More recently, autonomous coding agents have emerged ( 19 ; 38 ) . The closest work to ours is Horikawa et al. ( 14 ) , who analyzed the impact of agent-driven refactoring on various code metrics. They did not observe any significant improvement in readability. However, they collected commits with commit messages containing “refactor”. This keyword is broad, and refactoring involves many operations including performance optimization ( 35 ) , code restructuring ( 22 ) , security enhancement ( 11 ; 1 ) , and modularity improvement ( 40 ) , beyond just readability. In this study, we focus on readability-improving commits made by agents and examine the impact on readability metrics established by Oliveira et al. ( 23 ) . 3. Data Collection In this section, we describe the data mining and data filtering as outlined in Figure 1 . 3.1. Data Mining To study the relationship between code readability and AI Agents, we start with the AIDev dataset ( 19 ) . This dataset contains 932,791 pull requests (PRs) created by five coding agents from over 61,000 repositories. However, the AIDev dataset only includes commit data for repositories with more than 100 GitHub stars. To gather a more comprehensive set of data points, we leverage the GitHub REST API 1 1 1 https://docs.github.com/en/rest to mine all commits based on the provided metadata. In total, we collect 1,382,314 agentic commits. 3.2. Identifying Commits improving Readability We then employ a keyword-based filtering approach on commit messages to extract commits related to readability improvement. Specifically, we performed case-insensitive filtering on commit messages using eight keywords derived from prior work [6]: “readability”, “readable”, “understandability”, “understandable”, “clarity”, “legibility”, “easier to read”, and “comprehensible”. This set of keywords was selected to broadly capture different expressions referring to improvements in code readability and understandability. For RQ2 and RQ3, to ensure that the extracted commits correspond to source code readability improvements rather than documentation or configuration changes,