code_transformed : The Influence of Large Language Models on Code
Yuliang Xu, Siming Huang, Mingmeng Geng, Yao Wan, Xuanhua Shi, Dongping Chen · 2025
Read code_transformed : The Influence of Large Language Models on Code by Yuliang Xu, Siming Huang, Mingmeng Geng, Yao Wan, Xuanhua Shi, Dongping Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-12014.txt.
Introduction
As AI code writers spread, they may be quietly changing how human-written code looks. Yuliang Xu and co-authors measured that change in this 2025 study. Code is one of the basic ways people talk to machines, so a shift in its style matters. The team asked whether model-written code has traits that are now spreading into public code.
They mined more than twenty thousand public code repositories tied to research papers from 2020 to 2025. Naming habits, complexity, upkeep ease, and similarity were tracked over time. One clear trend: snake_case names, with words joined by underscores, rose sharply in Python code after chat models took off. That share climbed from about forty-one percent in early 2023 to about half by late 2025. The study also probes how models tackle algorithm problems.
Catalog record
| CREATOR | Yuliang Xu, Siming Huang, Mingmeng Geng, Yao Wan, Xuanhua Shi, Dongping Chen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-12014
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
code_transformed : The Influence of Large Language Models on Code Yuliang Xu, Siming Huang, Mingmeng Geng, Yao Wan, Xuanhua Shi, Dongping Chen Abstract Coding remains one of the most fundamental modes of interaction between humans and machines. With the rapid advancement of Large Language Models (LLMs), code generation capabilities have begun to significantly reshape programming practices. This development prompts a central question: Have LLMs transformed code style, and how can such transformation be characterized? In this paper, we present a pioneering study that investigates the impact of LLMs on code style, with a focus on naming conventions, complexity, maintainability, and similarity. By analyzing code from over 19,000 GitHub repositories linked to arXiv papers published between 2020 and 2025, we identify measurable trends in the evolution of coding style that align with characteristics of LLM-generated code. For instance, the proportion of snake_case variable names in Python code increased from 47% in Q1 2023 to 51% in Q1 2025. Furthermore, we investigate how LLMs approach algorithmic problems by examining their reasoning processes. Given the diversity of LLMs and usage scenarios, among other factors, it is difficult or even impossible to precisely estimate the proportion of code generated or assisted by LLMs. Our experimental results provide the first large-scale empirical evidence that LLMs affect real-world programming style. 1 1 1 We release all the experimental dataset and source code at: https://github.com/ignorancex/LLM_code . † † † Equal Contribution. * Corresponding Authors. ‡ Project Lead. 1 Introduction The widespread use of Large Language Models (LLMs) and LLM-powered tools (such as Copilot and Cursor) in coding has been confirmed through questionnaire surveys ( Liang et al. 2024a ; Daigle and Staff 2024 ; Peslak and Kovalchick 2024 ; Li et al. 2025 ) . Our paper takes a different perspective: Can the transformation driven by LLMs be observed in real-world code? (a) An example of code rewritten by LLMs. (b) Frequency of the variable name max_length in our human-written code dataset and LLM-processed code, as well as its coverage across the GitHub repositories we collected. Figure 1: LLMs’ preference for longer variable names and snake_case naming patterns. Figure 1(a) presents a motivating example showing that code generated by LLMs differs in style from human-written code ( Wang et al. 2024 ; Park et al. 2025 ) . For instance, the short variable names (i.e., count and summ ) are replaced with longer and more descriptive names following the snake_case convention (i.e., current_length and total_magical_subarrays ). Figure 1(b) shows that compared to humans, LLMs are more likely to use the variable name max_length , and its prevalence in GitHub repositories has also increased. Previous research has explored the influence of LLMs in text and speech domains ( Liang et al. 2024b ; Geng et al. 2024 ) . Thus, we could potentially adopt a similar approach to observe the impact of LLMs on code. Despite significant advancements, the rapid adoption of LLMs has raised concerns about code integrity Wadhwa et al. 2024 , potential copyright infringement Wan et al. 2024 , and broader ethical or legal implications Xu et al. 2024b . These issues have motivated efforts to trace and attribute the influence of LLM-assisted programming. Therefore, we conduct a pioneering study to investigate the influence of LLMs on code, focusing on naming patterns, complexity, maintainability, and code similarity. In addition, we examine the thinking process of reasoning models to better comprehend the way LLMs tackle coding tasks. From the perspective of naming patterns , we first categorize the names of variables, functions, and files into several distinct formats: single letter , lower-case , UPPERCASE , camelCase , snake_case , PascalCase , and endsWithDigits . By analyzing the names of variables and functions in GitHub repository code, we observe a clear increase in the usage of LLM-preferred naming styles. From the perspective of code complexity and maintainability , we simulate code generated and rewritten by LLMs, extract subsets for analysis, and compare them with human-written code from GitHub. Our results indicate that LLM-rewritten code tends to be more concise under certain metrics—notably, cyclomatic complexity in Python. However, this improvement is less pronounced in stylistic aspects such as naming conventions. Additionally, no clear trend is observed in the GitHub code, suggesting that LLMs may not differ substantially from human developers in these dimensions. From the perspective of code similarity , the rewritten code exhibits relatively high similarity to the original, especially compared to code generated directly by LLM. This observation further highlights that different usage scenarios, such as code rewriting versus direct generation, can produce different outcomes. These findings may offer valuable insights for future efforts to detect and distinguish how LLMs are utilized, whether for assisted programming. Based on the above findings, we further explore whether the generated content influences the subsequent code generation abilities of LLMs. By analyzing the models’ thinking process, we find that their outputs do not always align with the expected algorithmic approaches for the given problems. We believe the findings in our work will enhance knowledge about LLMs’ programming abilities and coding styles, providing novel insights for assessing and monitoring their broader impacts. 2 Background 2.1 Position of Our Work Comparisons between code generated by LLMs and that written by humans can be conducted from multiple perspectives. Prior research has investigated various methods to distinguish between the two, such as leveraging perplexity scores ( Xu and Sheng 2024 ) and manually designed features ( Bulla et al. 2024 ; Park et al. 2025 ) . However, rather than focusing on differentiating LLM-generated code from human-written code, our work considers a more realistic and increasingly common scenario: LLM-assisted code authoring, where human developers and language models collaboratively produce code. In this paper, we investigate coding style from observable perspectives, with detailed explanations provided in the next subsection. Additionally, we also consider comparing the similarity between human-written code and code that has been rewritten or generated by LLMs using cosine similarity and Jaccard similarity. In order to better grasp how LLMs generate code, we also carefully analyze the reasoning chains to see if they think about the algorithms that the problems were designed to elicit. Figure 2: The process of our experiments. 2.2 Code Style Measurements Naming Patterns. As we mentioned before, the naming in code generated by LLMs has its own characteristics ( Park et al. 2025 ) . Therefore, we categorize variable, function, and file names into several distinct formats ( e.g. snake_case ). The length of the names has also been considered. Cyclomatic Complexity. Cyclomatic complexity ( McCabe 1976 ) is a metric used to measure the number of linearly independent paths in the code. Graylin et al. 2009 explore the relationship between cyclomatic complexity and lines of code. Some researchers use this method to analyze code generated by LLMs ( Dou et al. 2024 ) . Given the control-flow graph (CFG) of a code snippet, let E denote the number of edges, n the number of nodes, and P the number of connected components. The cyclomatic complexity is calculated by G=E-N+2P . For a single connected component ( P=1 ), it simplifies to the number of decision points plus one. Each occurrence of if , for , while , etc.,