Beyond Functional Correctness: Design Issues in AI IDE-Generated Large-Scale Projects DOI: XXXXXXX.XXXXXXX Journal: TOSEM Volume: 0 0 0 0 CCS: Software and its engineering Software development techniques

Syed Mohammad Kashif, Ruiyin Li, Peng Liang, Amjed Tahir, Qiong Feng, Zengyang Li, Mojtaba Shahin · 2026

Read Beyond Functional Correctness: Design Issues in AI IDE-Generated Large-Scale Projects DOI: XXXXXXX.XXXXXXX Journal: TOSEM Volume: 0 0 0 0 CCS: Software and its engineering Software development techniques by Syed Mohammad Kashif, Ruiyin Li, Peng Liang, Amjed Tahir, Qiong Feng, Zengyang Li, Mojtaba Shahin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-06373.txt.

Introduction

Kashif and six colleagues test Cursor, an AI coding tool, on large-scale software builds using a Feature-Driven Human-In-The-Loop plan. They build 10 projects across three domains, guided by curated briefs. Manual checks give 91 percent average correctness, with projects at 16,965 lines across 114 files. CodeScene finds 1,305 issues in 9 categories; SonarQube finds 3,193 in 11 categories. Top issues are duplication, complexity, large methods, framework violations, exception handling, and accessibility. These violate SRP, SoC, and DRY principles, posing maintainability risks that demand review by experienced developers.

Catalog record

CREATORSyed Mohammad Kashif, Ruiyin Li, Peng Liang, Amjed Tahir, Qiong Feng, Zengyang Li, Mojtaba Shahin
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE119065
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.06373

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Beyond Functional Correctness: Design Issues in AI IDE-Generated Large-Scale Projects DOI: XXXXXXX.XXXXXXX Journal: TOSEM Volume: 0 0 0 0 CCS: Software and its engineering Software development techniques Syed Mohammad Kashif, Ruiyin Li, Peng Liang, Amjed Tahir, Qiong Feng, Zengyang Li, Mojtaba Shahin Abstract . New generation of AI coding tools, including AI-powered IDEs (AI IDEs) equipped with agentic capabilities, can generate code within the context of the project. These AI IDEs are increasingly perceived as capable of producing project-level code at scale. However, there is limited empirical evidence on the extent to which they can generate large-scale software systems and what design issues such systems may exhibit. To address this gap, we conducted an empirical study to explore the capability of Cursor in generating large-scale projects and to evaluate the design quality of projects generated by Cursor as an AI IDE. First, we propose a Feature-Driven Human-In-The-Loop (FD-HITL) framework that systematically guides project generation from curated project descriptions. We generated 10 projects using Cursor with the FD-HITL framework across three application domains (mobile, Web, and utility) and multiple technologies (e.g., React, Spring Boot, Django, and React Native). We assessed the functional correctness of these projects through manual evaluation, obtaining an average functional correctness score of 91%. Next, we analyzed the generated projects using two static analysis tools, CodeScene and SonarQube, to detect design issues. We identified 1,305 design issues categorized into 9 categories by CodeScene and 3,193 issues in 11 categories by SonarQube. Our findings show that (1) when used with the FD-HITL framework, Cursor can generate functional large-scale projects averaging 16,965 lines of code (LoC) and 114 files; (2) the generated projects nevertheless contain design issues that may pose long-term maintainability and evolvability risks, requiring careful review by experienced developers; (3) the most prevalent issues include Code Duplication , high Code Complexity , Large Methods , Framework Best-Practice Violations , Exception-Handling Issues and Accessibility Issues ; and (4) these design issues violate design principles such as the Single Responsibility Principle (SRP) , Separation of Concerns (SoC) , and Don’t Repeat Yourself (DRY) . Finally, we contributed DIinAGP, a curated dataset of 10 project descriptions and the corresponding Cursor-generated projects available at https://github.com/Kashifraz/DIinAGP . 1. Introduction The integration of Artificial Intelligence (AI) techniques, especially those employing Large Language Models (LLMs), has transformed the software development paradigm. LLMs have been widely adopted to develop tools that assist software engineers in their development tasks. Developers are increasingly relying on AI code generation tools to write code ( 14 ; 24 ; 68 ) . The most recent Stack Overflow Developer Survey (2025) also indicates significant adoption of AI code generation tools in practice ( 58 ) . Previous LLM-assisted AI code generation tools, such as GitHub Copilot ( 10 ) and CodeWhisperer ( 9 ) , and general-purpose LLMs (e.g., ChatGPT ( 45 ) and Gemini ( 15 ) ), can generate code snippets from natural language descriptions with minimal human intervention ( 13 ; 62 ) . These code snippets have been reported to improve developers’ productivity and reduce development time and effort ( 66 ; 35 ) . However, previous AI code generation tools can only generate code snippets ( 60 ; 28 ) and lack the capability to generate project-level code directly in the development environment ( 60 ; 29 ) . This is partly because these tools are not directly integrated into the development environment or lack sufficient development context ( 34 ; 17 ) . Recently, a new generation of code generation tools has been introduced, including AI-powered IDEs (e.g., Cursor ( 12 ) , Claude Code ( 7 ) , and OpenAI Codex ( 46 ) ) ( 32 ) and autonomous systems for software development (e.g., MetaGPT ( 23 ) and AgileCoder ( 44 ) ). These tools have demonstrated the ability to perform development tasks at the project level within the project context, which can assist developers in building more complex software systems ( 32 ) . In this study, we primarily focus on AI IDEs due to their agentic capabilities, such as interacting with the project environment (e.g., reading codebases, running tools, planning changes) ( 32 ) . Specifically, we focus on Cursor for this study, as this AI IDE has been used in prior research ( 22 ; 30 ) . In addition, a recent large-scale empirical study also found that Cursor-generated pull requests (PRs) are predominantly feature implementation PRs ( 32 ) . Developers often operate AI IDEs using Vibe Coding , which is a new programming approach in which they generate code without fully understanding the output, prioritizing speed and experimentation, which can affect code quality ( 19 ) . Despite the potential of these AI IDEs for project-level code generation, two research gaps remain. (1) The existing literature has not explored or evaluated the potential of these AI IDEs for large-scale project generation. Most studies that examine end-to-end full-project generation focus on generating simple (in terms of architectural components and technology stacks) and small projects (in terms of lines of code (LoC)) ( 23 ; 44 ; 51 ) . The descriptions for generating projects are also short and simple, and do not capture the complexity of typical industrial-level projects ( 23 ; 51 ) . (2) Existing studies have extensively evaluated AI-generated code, exploring their quality issues ( 59 ; 47 ) and security vulnerabilities ( 70 ; 49 ) . He et al. also found that the adoption of Cursor leads to an increase in project-level development velocity but also increases code complexity in GitHub projects ( 22 ) . However, there is a lack of empirical design quality evaluation of project-level code. Since project-level code is generated at scale within the project development context, it is more likely that high-level design issues will arise. To this end, our study aims to explore the use of Cursor for generating large-scale projects and to evaluate the design quality of the Cursor-generated projects. We define a large-scale project as a software system that contains at least 8K lines of code (LoC), follows specific architectural styles with multiple architectural components, and is built with a technology stack (e.g., MERN, LAMP) similar to industrial software systems. We address the following two Research Questions (RQs): • RQ1: To what extent can Cursor generate large-scale projects? • RQ2: What design issues are found in large-scale projects generated by Cursor? To address our RQs, we designed a feature-driven, Human-In-The-Loop (FD-HITL) framework to systematically generate large-scale projects using Cursor. First, we curated 10 project descriptions to generate the projects using Cursor (specifically Cursor Pro). We employed two static analysis tools, CodeScene ( 8 ) and SonarQube ( 56 ) , to detect design issues in the generated projects. We also manually checked the identified design issues and removed 1,612 false positives from the issues identified by SonarQube. We identified 1,305 design issues by CodeScene and 3,193 by SonarQube. We employed quantitative and qualitative techniques to analyze the identified design issues and obtained 9 categories of design issues by CodeScene and 11 by SonarQube. We also reported 133 overlapping design issues between CodeScene and SonarQube. Our findings show that: (1) Cursor can generate functional large-scale projects

Read the complete text