Beyond Strict Rules: Assessing the Effectiveness of Large Language Models for Code Smell Detection
Saymon Souza, Amanda Santana, Eduardo Figueiredo · 2026
Read Beyond Strict Rules: Assessing the Effectiveness of Large Language Models for Code Smell Detection by Saymon Souza, Amanda Santana, Eduardo Figueiredo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-09873.txt.
Introduction
Saymon Souza, Amanda Santana, and Eduardo Figueiredo test whether language models can smell bad code. Their 2026 arXiv paper assesses large language models for code smell detection, moving beyond strict rule-based checkers. Code smells are warning signs of quality problems that raise maintenance costs.
Language models have shown skill across many software engineering tasks. Their use for smell detection is still little explored. The authors measure how effective the models are in practice. A capable detector tool could warn developers very early on, before smelly code hardens into expensive, unreliable systems.
Catalog record
| CREATOR | Saymon Souza, Amanda Santana, Eduardo Figueiredo |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 64 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-09873
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Beyond Strict Rules: Assessing the Effectiveness of Large Language Models for Code Smell Detection Saymon Souza, Amanda Santana, Eduardo Figueiredo Abstract Code smells are symptoms of potential code quality problems that may affect software maintainability, thus increasing development costs and impacting software reliability. Large language models (LLMs) have shown remarkable capabilities for supporting various software engineering activities, but their use for detecting code smells remains underexplored. However, unlike the rigid rules of static analysis tools, LLMs can support flexible and adaptable detection strategies tailored to the unique properties of code smells. This paper evaluates the effectiveness of four LLMs – DeepSeek-R1, GPT-5 mini, Llama-3.3, and Qwen2.5-Code – for detecting nine code smells across 30 Java projects. For the empirical evaluation, we created a ground-truth dataset by asking 76 developers to manually inspect 268 code-smell candidates. Our results indicate that LLMs perform strongly for structurally straightforward smells, such as Large Class and Long Method . However, we also observed that different LLMs and tools fare better for distinct code smells. We then propose and evaluate a detection strategy that combines LLMs and static analysis tools. The proposed strategy outperforms LLMs and tools in five out of nine code smells in terms of F1-Score. However, it also generates more false positives for complex smells. Therefore, we conclude that the optimal strategy depends on whether Recall or Precision is the main priority for code smell detection. I Introduction Code smells are symptoms or indicators of potential code quality problems that may affect software maintainability and reliability [ 17 ] . Code maintainability is essential because it refers to how easily code can be understood, changed, and improved [ 51 , 58 ] . In this context, previous research has shown that code smells can increase development costs [ 1 , 39 ] , reduce software reliability [ 67 , 64 ] , and lead to software defects [ 38 , 35 , 24 , 20 ] . Detecting and refactoring code smells can be challenging because manual reviews require advanced skills, are expensive and slow, while automated tools are imprecise and still require human interpretation [ 39 ] . Common examples of code smells include methods that become complex because they take on too many responsibilities, intense communication between classes, or the same code snippets repeated in different places [ 17 ] . Recent advances in Large Language Models (LLMs) have sparked interest in their use for coding problems [ 2 , 34 , 30 ] . In fact, these models have shown promising capabilities for generating code [ 23 ] , repairing bugs [ 9 ] , and supporting software testing [ 60 ] , but the challenges of using LLMs to detect code smells remain underexplored and lack relevant benchmarks [ 63 ] . In contrast to traditional automated detection tools that adhere to strict rules, such as JDeodorant [ 56 ] and PMD [ 41 ] , LLMs can provide an innovative approach to detecting code smells. Their ability to understand complex contexts may enable them to adopt flexible, adaptable strategies for different types of code smells. In fact, some preliminary studies have investigated the use of LLMs to detect and refactor code smells [ 50 , 63 , 22 ] . However, to the best of our knowledge, they do not provide strong empirical evidence to support, for instance, the claim that LLMs perform better than traditional code smell detection tools. Moreover, early investigations of LLMs for code smell detection typically employ a small set of smell types and simple code samples [ 57 , 11 , 27 ] . Thus, it is essential to further investigate the effectiveness and limitations of LLMs for detecting code smells in real-world software projects [ 63 ] . Such software projects introduce numerous challenges for LLMs, as they require navigating large, complex codebases, adhering to diverse coding standards, and ensuring compatibility with existing systems [ 33 ] . These characteristics make automated detection of code smells particularly difficult, especially when compared to controlled or synthetic code examples. Evaluating how effectively LLMs can detect code smells under these realistic conditions is therefore crucial, as it helps determine whether they can serve as practical alternatives, or valuable complements, to traditional detection tools [ 50 ] . More importantly, we need to empirically determine not only how LLMs compare with other automated techniques, but also how closely their detections align with human judgments of what constitutes a code smell. This paper evaluates the effectiveness of LLMs by using four models, namely DeepSeek-R1, GPT-5 mini, Llama-3.3, and Qwen2.5-Coder, to detect nine code smells in 30 real-world Java projects. Our goal is to understand which code smells LLMs successfully detect and when they fail. To perform our empirical evaluation, we first expanded a large dataset of 30 top-starred Java projects mined from GitHub [ 45 ] with LLM and human evaluations. We then created ground truth by having 76 developers manually inspect 268 code-smell candidates. Using the newly expanded dataset as the foundation of our investigation, we conducted three complementary analyses to assess the effectiveness of LLMs in detecting code smells. The main analysis evaluates how accurately each LLM identifies code smells relative to the human-validated ground truth derived from developer evaluations. Building on this foundation, the second analysis compares LLM performance with that of traditional static analysis tools to identify where LLMs offer advantages or show limitations. Finally, the third analysis examines whether combining both detector types using a voting-based strategy can further improve detection accuracy and coverage. Our results indicate that LLMs perform strongly for structurally simple smells, such as Large Class and Long Method , especially when multiple detection strategies are combined via voting. This proposed strategy that aggregates outputs of four LLMs and two static analysis tools maximizes recall and F1-score. For more subjective or context-dependent smells, such as Feature Envy and Refused Bequest , LLMs provide mixed results, although specialized tools or carefully selected individual LLMs remain the most effective. The results also show that no single strategy outperforms the others across all smells. In most cases, the optimal strategy varies depending on the smell type. In particular, the combined strategy consistently improves recall and provides robust detection of common, easily quantifiable smells, making it an attractive choice for practitioners seeking greater code-smell coverage. However, this combined strategy may generate more false positives for complex smells. As a result, the choice between a combined and an individual detection strategy should reflect the project’s tolerance for such trade-offs and its specific code quality goals. The contributions of this work can be summarized as follows. • We created a ground truth of code smells in 30 real-world Java projects mined from GitHub. This ground truth can be used to compare the effectiveness of different strategies to detect code smells, including other static analysis tools and LLMs beyond those evaluated in our study. • We demonstrate that a combined strategy offers the best recall and F1-score for detecting 5 out of 9 code smells. However, specialized tools or LLMs remain preferable for more subjective cases, underscoring the importance of selecting detection strategies based on the smell type and the desired trade-off between Recall and Precision.