Variability-Aware Detection and Repair of Compilation Errors Using Foundation Models in Configurable Systems

Rohit Gheyi 3 , Lucas Albuquerque 3 ,Márcio Ribeiro 2 ,Eduardo Almeida 6 ,Danyllo Albuquerque 4 , Mirko Perkusich 4 · 2026

Read Variability-Aware Detection and Repair of Compilation Errors Using Foundation Models in Configurable Systems by Rohit Gheyi 3 , Lucas Albuquerque 3 ,Márcio Ribeiro 2 ,Eduardo Almeida 6 ,Danyllo Albuquerque 4 , Mirko Perkusich 4 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-16755.txt.

Introduction

Rohit Gheyi and a team of five co-authors hunt build flaws that hide in rare setups. Big software can switch parts on or off by flags. Some mixes of flags break the build at compile time. Tests oft miss those rare mixes. Their 2026 preprint pairs big AI models with this hunt.

The models scan code with its flag rules in view. They flag mixes that are apt to break. They then draft fixes for the breaks they find. This could save teams from late, costly build shocks. It suits cars, phones, and cloud tools with many flag paths.

Catalog record

CREATORRohit Gheyi 3 , Lucas Albuquerque 3 ,Márcio Ribeiro 2 ,Eduardo Almeida 6 ,Danyllo Albuquerque 4 , Mirko Perkusich 4
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE80 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-16755

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Variability-Aware Detection and Repair of Compilation Errors Using Foundation Models in Configurable Systems Rohit Gheyi 3 , Lucas Albuquerque 3 ,Márcio Ribeiro 2 ,Eduardo Almeida 6 ,Danyllo Albuquerque 4 , Mirko Perkusich 4 Abstract Modern software systems often rely on conditional compilation to support optional features and multiple deployment scenarios. In configurable systems, compilation errors may arise only under specific combinations of features, remaining hidden during development and testing. Such variability-induced errors are difficult to detect in practice, as traditional compilers analyze only a single configuration at a time, while existing variability-aware tools typically require complex setup and incur high analysis costs. In this article, we present an empirical study on the use of foundation models to detect and fix compilation errors caused by feature variability in configurable C systems. We evaluate two models, namely GPT-OSS-20B and Gemini 3 Pro , and compare them with TypeChef , a state-of-the-art variability-aware parser. Our evaluation considers two complementary settings: 5,000 small configurable systems designed to systematically exercise variability-induced compilation behavior, comprising both systems with and without compilation errors, and 14 real-world GitHub commits, as well as an additional set of mutation testing scenarios (42). Our results show that foundation models can effectively identify variability-induced compilation errors. On small configurable systems, GPT-OSS-20B achieved a precision of 0.97, recall of 0.90, and accuracy of 0.94, substantially increasing detection coverage compared to TypeChef , and exhibiting performance comparable to Gemini 3 Pro . For compilation error repair, GPT-OSS-20B produced compilable fixes in over 70% of the cases. In the analysis of real commits, ChatGPT-5.2 detected all injected faults except for two cases and identified a potential real compilation bug in a Linux commit with more than 1,000 modified lines. Overall, our findings indicate that current state-of-the-art foundation models provide a practical and low-effort complement to traditional variability-aware analyses, offering accurate detection and human-readable fixes, while still facing challenges related to execution time, cost, and the limited context window when analyzing large configurable systems. I Introduction In configurable systems, compilation errors may arise only under specific combinations of features, remaining latent and undetected during standard development and testing. As a result, ensuring compilation correctness becomes both difficult and costly in practice. This challenge intensifies when variability annotations are poorly structured or evolve over time, a common situation in long-lived systems [ 1 ] . Prior studies have shown that such variability-related complexity negatively impacts code quality, maintainability, and developer productivity [ 2 , 3 ] . Although the use of conditional compilation with #ifdef directives is sometimes criticized, it remains widely adopted in practice due to its flexibility and low overhead for managing variability [ 4 ] . However, traditional compilers are fundamentally limited in this context, as they analyze only a single configuration at a time. Consequently, the literature reports a large number of variability-related bugs in widely used configurable systems [ 5 , 6 , 7 , 8 , 9 ] . To mitigate these issues, variability-aware parsers such as TypeChef [ 10 ] and SuperC [ 11 ] have been proposed to analyze multiple configurations simultaneously. Despite their effectiveness, these tools require sophisticated setup, incur substantial analysis costs, and still fail to detect certain classes of compilation errors. Recently, large language models (LLMs) have demonstrated strong capabilities in program understanding, code generation, and software review [ 12 , 13 , 14 ] . These advances have sparked growing interest in applying LLMs to software engineering tasks such as testing, debugging, and program analysis [ 15 , 16 ] . Yet, their role in compilation-related tasks remains underexplored. A recent survey provides a systematic overview of LLM-enabled compilation, proposing a taxonomy of approaches and highlighting their potential to broaden compiler development [ 17 ] . However, to the best of our knowledge, there is still no systematic empirical study evaluating whether LLMs can effectively detect and help fix compilation errors induced by configuration variability in C systems [ 17 ] . In this article, we present an empirical investigation into the use of foundation models for variability-aware compilation analysis in configurable C systems. Specifically, we study to what extent such models can detect and fix compilation errors caused by feature variability, a problem that has traditionally been addressed only through compiler-based and variability-aware static analyses. First, we report on our experience using ChatGPT-5.2 to synthesize a controlled benchmark of 5,000 small configurable systems, comprising both systems with and without variability-induced compilation errors. We make this dataset publicly available to support benchmarking, replication, and future research on variability-aware analysis and learning-based approaches [ 18 ] . We then use this benchmark to assess model performance on configurable systems designed to systematically exercise variability-induced compilation behavior. Second, we move to a more realistic scenario by analyzing 14 Git commit diffs from real-world C projects on GitHub and evaluating whether foundation models can assist developers during software evolution. To further strengthen this analysis, we apply mutation testing [ 19 , 20 ] to 42 real-world commits, injecting compilation faults to assess the models’ ability to detect errors introduced by code changes. We evaluate both open-weight and proprietary foundation models, namely GPT-OSS-20B and Gemini 3 Pro , on a dataset of 5,000 configurable systems, and compare their performance against TypeChef , a state-of-the-art variability-aware parser. Our results show that foundation models can effectively detect and, in many cases, fix compilation errors induced by variability. In the controlled evaluation on small configurable systems, GPT-OSS-20B achieved a good performance, with a precision of 0.97, recall of 0.90, accuracy of 0.94, and F 1 -score of 0.93, substantially expanding the coverage of detected errors compared to TypeChef . The proprietary model Gemini 3 Pro , evaluated on a 7.1% sample of the dataset, showed performance comparable to the open-weight model, achieving high precision (1.00) and recall (0.88), which translated into an overall accuracy of 0.94 and an F 1 -score of 0.93. With respect to error repair, GPT-OSS-20B produced correct fixes in 70.2% of the cases where a fix was required according to the compiler-based baseline. Considering all cases in which the model generated code, the resulting systems compiled successfully in 73.9% of the instances. Beyond small systems, our change-based analysis on 56 real-world GitHub commits successfully detected all injected compilation faults introduced via mutation testing except for two cases using ChatGPT-5.2 . Moreover, the models identified a potential real compilation bug in a Linux commit involving more than 1,000 modified lines of code. From a practical perspective, our study highlights important trade-offs. While foundation models require minimal setup effort, naturally evolve with the programming language, and provide human-readable explanations and fixes, they also face challenges related to execution time, monetary cost, and limitations on the context window. Consequently, we argue that foundation models should be viewed as a complementary technique rather than a replacement for traditional variability-aware analyses. In summary, the main contributions of this article are as follows: • Empirical evaluation of foundation models for variability-aware compilation. We conduct an empirical study investigating to what extent GPT-OSS-20B and Gemini 3 Pro can detect and fix compilation errors induced by feature variability in configurable C systems (Sections IV and V ). • Change-based analysis on real-world commits. We perform a

Read the complete text