Byam: Fixing Breaking Dependency Updates with Large Language Models

Frank Reyes, May Mahmoud, Federico Bono, Sarah Nadi, Benoit Baudry, Martin Monperrus · 2025

Read Byam: Fixing Breaking Dependency Updates with Large Language Models by Frank Reyes, May Mahmoud, Federico Bono, Sarah Nadi, Benoit Baudry, Martin Monperrus (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-07522.txt.

Introduction

Frank Reyes and five co-authors use language models to fix code broken by dependency updates, in work from 2025. When a library changes its interface, client builds fail in tedious ways. Their system, Byam, prompts models with build output, the breaking change itself and the faulty line, then asks for a repair, tested on a Java benchmark of such breaks.

Five models were compared at build, file and single-error levels. The best, a reasoning model, fully repaired about a quarter of broken builds and fixed over three quarters of individual compile errors. Richer prompts clearly helped. The authors see real potential for automated upkeep of client code.

Catalog record

CREATORFrank Reyes, May Mahmoud, Federico Bono, Sarah Nadi, Benoit Baudry, Martin Monperrus
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE61 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-07522

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Byam: Fixing Breaking Dependency Updates with Large Language Models Frank Reyes, May Mahmoud, Federico Bono, Sarah Nadi, Benoit Baudry, Martin Monperrus Abstract Application Programming Interfaces (APIs) facilitate the integration of third-party dependencies within the code of client applications. However, changes to an API, such as deprecation, modification of parameter names or types, or complete replacement with a new API, can break existing client code. These changes are called breaking dependency updates ; It is often tedious for API users to identify the cause of these breaks and update their code accordingly. In this paper, we explore the use of Large Language Models (LLMs) to automate client code updates in response to breaking dependency updates. We evaluate our approach on the BUMP dataset, a benchmark for breaking dependency updates in Java projects. Our approach leverages LLMs with advanced prompts, including information from the build process and from the breaking dependency analysis. We assess effectiveness at three granularity levels: at the build level, the file level, and the individual compilation error level. We experiment with five LLMs: Google Gemini-2.0 Flash, OpenAI GPT4o-mini, OpenAI o3-mini, Alibaba Qwen2.5-32b-instruct, and DeepSeek V3. Our results show that LLMs can automatically repair breaking updates. Among the considered models, OpenAI’s o3-mini is the best, able to completely fix 27% of the builds when using prompts that include contextual information such as the buggy line, API differences, error messages, and step-by-step reasoning instructions. Also, it fixes 78% of the individual compilation errors. Overall, our findings demonstrate the potential for LLMs to fix compilation errors due to breaking dependency updates, supporting developers in their efforts to stay up-to-date with changes in their dependencies. ∎ 1 Introduction Modern software development heavily relies on third-party software libraries to improve developers’ productivity and time-to-market. The set of libraries used by a client project are referred to as the dependencies of that project. Developers access a library’s functionality through its provided Application Programming Interfaces (APIs). These APIs evolve over time to introduce new functionality, fix bugs, or address security vulnerabilities ( Lamothe et al., 2021 ) . The evolution of library APIs can cause client code to fail. This is due to modifications such as changes in a parameter name or type, method deprecation, or even complete removal of classes and modules ( Lamothe et al., 2021 ; Reyes et al., 2024b ; Reyes et al., 2024a ) . Failures that are due to API updates are referred to as breaking dependency updates . When a new version of a dependency results in a breaking dependency update, the developers using the library (i.e., the client developers ) have to figure out how to update their code to fix the errors they face. This is known to be a tedious and time consuming maintenance task ( Zhang et al., 2022 ; Ochoa et al., 2022 ) . (a) Difference in Maven build file, causing a breaking update. ⬇ 113 … 114 // create an instance of fop factory 115 FopFactory fopFactory = FopFactory . newInstance (); 116 // a user agent is needed for transformation 117 FOUserAgent foUserAgent = fopFactory . newFOUserAgent (); 118 … (b) The broken code with compilation failure after breaking update. [ERROR] /billy/billy-gin/src/main/java/com/premiumminds/billy/gin/services/ impl/pdf/FOPPDFTransformer.java:[115,43] no suitable method found for newInstance(no arguments) method org.apache.fop.apps.FopFactory.newInstance(org.apache.fop.apps .FopFactoryConfig) is not applicable (actual and formal argument lists differ in length) method org.apache.fop.apps.FopFactory.newInstance(java.io.File) is not applicable (actual and formal argument lists differ in length) method org.apache.fop.apps.FopFactory.newInstance(java.net.URI) is not applicable (actual and formal argument lists differ in length) method org.apache.fop.apps.FopFactory.newInstance(java.net.URI,java. io.InputStream) is not applicable (actual and formal argument lists differ in length) (c) Compilation error information from the logs. Figure 1 : Example of a breaking dependency update when updating org.apache.xmlgraphics (dependency) from version 1.0 (old version) to 2.2 (new version) in the project billy . Figure 1 shows an example of a breaking dependency update in project billy 1 1 1 https://github.com/premium-minds/billy/pull/300 , where the org . apache . xmlgraphics dependency was updated from version 1.0 to 2.2 in commit 36859167815292 f279e570d39dd2ddbcf1622dc6 . The org . apache . xmlgraphics library has a method newInstance () , which has zero parameters in version 1.0 and was changed in the new version 2.0. This led to a compilation error in billy , as shown at the bottom of the figure. Breaking dependency updates are common in software projects ( Brito et al., 2018b ) , and often discourage developers from upgrading their dependencies ( Dietrich et al., 2019 ; Ochoa et al., 2022 ; Venturini et al., 2023 ) . However, keeping dependencies up to date is crucial for security, as outdated libraries may contain vulnerabilities ( Brito et al., 2018b ; Xavier et al., 2017 ; Brito et al., 2020 ; Larios Vargas et al., 2020 ; Salza et al., 2018 ) . Most prior work on breaking dependency updates focused on analyzing their frequency, causes, and identifying breaking changes in library code ( Brito et al., 2018b ; Xavier et al., 2017 ; Brito et al., 2020 ; Brito et al., 2018a ; Mujahid et al., 2020 ) . There is scarce work on automating client code fixes. Xing et al. ( Xing and Stroulia, 2007 ) proposed rule-based program transformation techniques, limited to predefined patterns and small adaptation examples. Other approaches improved rule synthesis by incorporating additional library usage examples ( Brito et al., 2018a ) or recommendations for API method replacements based on source code analysis ( Dagenais and Robillard, 2009 ) . These approaches struggle to handle the widely diverse causes of breaking updates, such as method signature alterations, and removed classes. This is the problem we address in this paper. In this work, we explore the use of Large Language Models (LLMs) to automate the generation of a patch to fix a breaking dependency update. LLMs are neural networks trained on large datasets of text and code ( Vaswani, 2017 ; Chang et al., 2024 ) . They have shown great effectiveness in software engineering tasks like code generation and bug fixing ( Nam et al., 2024 ; Yin et al., 2024 ; Hidvégi et al., 2024 ) . We design and evaluate Byam, a pipeline that leverages LLMs to update client code in response to breaking dependency updates. We focus on breaking dependency updates that lead to compilation errors, which are the most common types of causes for breakage ( Reyes et al., 2024b ; Reyes et al., 2024a ) . Byam analyzes compilation errors caused by a dependency update, formulates prompts that incorporate relevant build and dependency information, and queries an LLM to generate updated code. We assess the effectiveness of Byam using the BUMP dataset ( Reyes et al., 2024b ) , which provides reproducible cases of breaking dependency updates in Java projects. Our experiments systematically evaluate different LLMs, prompt design strategies, and information granularity levels. We evaluate Byam with five different LLMs: Google Gemini-2.0 Flash, OpenAI GPT4o-mini, OpenAI o3-mini, Alibaba Qwen2.5-32b-instruct, and DeepSeek V3. We assess its effectiveness at three levels of granularity: build, file, and individual error. At the build level, we measure the number of fully fixed builds, those still failing due to compilation errors, and cases where compilation was resolved but test failures emerged. At the

Read the complete text