Towards Knowledge Alignment in Code LLMs: Contrastive Unlearning for Evolving APIs

Huy Q. Tran, Dang H. Vu, Tuyen N. Dinh, Anh H. D. Nguyen, Anh N. H. Vu, Anh M. T. Bui ∗, Phuong T. Nguyen · 2026

Read Towards Knowledge Alignment in Code LLMs: Contrastive Unlearning for Evolving APIs by Huy Q. Tran, Dang H. Vu, Tuyen N. Dinh, Anh H. D. Nguyen, Anh N. H. Vu, Anh M. T. Bui ∗, Phuong T. Nguyen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-30810.txt.

Introduction

Huy Tran and six colleagues fix a bad habit of code-generating language models: calling library functions that no longer exist. Models trained on old code keep emitting deprecated application programming interfaces, breaking software. Fine-tuning is too blunt when only a sliver of knowledge must change. Unlearning methods can suppress outdated calls, but they rarely teach the model what to write instead, leaving mismatched or half-finished code.

Their method, CURE, performs contrastive unlearning. It jointly discourages each deprecated interface and encourages its valid replacement, steering the model across the swap rather than silencing the old habit. On a recent benchmark of deprecated interfaces, CURE reduced outdated usage and improved correct replacement at once. It substantially outperformed two state-of-the-art baselines. General code generation ability stayed intact. The authors argue suppression plus replacement is the right shape for keeping models aligned with evolving libraries.

Catalog record

CREATORHuy Q. Tran, Dang H. Vu, Tuyen N. Dinh, Anh H. D. Nguyen, Anh N. H. Vu, Anh M. T. Bui ∗, Phuong T. Nguyen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2606.30810

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Towards Knowledge Alignment in Code LLMs: Contrastive Unlearning for Evolving APIs Huy Q. Tran, Dang H. Vu, Tuyen N. Dinh, Anh H. D. Nguyen, Anh N. H. Vu, Anh M. T. Bui ∗, Phuong T. Nguyen Abstract Large Language Models (LLMs) have recently achieved strong performance in code generation. However, due to knowledge cut-off and the rapid evolution of software libraries, they often generate deprecated API usages that lead to unreliable and incompatible code. Existing fine-tuning methods lack selectivity when only a small portion of model knowledge requires modification. Recent model-level approaches, such as machine unlearning and model editing, offer a promising direction for modifying parametric knowledge. However, their use for deprecated API mitigation remains largely unexplored. Moreover, existing methods primarily suppress outdated APIs, but do not explicitly steer models toward correct replacements, often leading to mismatched or incomplete generations. To address this limitation, we developed CURE , a contrastive unlearning approach that shifts unlearning from purely suppressing outdated knowledge to explicitly promoting correct API replacements. Concretely, CURE jointly discourages deprecated APIs while encouraging their valid alternatives, enabling more reliable adaptation to evolving software libraries. The experiments on recent deprecated API benchmark dataset show that CURE not only reduces deprecated API usage but also improves correct API replacement, while preserving general code generation performance. CURE substantially outperforms two SOTA baselines with respect to different quality metrics. These findings highlight the importance of combining suppression with replacement when adapting LLMs to evolving software ecosystems. I Introduction Large Language Models (LLMs) have recently shown strong performance in software engineering tasks, particularly code generation and code completion [ 22 , 2 , 11 , 29 , 12 ] . These capabilities are enabled by pretraining on massive public code corpora [ 7 ] , which also exposes models to outdated coding patterns, including deprecated APIs frequently appearing in historical repositories [ 28 ] . As a result, Code LLMs may continue to generate obsolete API calls that no longer align with modern codebases, reducing code reliability. This issue stems not only from knowledge cutoff, but also from outdated patterns embedded in the model’s parametric memory [ 34 ] , which can bias Code LLMs toward deprecated APIs even when valid alternatives exist. In rapidly evolving libraries, such behavior may introduce compatibility issues and increase maintenance effort [ 33 , 16 ] . Although fine-tuning can update model knowledge, it is often inefficient for localized API-level modifications since it globally updates model parameters. The key challenge is therefore to selectively remove outdated API knowledge while preserving the model’s general code generation capability [ 18 ] . Existing efforts to mitigate outdated API usage follow two main directions. Inference-time strategies, e.g., prompt augmentation and output-level API replacement [ 28 ] , attempt to steer model outputs toward updated APIs without modifying the internal knowledge. However, these approaches assume that each input context can be accurately mapped to a specific replacement API, which may not be realistic in real-world scenarios. More direct model-level approaches, including model editing [ 19 , 20 ] and machine unlearning [ 8 , 15 , 5 ] , modify the model’s parametric memory to suppress undesirable knowledge. However, their application to deprecated API mitigation remains limited. Lin et al. [ 20 ] propose a dynamic editing approach that updates the model whenever a deprecated API is generated. This strategy operates locally and may not generalize well to unseen contexts. Recent approaches [ 15 , 5 ] mainly focus on suppressing undesirable knowledge, without explicitly guiding the model toward correct API replacements required for evolving libraries. In this paper, we propose CURE , a C ontrastive U nlearning approach for deprecated API RE placement. The key idea is to shift unlearning from a purely suppressive process to a replacement-oriented mechanism. Specifically, CURE builds upon a parametric machine unlearning method, such as PROD [ 15 ] or SimNPO [ 9 ] , to reduce the likelihood of generating deprecated APIs. On top of this, we introduce a contrastive objective that explicitly promotes correct API alternatives, enabling the model not only to forget outdated APIs but also to adopt valid replacements. We evaluate CURE on a recent deprecated API benchmark across four representative Code LLMs: DeepSeek-Coder-1.3B [ 10 ] , StarCoder2-3B [ 21 ] , CodeLlama-7B [ 25 ] and CodeGen-2B [ 23 ] . Experimental results show that CURE not only reduces deprecated API generation, but also improves correct API replacement compared to standard unlearning methods. Evaluation on HumanEval [ 4 ] further indicates that CURE preserves general code generation capability without noticeable degradation. The main contributions of this work are as follows: • We propose CURE , a contrastive unlearning framework for mitigating deprecated API usage in Code LLMs through replacement-oriented learning. • We introduce a contrastive objective that explicitly aligns deprecated APIs with their corresponding updated alternatives under the same code context. • Experiments on a recent deprecated API benchmark and HumanEval [ 4 ] show that CURE consistently reduces deprecated API generation, improves correct API replacement, and preserves general code generation capability compared to standard unlearning methods. • We release our replication package to support future research [ 1 ] . II Related work Modern software systems evolve continuously, causing APIs to be frequently revised and deprecated in favor of improved alternatives [ 17 ] . This is particularly challenging for Code LLMs, whose static training corpora often encode outdated API knowledge, leading to deprecated API generation. Wang et al. [ 27 ] report that 37.4% of API predictions produced by GPT-3.5 involve deprecated APIs. Existing solutions mainly follow two directions: inference-level methods such as REPLACEAPI and INSERTPROMPT [ 27 ] , modifying outputs without updating model, and model-level approaches including model editing [ 20 ] and reinforcement learning [ 30 ] , which directly adapt knowledge within the model. Closely related to these efforts, machine unlearning has recently emerged as a promising direction for removing undesirable knowledge from LLMs [ 3 ] . Existing approaches can be broadly categorized into exact and approximate unlearning [ 5 , 13 ] . While exact methods retrain models from scratch to ensure complete data removal, approximate methods provide a more practical alternative through gradient-based parameter updates, including GA [ 13 ] , DPO-based unlearning [ 24 ] , NPO [ 32 ] , and SimNPO [ 9 ] . More recently, PROD [ 15 ] introduces token-level suppression for precise code unlearning, while CODEEraser [ 5 ] selectively removes sensitive memorized code segments. Despite these advances, applying unlearning techniques to mitigate deprecated API usage in Code LLMs remains largely underexplored. III Proposed Method In this section, we first formulate the unlearning problem for mitigating deprecated APIs, then present our proposed approach CURE , built upon a gradient-based parametric unlearning method by incorporating a contrastive objective to promote correct replacements by contrasting deprecated and valid API usages under the same context. III-A Problem Formulation We formulate unlearning as a post-training procedure that aims to reduce the model’s tendency to generate deprecated APIs by modifying its parametric memory. Let \pi_{\theta} denote a language model parameterized by \theta . We denote \mathcal{D}_{f}=\{x_{i},y^{-}_{i}\}_{i=1}^{N} as the forgetting set, which consists of code samples associated with deprecated APIs.

Read the complete text