MPLinker: Multi-template Prompt-tuning with Adversarial Training for Issue-commit Link Recovery

Bangchao Wang, Yang Deng, Ruiqi Luo, Peng Liang, Tingting Bi · 2025

Read MPLinker: Multi-template Prompt-tuning with Adversarial Training for Issue-commit Link Recovery by Bangchao Wang, Yang Deng, Ruiqi Luo, Peng Liang, Tingting Bi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-19026.txt.

Introduction

Software teams lose track of which code change fixed which reported issue, and rebuilding those links by hand is slow. Bangchao Wang and co-authors present a tool in 2025 that recovers issue-to-commit links more accurately. Older methods turn the task into plain sorting and miss much of the meaning stored in pre-trained language models.

Their tool, MPLinker, reframes the task as filling in a blank, using several prompt templates to draw out the model's knowledge. Extra training with crafted hard examples makes it more robust and less prone to overfitting. On six open-source projects, it beat prior methods on every measure, with scores in the mid-nineties. The authors share their code and data for others to check.

Catalog record

CREATORBangchao Wang, Yang Deng, Ruiqi Luo, Peng Liang, Tingting Bi
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE67 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-19026

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MPLinker: Multi-template Prompt-tuning with Adversarial Training for Issue-commit Link Recovery Bangchao Wang, Yang Deng, Ruiqi Luo, Peng Liang, Tingting Bi Abstract In recent years, the pre-training, prompting and prediction paradigm, known as prompt-tuning, has achieved significant success in Natural Language Processing (NLP). Issue-commit Link Recovery (ILR) in Software Traceability (ST) plays an important role in improving the reliability, quality, and security of software systems. The current ILR methods convert the ILR into a classification task using pre-trained language models (PLMs) and dedicated neural networks. these methods do not fully utilize the semantic information embedded in PLMs, resulting in not achieving acceptable performance. To address this limitation, we introduce a novel paradigm: Multi-template Prompt-tuning with adversarial training for issue-commit Link recovery (MPLinker). MPLinker redefines the ILR task as a cloze task via template-based prompt-tuning and incorporates adversarial training to enhance model generalization and reduce overfitting. We evaluated MPLinker on six open-source projects using a comprehensive set of performance metrics. The experiment results demonstrate that MPLinker achieves an average F1-score of 96.10%, Precision of 96.49%, Recall of 95.92%, MCC of 94.04%, AUC of 96.05%, and ACC of 98.15%, significantly outperforming existing state-of-the-art methods. Overall, MPLinker improves the performance and generalization of ILR models, and introduces innovative concepts and methods for ILR. The replication package for MPLinker is available at https://github.com/WTU-intelligent-software-development/MPLinker . Journal: Journal of Systems and Software 1 Introduction Software Traceability (ST) activities link different software artifacts produced in software development to enhance developers’ observability in practice, which supports diverse software engineering activities such as impact analysis, security assurance, and safety validation of software systems [ 1 ] . In particular, trace links between issues and commits (Issue-commit Link Recovery (ILR)) play a key role in bug localization [ 2 ] , defect prediction [ 3 ] , and other software maintenance tasks. In software development, developers commonly establish links between issues and commits by recording the relevant commit after resolving an issue. However, this process is not mandatory, relying on developers’ voluntary actions, and manual link creation is time-intensive and demanding. As a result, developer negligence or other factors during the development process may lead to numerous missing links, impacting the overall quality and reliability of software systems. To alleviate the above problems, researchers have increasingly focused on studying automated ILR methods, particularly learning-based ILR methods [ 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 ] . Ruan et al. [ 8 ] introduced word embeddings and Recurrent Neural Networks (RNN) to capture semantic information from issues and commits. Similarly, Xie et al. [ 9 ] employed neural networks for word embeddings and integrated knowledge graph mining to extract semantic context from code. However, these methods require large-scale data and do not apply to projects with poor quality or small scales. With advancements in Natural Language Processing (NLP), pre-trained language models (PLMs) like BERT and RoBERTa have been applied to software development tasks for artifact representation [ 12 ] [ 13 ] . Existing fine-tuning-based ILR methods leverage the text understanding capabilities of PLMs by fine-tuning them to perform downstream ILR tasks. These pre-trained models are trained on large-scale, general corpora, giving them strong language understanding capabilities. For example, RoBERTa is trained using masked language model (MLM), while BERT incorporates both MLM and next sentence prediction during pre-training (as illustrated in Figure 1 .a and Figure 1 .b, respectively). Figure 1 .c outlines the conventional fine-tuning approach for ILR, where PLMs serve as encoders for commits and issues. Additional neural network layers, such as fusion layers or fully connected layers, are often introduced, along with task-specific loss functions, to train the model effectively [ 1 , 14 , 15 ] . However, a significant challenge with these methods arises from the mismatch between the pre-training tasks and the downstream ILR objectives. This misalignment can hinder the model’s ability to fully capture the semantic relationships between issues and commits, while also limiting the effective use of generalized knowledge embedded in PLMs [ 16 , 17 ] . Recently, a novel pre-training, prompting, and prediction paradigm, known as prompt-tuning, has achieved good performance in various tasks, including disease classification [ 18 ] , relation extraction [ 19 ] , and news recommendation [ 16 ] . This new paradigm reorganizes downstream tasks into PLM tasks by prompting templates and label words [ 19 , 20 ] . Inspired by this paradigm, we propose a Multi-template Prompt-tuning method with adversarial training for issue-commit Link recovery (MPLinker). As shown in Figure 1 .d, MPLinker integrates various templates to enhance the model’s generalization and adopts adversarial training to mitigate the model overfitting. The input to the model consists of commit, issue, templates (e.g., “The link is \left[MASK\right] "), and label words (“correct", “incorrect", etc.). The MLM in PLM then predicts “ \left[MASK\right] " to determine whether or not there are links. The following Research Questions (RQs) are proposed to investigate the practicality and effectiveness of the prompt-tuning paradigm in the ILR task: RQ1: Which prompt-tuning architectures can perform better in MPLinker for the ILR task? RQ1 compares the effectiveness of three prompt-tuning architectures, Single-template Prompt, Multi-template Prompt, and CLSPrompt in ILR. The experiment evaluated how well MPLinker performs ILR using MLM for cloze tasks. Additionally, the experiments confirmed that our proposed multi-template prompt can leverage the advantages of each template to improve overall ILR performance, reduce the occurrence of outliers, and achieve more stable and reliable results. RQ2: Which PLMs are most effective for prompt-tuning in MPLinker for ILR task? RQ2 compared RoBERTa, BERT, and GPT-2 to verify the importance of consistency between LLM training and downstream ILR tasks. The results indicate that RoBERTa, which utilizes MLM for cloze tasks during training, performs best among all LLMs. BERT performs better than GPT-2, although it has less stability than GPT-2. RQ3: How does adversarial training contribute to the MPLinker? RQ3 evaluated the contribution of adversarial training to the performance of MPLinker in ILR through ablation experiments. The results demonstrate that incorporating adversarial training significantly improves the performance of ILR by enhancing link diversity and mitigating model overfitting. RQ4: How effective is MPLinker in ILR compared to state-of-the-art methods? RQ4 compared MPLinker with the state-of-the-art methods (FRLink [ 5 ] , DeepLink [ 8 ] , hybrid-linker [ 7 ] and BTLink [ 14 ] ) on the ILR task. The results show that MPLinker significantly enhances ILR performance, revealing prompt-tuning potential for ILR task. In summary, the main contributions of this work are the following: 1. To the best of my knowledge, it is the first attempt to convert the ILR task into a cloze task through templates and propose a prompt-tuning-based ILR method. Moreover, this paper presents three prompt strategies for ILR and evaluates the impact of three PLMs on ILR performance. 2. MPLinker with multi-template is proposed to integrate predictions from various templates to determine whether issue and commit are linked. Since there is no prior knowledge about which template performs best, MPLinker with multi-template mitigates biases introduced by different templates, leading to fairer and more reliable results. 3. MPLinker incorporates adversarial training into the process by introducing noise into the text to perturb the issue and commit

Read the complete text