Automated Bug Triaging using Instruction-Tuned Large Language Models
Kiana Kiashemshaki, Arsham Khosravani, Alireza Hosseinpour, Arshia Akhavan · 2025
Read Automated Bug Triaging using Instruction-Tuned Large Language Models by Kiana Kiashemshaki, Arsham Khosravani, Alireza Hosseinpour, Arshia Akhavan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-21156.txt.
Introduction
Kiana Kiashemshaki, Arsham Khosravani, Alireza Hosseinpour, and Arshia Akhavan automate the assignment of bug reports to developers in a 2025 study. In large projects, triaging each new issue by hand is slow and inconsistent. Sending a bug to the wrong person wastes days before real fixing starts.
Their lightweight framework fine-tunes a language model with small adapter weights and constrains its decoding to valid candidates. Tests on two open-source projects show strong shortlist quality, with the right developer usually inside the top few names, even when the single best guess is modest. Accuracy jumps on recent snapshots, favoring human-in-the-loop use.
Catalog record
| CREATOR | Kiana Kiashemshaki, Arsham Khosravani, Alireza Hosseinpour, Arshia Akhavan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-21156
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Automated Bug Triaging using Instruction-Tuned Large Language Models Kiana Kiashemshaki, Arsham Khosravani, Alireza Hosseinpour, Arshia Akhavan Abstract Manual bug triaging is slow, inconsistent, and increasingly unsustainable in large-scale software projects. We present an instruction-tuned, project-specific large language model (LLM) for automated bug triaging that integrates candidate-constrained decoding to guarantee valid developer assignments. Our framework fine-tunes a LoRA adapter on the 8B-parameter DeepSeek-R1-Distill-Llama-8B model using a conversational JSONL format derived from EclipseJDT and Mozilla issue-tracker data. The approach requires no handcrafted features, graph construction, or complex preprocessing, enabling rapid adaptation to new projects with minimal engineering effort. At inference time, candidate-constrained decoding restricts outputs to a known roster and produces ranked Top- K K developer recommendations. Using the same multi-year temporal window and filtering protocol as prior graph-based work, our model attains Top-1/Hit@10 of 0.156/0.475 on EclipseJDT and 0.013/0.753 on Mozilla . The method is compute-efficient leveraging 4-bit NF4 quantization and fully reproducible, with code, prompts, and per-issue predictions available for replication. These results show that lightweight, instruction-tuned LLMs can generate practical, high-quality shortlists for real-world bug triaging and motivate hybrid extensions to improve exact Top-1 assignment. 1 1 1 All code, prompts, and predictions will be released in an open repository. I Introduction In software development, when a new issue is reported through a project’s issue tracking system (e.g., Jira, Bugzilla, or GitHub), it is typically reviewed by a project maintainer also known as a triager who verifies the report, identifies duplicates, sets its severity, and assigns it to the appropriate developer [ 1 , 2 ] . This process, known as bug triaging , plays a critical role in ensuring timely and accurate issue resolution. When triaging is ineffective, it can lead to bug tossing , where issues are repeatedly reassigned before reaching the EclipseJDT project, some bugs have required more than a dozen reassignments and taken up to 100 days to resolve [ 1 ] , significantly increasing maintenance costs and delaying fixes [ 3 , 4 ] . Effective triaging is inherently challenging. It requires deep knowledge of the project, its codebase, and the expertise and availability of contributors [ 5 , 6 ] . These challenges are especially pronounced in large-scale open-source projects, where contributor roles are fluid and expertise is not always clearly defined, making manual bug assignment error-prone [ 2 ] . As software projects have scaled, the number of reported issues has grown dramatically, making manual bug triaging increasingly unsustainable. Projects like EclipseJDT, Mozilla, and Google Chromium receive thousands of new reports each month [ 7 , 2 ] . This volume has driven interest in automated bug triaging solutions, which can be broadly grouped into two categories [ 8 ] : (1) text classification-based approaches and (2) graph-based approaches. Text classification-based methods treat bug triaging as a supervised classification problem, using textual elements of bug reports (e.g., titles, descriptions) as features and the assigned developer as the label. Recent transformer-based models have shown strong results by capturing contextual semantics [ 9 , 10 , 11 , 12 ] . However, these models face key limitations: (i) incorporating all available bug text can introduce noise [ 13 ] , preventing convergence, and (ii) transformer models have token span limits that restrict how much context they can use [ 14 ] . Moreover, report structures vary across projects, impacting generalizability [ 2 , 13 , 9 ] . Despite these advances, relying solely on textual features can be insufficient. To address this, researchers have explored graph-based methods that model relationships between bugs, developers, and other artifacts [ 2 , 15 , 1 , 16 , 8 , 17 ] . Graph neural networks (GNNs) [ 18 ] learn node embeddings that capture structural, temporal, and semantic correlations, leveraging the insight that a developer’s expertise is reflected in the bugs they have resolved [ 1 ] . While effective, these methods can be computationally expensive, rely on partial graph sampling (e.g., random walks [ 1 ] ), and depend on explicit bug–developer links, which are often sparse or noisy [ 8 ] . A common limitation across both categories is their reliance on costly retraining and feature engineering to keep up with evolving developer activity and collaboration patterns [ 8 , 1 , 10 , 11 , 9 ] . They also assume that the listed assignee is the actual resolver, despite evidence of frequent discrepancies, leading to noisy labels [ 2 ] . While prior work has shown strong results with transformer encoders and graph-based models, LLMs for triaging are either used zero-shot or as components inside larger pipelines, with limited evidence on instruction-tuned, project-specific models that constrain outputs to valid assignees. The impact of such candidate-constrained decoding on ranked recommendations (Hit@K) has not been systematically studied under large, multi-year project datasets. In this work, we investigate the use of large language models (LLMs) for bug triaging. We construct datasets from the EclipseJDT and Mozilla projects, following the same multi-year temporal windows and filtering protocol introduced in NCGBT [ 8 ] . After cleaning and formatting them into JSONL conversational prompts, we fine-tune a LoRA adapter on top of the DeepSeek-R1-Distill-Llama-8B model. During inference, we apply candidate-constrained decoding to restrict outputs to valid assignee identifiers, producing a ranked Top- K list. We report Top-1 and Hit@K results for both projects and position these findings in direct comparison with prior text classification- and graph-based methods. Our results highlight the scalability and low engineering overhead of LLM-based triaging in practice. The remainder of this paper is organized as follows: Section II reviews related work, Section III details the methodology, Section IV presents the results, and Section VIII concludes the paper. Figure 1 illustrates the typical lifecycle of a bug. Fig. 1: States of a bug. Accurate triaging prevents repeated reassignment and delays. II Related Work Automatic bug triaging plays a pivotal role in modern software maintenance workflows by reducing manual effort and accelerating the resolution of reported defects [ 10 , 11 ] . Over the years, researchers have proposed a range of techniques to automate the bug assignment process. In this section, we categorize prior work into text classification-based , graph-based , and LLM-based methods, and discuss each in detail. Text Classification-Based Methods These approaches typically treat elements of bug reports (e.g., titles and descriptions) as structured text input. This text is transformed into feature representations and used to train machine learning (ML) or deep learning (DL) models, with developers treated as classification labels. Earlier work relied on classic ML techniques such as Naive Bayes [ 19 ] , Support Vector Machines (SVM) [ 20 ] , Term Frequency Inverse Document Frequency (TF–IDF) [ 21 ] , and Latent Dirichlet Allocation (LDA) [ 22 ] . However, these methods have limited semantic understanding of textual features and struggle to capture the contextual nuances of bug report information [ 15 , 2 , 13 , 9 ] . Recent studies address this limitation by leveraging deep learning models capable of capturing context-dependent semantic features [ 9 , 10 , 11 ] . Lee et al. [ 11 ] proposed a lightweight framework that fine-tunes pre-trained models like RoBERTa [ 23 ] and DeBERTa [ 24 ]