Multi-Artifact Analysis of Self-Admitted Technical Debt in Scientific Software CCS: Software and its engineering Maintaining software CCS: Software and its engineering Software evolution

Eric L. Melin, Nasir U. Eisty, Gregory Watson, Addi Malviya-Thakur · 2026

Read Multi-Artifact Analysis of Self-Admitted Technical Debt in Scientific Software CCS: Software and its engineering Maintaining software CCS: Software and its engineering Software evolution by Eric L. Melin, Nasir U. Eisty, Gregory Watson, Addi Malviya-Thakur (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-10850.txt.

Introduction

Eric L. Melin, Nasir U. Eisty, Gregory Watson, and Addi Malviya-Thakur study self-admitted technical debt in scientific software. Developers leave notes in code admitting shortcuts; that is the debt. In scientific code, such debt can threaten the validity and reproducibility of results. A shortcut in a solver or a pipeline can bend a finding without anyone noticing.

The study identifies and sorts this debt across several artifacts, not just code comments. It defines scientific debt, a domain-specific form, and tests how far standard debt categories capture it. The results matter for labs that build on shared code. Knowing where debt hides is the first step to paying it down before it corrupts results.

Catalog record

CREATOREric L. Melin, Nasir U. Eisty, Gregory Watson, Addi Malviya-Thakur
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-10850

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Multi-Artifact Analysis of Self-Admitted Technical Debt in Scientific Software CCS: Software and its engineering Maintaining software CCS: Software and its engineering Software evolution Eric L. Melin, Nasir U. Eisty, Gregory Watson, Addi Malviya-Thakur Abstract . Context: Self-admitted technical debt (SATD) occurs when developers acknowledge shortcuts in code. In scientific software (SSW), such debt poses unique risks to the validity and reproducibility of results. Objective: This study aims to identify, categorize, and evaluate scientific debt , a specialized form of SATD in SSW, and assess the extent to which traditional SATD categories capture these domain-specific issues. Method: We conduct a multi-artifact analysis across code comments, commit messages, pull requests, and issue trackers from 23 open-source SSW projects. We construct and validate a curated dataset of scientific debt , develop a multi-source SATD classifier, and conduct a practitioner validation to assess the practical relevance of scientific debt . Results: Our classifier performs strongly across 900,358 artifacts from 23 SSW projects. SATD is most prevalent in pull requests and issue trackers, underscoring the value of multi-artifact analysis. Models trained on traditional SATD often miss scientific debt , emphasizing the need for its explicit detection in SSW. Practitioner validation confirmed that scientific debt is both recognizable and useful in practice. Conclusions: Scientific debt represents a unique form of SATD in SSW that that is not adequately captured by traditional categories and requires specialized identification and management. Our dataset, classification analysis, and practitioner validation results provide the first formal multi-artifact perspective on scientific debt , highlighting the need for tailored SATD detection approaches in SSW. Notice: This manuscript has been authored by UT-Battelle, LLC, under contract DE-AC05-00OR22725 with the US Department of Energy (DOE). The US government retains and the publisher, by accepting the article for publication, acknowledges that the US government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for US government purposes. DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (https://www.energy.gov/doe-public-access-plan). 1. Introduction In software development, the saying “done is better than perfect” highlights the unavoidable balance between maintaining software quality and meeting deadlines. This mindset often results in compromises, where speed takes priority over robustness. Technical debt (TD) ( 11 ) arises when practitioners choose expedient solutions rather than thorough and maintainable ones. Development teams often work under pressure due to limited resources, strict timelines, and evolving requirements ( 16 ) . Hence, developers may intentionally or unintentionally incur TD to expedite development, enabling teams to meet these deadlines, often at the cost of compromising the long-term integrity of the software system ( 34 ) . Similarly, self-admitted technical debt (SATD) ( 39 ) , a subset of TD, refers to instances where developers explicitly acknowledge these compromises, generally documented in code comments, commit messages, pull requests, or issue tracker entries ( 30 ) . Based on the content of the acknowledged compromise, SATD instances can be categorized into numerous types ( 3 ) . While prior studies of SATD have largely examined general-purpose software systems, scientific software (SSW) presents unique challenges that warrant specialized attention ( 19 ; 37 ) . SSW supports critical domains such as climate modeling, energy systems, nuclear simulations, and biomedical research, where accuracy and reliability are essential for scientific discovery, reproducibility, policy decisions, and safety-critical outcomes. Unlike traditional software, shortcomings in scientific code may compromise not only maintainability but also the validity of scientific results. Pressures stemming from evolving scientific requirements, reliance on complex numerical methods, and the demand to rapidly disseminate findings create conditions for scientific debt ( 7 ) , a specialized form of SATD. Consequently, dedicated approaches are needed to identify and categorize SATD in SSW to capture these unique risks and trade-offs. Therefore, the goal of this study is to identify, categorize, and evaluate scientific debt , and to assess the extent to which existing SATD taxonomies capture the unique challenges and trade-offs inherent to SSW development. By examining scientific debt within the context of critical domains such as climate modeling, energy systems, and biomedical research, this work seeks to enhance understanding of how domain-specific constraints, accuracy requirements, and evolving scientific objectives influence the manifestation and management of TD in SSW. To guide our study, we investigate the following research questions (RQs): RQ1: How well does a multi-source SATD classifier detect and categorize SATD instances across SSW artifacts? SSW artifacts, such as source code comments, commit messages, issue sections, and pull requests, capture SATD in different ways. We evaluate multiple candidate models to determine how well a classifier trained across these sources generalizes and select the strongest-performing model for downstream multi-artifact analysis. RQ2: What is the prevalence of different SATD types across code comments, commit messages, pull requests, and issue tracking sections in SSW? Different artifact types may surface distinct kinds and quantities of SATD. For example, code/design debt in code comments, requirement debt in issues, or scientific debt in discussions around results. Measuring their distribution is needed to guide both automated detection approaches and debt management practices. RQ3: Can traditional SATD types capture scientific debt instances? Traditional SATD categories, such as code/design , documentation , test , and requirement debt , were developed with industrial software in mind. SSW, however, introduces unique challenges, including domain-specific assumptions, computational accuracy concerns, and evolving scientific knowledge. Investigating whether existing SATD types adequately capture these cases, or whether scientific debt should be treated as a distinct category, is vital for extending SATD research to the scientific domain. Unlike RQ1, which focuses on model performance, RQ3 is designed to assess cross-class generalization. Specifically, how an existing SATD classifier behaves when confronted with scientific debt. To reflect this intent, we intentionally use a standard baseline model rather than the best-performing model identified in RQ1, allowing us to evaluate the limitations of traditional SATD types and classifiers in the context of SSW. The key contributions and main findings of this study can be summarized as follows: • We identify and categorize scientific debt across multiple artifact types in SSW, providing the first formal multi-artifact analysis of this SATD subtype. • We construct the first multi-artifact dataset for detecting scientific debt in SSW. • We validate the practical relevance of scientific debt through a practitioner study with domain experts. • We examine whether traditional SATD categories are sufficient to capture scientific debt or whether a dedicated category is necessary. • We release a replication package, including all code and datasets, to support reproducibility and future research. 2. Background and Related Works 2.1. SATD Taxonomy SATD refers to instances where developers explicitly acknowledge the presence of TD within code comments or documentation. Early work of Potdar and Shihab ( 39 ) first formalized the concept of SATD, while Maldonado and Shihab ( 33 ) proposed an initial taxonomy consisting of five types of SATD. Subsequent research refined and expanded these categories ( 42 ) . Alves et al. ( 3 ) broadened the understanding of TD through a systematic mapping study that characterized debt types beyond self-admitted instances and across multiple software artifacts. Their ontology introduced thirteen categories of debt, organized by debt

Read the complete text