From Machine Learning Documentation to Requirements: Bridging Processes with Requirements Languages

Yi Peng (🖂), Hans-Martin Heyn, Jennifer Horkoff · 2025

Read From Machine Learning Documentation to Requirements: Bridging Processes with Requirements Languages by Yi Peng (🖂), Hans-Martin Heyn, Jennifer Horkoff (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-15340.txt.

Introduction

Yi Peng, Hans-Martin Heyn, and Jennifer Horkoff ask how much requirements gold is hidden in the documentation that machine learning teams already write. Before an ML component can be integrated and verified, its requirements, covering both the model and its data, must be stated clearly. Model cards and data sheets are written for transparency, not for engineers, yet they describe intended use, limits, and data provenance in detail.

The authors mine twenty public model cards and data sheets and find a significant amount of requirements-relevant information inside. They then test how well three established requirements notations can carry that information without loss. Bridging the two worlds could spare teams from re-eliciting facts that were documented all along.

Catalog record

CREATORYi Peng (🖂), Hans-Martin Heyn, Jennifer Horkoff
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-15340

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

From Machine Learning Documentation to Requirements: Bridging Processes with Requirements Languages Yi Peng (🖂), Hans-Martin Heyn, Jennifer Horkoff Abstract In software engineering processes for machine learning (ML)-enabled systems, integrating and verifying ML components is a major challenge. A prerequisite is the specification of ML component requirements, including models and data, an area where traditional requirements engineering (RE) processes face new obstacles. An underexplored source of RE-relevant information in this context is ML documentation such as ModelCards and DataSheets. However, it is uncertain to what extent RE-relevant information can be extracted from these documents. This study first investigates the amount and nature of RE-relevant information in 20 publicly available ModelCards and DataSheets. We show that these documents contain a significant amount of potentially RE-relevant information. Next, we evaluate how effectively three established RE representations (EARS, Rupp’s template, and Volere) can structure this knowledge into requirements. Our results demonstrate that there is a pathway to transform ML-specific knowledge into structured requirements, incorporating ML documentation in software engineering processes for ML systems. 1 Introduction Machine learning (ML)-enabled systems are increasingly used across a wide variety of domains, including healthcare, automotive, and manufacturing [ 21 ] . However, their non-deterministic behavior and heavy reliance on data complicates software engineering (SE) processes, especially in managing customer expectations, data quality, and ensuring transparency in system design [ 3 ] . Requirements Engineering (RE) plays a central role in addressing these challenges by translating stakeholder needs through clear and actionable requirements [ 18 ] . With ML components as part of complex software systems, there is a need to re-consider where requirements-related information comes from, and what types of new artifacts or boundary objects we should be considering as part of SE processes for ML systems [ 24 ] . Defining and integrating processes to capture model and data requirements, which cover aspects such as robustness, interpretability, fairness, and data quality [ 41 ] , remains a challenge in SE for ML systems [ 27 ] . In parallel, the ML community has introduced documentation artifacts such as ModelCards [ 35 ] and DataSheets [ 12 ] to document component-level information. These artifacts are particularly popular in open-source platforms such as Hugging Face ( https://huggingface.co/models ), yet they remain loosely structured and inconsistently adopted which limits their integration into software development processes [ 8 ] . Nonetheless, we argue that these artifacts could be a potential source of information for RE because they document information about intended use, performance limitations, and biases, which directly relate to requirements, including functional requirements (FRs), non-functional requirements (NFRs), and constraints. Structuring this information through RE languages may be a key step towards integrating these informal artifacts into SE for ML systems processes. The overall goal of this study is to assess the potential of ModelCards and DataSheets to serve as a bridge between informal ML component descriptions and more structured SE processes. To determine if these artifacts can serve this role, we must first validate that they contain valuable requirements-relevant information in practice. This step is addressed by research question (RQ) 1: RQ1 How much requirements-relevant information is contained in ModelCards and DataSheets? RQ1.1 What are the issues in how this information is documented from an RE perspective? If the presence of this information is confirmed, the next step is to evaluate whether existing RE languages are suitable for structuring it. RQ2 is formulated to investigate this: RQ2 How effective are existing requirements representation languages (e.g., EARS, Rupp’s Template, Volere) in capturing requirements-relevant information from ModelCards and DataSheets? RQ2.1 What types of requirements-relevant information are not well captured by these techniques in the context of ModelCards and DataSheets? We investigated these questions in two steps: First, we analyze a sample of 20 diverse ModelCards and DataSheet, revealing that these artifacts contain significant RE-relevant information. However, they also exhibit redundancies and inconsistent levels of detail, which may stem from the varied adoption of these documentation practices. Second, we evaluate how well three well-known RE languages —EARS [ 32 ] , Rupp’s Template [ 45 ] , and Volere [ 43 ] —capture this RE-relevant content, finding that while EARS and Rupp’s Template can extract much of the RE-relevant information, only Volere captures implementation-level details relevant for integrating ML models and data into broader system specifications. Overall, our findings contribute to the integration of ML documentation artifacts into the software engineering life cycle, aiming to improve communication between ML and SE teams and enhance the engineering of successful ML-systems. The paper is organized as follows: Sec. 2 presents background and related work, with methodology in Sec. 3 . We report the results in Sec. 4 and discuss implications, future work, and threats in Sec. 5 . Sec. 6 concludes this paper. 2 Background and Related Work 2.1 RE Languages and Templates Structured RE languages and templates enhance clarity, traceability, and consistency in requirements documentation. Among them, EARS, Rupp’s template, and Volere are widely adopted [ 15 , 13 ] . Fig. 1 shows the core structures of EARS, Rupp’s Template and Volere. (a) (c) (b) Figure 1 : (a) EARS [ 33 ] , (b) Rupp’s template [ 34 ] and (c) Volere [ 44 ] . EARS is a lightweight, structured natural language format, applied successfully in industry [ 32 ] . It uses patterns such as ubiquitous, event-driven, state-driven, optional feature, and unwanted behavior to reduce ambiguity and improve readability [ 14 ] . Rupp’s Template [ 45 ] , established by the International Requirements Engineering Institute (IREB) as a de facto standard for the syntactic specification of system requirements [ 34 ] , specifies nine patterns: three for FRs, three for NFRs, and three for conditional clauses. FRs are grouped under FunctionMASTER, classified by activity type: system, user interaction, and interface. NFRs are expressed using PropertyMASTER, EnvironmentMASTER, and ProcessMASTER. Conditional logic is supported by LogicMASTER (“if”), EventMASTER (“as soon as”), and TimeMASTER (“as long as”). The template encourages fine-grained specification using modal verbs and optional details [ 15 ] . Volere [ 42 ] focuses on traceability of requirements and structured metadata (e.g., rationale, acceptance criteria). It categorizes NFRs (33 types) and constraints (8 types), and includes sections on external factors (e.g., assumptions, risks). With over 20,000 downloads of its online template materials reported on its official website, the Volere template demonstrates significant practical use. These three RE languages were selected for their popularity and distinct strengths: EARS provides a lightweight, structured syntax for atomic requirements; Rupp’s template offers a systematic approach for capturing variability and fine-grained system behavior, and Volere ensures traceability and completeness. Together, they offer a comprehensive lens to assess how information in ML documentation can be utilized as a source of requirements. 2.2 RE for ML Systems Vogelsang and Borg [ 52 ] were among the first to define characteristics and challenges unique to RE for ML-based systems by interviewing data scientists, observing that the development paradigm for ML requires rethinking of RE. Since then, frameworks such as RE for human-centered AI [ 1 ] , perspective-driven RE [ 51 ] , as well as extensions of goal-orientated RE [ 7 ] have emerged. While these approaches guide requirement elicitation and modeling, they tend to remain

Read the complete text