A Conceptual Framework for Requirements Engineering of Pretrained-Model-Enabled Systems

Dongming Jin 1,2 , Zhi Jin* 1,2 , Linyu Li 1,2 , Xiaohong Chen 3 · 2025

Read A Conceptual Framework for Requirements Engineering of Pretrained-Model-Enabled Systems by Dongming Jin 1,2 , Zhi Jin* 1,2 , Linyu Li 1,2 , Xiaohong Chen 3 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-13095.txt.

Introduction

Jin and co-authors argue that requirements work must be rethought for software built around large pretrained models. Such systems behave unlike older software: what they can do has fuzzy edges, shifts with context, and keeps changing. Classic methods assume functions can be split apart and future behavior guessed. Those ideas break here.

The paper offers a framework built for model-enabled systems. It also sketches research paths inside that framework. The authors aim to guide both scholars and builders facing these systems. Clear specs, they argue, remain possible, but only with methods that accept emergent behavior.

Catalog record

CREATORDongming Jin 1,2 , Zhi Jin* 1,2 , Linyu Li 1,2 , Xiaohong Chen 3
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE20 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-13095

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Conceptual Framework for Requirements Engineering of Pretrained-Model-Enabled Systems Dongming Jin 1,2 , Zhi Jin* 1,2 , Linyu Li 1,2 , Xiaohong Chen 3 Abstract Recent advances in large pretrained models have led to their widespread integration as core components in modern software systems. The trend is expected to continue in the foreseeable future. Unlike traditional software systems governed by deterministic logic, systems powered by pretrained models exhibit distinctive and emergent characteristics, such as ambiguous capability boundaries, context-dependent behavior, and continuous evolution. These properties fundamentally challenge long-standing assumptions in requirements engineering, including functional decomposability and behavioral predictability. This paper investigates this problem and advocates for a rethinking of existing requirements engineering methodologies. We propose a conceptual framework tailored to requirements engineering of pretrained-model-enabled software systems and outline several promising research directions within this framework. This vision helps provide a guide for researchers and practitioners to tackle the emerging challenges in requirements engineering of pretrained-model-enabled systems. I Introduction With the recent breakthroughs in large pretrained models ( e.g., large language models and multi-modal models), these models are no longer mere auxiliary tools but are integrated as core components in modern software systems [ 1 ] . Within such systems, pretrained models typically serve as reasoning engines, knowledge bases, or intelligent agents [ 2 ] [ 3 ] . We refer to such software systems as pretrained-model-enabled systems in this paper. These systems have already shown significant potential to impact various application domains, e.g., social simulation [ 4 ] , research assistant [ 5 ] , and software engineering [ 6 ] . This trend is expected to continue in the foreseeable future. Unlike traditional software systems governed by deterministic logic and rule-based behaviors [ 7 ] , pretrained-model-enabled systems exhibit three distinctive and emergent characteristics. ❶ Ambiguous capability boundaries: their capabilities are limited by pretrained models rather than explicitly defined functions. Moreover, an understanding of what pretrained models can do is unclear. ❷ Context sensitive behaviors: their behaviors heavily depend on prompt design and user input context. Even minor modifications may lead to qualitatively different responses [ 8 ] . ❸ Continuous evolution: Pretrained models are frequently updated through fine-tuning [ 9 ] , parameter-efficient adaptation [ 9 ] to fix issues, incorporate new knowledge, or improve alignment. Each update may shift system behaviors in unforeseen ways. These characteristics challenge the foundational assumptions of traditional requirements engineering. Specifically, current mainstream methodologies ( e.g., waterfall or agile practices [ 10 ] ) assume that system behaviors are predictable, specifiable, and verifiable, thereby treating requirements as deterministic specifications [ 11 ] . However, ambiguous capability boundaries hinder stakeholders from defining a precise feature set. The context-sensitive characteristic undermines the assumption that inputs and outputs can be exhaustively enumerated and verified. The evolution characteristic requires continuous monitoring and tracking of model capabilities, prompts, and data sources [ 12 ] . We investigate this problem and identify six open challenges that highlight the disconnect between traditional requirements engineering practice and the reality of pretrained-model-enabled systems. (1) Breakdown in requirements elicitation: stakeholders rarely possess a clear and shared picture of a pretrained model’s capabilities and limitations, which makes conventional elicitation techniques ineffective [ 12 ] . (2) Intent-capability misalignment: there is a mismatch between granularity in human intent and model capability description, which makes it difficult to select pretrained models that can satisfy user intent [ 13 ] [ 14 ] . (3) Inadequate specification: natural language requirements specifications lack structural rigor and semantic guarantees, which limits their effectiveness as precise and robust specifications. (4) Specification quality assurance gap: prompts act as requirements specifications, but there is no established method to validate their completeness, consistency or testability, which leaves stakeholders uncertain whether the specifications are fit for purpose [ 15 ] . (5) Emergence of under-specified non-functional requirements: the introduction of pretrained models brings new quality dimensions ( e.g., hallucination rate, fairness, and explainability) that expand the traditional scope of non-functional requirements, but they still lack common definitions, objective metrics, and mature engineering practices. (6) Unmanaged evolution of model artifacts: prompts, model checkpoints, and contextual configurations change rapidly without robust versioning or traceability, which leads to configuration drift and audit gaps [ 16 ] . To address these emerging challenges, we propose a conceptual framework that restructures the requirements engineering life cycle into six interconnected activities, as illustrated in Figure 1 . • Model Capability Discovery: uncovers and bounds what pretrained models can do based on their ecosystems ( e.g., the HuggingFace ecosystem). • User Intent Elicitation: elicits stakeholder goals, constraints, and success criteria through interactive, model capability-driven techniques. • Intent-capability alignment: matches the elicited intents to candidate models whose capabilities best satisfy the desired functionality. • Prompt-as-specification: elevates prompts to first-class and executable requirements specification, giving them explicit structure and semantics. • Behavior-driven specification validation: validates prompt specifications via statistical testing, scenario simulations, and human-in-the-loop reviews to ensure completeness, consistency, and robustness. • Model Artifacts Management: versions and traces prompts, model checkpoints, and fine-tuning datasets to control evolution, prevent configuration drift, and maintain auditability. Building on this conceptual framework, we envision several promising research directions for requirements engineering of pretrained-model-enabled systems. (1) Pretrained-model Capability Ontologies: constructing structured ontologies that formally capture, organize, and evolve pretrained-model capabilities. (2) Requirements-Driven Model Selection : devising methods that align stakeholder intents with candidate models through automated capability matching and recommendation. (3) Prompt Specification Languages: developing syntactically and semantically constrained languages for prompt specifications. (4) Emergent Quality Modeling : defining, measuring, and integrating novel quality attributes ( e.g., hallucination risk) into requirements engineering artifacts and processes. (5) Life cycle Traceability: establishing robust mechanisms for versioning, provenance, and audit mechanisms for prompts, model checkpoints, and fine-tuning datasets. We hope this work lays the groundwork for a systematic exploration of requirements engineering of pretrained-model-enabled systems. The remainder of this paper is organized as follows. In section II , we discuss existing challenges when traditional requirements engineering methodologies are applied to pretrained-model-enabled systems. In section III , we present our conceptual framework for requirements engineering of pretrained-model-enabled systems. Section IV outlines promising research directions, and section V concludes the paper. II Open Challenges The characteristics of pretrained-model-enabled systems present challenges not typically encountered in traditional requirements engineering. This section analyzes six concrete challenges, which span the entire requirements engineering life cycle ( i.e., elicitation, analysis, specification, validation, and management). Challenge 1: Requirements elicitation under opaque model capabilities. In traditional systems, stakeholders and engineers collaborate to define requirements based on explicit, decomposable functionalities. However, the pretrained-model-enabled systems’ core behaviors stem from latent knowledge encoded in the model rather than predefined features. This opacity makes it nearly impossible for stakeholders to articulate clear boundaries of “what the system can or cannot do”. Traditional elicitation techniques, such as structured interviews, use-case workshops, or user stories, rely on stakeholders’ understanding of pretrained models’ capability. In practice, stakeholders may overestimate model capabilities or underestimate their limitations. Challenge 2: Intent-capability misalignment. Even when stakeholder intents are partially elicited, mapping them to a suitable pretrained model remains non-trivial. Current model documentation ( e.g., model cards and technical reports) is often too coarse-grained to align with task-specific stakeholder requirements. For example, a model card might claim “strong performance on question answering”

Read the complete text