A Metrics-Oriented Architectural Modelto Characterize Complexity onMachine Learning-Enabled Systems

Renato Cordeiro Ferreira · 2025

Read A Metrics-Oriented Architectural Modelto Characterize Complexity onMachine Learning-Enabled Systems by Renato Cordeiro Ferreira (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-08153.txt.

Introduction

Renato Cordeiro Ferreira takes a first step toward a metrics-based model of complexity in machine learning systems. The paper from twenty twenty-five asks how such systems' complexity can be managed well. Software that wraps learned models mixes code, data, and training loops. That mix breeds complexity that plain software gauges miss. Ferreira's aim is a model that guides design calls as systems grow.

This stage extends a reference architecture for such systems so it can host metric collection. The extended frame describes the parts whose complexity must be tracked. Later stages will set the metrics themselves. The author frames the work as groundwork for inception and growth choices. It serves teams that must keep learning systems sound as they scale.

Catalog record

CREATORRenato Cordeiro Ferreira
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE16 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-08153

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Metrics-Oriented Architectural Modelto Characterize Complexity onMachine Learning-Enabled Systems Thanks: This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior – Brasil (CAPES) – Finance Code 001. It also received support from the MARIT-D project, co-funded from the Internal Security Fund – Police programme under grant agreement no. 101114216. Renato Cordeiro Ferreira Abstract How can the complexity of ML-enabled systems be managed effectively? The goal of this research is to investigate how complexity affects ML-Enabled Systems (MLES). To address this question, this research aims to introduce a metrics-based architectural model to characterize the complexity of MLES. The goal is to support architectural decisions, providing a guideline for the inception and growth of these systems. This paper showcases the first step for creating the metrics-based architectural model: an extension of a reference architecture that can describe MLES to collect their metrics. I Introduction Complexity has been a subject of discussion since the early days of the Software Engineering field [ 1 ] . In the book [ 1 ] , [ 1 ] introduces the concept of essential and accidental complexity for software [ 1 ] : the essential exists intrinsically to satisfy the requirements of the problem solved, whereas the accidental may originate from any external factors that influence the solution chosen. Under this definition, an ML-enabled system (MLES) has high essential complexity: it requires extra components in its architecture to support data processing and model handling [ 2 , 3 , 4 ] . According to Gartner’s report [ 5 ] , only around 54% of AI projects successfully reach production. Besides technical challenges [ 6 ] , many factors influence the ability to deliver, such as infrastructure, developer experience, development processes, and team composition. All of them reflect into the software architecture [ 1 ] , thus becoming potential sources of accidental complexity. This PhD research outlines a metrics-oriented architectural model to characterize the complexity of ML-enabled systems . The goal is to use metrics to identify where complexity emerges in the software architecture of MLES. Hopefully, this technique will assist developers to manage complexity, allowing them to reach production more often. II Related Literature While there are many recent surveys summarizing important gaps in the SE4AI literature, only [ 7 ] highlights handling complexity as an open challenge for MLE [ 7 ] . However, this idea is also supported by [ 8 ] , who outlines challenges for sustainable AI [ 8 ] . In an interview study with 18 professional ML engineers [ 9 ] , [ 9 ] paper describes how participants “expressed an aversion to complexity” , implying that practitioners consider it an issue. The survey by [ 10 ] further explores possible origins of complexity in MLES [ 10 ] , which become visible in the software architecture: • they are harder to modularize, since data modeling and data processing are highly coupled [ 11 ] ; • they require an understanding of ML principles and techniques, with implications to the number of ML models required, their size, versioning, and tracking [ 12 , 13 ] ; • they often handle big data, which leads to the adoption of domain-specific distributed data processing patterns [ 14 ] . The field study by [ 15 ] discusses the impact of such complexity for practitioners [ 15 ] . It was held with 11 participants with at least two years of experience with ML. Seasoned engineers described that developing MLES “required skills held only by certain ‘high priests”’ , while debugging them was “something akin to magic and even voodoo” [ 15 ] . Such challenges become even more critical since MLES need to be updated and improved continuously [ 11 ] . The many surveys in the literature show a general lack of studies regarding metrics for MLES. Recently, [ 16 ] were one of the first to publish about the subject, proposing a set of metrics to measure how automated is an MLES. Henceforth, this research intends to further explore this gap, thus understanding how metrics can help to manage the architectural complexity of MLES. III Research Questions To achieve the goal of this research, this PhD proposes two main research questions. They follow the SMART principle [ 17 ] , i.e., they should be Specific, Measurable, Achievable, Relevant, and Time-Bound . RQ1: What are the measurable dimensions of complexity in the architecture of MLES? Section III aims to explore metrics related to complexity available in the literature. Software Metrics have been thoroughly researched by Software Engineering [ 18 ] . However, MLES also have their data and model dimensions [ 19 ] , which affect the software architecture. Finding metrics that measure the data- and model-related complexity beyond code is the biggest potential challenge for answering section III . RQ2: How can complexity metrics be operationalized over the architecture of MLES? Section III aims to define a process to collect the metrics found in section III . Metrics may be related to different abstraction levels of a system: from code snippets, to components, to services [ 18 ] . Moreover, they can have varying levels of different quality attributes [ 20 , 21 ] , such as validity, reliability, computability, intuitiveness, ease of implementation, and independence of other metrics. As a consequence, only some metrics found in section III may be practical or useful to collect. In particular, if two metrics provide similar information, it will be preferable to use the best according to the quality attributes. Therefore, the challenge for answering section III is twofold: creating a dataset of representative MLES to collect the metrics, and then choosing which subset of metrics to collect. Measuring a metric may require different levels of access to the system (e.g., having the codebase available for processing) or rely on different representations of the system (e.g., creating a graph describing its data flow). As a consequence, only some metrics found in section III may be practical to collect. In particular, if two metrics provide similar information, it may be preferable to use the simpler. IV Methodology Figure 1 summarizes four steps, grouped into two phases, of the methodology proposed for this research. Fig. 1: Research Methodology . The methodology is divided into two phases, each addressing a research question from section III . This methodology categorizes as empirical software engineering with a mixed-method approach. IV-A Understanding Metrics for Complexity To answer section III , this research will start studying the complexity of software systems and metrics . For that, section IV-A has been divided into two steps: (1) to analyze the existing literature on software systems complexity; and (2) to categorize the metrics related to complexity from the literature. IV-A 1 Systematic Literature Review (SLR) Making an SLR ensures a reproducible way to analyze the literature [ 22 ] , as it is the goal of section IV-A . Initially, the SLR will focus on finding references about complexity metrics , i.e., metrics that were created to measure some aspect of complexity. On the one hand, this subject has been thoroughly explored by the software engineering literature [ 18 ] . On the other hand, MLES have their

Read the complete text