A Tale of Two Systems:Characterizing Architectural Complexity onMachine Learning-Enabled Systems

Renato Cordeiro Ferreira · 2025

Read A Tale of Two Systems:Characterizing Architectural Complexity onMachine Learning-Enabled Systems by Renato Cordeiro Ferreira (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-11295.txt.

Introduction

Renato Cordeiro Ferreira wrote a paper in two thousand twenty-five about the architectural complexity of systems that embed machine learning. Such systems mix data pipelines, models, and software, and their design can grow tangled. The guiding question is how that complexity can be measured and managed before it harms the system.

The paper sets out a research plan to build a metrics-based model of architecture for these systems. It presents two real systems side by side as case studies: one called SPIRA and one called Ocean Guard. Their architecture descriptions provide the raw material from which the metrics model will be drawn. The aim is a guideline that supports design choices as such systems are started and grown.

Catalog record

CREATORRenato Cordeiro Ferreira
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE15 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-11295

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Tale of Two Systems:Characterizing Architectural Complexity onMachine Learning-Enabled Systems Renato Cordeiro Ferreira Abstract How can the complexity of ML-enabled systems be managed effectively? The goal of this research is to investigate how complexity affects ML-Enabled Systems (MLES). To address this question, this research aims to introduce a metrics-based architectural model to characterize the complexity of MLES. The goal is to support architectural decisions, providing a guideline for the inception and growth of these systems. This paper brings, side-by-side, the architecture representation of two systems that can be used as case studies for creating the metrics-based architectural model: the SPIRA and the Ocean Guard MLES. 1 Introduction Complexity has been a subject of discussion since the early days of the Software Engineering field [ 3 ] . In the book “Mythical Man Month”, Brooks introduces the concept of essential and accidental complexity for software [ 3 ] : the essential exists intrinsically to satisfy the requirements of the problem solved, whereas the accidental may originate from any external factors that influence the solution chosen. Under this definition, an ML-enabled system (MLES) has high essential complexity: it requires extra components in its architecture to support data processing and model handling [ 1 , 2 ] . As a consequence, this also creates more opportunities for introducing accidental complexity into the system. While there are many recent surveys summarizing important gaps in the SE4AI literature, only Giray et al. highlights handling complexity as an open challenge for MLE [ 9 ] . In an interview study with 18 professional ML engineers [ 17 ] , Shankar et al. paper describes how participants “expressed an aversion to complexity” , implying that practitioners consider it an issue. This proposal presents a plan to create a metrics-oriented architectural model to characterize the complexity of MLES . By using metrics to identify where complexity emerges in the software architecture of MLES, this research aims to provide a method to avoid pitfalls that make them fail to reach production. 2 Research Questions To achieve the goal of this research, this PhD proposes two main research questions. They follow the SMART principle [ 18 ] , i.e., they should be Specific, Measurable, Achievable, Relevant, and Time-Bound . RQ1: What are the measurable dimensions of complexity in the architecture of MLES? Section 2 aims to explore metrics related to complexity available in the literature. Software Metrics have been thoroughly researched by Software Engineering [ 5 ] . However, MLES also have their data and model dimensions [ 16 ] , which affect the software architecture. Finding metrics that measure the data- and model-related complexity beyond code is the biggest potential challenge for answering section 2 . RQ2: How can complexity metrics be operationalized over the architecture of MLES? Section 2 aims to define a process to collect the metrics found in section 2 . Metrics may be related to different abstraction levels of a system: from code snippets, to components, to services [ 5 ] . Moreover, they can have varying levels of different quality attributes [ 12 ] , such as validity, reliability, computability, intuitiveness, ease of implementation, and independence of other metrics. As a consequence, only some metrics found in section 2 may be practical or useful to collect. In particular, if two metrics provide similar information, it will be preferable to use the best according to the quality attributes. Therefore, the challenge for answering section 2 is twofold: creating a dataset of representative MLES to collect the metrics, and then choosing which subset of metrics to collect. Measuring a metric may require different levels of access to the system (e.g., having the codebase available for processing) or rely on different representations of the system (e.g., creating a graph describing its data flow). As a consequence, only some metrics found in section 2 may be practical to collect. In particular, if two metrics provide similar information, it may be preferable to use the simpler. 3 Methodology Figure 1 summarizes four steps, grouped into two phases, of the methodology proposed for this research. Figure 1: Research Methodology . The methodology is divided into two phases, each addressing a research question from section 2 . This methodology categorizes as empirical software engineering with a mixed-method approach. 3.1 Understanding Metrics for Complexity To answer section 2 , this research will start studying the complexity of software systems and metrics . For that, section 3.1 has been divided into two steps: (1) to understand how practitioners address complexity in their systems; and (2) to describe and categorize related metrics based on the literature. Survey Research with Industry. Survey research proposes selecting a representative sample from a well-defined population, identifying characteristics from it (via questionnaires or structured interviews), and then applying data analysis techniques to generalize the results [ 4 ] . This method helps to answer a question about a particular target population, such as professional developers working on software companies, without polling every member of the population. A survey allows a practical way to map how practitioners address complexity while creating MLES, as is the goal of section 3.1 . This approach is inductive rather than deductive , i.e., first it proposes to understand what practitioners consider while designing their systems, then it tries to create a theory around it. Since MLES have their data and model dimensions [ 16 ] , these characteristics will likely reflect during the characterization of the complexity of MLES. Ontology Design. Making an ontology results in a well-defined categorization of the metrics [ 15 ] , as it is the goal of section 3.1 . The ontology will focus on describing different types of metrics [ 5 ] , as well as possible quality attributes associated with them [ 12 ] . Some properties will be related to the applicability of the metrics, which will be explored only on section 3.2 . Therefore, the knowledge base will be fully complete only by the end of the research. An ontology can be created via ontology design [ 15 ] . An ontology is the structure behind a knowledge base, including classes , properties , and restrictions [ 15 ] . The ontology will focus on describing different types of metrics [ 5 ] , as well as possible quality attributes associated with them [ 12 ] . 3.2 Designing a Metrics-Oriented Architectural Model To answer section 2 , this research will focus on selecting a subset of the metrics from section 3.1 to compose the metrics-oriented architectural model . For that, section 3.2 has been divided into two steps: (1) to find how to collect the metrics on a known MLES; and (2) to validate the metric collection process on industry MLES. Exploratory Case Study. Making an exploratory case study provides a framework to propose an answer to section 2 [ 4 ] . The exploratory case study will have the following study proposition: “What is a process to operationalize the collection of complexity metrics on an MLES?” , as it is the goal of section 3.2 . To achieve that, it will rely on two typical cases [ 4 ] : the SPIRA MLES [ 7 , 13 ] and the Ocean Guard MLES

Read the complete text