Generating representative macrobenchmark microservice systems from distributed traces with Palette
Vaastav Anand, Matheus Stolet, Jonathan Mace, Antoine Kaufmann · 2025
Read Generating representative macrobenchmark microservice systems from distributed traces with Palette by Vaastav Anand, Matheus Stolet, Jonathan Mace, Antoine Kaufmann (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-06448.txt.
Introduction
Vaastav Anand and three colleagues present a 2025 systems paper on a chicken-and-egg problem in cloud computing research. New ideas for microservice systems need realistic test systems to prove themselves, but real production systems are closed, so researchers settle for toy examples or simulations. The authors propose building test systems from distributed traces, the call records that large internet firms have released.
Their system, Palette, first compresses a trace into a causal model capturing which services call which, in what order, with what timing and branching odds. From that model it generates full benchmark systems that mirror the original's scale and behavior. The authors argue trace-grown benchmarks can replace the hand-built stand-ins the field relies on.
Catalog record
| CREATOR | Vaastav Anand, Matheus Stolet, Jonathan Mace, Antoine Kaufmann |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 24 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-06448
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Generating representative macrobenchmark microservice systems from distributed traces with Palette Vaastav Anand, Matheus Stolet, Jonathan Mace, Antoine Kaufmann Abstract . Microservices are the dominant design for developing cloud systems today. Advancements for microservice need to be evaluated in representative systems, e.g. with matching scale, topology, and execution patterns. Unfortunately in practice, researchers and practitioners alike often do not have access to representative systems. Thus they have to resort to sub-optimal non-representative alternatives, e.g. small and oversimplified synthetic benchmark systems or simulated system models instead. To solve this issue, we propose the use of distributed trace datasets, available from large internet companies, to generate representative microservice systems. To do so, we introduce a novel abstraction of a system topology which uses Graphical Causal Models (GCMs) to model the underlying system by incorporating the branching probabilities, execution order of outgoing calls to every dependency, and execution times. We then incorporate this topology in Palette, a system that generates representative flexible macrobenchmarks microservice systems from distributed traces. 1. Introduction Modern cloud systems are developed as microservice systems. They have been adopted by many large companies such as Facebook ( huye2023lifting ) , Netflix ( cockcroft2016microservices ) , Uber ( uber2015soa ) , among others ( cockcroft2016evolution ; mazdak2017infrastructure ) due to the ability of microservices to be developed, deployed, and scaled independently. Validating and testing new advancements for microservices often requires developers to experiment with interventions—that is change aspects of the system with new design decisions, architectural choices, algorithms, backend components, and other strategies—in order to estimate and assess their impact on the system. Ideally, developers and researchers would execute these intervention experiments on production systems to make claims about the technique’s scalability and generalizability. However, access to production systems is limited to a select few. Even if one could procure access, the scope of experimentation is limited to minimize disruptions. Despite the limited access to production systems, distributed traces are often readily available and researchers can use them to glean insight into these systems. Distributed traces capture rich structural and temporal information about the execution of the system, such as latency, execution patterns, branch probabilities, and call probabilities. We posit that distributed traces can support general purpose intervention experiments given the rich volume of system behavior they capture. However, we believe that currently there are three key challenges that prevent distributed traces from being used for intervention experiments. First, while distributed traces capture a variety of metrics, they are observations of the intrinsic behavior of the system. The system behavior that caused these observations is not explicitly captured in distributed traces. Thus, we need a mechanism that can use the distributed traces to derive a model of the intrinsic behavior of the system. Second, currently there is no mechanism to convert intervention experiments into targeted modifications of the system without modifying the rest of the system behavior. The fundamental tenet of an intervention experiment requires that all the other factors in the system must be held constant to isolate the impact of the intervention. Third, converting distributed traces to a runnable system for supporting general purpose intervention experiments is non-trivial as different intervention experiments might wish to preserve different characteristics of the systems. To overcome these challenges, we propose Palette, a system designed for natively supporting intervention experiments using distributed traces. Palette provides a new abstraction called system topology that models system behavior captured by generating a graphical causal model (GCM). To support targeted modifications for interventions, Palette provides a primitive set of operations that allow developers to make localized changes in the system topology for a given intervention. Finally, Palette provides a generation mechanism for converting a system topology into runtime components. Our proposed solution leverages the GCM at runtime to model the causal dependencies from the original system and uses it to more faithfully sample metrics for execution behavior, such as the amount of work to be done in a service or the payload size to be used between two services given the current state of the system. 2. Background and Motivation 2.1. Research Use Cases Microservices have a large design space owing to its hetereogeneous nature. Consequently, the set of all possible interventions a researcher can perform is also large. For instance, researchers building a new network stack may be interested in confirming that their design maintains low tail latency under load so that requests meet their SLOs. To test their hypothesis they need a system that matches the characteristics of a real system such as request size, service execution times, and call graph depth and width. Request size matters because systems optimize differently based on its average. Service execution is critical because services with long execution times may see negligible benefits from a faster network stack. Similarly, benefits may be amplified in deep call graphs or offset by slow services in the critical path. Therefore, generated systems should be able to modify and preserve these properties so that interventions can be added while still maintaining realistic performance characteristics. Different use-cases, as evidenced in Table 1 , will care about different properties when introducing interventions to the system. No single point solution system derived from traces can support all possible interventions correctly. Use-case Meaningful properties \mu Service Performance ( zhang2024mucache ; sriraman2018mutune ; sriraman2019softsku ) Varying graph sizes, execution paths, topology Network Stacks ( kaufmann:tas ; fried2024making ; zhang2021demikernel ) Request sizes, service execution time, call depth, and call width Congestion Control ( kumar2020swift ; zhou2021fasttune ) Varying topology, timeout values Res. Management ( somashekar2024oppertune ; qiu2020firm ) Varying execution paths, large service graphs Tracing Framework ( las2019sifter ; zhang2023benefit ; fonseca2007x ; toslali2021automating ) Varying execution paths, large number of services RPC Framework ( kogias:r2p2 ; kalia:erpc ) Varying request sizes Table 1. Research use-cases and the ideal properties from a benchmark system. 2.2. The Role of Distributed Tracing Distributed tracing is a critical monitoring component in modern cloud systems. It provides troubleshooting support for developers and operators during incident root-cause analysis and post-mortems. Distributed tracing supports this by generating an execution trace of each request across all components of the system. We believe that the data-rich nature of distributed trace datasets represents an opportunity for addressing the structural diversity issue in existing open-source microservice systems for research use-cases. Trace Details. A distributed trace contains the partially-ordered list of all APIs (referred to as spans) executed by the system to service the request. For each span, the tracing framework captures the total amount of execution time and the specific service (or component) at which the span was executed. The tracing frameworks encodes caller-callee relationships between APIs as parent-child relationships. Distributed Tracing Workload Generation. Distributed tracing can also be used to generate realistic workloads to test solutions at large scale because of its rich collection of execution data ( sajal2024traceupscaler ; du2024microservicegraphgenerator ) . 2.3. Graphical Causal Models (GCMs) Graphical Causal Models ( elwert2013graphical ) are directed acyclic graphs which encode causal relationships between the different nodes in the graph. Each node represents a variable, i.e. some observable data, and the edges represent causality. A directed edge between two nodes signifies that a variable influences the value of the other variable. These models are useful for understanding the causal