Iridescent: A Framework Enabling Online System Implementation Specialization

Vaastav Anand, Deepak Garg, Antoine Kaufmann · 2025

Read Iridescent: A Framework Enabling Online System Implementation Specialization by Vaastav Anand, Deepak Garg, Antoine Kaufmann (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-16690.txt.

Introduction

Vaastav Anand, Deepak Garg, and Antoine Kaufmann present a 2025 framework that tunes a running software system to its actual work. Tailoring a system to its hardware and load can speed it greatly. In practice the tailoring is hard: the best form is unclear, narrow forms lose range, and loads keep shifting. Their framework, Iridescent, searches for the best form while the system runs.

Developers mark the space of allowed variants up front. The framework then builds and trials variants live, using whole-system speed as the guide. Fast build tools make the swaps cheap enough to run in production. Tests show the approach works, helps, and is easy to use. Live tailoring could suit cloud services whose loads swing by hour.

Catalog record

CREATORVaastav Anand, Deepak Garg, Antoine Kaufmann
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-16690

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Iridescent: A Framework Enabling Online System Implementation Specialization Vaastav Anand, Deepak Garg, Antoine Kaufmann Abstract . Specializing systems to specifics of the workload they serve and platform they are running on often significantly improves performance. However, specializing systems is difficult in practice because of compounding challenges: i) complexity for the developers to determine and implement optimal specialization; ii) inherent loss of generality of the resulting implementation, and iii) difficulty in identifying and implementing a single optimal specialized configuration for the messy reality of modern systems. To address this, we introduce Iridescent, a framework for automated online system specialization guided by observed overall system performance. Iridescent lets developers specify a space of possible specialization choices, and then at runtime generates and runs different specialization choices through JIT compilation as the system runs. By using overall system performance metrics to guide this search, developers can use Iridescent to find optimal system specializations for the hardware and workload conditions at a given time. We demonstrate feasibility, effectivity, and ease of use. 1. Introduction Specializing system implementations to workload characteristics and hardware can significantly improve performance and efficiency ( Farshin et al. 2021 ; Kalia et al. 2019 ; Van Aken et al. 2017 ; Miano et al. 2022 ; Sriraman and Wenisch 2018 ; Alipourfard et al. 2017 ; Karthikeyan et al. 2023 ; Rzadca et al. 2020 ; Ghasemirahni et al. 2024 ; Wu et al. 2024 ; Ghasemirahni et al. 2022 ; Deng et al. 2020 ; Sharif et al. 2018 ; Bhatia et al. 2004 ; Molnár et al. 2016 ; Pan et al. 2024 ; Sriraman et al. 2019 ; Dean et al. 1995 ; Consel and Noël 1996 ) . Achieving these benefits requires manual modification the system implementations and recompilation. Part of the performance benefit arises from cascading compiler optimizations, e.g. by removing a feature enabling the compiler to eliminate dead code, in turn enabling further optimizations ( Wintermeyer et al. 2020 ; Panchenko et al. 2019 ; Farshin et al. 2021 ) . Today, system specialization for performance is manual, developer-driven, and iterative. Building optimal systems in practice is a challenge because of three compounding factors: First, specialization is difficult for developers to implement, as it requires trial-and-error and maintaining multiple versions of key code paths. Next, specialization fundamentally comes at the cost of generality — a specialized system either performs poorly outside its regime, or completely fails. Finally, predicting performance implications of specialization choices for different workloads and hardware is almost impossible. To make matters worse, workload and platform conditions are dynamic and optimal specialization choices depend on them. We propose a fundamentally different approach: automated online system specialization guided by observed overall system performance . Our goal is to specialize performance critical system systems at runtime to the exact momentary workload and hardware conditions, without the human developer on the critical path . To enable this, we re-distribute tasks between ahead of time development and runtime operation. Ahead of deployment, rather than choosing a concrete set of specializations, developers specify the space of possible specializations , along with a search strategy. After deployment, our runtime iteratively generates different specialized code versions, choosing the current optimum based on observed overall system performance. This turns compile-time specialization into runtime parameter tuning, suitable for search-based auto-tuning ( Van Aken et al. 2017 ; Sriraman and Wenisch 2018 ) . To enable this, we introduce Iridescent, a framework for practical development of fast and efficient systems with online specialization. Iridescent enables both the incremental specialization of existing code-bases with minimal modifications, and in-depth design for specialization for maximizing performance. Iridescent builds on LLVM to target systems written in typical (non-managed) languages such as C(++) or Rust. Concretely, Iridescent supports the developer in separating the code-base into a specialized, performance critical core, and the remaining generic code. Iridescent then provides the developer with a specialization API to annotate possible specializations in the code thereby specifying the space of possible specializations. Additionally, Iridescent provides hooks to optionally customize JIT code generation in depth. Finally, Iridescent provides the runtime API to control the exploration of different specialized implementations as the system runs. We integrate Iridescent with multiple systems and show that with Iridescent enabled specializations, systems can gain a performance boost of upto 30 \times . Moreover, with Iridescent, developers can easily configure the system to automatically explore the specialization space to adaptively find the best performing specialization for different workloads at runtime. 2. Motivation & Challenges 2.1. Illustrative Example: Matrix Multiply Server We illustrate the benefits and challenges of specialization with an example system: a server executing square matrix multiplications for clients. To optimize cache locality we use a blocked matrix multiplication ( Carr and Kennedy 1989 ; Wolfe 1987 ; Wolfe 1989 ) . We identify one key workload parameter, N the matrix size, and a key configuration parameter, B the block size. Code specialization improves performance. Both parameters offer a substantial opportunity for optimization through specialization. Unsurprisingly, we find fixing B as a compile-time constant instead of leaving it a variable improves throughput by up to 6.5\times , by enabling the compiler to unroll and vectorize the inner loops. Note that B is an internal parameter that may affect performance, but any valid choice results in correct behavior with every workload. Similarly, assumptions about the workload, N , can also simplify the algorithm, e.g. by assuming that N is a multiple of B we avoid the need for copying and 0-padding partial blocks. Machine / Workload N=1024 N=256 N=64 IceLake 32 32 32 IvyBridge 16 16 4 CoffeeLake 32 4 4 AlderLake-p 32 4 2 AlderLake-e 64 4 4 Table 1 . Optimal configurations for our block matrix multiply, across 5 hardware platforms and 3 workloads. Optimal configuration depends on workload and HW. We now compare different block sizes for different workloads (matrix sizes) on 5 different processors. As Table 1 shows, different block sizes yield optimal performance for different workload and processor combinations. Picking a fixed B ahead of a practical deployment in a dynamic environment will not yield optimal performance. Moreover, even with single single-size workloads it is difficult to predict what block size will be ideal for a concrete processor. 2.2. Code Specialization is Effective but Complex There is a long line of prior work that has established the performance and efficiency benefits of specializing code, and systems code more specifically. For example, interpreted languages may generate specialized instances of functions with constant parameters and optimize them accordingly ( de Assis Costa et al. 2014 ; Lima et al. 2020 ) . This is a generic optimization and typically done automatically and transparently by the runtime. Other optimizations are specific to individual systems and concrete concerns, such as inlining small table lookups in software network functions as if statements ( Miano et al. 2022 ) . As most compiler optimizations, these approaches are typically guided by simple heuristics around local metrics (e.g. frequency of the same parameter value, or table size) based on developer intuition. Specialization effects often cascade. A key aspect of specialization techniques is that the effects combine. For example, PacketMill ( Farshin et al. 2021 ) first de-virtualizes function pointers ( Kohler

Read the complete text