Late Breaking Results: A Diffusion-Based Framework for Configurable and Realistic Multi-Storage Trace Generation

Seohyun Kim, Junyoung Lee, Jongho Park, Jinhyung Koo Sungjin Lee, Yeseong Kim · 2025

Read Late Breaking Results: A Diffusion-Based Framework for Configurable and Realistic Multi-Storage Trace Generation by Seohyun Kim, Junyoung Lee, Jongho Park, Jinhyung Koo Sungjin Lee, Yeseong Kim (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-01919.txt.

Introduction

Seohyun Kim leads five co-authors with DiTTO, a diffusion based tool that forges realistic storage traces for many devices at once. Storage traces log reads and writes over time and guide system design. Real traces are hard to share and hard to tune to a test need. Diffusion models build data by refining noise, and here they capture time patterns and cross device links.

Users can set precise configs to steer the forged traces. Tests show high fidelity, meaning close match to real traces, plus strong diversity across runs. Alignment with the guiding configs lands within a small error margin. The tool could supply shareable test data where real logs stay private or scarce. It also lets designers stress edge cases on demand.

Catalog record

CREATORSeohyun Kim, Junyoung Lee, Jongho Park, Jinhyung Koo Sungjin Lee, Yeseong Kim
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE12 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-01919

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Late Breaking Results: A Diffusion-Based Framework for Configurable and Realistic Multi-Storage Trace Generation Thanks: This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korean government (MSIT) (No.2022-0-00991, 1T-1C DRAM Array Based High-Bandwidth, Ultra-High Efficiency Processing-in-Memory Accelerator). This work was also supported by the Technology Innovation Program (RS-2024-00445759, Development of Navigation Technology Utilizing Visual Information Based on Vision-Language Models for Understanding Dynamic Environments in Non-Learned Spaces) funded by the Ministry of Trade, Industry & Energy (MOTIE, Korea) Seohyun Kim, Junyoung Lee, Jongho Park, Jinhyung Koo Sungjin Lee, Yeseong Kim Abstract We propose 𝖣𝗂𝖳𝖳𝖮 \mathsf{DiTTO} , a novel diffusion-based framework for generating realistic, precisely configurable, and diverse multi-device storage traces. Leveraging advanced diffusion techniques, 𝖣𝗂𝖳𝖳𝖮 \mathsf{DiTTO} enables the synthesis of high-fidelity continuous traces that capture temporal dynamics and inter-device dependencies with user-defined configurations. Our experimental results demonstrate that 𝖣𝗂𝖳𝖳𝖮 \mathsf{DiTTO} can generate traces with high fidelity and diversity while aligning closely with guided configurations with only 8% errors. I Introduction Understanding and optimizing the performance of distributed storage systems require detailed workload analysis in various environments such as RAID arrays and CephFS. Realistic workload traces, which capture read and write operations across multiple devices, are crucial for identifying performance bottlenecks and evaluating optimization strategies such as caching policies and load balancing. However, collecting real-world traces is costly and often impractical due to high instrumentation overheads and privacy concerns. This lack of accessible traces hinders the development of data-driven storage optimizations and limits the evaluation of emerging techniques. To mitigate this, synthetic trace generation tools such as SPEC Storage allows for controlled workload simulation. While useful, these tools rely on predefined templates and fail to capture the complex, evolving behavior of real-world workloads. Machine learning-based methods [ 1 ] attempt to improve realism but are still constrained by excessive feature engineering by human experts to identify application-specific characteristics. In this paper, we propose \mathsf{DiTTO} ( Di ffusion-based T race generation and T emporal O utpainting), a diffusion-based generative framework for producing realistic and configurable multi-device storage traces. Unlike conventional generative approaches that rely on predefined patterns or static sampling, our framework provides precise, user-controlled , and arbitrary-length trace generation, enabling workloads to be synthesized according to quantifiable constraints such as read/write ratios, access burstiness, and temporal dependencies. \mathsf{DiTTO} transforms raw access logs, including timestamps, operation types, and device identifiers, into structured multi-channel representations, enabling diffusion models to capture complex temporal dependencies and cross-device correlations. We introduce a workload conditioning mechanism that explicitly embeds quantifiable user-defined parameters into the generative process, providing fine-grained control over trace characteristics. Unlike conventional text-based conditioning in image diffusion models [ 2 ] , our approach ensures explicit and constrained output control. Additionally, we design a sparsity-aware training approach to ensure that long periods of inactivity and bursty access patterns are accurately modeled. Finally, by leveraging outpainting techniques, a generative technique that extends content while preserving contextual coherence, \mathsf{DiTTO} produces traces of arbitrary length while maintaining contextual consistency. In this paper, we make the following contributions. 1) We propose a diffusion-based synthetic trace generation framework that accurately captures multi-device workload patterns through structured representations. To the best of our knowledge, it is the first work that utilizes the diffusion technique for the storage trace generation. 2) We present a novel quantifiable workload conditioning mechanism that ensures quantifiable control over key workload properties such as read/write ratios. 3) We design sparsity-aware training and outpainting-based trace generation techniques to generate workloads with long inactive periods and extended trace durations. 4) We show that \mathsf{DiTTO} can generate high-fidelity storage traces that replicate real-world workload characteristics while preserving trace diversity with less than 8% error. Fig. 1: Overview of \mathsf{DiTTO} II Proposed Technique: \mathsf{DiTTO} We propose \mathsf{DiTTO} , a novel diffusion-based framework for generating realistic, configurable, and diverse multi-device storage traces. The framework captures intricate temporal dynamics and inter-device dependencies while allowing users to specify workload characteristics. The overall pipeline of \mathsf{DiTTO} is illustrated in Fig. 1 , consisting of three main stages: \mathbin{\vphantom{\circledast}\text{\vtop{\halign{#\cr\smash{\raisebox{-2.58334pt}{\scalebox{2.30}{$\bullet$}}}\cr\smash{{\color[rgb]{1,1,1}\bf\footnotesize$1$}}\cr}}}} transforming traces into structured image representations, \mathbin{\vphantom{\circledast}\text{\vtop{\halign{#\cr\smash{\raisebox{-2.58334pt}{\scalebox{2.30}{$\bullet$}}}\cr\smash{{\color[rgb]{1,1,1}\bf\footnotesize$2$}}\cr}}}} encoding numeric workload characteristics for explicit conditioning, and \mathbin{\vphantom{\circledast}\text{\vtop{\halign{#\cr\smash{\raisebox{-2.58334pt}{\scalebox{2.30}{$\bullet$}}}\cr\smash{{\color[rgb]{1,1,1}\bf\footnotesize$3$}}\cr}}}} generating arbitrarily long traces using a diffusion-based model enhanced with outpainting. Transforming system traces into image-based representations. \mathsf{DiTTO} first converts raw per-device access logs into structured multi-channel image representations, enabling the diffusion model to learn workload patterns effectively. Each input trace consists of timestamped read and write operations mapped to corresponding storage devices. To construct the image representation, we segment time into fixed intervals along the x-axis, while the y-axis corresponds to different storage devices. Each pixel encodes the presence of a read or write event at a specific time interval and device, with separate channels distinguishing operation types. Storage workloads exhibit extreme sparsity, where access events are interspersed with long periods of inactivity. To mitigate this, we introduce local Gaussian noise augmentation, which creates smooth intensity gradients around access events. This prevents excessive sparsity in the image representation, improving the diffusion model to learn contextual relationships across time and devices. Encoding numeric workload characteristics. A key requirement for synthetic trace generation is the ability to precisely control workload properties including read/write ratios, request rates, and device utilization patterns. To achieve this, we introduce CHIP (Contrastive Hyperconfiguration-Image Pretraining), a novel embedding mechanism that conditions the model on user-defined workload parameters. CHIP operates by mapping numeric workload configurations into a latent space that aligns with the image-based representations of system traces. This is achieved via contrastive learning [ 3 ] , where the model learns to associate similar workload characteristics with similar trace representations. Given a workload configuration, CHIP generates an embedding that guides the diffusion process, ensuring that generated traces adhere to the specified workload properties. Unlike conventional text-based conditioning in image diffusion models, CHIP explicitly enforces quantifiable constraints , making it well-suited for structured data synthesis. Generating continuous traces. Since real-world storage workloads span arbitrary durations, \mathsf{DiTTO} must generate long sequences while preserving workload characteristics. To achieve this, we incorporate outpainting, a generative technique that extends trace durations while maintaining contextual coherence. The outpainting works by conditioning the diffusion model on the final states of previously generated traces, allowing seamless extension over time. Specifically, when generating a new trace segment, the model samples latent variables from the boundary of the prior output, ensuring that patterns such as periodic bursts, coordinated device accesses, and workload fluctuations remain consistent. This prevents abrupt transitions and maintains statistical fidelity across extended traces. III Experimental Results Fig. 2: Generation error and t-SNE Fig. 3: (a), (b) Comparison between the original and generated traces, (c) Extended trace with collected data and configuration (Outpainting) Fig. 4: Distribution of spatial locality for different configuration groups We implemented \mathsf{DiTTO} using Pytorch 2.4 and evaluated the efficacy using Alibaba Block IO trace dataset. One of the key features of \mathsf{DiTTO} is its ability to generate realistic traces that closely adhere to user-specified configurations, such as read/write ratios and workload intensity. Figure 2 (a) presents the evaluation results for 50 generated traces, reporting the errors in how well the traces follow the user configurations. The results show that \mathsf{DiTTO} achieves an average error of less than 8% for read/write ratios and 2% for

Read the complete text