PRIME: Pseudo-Random Integrated Multi-Part Entropy for Adaptive Packet Spraying in AI/ML Data centers

Ashkan Sobhani, Sogand Sadrhaghighi, Xingjun Chu · 2025

Read PRIME: Pseudo-Random Integrated Multi-Part Entropy for Adaptive Packet Spraying in AI/ML Data centers by Ashkan Sobhani, Sogand Sadrhaghighi, Xingjun Chu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-23012.txt.

Introduction

Ashkan Sobhani and co-authors redesign how AI supercomputers spray packets across their networks, arguing the current recipes quietly clog themselves. Training clusters generate bursty, long-lived, low-variety flows that defeat classic Ethernet load balancing. Packet spraying, sending one flow down many paths, is the industry's chosen fix, endorsed by the Ultra Ethernet Consortium.

The authors show existing spraying schemes inflate buffers over time, especially when acknowledgment coalescing makes their routing information stale, and under lopsided conditions like failing links. Their design, PRIME, builds pseudo-random multi-part entropy into the spraying decision to stay adaptive. In data centers where a stalled network idles a fortune in chips, such plumbing is strategy.

Catalog record

CREATORAshkan Sobhani, Sogand Sadrhaghighi, Xingjun Chu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-23012

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

PRIME: Pseudo-Random Integrated Multi-Part Entropy for Adaptive Packet Spraying in AI/ML Data centers Ashkan Sobhani, Sogand Sadrhaghighi, Xingjun Chu Abstract Large-scale distributed training in production data centers place significant demands on network infrastructure. In particular, significant load balancing challenges arise when processing AI/ML workloads, consisting of low-entropy, bursty and long-lived flows. Existing solutions designed for Ethernet, such as Equal-Cost Multi-Path ( ECMP ) struggle to maintain high network utilization. While major industry players ( e.g. , Ultra Ethernet Consortium) and parts of academia have proposed packet spraying to enhance AI/ML workload performance, we argue that existing packet spraying solutions lead to buffer inflation over time, negatively affecting network performance. Specifically, when ACK coalescing is used, these solutions lead to stale information, degrading network performance. Additionally, in asymmetric network conditions- such as mix of ordered an unordered traffic, or link degradation and failures- existing packet spraying solutions often lead to increased tail latency. In this paper, we present the design and evaluation of PRIME , a pseudo-randomized round-robin approach to packet spraying that considers the network topology to optimize load distribution and performance. PRIME uses congestion as an indicator to re-balance the load. To this extent, PRIME takes into account various congestion signals, accounting for congestion severity, and their decay times to avoid network hotspots. We extensively evaluated PRIME using large-scale production-level simulator. Our results indicate that, compared to existing solutions, PRIME leads to up to 15 % 15\% improvement for permutation traffic and up to 27 % 27\% improvement in network degradation scenarios. I Introduction Motivation. The rapid growth in AI’s computational demands necessitates distributed training to efficiently scale across large GPU clusters and leverage parallel processing for faster training time [ 1 ] . To speed up the training process significant research has been conducted in various areas, e.g. , parallelization strategies [ 2 ] , topology optimization [ 3 ] , etc . Communication, however, is the main bottleneck in distributed training [ 4 ] , with congestion and unbalanced load being the main culprit behind slow downs. While congestion control and load-balancing are two of the most widely studied problems in the context of data centers [ 5 , 6 , 7 ] , AI/ML workloads have different characteristics compared to traditional data center workloads [ 8 ] , requiring different congestion control and load-balancing mechanisms. In particular, traditional data center traffic features a large number of short-lived, small data flows (known as mice flows) with diverse destinations and random start/stop time [ 6 ] . On the other hand, AI/ML traffic consists of fewer long-lived, large data flows (known as elephant flows) that start synchronously and follow specific patterns, such as all-reduce or all-to-all communication [ 9 , 10 ] . As such, while traditional data traffic exhibits high entropy due to randomness, AI/ML workloads has lower entropy with predictable flow patterns. Consequently, optimizing job completion time in AI/ML training requires minimizing tail flow completion time ( FCT ) by using an effective load balancing mechanism to prevent resource bottlenecks and optimize GPU utilization in data centers [ 11 ] . Equal-Cost Multi-Path ( ECMP ) routing [ 12 ] is a standard flow-level load balancing algorithm, which randomly assigns flows to different paths. While ECMP is widely adopted in traditional data centers, it suffers from hash collisions and is not able to adapt to network asymmetry ( e.g. , link failure), both of which increase the tail latency of AI/ML workloads. To address these limitations, some works, e.g. , CONGA [ 5 ] and Letflow [ 13 ] , split a flow into smaller bursts, called flowlets, and dynamically route them across multiple paths. These mechanisms, however, are not flexible as the flowlet gap is set to a fixed value [ 14 ] , affecting the path utilization. Additionally, regardless of the design merits, it is expensive and time-consuming to adopt schemes that require new switch processing, and impractical to rely on designs with centralized controllers ( e.g. , Hedera [ 15 ] ). In the wake of this problem, both academia and major industry players, e.g. , Ultra Ethernet Consortium [ 16 , 17 ] , adopt packet spraying to enhance AI/ML workload performance. Packet spraying reduces collisions by distributing packets of every flow across multiple paths, thereby lowering the likelihood of congestion on any single path and enhancing overall network efficiency for AI/ML traffic [ 18 ] . Nonetheless, random packet spraying results in bursts and queue build up, increasing tail latency. To mitigate these bursts, some approaches, employ Adaptive Routing ( AR ) in which packets are sprayed based on switches’ local queue information [ 19 ] . AR solutions, however, suffer from collisions on higher tiers of the network, since switches decide on the path locally without having visibility over the entire path. To account for the entire path, some studies, such as STrack [ 20 ] and REPS [ 7 ] , distribute packets across different paths based on entropy values ( EV s) derived from a hash function applied to the flow identifier. STrack , however, suffers from stale congestion information and over penalizing the congested paths. The former is due to STrack not using any decay functions to account for drainage of buffers over time and the latter is due to using accumulative penalties. REPS addresses these limitations by employing time-based decay function and non-cumulative penalties. REPS , however, cannot generate perfectly uniform EV s, leading to buffer inflation over time. Additionally, REPS effectiveness is limited when ACK coalescing is used, as fewer ACK s reduce the granularity of congestion signals, leading to stale congestion information and load imbalance in the network. Finally, REPS suffers from performance degradation in the presence of network asymmetry. Our approach. AI/ML workloads requires a LB solution that sustain high network utilization under various scenarios and is robust to ACK coalescing. In particular, the problem we are addressing in this work is, how to have a robust load balancing mechanism that maintains high network utilization for AI/ML workloads while remaining resilient to reduced congestion visibility caused by ACK coalescing by proposing PRIME , a load balancing approach that sprays packets of a flow across different available paths. PRIME is a hybrid switch-host approach, however, unlike complex in-network load balancing implementations, it does not require any switch support besides similar support to ECMP . On the host-side, PRIME is simple and can be implemented in the NIC hardware with minimal memory/area footprint. PRIME ’s design is based on three main insights. Firstly, EV s can be generated based on the network topology in a partially deterministic and controlled randomized manner. Secondly, when congestion is detected on a certain path, the path shouldn’t be taken by any packets until the congestion is resolved. Thirdly, an effective load balancing solution considers not only the severity of congestion but also the time congestion signals were received, avoiding stale information in path selection. To increase the scalability of the proposed LB , PRIME generates a multi-part EV ( MP-EV ), where each part is an EV that specifies the index of the next-hop switch uplink port. The number of parts depends on the network topology, e.g. , a

Read the complete text