UCCL-Zip : Lossless Compression Supercharged GPU Communication

Shuang Ma † ChonLam Lao ‡ Zhiying Xu § Zhuang Wang § Ziming Mao ¶ Delong Meng § Zhen Jia § Jun Wu § Ion Stoica ¶ Yida Wang § Yang Zhou † † UC Davis ‡ Harvard University § Amazon Web Services ¶ UC Berkeley · 2026

Read UCCL-Zip : Lossless Compression Supercharged GPU Communication by Shuang Ma † ChonLam Lao ‡ Zhiying Xu § Zhuang Wang § Ziming Mao ¶ Delong Meng § Zhen Jia § Jun Wu § Ion Stoica ¶ Yida Wang § Yang Zhou † † UC Davis ‡ Harvard University § Amazon Web Services ¶ UC Berkeley (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-17172.txt.

Introduction

Ma and ten colleagues build lossless compression into GPU links, avoiding the numerical errors of quantization or lossy compression. Their UCCL-Zip design covers point-to-point and collective communication without changing user-facing APIs. A split-send pipeline overlaps squeezing and transfer, while fused execution folds squeezing into NCCL's persistent kernel model. In real workloads, it speeds RL weight synchronization by up to 47.5 percent and cuts vLLM inference latency by up to 10 percent, with no application changes.

Catalog record

CREATORShuang Ma † ChonLam Lao ‡ Zhiying Xu § Zhuang Wang § Ziming Mao ¶ Delong Meng § Zhen Jia § Jun Wu § Ion Stoica ¶ Yida Wang § Yang Zhou † † UC Davis ‡ Harvard University § Amazon Web Services ¶ UC Berkeley
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54096
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.17172

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

UCCL-Zip : Lossless Compression Supercharged GPU Communication Shuang Ma † ChonLam Lao ‡ Zhiying Xu § Zhuang Wang § Ziming Mao ¶ Delong Meng § Zhen Jia § Jun Wu § Ion Stoica ¶ Yida Wang § Yang Zhou † † UC Davis ‡ Harvard University § Amazon Web Services ¶ UC Berkeley Abstract . The rapid growth of large language models (LLMs) has made GPU communication a critical bottleneck. While prior work reduces communication volume via quantization or lossy compression, these approaches introduce numerical errors that can degrade convergence, accuracy, and stability. We present UCCL-Zip , a unified design that integrates lossless compression directly into GPU communication primitives. UCCL-Zip supports both point-to-point (P2P) and collective communication without modifying user-facing APIs or compromising numerical correctness. For P2P communication, Uzip-P2P employs a split-send pipeline that exposes transmissible data early and overlaps compression with communication, while preserving high GPU efficiency by operating on large data blocks. For collective communication, Uzip-NCCL integrates compression into NCCL’s persistent kernel model via fused execution, eliminating redundant memory traffic and kernel launches. In real workloads, UCCL-Zip accelerates RL weight synchronization by up to 47.5% and reduces vLLM end-to-end inference latency by up to 10%, all without application changes. 1. Introduction The rapid growth of model size and context length in large language models (LLMs) ( 9 ; 15 ; 11 ; 40 ) has made multi-node, multi-GPU execution indispensable for both training and inference. In such settings, large volumes of data, including gradients, model parameters, activations, and intermediate states, are exchanged across GPUs within and across nodes. Collective communication ( 27 ; 2 ; 43 ) and point-to-point (P2P) communication are two widely used communication primitives in distributed GPU systems. Collective communication is commonly used in regular and structured communication patterns, such as tensor parallelism (TP), pipeline parallelism (PP), and other distributed training paradigms ( 36 ; 42 ; 31 ) . P2P communication provides greater flexibility and is commonly used in scenarios such as KV cache transfer ( 46 ; 39 ; 33 ; 35 ) and asynchronous weight updates in reinforcement learning workloads. As a result, GPU communication performance has become a critical bottleneck ( 5 ; 41 ; 6 ) , directly impacting the efficiency and scalability of LLM training and inference. To reduce communication overhead, prior work has extensively explored quantization and other lossy compression techniques to reduce data volume ( 1 ; 48 ; 30 ; 23 ; 50 ; 44 ) . While effective in improving bandwidth utilization, these approaches inevitably introduce numerical errors. These errors can slow convergence during training or degrade model accuracy at inference time, and may also cause training–inference mismatch in RL ( 18 ) . More recently, lossless compression techniques on GPUs have begun to emerge ( 34 ; 19 ; 17 ; 52 ; 37 ; 3 ) , demonstrating the potential to reduce data size without sacrificing numerical fidelity. This observation opens up a promising opportunity: applying lossless compression to GPU communication. However, despite its potential, this direction remains largely underexplored, particularly in the context of tightly integrated communication pipelines. (a) Baseline. (b) Uzip-P2P . (c) Uzip-NCCL . Figure 1 . Overview of UCCL-Zip . (a) Naive design suffers from lack of overlap and additional kernel overhead. (b) Uzip-P2P enables early transmission and overlaps compression with communication; detailed steps are shown in Figure 4 . (c) Uzip-NCCL integrates compression into the persistent kernel, eliminating redundant memory traffic and extra kernel launches. Effectively hiding the overhead of lossless compression within the communication pipeline remains a key challenge. On modern GPUs, compression latency does not scale linearly with input size due to their highly parallel execution model, where throughput is maximized only when sufficient parallelism is exposed to fully utilize hardware resources ( 38 ) . As a result, compressing smaller chunks often incurs similar latency to larger ones, undermining conventional fine-grained pipelining strategies that partition data into small blocks and overlap compression with transmission. In such designs, excessive fragmentation can even amplify overheads, leading to degraded end-to-end performance. As shown in Figure 1(a) , this challenge is further exacerbated in collective communication. Libraries such as NCCL launch a single kernel per message to orchestrate computation and communication. Data is processed at a fine granularity, typically on the order of a few kilobytes per chunk, and staged through internal buffers to enable overlap. Directly inserting a standalone compression stage into this workflow introduces additional kernel launches and redundant memory traffic, which can negate the benefits of compression. In this work, we present UCCL-Zip , a unified design that integrates lossless compression directly into GPU communication primitives, thereby addressing the fundamental tension between compression overhead and communication efficiency. UCCL-Zip seamlessly supports both collective and P2P communication, enabling bandwidth reduction without modifying user-facing APIs or compromising numerical correctness. Our approach consistently improves effective bandwidth and, in many cases, approaches or exceeds the hardware-imposed network limit on modern GPU clusters. For P2P communication, Uzip-P2P employs a compression-aware split-send pipeline that aligns with the structure of floating-point compression (Section 3.2 ). As shown in Figure 1(b) , Uzip-P2P processes data in sufficiently large chunks to maintain high GPU utilization, and decouples compression stages to expose data that can be transmitted early. This allows a significant portion of the payload to be sent over the network before full compression is complete, effectively overlapping communication with compression. For collective communication, such as AllReduce, Uzip-NCCL integrates compression into NCCL’s persistent kernel execution model (Section 3.4 ). As shown in Figure 1(c) , we redesign the compression workflow by introducing localized frequency tables that enable block-wise, on-the-fly entropy modeling, eliminating the need for global preprocessing. We then fully fuse compression into the compute–communication pipeline, avoiding extra kernel launches and reducing redundant HBM accesses. By aligning compression with the execution granularity of the collective loop, compression and communication proceed in a tightly coupled streaming fashion without introducing additional synchronization or kernel launches. UCCL-Zip is designed to be portable across heterogeneous hardware platforms. It supports multiple GPU architectures, including AMD and NVIDIA devices, and operates over diverse RDMA-capable interconnects such as NVIDIA CX7, Broadcom Thor-2, and AWS EFA NICs. In addition, UCCL-Zip extends DietGPU ( 34 ) to support float8 compression, enabling compatibility with a wide range of floating-point formats commonly used in modern machine learning workloads, including bfloat16 , float16 , float32 , float8_e4m3fn , and float8_e5m2 . This broad compatibility enables UCCL-Zip to be readily deployed in existing training and inference systems without requiring changes to data representation or communication backends. We evaluate UCCL-Zip across a diverse set of GPU clusters, including multi-node RDMA environments with EFA and RoCEv2 (Section 5 ). In reinforcement learning training, Uzip-P2P improves weight synchronization throughput by up to 47.5% on large tensors. In distributed LLM inference with vLLM, Uzip-NCCL reduces end-to-end inference latency by up to 10%. These benefits are achieved transparently, without modifying application code. We also reveal the architecture incompatibility of the widely-used NCCL with lossless compression, and identify concrete future directions in designing a better collective library (e.g., two-shot collectives and increasing chunk sizes). UCCL-Zip has been open-sourced at https://github.com/uccl-project/uccl/tree/main/p2p. 2. Background and Motivation 2.1. Basic GPU Lossless Compression Design Tensor Type Size (MB) Data

Read the complete text