ZCCL: Significantly Improving Collective Communication With Error-Bounded Lossy Compression

Jiajun Huang, Sheng Di, Xiaodong Yu, Yujia Zhai, Zhaorui Zhang, Jinyang Liu, Xiaoyi Lu, Ken Raffenetti, Hui Zhou, Kai Zhao, Khalid Alharthi, Zizhong Chen, Franck Cappello, Yanfei Guo, Rajeev Thakur · 2025

Read ZCCL: Significantly Improving Collective Communication With Error-Bounded Lossy Compression by Jiajun Huang, Sheng Di, Xiaodong Yu, Yujia Zhai, Zhaorui Zhang, Jinyang Liu, Xiaoyi Lu, Ken Raffenetti, Hui Zhou, Kai Zhao, Khalid Alharthi, Zizhong Chen, Franck Cappello, Yanfei Guo, Rajeev Thakur (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-18554.txt.

Description

Jiajun Huang and a large team tackle slow communication on supercomputers in this two thousand twenty-five paper. Big simulations spread work across many machines that must swap data in lockstep. These collective swaps can choke overall speed as jobs grow. Compressing the messages helps, but off-the-shelf compressors give weak gains and unbounded errors. The team built a tool, ZCCL, around lossy compression with a strict error bound.

Their method shrinks messages a great deal while keeping every error inside the promised limit. That promise matters for science, where wild errors could spoil results. Tests show large cuts in communication time. The tool slots into the standard MPI library used by most supercomputer codes. It should interest teams that run huge parallel simulations.

Catalog record

CREATORJiajun Huang, Sheng Di, Xiaodong Yu, Yujia Zhai, Zhaorui Zhang, Jinyang Liu, Xiaoyi Lu, Ken Raffenetti, Hui Zhou, Kai Zhao, Khalid Alharthi, Zizhong Chen, Franck Cappello, Yanfei Guo, Rajeev Thakur
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-18554

Browse the full library · About the Tower of Babel