Addressing the Collaboration Dilemma in Low-Data Federated Learning via Transient Sparsity

Qiao Xiao, Boqian Wu, Andrey Poddubnyy, Elena Mocanu, Phuong H. Nguyen, Mykola Pechenizkiy, Decebal Constantin Mocanu · 2025

Read Addressing the Collaboration Dilemma in Low-Data Federated Learning via Transient Sparsity by Qiao Xiao, Boqian Wu, Andrey Poddubnyy, Elena Mocanu, Phuong H. Nguyen, Mykola Pechenizkiy, Decebal Constantin Mocanu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-00932.txt.

Description

Qiao Xiao and co-authors tackle a weak point in federated learning, where many devices train one shared model without sharing raw data. Their 2025 paper studies settings where each client holds little data and the data differ across clients. In such cases, joint work can stall instead of helping.

The authors name a pattern they call layer-wise inertia: middle layers of the shared model barely change after early rounds. They show the pattern across many data sets and model shapes. Their fix, called LIPS, adds brief bursts of sparsity to wake those layers up. Tests show better merging of updates and stronger overall results in varied federated settings.

Catalog record

CREATORQiao Xiao, Boqian Wu, Andrey Poddubnyy, Elena Mocanu, Phuong H. Nguyen, Mykola Pechenizkiy, Decebal Constantin Mocanu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-00932

Browse the full library · About the Tower of Babel