Fractional Order Distributed Optimization

Andrei Lixandru 1, Marcel van Gerven 1 and Sérgio Pequito 2 1- Department of Machine Learning and Neural Computing, Donders Institute, Radboud University, The Netherlands 2- Department of Electrical and Computer Engineering, Instituto Superior Técnico, University of Lisbon, Portugal · 2024

Read Fractional Order Distributed Optimization by Andrei Lixandru 1, Marcel van Gerven 1 and Sérgio Pequito 2 1- Department of Machine Learning and Neural Computing, Donders Institute, Radboud University, The Netherlands 2- Department of Electrical and Computer Engineering, Instituto Superior Técnico, University of Lisbon, Portugal (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-02546.txt.

Description

Andrei Lixandru, Marcel van Gerven, and Sérgio Pequito present a 2024 paper on distributed optimization, the math behind training models across many machines. Such methods power federated learning, where data stay on local devices. They often stall on ill-shaped problems or trade speed against stability. The team introduces FrODO, a method that adds fractional-order memory terms, which weigh past steps with a long tail.

The authors prove the method converges at a steady rate on any strongly connected network. In tests it converged up to four times faster than rivals on ill-shaped problems. It also sped federated neural network training by two to three times while staying stable. The paper offers both theory and evidence for the design. The authors aim it at large, spread-out learning systems.

Catalog record

CREATORAndrei Lixandru 1, Marcel van Gerven 1 and Sérgio Pequito 2 1- Department of Machine Learning and Neural Computing, Donders Institute, Radboud University, The Netherlands 2- Department of Electrical and Computer Engineering, Instituto Superior Técnico, University of Lisbon, Portugal
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE14 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-02546

Browse the full library · About the Tower of Babel