Fractional Order Distributed Optimization
Andrei Lixandru 1, Marcel van Gerven 1 and Sérgio Pequito 2 1- Department of Machine Learning and Neural Computing, Donders Institute, Radboud University, The Netherlands 2- Department of Electrical and Computer Engineering, Instituto Superior Técnico, University of Lisbon, Portugal · 2024
Read Fractional Order Distributed Optimization by Andrei Lixandru 1, Marcel van Gerven 1 and Sérgio Pequito 2 1- Department of Machine Learning and Neural Computing, Donders Institute, Radboud University, The Netherlands 2- Department of Electrical and Computer Engineering, Instituto Superior Técnico, University of Lisbon, Portugal (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-02546.txt.
Description
Andrei Lixandru, Marcel van Gerven, and Sérgio Pequito present a 2024 paper on distributed optimization, the math behind training models across many machines. Such methods power federated learning, where data stay on local devices. They often stall on ill-shaped problems or trade speed against stability. The team introduces FrODO, a method that adds fractional-order memory terms, which weigh past steps with a long tail.
The authors prove the method converges at a steady rate on any strongly connected network. In tests it converged up to four times faster than rivals on ill-shaped problems. It also sped federated neural network training by two to three times while staying stable. The paper offers both theory and evidence for the design. The authors aim it at large, spread-out learning systems.
Catalog record
| CREATOR | Andrei Lixandru 1, Marcel van Gerven 1 and Sérgio Pequito 2 1- Department of Machine Learning and Neural Computing, Donders Institute, Radboud University, The Netherlands 2- Department of Electrical and Computer Engineering, Instituto Superior Técnico, University of Lisbon, Portugal |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 14 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-02546