Position: Federated Foundation Language Model Post-Training Should Focus on Open-Source Models
Nikita Agrawal, Simon Mertel, Ruben Mayer · 2025
Read Position: Federated Foundation Language Model Post-Training Should Focus on Open-Source Models by Nikita Agrawal, Simon Mertel, Ruben Mayer (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-23593.txt.
Description
Nikita Agrawal, Simon Mertel, and Ruben Mayer argue, in a position paper, that federated post-training of large language models should be built on open-source models rather than black-box ones. Federated learning promises privacy-preserving improvement across many owners' data. Recent work, however, copies the centralized habit of post-training through closed models whose weights and architecture are hidden. Writing in twenty twenty-five, the authors contend that this habit clashes with federation's founding principles: data privacy and participant autonomy. A closed model at the center reintroduces exactly the dependency federation exists to remove.
The paper walks through the dimensions of openness, weights, architecture, data, and governance, and traces what each implies for federated practice. The authors label the piece an opinion with analysis, aimed at steering a young research area before black-box defaults harden into infrastructure.
Catalog record
| CREATOR | Nikita Agrawal, Simon Mertel, Ruben Mayer |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-23593