Position: Federated Foundation Language Model Post-Training Should Focus on Open-Source Models

Nikita Agrawal, Simon Mertel, Ruben Mayer · 2025

Read Position: Federated Foundation Language Model Post-Training Should Focus on Open-Source Models by Nikita Agrawal, Simon Mertel, Ruben Mayer (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-23593.txt.

Description

Nikita Agrawal, Simon Mertel, and Ruben Mayer argue, in a position paper, that federated post-training of large language models should be built on open-source models rather than black-box ones. Federated learning promises privacy-preserving improvement across many owners' data. Recent work, however, copies the centralized habit of post-training through closed models whose weights and architecture are hidden. Writing in twenty twenty-five, the authors contend that this habit clashes with federation's founding principles: data privacy and participant autonomy. A closed model at the center reintroduces exactly the dependency federation exists to remove.

The paper walks through the dimensions of openness, weights, architecture, data, and governance, and traces what each implies for federated practice. The authors label the piece an opinion with analysis, aimed at steering a young research area before black-box defaults harden into infrastructure.

Catalog record

CREATORNikita Agrawal, Simon Mertel, Ruben Mayer
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-23593

Browse the full library · About the Tower of Babel