: The First Large-Scale Open-Science Speech Foundation Model for English and Italian
Sara Papi, Marco Gaido, Luisa Bentivogli, Alessio Brutti, Mauro CettoloRoberto Gretter, Marco Matassoni, Mohamed Nabih, Matteo Negri Fondazione Bruno Kessler (FBK), Italy {spapi,mgaido,bentivo,brutti,cettolo · 2025
Read : The First Large-Scale Open-Science Speech Foundation Model for English and Italian by Sara Papi, Marco Gaido, Luisa Bentivogli, Alessio Brutti, Mauro CettoloRoberto Gretter, Marco Matassoni, Mohamed Nabih, Matteo Negri Fondazione Bruno Kessler (FBK), Italy {spapi,mgaido,bentivo,brutti,cettolo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-22759.txt.
Introduction
Nine researchers wrote a paper in two thousand twenty-five presenting an open speech model for English and Italian. Famous speech systems are closed: their training data and code are secret, which blocks checking and fair comparison. Open-science models lag behind in speech, unlike in other fields.
Their model family, FAMA, was trained on over one hundred fifty thousand hours of openly licensed speech. The team also released a cleaned, labeled set of sixteen thousand hours for both languages. FAMA matches existing systems on quality while running up to eight times faster. Code, data, and models are all openly released.
Catalog record
| CREATOR | Sara Papi, Marco Gaido, Luisa Bentivogli, Alessio Brutti, Mauro CettoloRoberto Gretter, Marco Matassoni, Mohamed Nabih, Matteo Negri Fondazione Bruno Kessler (FBK), Italy {spapi,mgaido,bentivo,brutti,cettolo |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-22759
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
: The First Large-Scale Open-Science Speech Foundation Model for English and Italian Sara Papi, Marco Gaido, Luisa Bentivogli, Alessio Brutti, Mauro CettoloRoberto Gretter, Marco Matassoni, Mohamed Nabih, Matteo Negri Fondazione Bruno Kessler (FBK), Italy {spapi,mgaido,bentivo,brutti,cettolo Abstract FAMA-medium (878M): https://hf.co/FBK-MT/fama-medium FAMA-small (479M): https://hf.co/FBK-MT/fama-small FAMA Data: https://hf.co/datasets/FBK-MT/fama-data FAMA Code: https://github.com/hlt-mt/FBK-fairseq Abstract The development of speech foundation models (SFMs) like Whisper and SeamlessM4T has significantly advanced the field of speech processing. However, their closed nature–with inaccessible training data and code–poses major reproducibility and fair evaluation challenges. While other domains have made substantial progress toward open science by developing fully transparent models trained on open-source (OS) code and data, similar efforts in speech processing remain limited. To fill this gap, we introduce FAMA, the first family of open science SFMs for English and Italian, trained on 150k+ hours of OS speech data. Moreover, we present a new dataset containing 16k hours of cleaned and pseudo-labeled speech for both languages. Results show that FAMA achieves competitive performance compared to existing SFMs while being up to 8 times faster. All artifacts, including codebase, datasets, and models, are released under OS-compliant licenses, promoting openness in speech technology research. 1 Introduction The development of speech foundation models (SFMs) has significantly advanced speech processing in the last few years, particularly in areas such as automatic speech recognition (ASR) and speech translation (ST). Popular SFMs such as OpenAI Whisper ( Radford et al., 2023 ) and Meta SeamlessM4T ( Barrault et al., 2023 ) have been released to the public in various sizes and with extensive language coverage. However, these models completely lack comprehensive accessibility to their training codebases and datasets, hindering their reproducibility and raising concerns about potential data contamination ( Dong et al., 2024 ) , thereby complicating fair evaluation. In other domains, multiple efforts towards building models that are more accessible, reproducible, and free from proprietary constraints have been made ( BigScience Workshop et al., 2022 ; Biderman et al., 2023 ; Liu et al., 2024 ; Sun et al., 2024 ; Deitke et al., 2024 ; Dai et al., 2024 ; Martins et al., 2025 ) . For instance, the OLMO project ( Groeneveld et al., 2024 ) has demonstrated the feasibility of training large language models (LLMs) using only open-source (OS) data ( Soldaini et al., 2024 ) , realizing an open-science 1 1 1 Open science involves ensuring transparency and accessibility at all stages of the scientific process ( Vicente-Saez and Martinez-Fuentes, 2018 ) , including publishing OS research papers, data, code, and any information needed to replicate the research. system ( White et al., 2024 ) for text processing. However, such comprehensive approaches are still lacking in the field of speech processing. Recent works towards this direction are represented by OWSM ( Peng et al., 2023 ) and its subsequent versions ( Peng et al., 2024 ) . OWSM, whose model weights and the codebase used for the training are released open source, reproduces a Whisper-style training using publicly available data. Despite representing a valuable initiative toward building an open-science system, there is still a step missing for creating the first SFM of this kind: leveraging only data that is not only publicly available but also released under an OS-compliant license ( Gaido et al., 2024a ) . Such effort would allow users complete access and control over the data used at every stage of the scientific process, promoting reproducibility ( Belz et al., 2023 ) , fair evaluation ( Balloccu et al., 2024 ) , and the ability to build upon prior research without any barriers ( Chesbrough, 2015 ) . Besides transparency and collaboration, these efforts also foster users’ trust by ensuring that data is not leveraged to build tools that can be used under conditions/purposes (e.g., commercial) for which the data was not intended ( White et al., 2024 ) . To fill this gap, we create FAMA , 2 2 2 Fama (from the Latin “fari” meaning “to speak” ) is the personification of the public voice in Roman mythology. the first family of large-scale open-science SFMs for English and Italian trained on over 150k hours of exclusively OS-compliant speech data. We leverage both already available OS datasets and create a new collection of ASR and ST psuedolabels for Italian and English comprising more than 16k hours of OS-compliant speech, along with automatically generated Italian and English translations for an additional 130k+ hours of speech. We also detail training and evaluation procedures and provide full access to training data to have complete control of the model creation and avoid data contamination issues. FAMA models achieve remarkable results, with up to 4.2 WER and 0.152 COMET improvement on average across languages compared to OWSM and remaining competitive in terms of ASR performance with the Whisper model family while being up to 8 times faster. All the artifacts used for realizing FAMA models, including codebase, datasets, and models themself, are released under OS-compliant licenses, promoting a more responsible creation of models in our community. Our approach would not only facilitate fair evaluation and comparison of SFMs but also encourage broader participation in speech technology development, leading to more inclusive and diverse applications. 2 The FAMA Framework 2.1 Training and Evaluation Data In compliance with the open-science ideology, we train and test our models only on OS-compliant data. The training set comprises both already publicly available OS datasets, and new pseudolabels created for this work, whose list is presented in Table 1 . Dataset #hours Label en it CommonVoice v18 ( Ardila et al., 2020 ) 1746 250 G CoVoST2 ( Wang et al., 2021b ) 420 28 G FLEURS ( Conneau et al., 2023 ) 7 9 G LibriSpeech ( Panayotov et al., 2015 ) 358 - G MOSEL ( Gaido et al., 2024a ) 66,301 21,775 A MLS ( Pratap et al., 2020 ) 44,600 247 G VoxPopuli-ASR ( Wang et al., 2021a ) 519 74 G YouTube-Commons ( our paper ) 14,200 1,828 A Total 128,152 24,211 G+A Table 1: List of both publicly available training data and the data created in this paper for English (en) and Italian (it). “G” stands for gold labels while “A” for automatically generated labels (transcripts). To create the new pseudolabels, we leveraged the speech content of YouTube-Commons, 3 3 3 https://hf.co/datasets/PleIAs/YouTube-Commons a dataset collecting YouTube videos released with the permissive CC-BY 4.0 license. The videos are automatically converted into wav files with one channel and a sampling rate of 16k Hz. Then, the audio is cleaned from music and non-speech phenomena and segmented using silero ( Team, 2024 ) , a lightweight VAD having low computational requirements. Lastly, the audio is split using SHAS ( Tsiamas et al., 2022 ) to obtain segments suitable for training of around 16 seconds on average. The resulting dataset contains automatic transcripts, which we created with Whisper large-v3 , 4 4 4 https://hf.co/openai/whisper-large-v3 for 14,200 hours of speech for English ( en ) and 1,828 for Italian ( it ). Including publicly available data (113,951 hours for en , and 22,383 hours for it ), the final ASR training set comprises 128,152 hours of en speech and 24,211 hours of it speech, with a total of