GigaAM: Efficient Self-Supervised Learner for Speech Recognition
Aleksandr, Alexandr, Georgii, Pavel, Fyodor · 2025
Read GigaAM: Efficient Self-Supervised Learner for Speech Recognition by Aleksandr, Alexandr, Georgii, Pavel, Fyodor (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-01192.txt.
Description
Aleksandr Kutsakov, Alexandr Maximenko, Georgii Gospodinov, Pavel Bogomolov, and Fyodor Minkin train a family of speech recognizers by self-supervision. Their pretraining masks spans of audio and predicts targets drawn from a speech recognition model itself. Chunkwise attention with randomly sampled chunk sizes serves two masters at once: full-context accuracy and streaming deployment. Scaling experiments vary model size and data volume.
The resulting GigaAM family includes a Russian speech recognizer the authors call state of the art. It beats a leading multilingual rival by fifty percent on their measure. Foundation and recognition models are released openly under a permissive license, with inference code. The work shows masked prediction with recognition-derived targets as a strong recipe. It also gives the Russian speech community a serious open baseline.
Catalog record
| CREATOR | Aleksandr, Alexandr, Georgii, Pavel, Fyodor |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 20 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-01192