GigaAM: Efficient Self-Supervised Learner for Speech Recognition

Aleksandr, Alexandr, Georgii, Pavel, Fyodor · 2025

Read GigaAM: Efficient Self-Supervised Learner for Speech Recognition by Aleksandr, Alexandr, Georgii, Pavel, Fyodor (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-01192.txt.

Description

Aleksandr Kutsakov, Alexandr Maximenko, Georgii Gospodinov, Pavel Bogomolov, and Fyodor Minkin train a family of speech recognizers by self-supervision. Their pretraining masks spans of audio and predicts targets drawn from a speech recognition model itself. Chunkwise attention with randomly sampled chunk sizes serves two masters at once: full-context accuracy and streaming deployment. Scaling experiments vary model size and data volume.

The resulting GigaAM family includes a Russian speech recognizer the authors call state of the art. It beats a leading multilingual rival by fifty percent on their measure. Foundation and recognition models are released openly under a permissive license, with inference code. The work shows masked prediction with recognition-derived targets as a strong recipe. It also gives the Russian speech community a serious open baseline.

Catalog record

CREATORAleksandr, Alexandr, Georgii, Pavel, Fyodor
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE20 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-01192

Browse the full library · About the Tower of Babel