Voxtral
2025
Read Voxtral (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-13264.txt.
Introduction
Voxtral is the title of a technical report released in two thousand twenty-five and credited to a very large team of authors. The hosted text consists of the title and the long author list, with no summary of contents. The name and the team's known line of work point to a speech model release: Voxtral is a family of audio models built to take in speech and answer in text. Readers should treat the entry as the release record for that model family rather than a study with findings.
Catalog record
| YEAR | 2025 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-13264
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Voxtral Abstract We present Voxtral Mini and Voxtral Small, two multimodal audio chat models. Voxtral is trained to comprehend both spoken audio and text documents, achieving state-of-the-art performance across a diverse range of audio benchmarks, while preserving strong text capabilities. Voxtral Small outperforms a number of closed-source models, while being small enough to run locally. A 32K context window enables the model to handle audio files up to 40 minutes in duration and long multi-turn conversations. We also contribute three benchmarks for evaluating speech understanding models on knowledge and trivia. Both Voxtral models are released under Apache 2.0 license. Webpage: https://mistral.ai/news/voxtral/ Model weights: https://huggingface.co/mistralai/Voxtral-Mini-3B-2507 https://huggingface.co/mistralai/Voxtral-Small-24B-2507 Evals: https://huggingface.co/collections/mistralai/speech-evals-6875e9b26c78be4a081050f4 1 Introduction This paper describes Voxtral Mini and Voxtral Small, a pair of multimodal language models trained to understand both speech and text, released with open-weights under an Apache 2.0 license. Voxtral is pretrained on a large-scale corpus of audio and text documents, and subsequently instruction tuned on real and synthetic data. It is capable of responding directly to audio (or text) and answering questions about audio files. With a 32K token context window, Voxtral is capable of processing audio files up to 40 minutes long. Compared with similarly sized models in the same evaluation setting, we find that Voxtral delivers strong audio reasoning capabilities without sacrificing text-only performance. Its performance is state-of-the-art for speech transcription and translation, outperforming other open-weights and closed models. In speech question-answering (QA) and summarization, it performs comparably with closed models of a similar price class, such as GPT-4o mini [ Hurst et al. 2024 ] and Gemini 2.5 Flash [ Comanici et al. 2025 ] . During evaluation of Voxtral and other models, we found that the existing ecosystem of speech evaluations lacked breadth and standardization; the majority of previous work focused on evaluation of transcription and translation quality, and less on other understanding tasks. In Section 3.4 , we present evaluations that measure a wider range of speech comprehension and reasoning tasks. Our primary contributions are: • Two open-weights audio models with state-of-the-art transcription and multilingual speech understanding for audio durations up to their 32K context window • Native function calling support with audio • Evaluation benchmarks that measure speech understanding and reasoning The report is structured as follows: First, we outline our modeling choices. Next, we describe methods for pretraining, post-training, and response quality enhancement. Finally, we present benchmark results along with architectural and data ablations. 2 Modeling Voxtral is based on the Transformer architecture [ Vaswani et al. 2017 ] , consisting of three components: an audio encoder to process speech inputs, an adapter layer to downsample audio embeddings, and a language decoder to reason and generate text outputs. The overall architecture is depicted in Figure 1 . Figure 1: Voxtral Architecture. The audio encoder processes the speech input, attending to 30-second chunks of audio independently. The audio embeddings are concatenated at the output, and downsampled by a factor of 4x in the audio-language adapter. The multimodal LLM decoder auto-regressively predicts text tokens, conditional on the audio and text inputs. 2.1 Audio Encoder The audio encoder is based on Whisper large-v3 [ Radford et al. 2023 ] . A raw audio waveform is first mapped to a log-Mel spectrogram [ Davis and Mermelstein 1980 ] with 128 Mel-bins and 160 hop-length. Within the Whisper encoder, the spectrogram passes through a convolutional stem that downsamples its temporal resolution by a factor of two, after which it is fed into a stack of bidirectional self-attention layers. The resulting audio embeddings have a frame rate of 50 Hz. Whisper has a fixed receptive field of 30 seconds. To accommodate audio sequences exceeding this duration, we compute the log-Mel spectrogram for the entire audio, but restrict the encoder to independently process each 30 second chunk. The absolute positional encodings are reset for each chunk, and chunks from the same audio are partitioned into a batch axis. Within the encoder’s attention layers, this approach is functionally equivalent to chunk-wise attention [ Zhang et al. 2023 ] , which mitigates the computational overhead for longer audio inputs and enhances length generalization. The embeddings computed from each chunk are concatenated at the output stage, forming a unified representation of the complete audio sequence. Due to its fixed receptive field, Whisper also pads short audios to 30 seconds. In Section 5.1 , we investigate removing this padding requirement to allow continuous audio lengths. However, empirical results demonstrated a decline in performance, even when tuning the encoder to adapt. Consequently, we maintain the practice of padding all audio inputs to the next multiple of 30 seconds. 2.2 Adapter Layer The high frame-rate of the audio encoder would result in long sequence-lengths through the language decoder. For example, a 30 minute audio at 50Hz has a sequence length of 90k tokens, leading to high memory and slow inference. To circumvent this, we append an additional MLP layer at the audio encoder outputs that is responsible for downsampling the audio embeddings. In Section 5.2 , we show a downsampling factor of 4x yields the best trade-off between sequence-length and performance. This results in an effective frame-rate of 12.5Hz, enabling Voxtral to gracefully handle audios up to 40 minutes with a context-length of 32k tokens. 2.3 Language Decoder We release two variants of Voxtral: Mini and Small. Voxtral Mini is built on top of Ministral 3B [ Mistral AI Team 2024 ] , an edge-focused model that delivers competitive performance with a small memory footprint. Voxtral Small leverages the Mistral Small 3.1 24B backbone [ Mistral AI Team 2025 ] , giving strong performance across a range of knowledge and reasoning tasks. Table 1 decomposes the number of parameters in each checkpoint based on the sub-components. Table 1: Parameter Counts. Number of parameters for Voxtral Mini and Small. Audio Encoder Audio Adapter Text Embeddings Language Decoder Total Mini 640M 25M 400M 3.6B 4.7B Small 640M 52M 670M 22.9B 24.3B 3 Methodology We train the model in three phases: pretraining, supervised finetuning, and preference alignment. Each phase is described separately below. Finally, we describe our evaluation protocol for speech understanding tasks. 3.1 Pretraining The pretraining stage of Voxtral is designed to introduce speech to the language decoder, complementary to the existing modality of text. Given an audio dataset with text transcriptions, we first chunk the audio into short segments together with their corresponding transcription, forming parallel audio-text pairs: (A_{1},T_{1}),(A_{2},T_{2}),\dots,(A_{N},T_{N}) . The segmentation boundaries are defined by upstream voice activity detection and diarization models. If transcripts are unavailable, we pseudo-label the audio with an ASR model. Similar to prior works [ Nguyen et al. 2025 , Zeng et al. 2024 ] , we define two patterns that combine audio and text into training samples for the model: audio-to-text repetition and cross-modal continuation . The audio-to-text repetition pattern is defined as an audio segment A_{n} followed by the corresponding transcription T_{n} . A training sample consists of a single audio-text pair (A_{n},T_{n}) . This formulation mimics speech recognition and is used to explicitly teach the model speech-to-text alignment. On the other hand, the cross-modal continuation pattern is designed to implicitly align the speech and text modalities through modality-invariant context modeling. Specifically, for each audio segment A_{n} , the corresponding