Acoustic-to-Articulatory Inversion of Clean Speech Using an MRI-Trained Model
Sofiane, Pierre-André, Yves · 2026
Read Acoustic-to-Articulatory Inversion of Clean Speech Using an MRI-Trained Model by Sofiane, Pierre-André, Yves (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-11845.txt.
Introduction
Azzouz and co-authors ask whether clean speech, recorded away from MRI noise, can drive articulatory inversion. MRI scanners corrupt their own audio, so inversion models usually train on denoised MRI speech. The team aligns clean and MRI recordings of the same speaker saying the same sentences. A model trained on denoised MRI speech is then tested on both kinds of audio. Clean speech works well, reaching an RMSE of 1.56 mm, close to the MRI-based score.
Catalog record
| CREATOR | Sofiane, Pierre-André, Yves |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 21258 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.11845
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Acoustic-to-Articulatory Inversion of Clean Speech Using an MRI-Trained Model Sofiane, Pierre-André, Yves Abstract Articulatory acoustic inversion reconstructs vocal tract shapes from speech. Real-time magnetic resonance imaging (rt-MRI) allows simultaneous acquisition of both the acoustic speech signal and articulatory information. Besides the complexity of rt-MRI acquisition, the recorded audio is heavily corrupted by scanner noise and requires denoising to be usable. For practical use, it must be possible to invert speech recorded without MRI noise. In this study, we investigate the use of speech recorded in a clean acoustic environment as an alternative to denoised MRI speech. To this end we compare two signals from the same speaker with identical sentences which are aligned using phonetic segmentation. A model trained on denoised MRI speech is evaluated on both denoised MRI and clean speech. We also assess a model trained and tested only on clean speech. Results show that clean speech supports articulatory inversion effectively, achieving an RMSE of 1.56 mm, close to MRI-based performance. \definecolor arytenoid-cartilagergb0.54, 0.17, 0.89 \definecolor epiglottisrgb0.25, 0.88, 0.82 \definecolor lower-liprgb0.00, 1.00, 0.00 \definecolor pharyngeal-wallrgb0.85, 0.66, 0.13 \definecolor soft-palate-midlinergb0.12, 0.56, 1.00 \definecolor tonguergb1.00, 0.55, 0.00 \definecolor upper-liprgb1.00, 0.00, 1.00 \definecolor vocal-foldsrgb1.00, 0.41, 0.71 Azzouz Vuissoz Laprie keywords Acoustic to articulatory inversion, speech production, clean speech, rt-MRI. † † address: 1 Université de Lorraine, CNRS, Inria, F-54000 Nancy, France 2 Université de Lorraine, Inserm, IADI U1254, F-54000 Nancy, France † † email: [email protected], [email protected], [email protected] 1 Introduction Acoustic-to-articulatory inversion aims to estimate the geometric shape of the vocal tract from the acoustic speech signal. Since the 1970s, several families of solutions have been proposed to address this complex inverse problem. With the availability of measured articulatory data, such as Electromagnetic Articulography (EMA) [ 27 ] and X-ray Microbeam (XRMB) [ 26 ] , it became possible to establish a direct relationship between the acoustic signal and vocal tract geometry. Statistical approaches emerged, including Gaussian Mixture Models (GMM) [ 24 ] and Hidden Markov Models (HMM) [ 9 ] . Later, neural networks significantly improved performance, with architectures such as Bi-LSTM [ 13 , 19 ] , Bi-GRNN [ 28 ] , and Temporal Convolutional Networks (TCN) [ 23 ] . Real-time Magnetic Resonance Imaging (rt-MRI) has been introduced as an alternative to EMA, as it overcomes many of its limitations and constraints by enabling visualization of the entire vocal tract from the glottis to the lips. A first MRI corpus was described in [ 20 ] ; however, it is limited by relatively low image resolution (68 × 68 pixels) and by heavily corrupted audio due to scanner noise. Some studies have directly applied inversion to MRI images [ 7 , 17 ] , while others have proposed generative models, including diffusion-based approaches [ 16 ] . In both cases, performance is typically evaluated at the pixel level of the reconstructed images. An alternative approach consists of automatically segmenting MRI images to extract articulator contours, as proposed in [ 22 ] , which achieves very promising results. In this framework, inversion is performed on articulatory contours rather than raw MRI images. Following this direction, [ 2 ] successfully inverted the tongue only, and [ 3 ] extended this approach to reconstruct the complete vocal tract. Furthermore, the use of higher-resolution images (136 × 136 pixels), combined with better-denoised audio, significantly improves data quality. These aspects will be detailed in Section 3 . The majority of previous works use Mel-Frequency Cepstral Coefficients (MFCCs) or Mel-Generalized Cepstral Line Spectral Pairs (MGC-LSP) [ 7 ] as input features. More recently, Self-Supervised Learning (SSL) methods have demonstrated remarkable performance in acoustic-to-articulatory inversion. In particular, HuBERT [ 10 ] stands out [ 6 ] , outperforming approaches such as wav2vec 2.0 [ 4 ] , WavLM [ 5 ] , as well as classical MFCCs features. Recent studies further confirm the superiority of HuBERT on XRMB data [ 1 ] . Moreover, these representations have proven effective in tasks such as speaker verification [ 25 ] , which is closely related to the objectives of the present study. Figure 1: Comparison of normal speech, MRI-denoised speech from our dataset, and denoised speech from [ 20 ] . The first two signals were produced by the same female speaker uttering “Après une heure.” All three audio files are provided as supplementary material. In this work, we focus on the impact of acoustic signal quality on articulatory inversion, particularly on the transition from noisy speech (recorded in the MRI environment) to clean speech. We use an rt-MRI database that shares similar characteristics with the one described in [ 11 ] , but with higher image resolution (136 × 136 pixels). Our objective is to evaluate the effect of using clean speech instead of noisy speech on the performance of a model trained on MRI data. To ensure temporal alignment between the two types of signals, phonetic alignment is applied. We analyze the impact of this substitution either on the test data only or on both the training and test sets. 2 Impact of MRI noise and denoising Our goal is to be able to use articulatory acoustic inversion in real-world applications, which means that the inversion must be able to use speech recorded in a quiet environment. However, speech recorded in an MRI machine, even if it is denoised before use, is still far from speech produced in a quiet environment. Figure 1 illustrates the difference between a signal recorded in a quiet room and two speech signals recorded in an MRI machine and then denoised, the first from the database we use in this work and the second from the database collected by [ 20 ] . It is clear that the denoising we used gives much better results, but despite this, the denoised signal is still quite different from normal speech (the energy is lower in the high frequencies in particular). It is therefore necessary to adapt the inversion system so that the results remain relevant. The second difference between normal speech and speech produced in an MRI machine comes from the Lombard effect [ 12 ] and the supine position. We do not take this effect into account because we do not have the data to address this issue, as it is currently impossible to have an MRI machine in a vertical position. 3 Dataset We used two different corpus recorded by the same native French female speaker. The first corpus was recorded at the Centre Hospitalier Régional Universitaire (CHRU) de Nancy and contains approximately 2.5 hours of data. It consists of 105 acquisitions, each lasting 80 seconds and containing 4000 MRI frames. The rt-MRI images have a spatial resolution of 136 × 136 pixels, with a temporal resolution of 50 ms per frame (20 fps). The in-plane pixel size is 1.62 mm. The corresponding audio was recorded using an optical microphone at a sampling rate of 16 kHz and was subsequently denoised using [ 18 ] . In addition, we have a phonetic segmentation into 44 phonemes. The alignment was first performed automatically using Astali [ 8 ] and then manually corrected by an expert. The second corpus was recorded in a clean environment, outside the MRI scanner. The same speaker was