Physics-Informed Neural Networks for Speech Production

Kazuya Yokota, Ryosuke Harakawa, Masaaki Baba, Masahiro Iwahashi · 2025

Read Physics-Informed Neural Networks for Speech Production by Kazuya Yokota, Ryosuke Harakawa, Masaaki Baba, Masahiro Iwahashi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-00428.txt.

Introduction

Kazuya Yokota, Ryosuke Harakawa, Masaaki Baba, and Masahiro Iwahashi bring physics-informed neural networks to the study of how speech is produced. These networks train directly on the equations of vocal-fold vibration and vocal-tract acoustics, promising insight into voice behavior for linguistics and medicine. Two obstacles stand out: vocal-fold collisions break smoothness, and the vibration period is unknown ahead of time.

A differentiable approximation smooths the collision problem, letting folds be analyzed inside the framework. Treating the period as a learnable parameter yields proper periodic solutions. Coupling between airflow at the glottis and tract acoustics is enforced as a hard constraint, with no extra loss terms. Forward and inverse tests confirm the method can estimate airflow, fold state, and pressure below the glottis from speech signals alone.

Catalog record

CREATORKazuya Yokota, Ryosuke Harakawa, Masaaki Baba, Masahiro Iwahashi
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-00428

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Physics-Informed Neural Networks for Speech Production Kazuya Yokota, Ryosuke Harakawa, Masaaki Baba, Masahiro Iwahashi Abstract The analysis of speech production based on physical models of the vocal folds and vocal tract is essential for studies on vocal-fold behavior and linguistic research. This paper proposes a speech production analysis method using physics-informed neural networks (PINNs). The networks are trained directly on the governing equations of vocal-fold vibration and vocal-tract acoustics. Vocal-fold collisions introduce nondifferentiability and vanishing gradients, challenging phenomena for PINNs. We demonstrate, however, that introducing a differentiable approximation function enables the analysis of vocal-fold vibrations within the PINN framework. The period of self-excited vocal-fold vibration is generally unknown. We show that by treating the period as a learnable network parameter, a periodic solution can be obtained. Furthermore, by implementing the coupling between glottal flow and vocal-tract acoustics as a hard constraint, glottis–tract interaction is achieved without additional loss terms. We confirmed the method’s validity through forward and inverse analyses, demonstrating that the glottal flow rate, vocal-fold vibratory state, and subglottal pressure can be simultaneously estimated from speech signals. Notably, the same network architecture can be applied to both forward and inverse analyses, highlighting the versatility of this approach. The proposed method inherits the advantages of PINNs, including mesh-free computation and the natural incorporation of nonlinearities, and thus holds promise for a wide range of applications. Note: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. I Introduction Speech production analysis based on physical models plays an important role in studies of vocal-fold behavior [ 1 , 2 ] , diagnosis of voice disorders [ 3 , 4 ] , and linguistics [ 5 , 6 ] . Studies on the vibratory modes of the vocal folds [ 1 , 7 ] and simulations predicting postsurgical changes in voice [ 8 , 9 ] illustrate the importance of physical modeling in speech research. Model-based studies often attempt to estimate vocal-fold or vocal-tract states directly from speech data [ 10 , 11 , 12 ] , enabling the inference of articulatory states from readily observable signals. However, many of these methods rely on the assumption of independence between the source (vocal-fold vibration) and the filter (vocal-tract acoustics) [ 13 ] . To achieve more realistic physical analyses, inverse methods that include vocal-fold dynamics are required. However, conventional solvers (e.g., finite-difference methods) are designed primarily for forward analyses and require dedicated algorithms for inverse analyses involving vocal-fold vibrations [ 14 ] . In addition, coupled analyses of the vocal folds and vocal tract are inherently multiphysical (structure-fluid-acoustic) problems, making the construction of such models complex. Traditional time and space discretization approaches also suffer from increased computational cost when performing detailed simulations [ 15 ] . Physics-informed neural networks (PINNs) [ 16 ] offer a promising approach to these challenges, having recently attracted attention as powerful numerical frameworks for inverse problems [ 17 , 18 ] . PINNs incorporate governing equations as loss function constraints, enabling mesh-free simulation [ 16 ] , natural handling of nonlinearities [ 19 ] , and inverse analysis using the same network as for forward analysis [ 16 ] . PINN applications in acoustics are expanding, including acoustic admittance estimation from noisy measurements [ 20 ] , acoustic impulse response reconstruction [ 21 ] , and sound pressure distribution estimation [ 22 ] , all demonstrating their potential for acoustic inverse problems. Nevertheless, PINN applications in speech analysis remain limited, with few reports apart from our previous work on vocal-tract acoustics [ 23 ] . Although PINNs based on physical models of speech production hold great promise for diagnosing voice disorders and for inverse mapping of articulatory mechanisms, to our knowledge, no studies have analyzed human speech production using PINNs that explicitly include vocal-fold vibrations. Several challenges arise when applying PINNs to speech-production analysis. First, glottal flow nondifferentiability and vanishing gradients are problematic. Glottal closure during vocal-fold collision introduces points where the time derivatives are undefined [ 7 ] . Moreover, because the flow becomes zero during closure, there are intervals in which the time derivative of the glottal flow is zero. Although PINNs, according to the universal approximation theorem [ 24 ] , can approximate continuous functions, these issues hinder backpropagation training [ 25 ] and degrade learning performance [ 26 , 27 ] . Second, the vocal-fold self-oscillation period is generally unknown. Owing to the spectral bias of PINNs [ 28 , 29 ] , shorter analysis time windows enable a higher frequency resolution [ 30 ] , making it desirable, similar to the shooting method [ 31 ] , to analyze only a single steady-state oscillation cycle. However, the unknown period of self-oscillation prevents the prior determination of collocation points in time. Additional challenges include the multiphysics coupling between the vocal folds and vocal tract, as well as high-frequency acoustic analysis of higher formants, which complicate designing PINNs for speech production. In this paper, we present the first PINNs for speech production to address these challenges. To handle glottal closure, we introduced differentiable approximation functions to facilitate stable learning. To cope with the unknown oscillation period, we treated the period as a learnable parameter of the network and introduced a time-scaling variable, enabling the automatic identification of the period without reconfiguring the collocation points. For glottal flow-vocal tract coupling, we demonstrated that implementing the interaction as a hard constraint eliminates the need for additional loss terms. As a foundational study, the Ishizaka–Flanagan two-mass model [ 1 ] was adopted for the vocal folds, and a one-dimensional acoustic model [ 32 , 23 ] was used for the vocal tract. The validity of the proposed method was confirmed through both forward analysis of vowel generation and inverse analysis, estimating vocal-fold vibration states and subglottal pressure from speech waveforms. In particular, for the inverse analysis, we demonstrate that, using nearly the same network as for the forward analysis, it is possible to simultaneously estimate the glottal flow, vocal-fold vibration waveform, and subglottal pressure from the speech signal without constructing complex algorithms specifically for inversion. The contributions of this study are as follows: 1. The proposal of a PINN that enables speech production analysis including vocal-fold vibration. 2. A methodology for constructing PINNs capable of handling glottal closure, unknown oscillation periods, and coupling between glottal flow and vocal-tract acoustics. 3. The demonstration of the first PINN to simultaneously perform inverse estimation of glottal flow, vocal-fold vibration waveform, and subglottal pressure from a speech signal. The remainder of this paper is organized as follows: Section II describes the governing equations of the vocal-fold and vocal-tract models. Section III introduces the PINN framework for speech production and adapts its architecture to the underlying physics. Section IV presents the forward and inverse analyses using the proposed method and validates the approach. Section V concludes the paper with a summary and discussion of the potential applications of the proposed PINNs in speech production. II Governing Equations of Speech Production This section describes the governing equations of vocal-fold vibration and vocal-tract acoustics used in this study. II-A Two-mass Model of Vocal Folds In this study, the Ishizaka–Flanagan

Read the complete text