Silent Speech Interfaces in the Era of Large Language Models: A Comprehensive Taxonomy and Systematic Review
Kele Xu, Yifan Wang, Ming Feng, Qisheng Xu, Wuyang Chen, Yutao Dou, Cheng Yang, Huaimin Wang · 2026
Read Silent Speech Interfaces in the Era of Large Language Models: A Comprehensive Taxonomy and Systematic Review by Kele Xu, Yifan Wang, Ming Feng, Qisheng Xu, Wuyang Chen, Yutao Dou, Cheng Yang, Huaimin Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-11877.txt.
Introduction
Xu and co-authors review silent speech interfaces, systems that decode speech intent without sound. Normal speech interfaces lean on the acoustic channel, which brings noise, privacy, and impairment problems. The review sorts sensing by where signals are tapped: brain oscillations, muscle activity, articulator motion, and active probing. It then tracks a shift toward language models as strong priors that resolve sparse, drifting biosignals. Modern systems, the authors note, now near the word error levels real use demands.
Catalog record
| CREATOR | Kele Xu, Yifan Wang, Ming Feng, Qisheng Xu, Wuyang Chen, Yutao Dou, Cheng Yang, Huaimin Wang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 74977 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.11877
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Silent Speech Interfaces in the Era of Large Language Models: A Comprehensive Taxonomy and Systematic Review Kele Xu, Yifan Wang, Ming Feng, Qisheng Xu, Wuyang Chen, Yutao Dou, Cheng Yang, Huaimin Wang Abstract Human-computer interaction has traditionally relied on the acoustic channel, a dependency that introduces systemic vulnerabilities to environmental noise, privacy constraints, and physiological speech impairments. Silent Speech Interfaces (SSIs) emerge as a transformative paradigm that bypasses the acoustic stage by decoding linguistic intent directly from the neuro-muscular-articulatory continuum. This review provides a high-level synthesis of the SSI landscape, transitioning from traditional transducer-centric analysis to a holistic intent-to-execution taxonomy. We systematically evaluate sensing modalities across four critical physiological interception points: neural oscillations, neuromuscular activation, articulatory kinematics (ultrasound/magnetometry), and pervasive active probing via acoustic or radio-frequency sensing. Critically, we analyze the current paradigm shift from heuristic signal processing to Latent Semantic Alignment. In this new era, Large Language Models (LLMs) and deep generative architectures serve as high-level linguistic priors to resolve the “informational sparsity” and non-stationarity of biosignals. By mapping fragmented physiological gestures into structured semantic latent spaces, modern SSI frameworks have, for the first time, approached the Word Error Rate usability threshold required for real-world deployment. We further examine the transition of SSIs from bulky laboratory instrumentation to “invisible interfaces” integrated into commodity-grade wearables, such as earables and smart glasses. Finally, we outline a strategic roadmap addressing the “user-dependency paradox” through self-supervised foundation models and define the ethical boundaries of “neuro-security” to protect cognitive liberty in an increasingly interfaced world. I Introduction Human-computer interaction (HCI) has traditionally been predicated on the acoustic channel. Conventional Automatic Speech Recognition (ASR) systems-evolving from classical stochastic modeling to contemporary end-to-end neural architectures-operate under the prerequisite of high-fidelity audio acquisition [ 114 , 148 ] . However, this fundamental reliance on airborne sound introduces systemic vulnerabilities: (i) Environmental Fragility: Performance degrades sharply in non-stationary noise or high-reverberation settings where the signal-to-noise ratio (SNR) is severely compromised [ 91 ] ; (ii) Social and Privacy Constraints: The necessity of audible vocalization leads to sensitive information leakage and “social friction” in public or shared spaces [ 34 ] ; and (iii) Inclusivity Barriers: Acoustic-dependent systems inherently exclude populations with severe phonation disorders, such as those suffering from laryngectomy-induced aphonia or neurodegenerative speech impairments [ 11 ] . Fig. 1: Total number of publications in the field of Silent Speech Interface (SSI). The statistics were retrieved from the Web of Science using the themes “Silent speech”, “Silent speech interface”, “SSI”, “Silent speech recognition”, “Subvocal speech”, “Mute speech recognition”, “Non-audible speech”, “Articulatory speech recognition”, “Articulatory-to-text”, and “Speech without sound”, covering the period from 2011 to 2025. Silent Speech Interfaces (SSIs) emerge as a transformative paradigm that bypasses the acoustic channel by decoding linguistic intent directly from the neuro-muscular-articulatory chain of speech production [ 31 , 118 , 47 , 39 , 174 ] . To further illustrate the rapid growth and increasing research attention in this area, As shown in Fig. 1 , the number of annual publications related to SSI has risen consistently between 2011 and 2025. The speech production process originates from cortical motor intent, propagates via efferent neuromuscular activation, and culminates in the physical kinematics of the vocal tract (e.g., the tongue, lips, and velum). By intercepting this chain at distinct physiological stages—spanning neural oscillations (ECoG/EEG), myoelectric excitation (sEMG), and articulatory trajectories (ultrasound, magnetometry, or computer vision)-SSIs effectively decouple communicative intent from audible pressure waves. This facilitates a direct mapping from physiological precursors to semantic or synthesized acoustic representations, effectively redefining the boundaries of speech communication [ 39 ] . Fig. 2: Framework of this survey paper. The current momentum in SSI research is propelled by three converging technological and societal imperatives: (i) High-Performance Neuroprosthetics: For individuals with profound speech loss, SSIs serve as a critical bridge to reclaiming communicative identity. Recent breakthroughs in high-density neural decoding have demonstrated the feasibility of synthesizing naturalistic speech from cortical activity with word error rates (WER) increasingly competitive with human-level performance [ 157 , 183 ] . (ii) Privacy-Preserving Ubiquity: The proliferation of pervasive AI has intensified the demand for discreet interaction modalities. SSIs enable ‘unvoiced’ commands, allowing users to engage with digital agents in silence, thereby mitigating the risk of acoustic eavesdropping in sensitive environments [ 72 ] . (iii) Acoustic Immunity and Robustness: By operating in the non-acoustic domain, SSIs offer an intrinsic resolution to the “cocktail party problem.” Modalities such as sEMG and active acoustic sensing (AAS) maintain signal integrity irrespective of ambient sound pressure levels, ensuring deterministic performance in extreme industrial or aeronautical contexts [ 67 , 176 ] . The technological landscape of SSIs has undergone a profound metamorphosis over the past decade, transitioning from bulky, laboratory-bound instrumentation to unobtrusive, commodity-grade wearables. Early research predominantly focused on high-precision but restrictive modalities, such as Ultrasound Tongue Imaging (UTI) [ 79 ] or Permanent Magnetic Articulography (PMA) [ 45 ] . However, recent innovations in active sensing and flexible electronics have catalyzed a shift toward consumer-centric form factors. State-of-the-art SSIs now leverage transducers integrated into standard earbuds [ 67 ] , smart glasses [ 176 ] , and wrist-worn devices [ 175 ] to track articulatory dynamics via acoustic echoes or inertial measurement units (IMUs). Notably, the emergence of contactless magnetometry and bio-impedance sensing has introduced high-sensitivity alternatives that circumvent the skin-coupling and motion-artifact challenges inherent in traditional sEMG. In parallel with hardware evolution, the integration of Large Language Models (LLMs) and deep generative architectures has revolutionized the SSI computational backend. A historical bottleneck in SSI recognition has been the “Silent Lombard Effect”-the systematic physiological variance in articulation that occurs in the absence of auditory feedback [ 42 ] . Contemporary frameworks address this via multi-modal contrastive learning and LLM-conditioned rescoring. By aligning fragmented physiological features with high-level linguistic priors in a unified latent space, these systems can robustly reconstruct missing articulatory information, achieving unprecedented accuracy in open-vocabulary, continuous speech tasks [ 9 , 135 ] . Despite these rapid advancements, the field lacks a comprehensive synthesis that bridges the gap between heterogeneous sensing modalities and the latest generative AI paradigms. This work provides a systematic review of the SSI state-of-the-art. We establish a technical taxonomy based on the speech production chain and critically evaluate the transition from traditional classification to modern Transformer-based and diffusion-based architectures. Furthermore, we explore emerging interaction paradigms in SSIs [ 103 ] , while addressing open challenges such as signal non-stationarity, cross-subject domain adaptation, and the ethical implications of “thought-to-speech” neurotechnology. The specific contributions of this review are as follows (as shown in Fig 2 ): • Unified Physiological Taxonomy: We provide a rigorous classification of SSI modalities based on their specific interception points within the neuro-muscular-articulatory chain, offering a comparative analysis of their physical constraints and information capacity. • Algorithmic Evolution Analysis: We synthesize the transition from heuristic feature engineering to deep representation learning, with a specific focus on cross-modal latent alignment, self-supervised pre-training, and LLM-driven decoding frameworks. • Multifaceted Application Scenarios: We delineate the diverse operational landscapes of SSIs, ranging from clinical restorative neuroprosthetics for speech disorders to secure communication in public spaces and high-noise industrial or aerospace contexts. • Benchmarking and Open Science: We provide a curated