Compact Neural TTS Voices for Accessibility

Kunal Jain, Eoin Murphy, Deepanshu Gupta, Jonathan Dyke, Saumya Shah Vasilieios Tsiaras, Petko Petkov, Alistair Conkie · 2025

Read Compact Neural TTS Voices for Accessibility by Kunal Jain, Eoin Murphy, Deepanshu Gupta, Jonathan Dyke, Saumya Shah Vasilieios Tsiaras, Petko Petkov, Alistair Conkie (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-17332.txt.

Introduction

Jain and co-authors present a study, published in 2025, on compact computer voices for accessibility tools. Screen readers voice text for blind users, and the voice must answer fast and sound decent. Old on-device methods are quick and tiny but robotic. Cloud neural voices sound natural yet lag, which breaks the flow of reading. Recent neural models can run on handhelds, but they still respond slowly and take too much storage for several voices.

The team works on shrinking neural voices so they can live on the device. Their goal is natural sound at the speed of the old methods, with a footprint small enough to preinstall many voices. Users differ in the voice they can follow best, so choice matters. The paper reports their approach and its trade-offs. It aims at voices that ship with the device, ready offline.

Catalog record

CREATORKunal Jain, Eoin Murphy, Deepanshu Gupta, Jonathan Dyke, Saumya Shah Vasilieios Tsiaras, Petko Petkov, Alistair Conkie
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE15 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-17332

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Compact Neural TTS Voices for Accessibility Kunal Jain, Eoin Murphy, Deepanshu Gupta, Jonathan Dyke, Saumya Shah Vasilieios Tsiaras, Petko Petkov, Alistair Conkie Abstract Contemporary text-to-speech solutions for accessibility applications can typically be classified into two categories: (i) device-based statistical parametric speech synthesis (SPSS) or unit selection (USEL) and (ii) cloud-based neural TTS. SPSS and USEL offer low latency and low disk footprint at the expense of naturalness and audio quality. Cloud-based neural TTS systems provide significantly better audio quality and naturalness but regress in terms of latency and responsiveness, rendering these impractical for real-world applications. More recently, neural TTS models were made deployable to run on handheld devices. Nevertheless, latency remains higher than SPSS and USEL, while disk footprint prohibits pre-installation for multiple voices at once. In this work, we describe a high-quality compact neural TTS system achieving latency on the order of 15 ms with low disk footprint. The proposed solution is capable of running on low-power devices. † † address: Apple Inc. 1 Introduction In recent years, text-to-speech (TTS) systems have emerged as crucial tools for enhancing accessibility across various domains. These systems convert written text into spoken words, providing essential support for individuals with visual impairments, reading disabilities, and other conditions that hinder text comprehension [ 1 ] . By enabling auditory consumption of information, TTS systems empower users to engage with digital content independently and effectively. For instance, screen readers equipped with TTS capabilities facilitate access to computers and smartphones for users with visual impairments, offering them the ability to navigate interfaces, read documents, and interact with online resources [ 2 ] . In educational settings, TTS can aid students with dyslexia, aphasia etc. by providing an alternative means to process and understand written material [ 3 ] . For a TTS system used in accessibility settings, other than the need to sound more natural [ 4 ] , additional performance considerations should be taken into account. Stricter latency requirements in these settings is motivated by [ 5 ] . For deployment in resource constrained environments, lower disk footprint and FLOPs also become important. Traditional TTS systems used statistical approaches like Hidden Markov Models [ 6 ] for modeling text to acoustic features. Database-driven unit selection methods to generate audios made up of concatenated speech units have also been widely used [ 7 ] . These systems lack naturalness and sound very robotic, which can be a sub-optimal experience for individuals. Hence, there has been a surge in the research and development of systems that sound very natural and produce almost a human-like speech. These recent approaches [ 8 ] are mostly deep learning based with large runtime computation and disk footprint requirement, which can be a hindrance to deployment on a low-powered device. We develop a compact neural network based TTS system that is not only resource efficient (occupies only about 18 MB) on device, but also computationally efficient in comparison to neural TTS systems typically used on edge devices. In the next section, we outline the baseline architecture used in typical neural TTS systems. These systems use high resources and are not directly usable as a system for accessibility purposes in a resource constrained setting. In the successive sections, we discuss the optimizations necessary to compress the system’s neural components, while meeting various considerations discussed earlier, for application in an accessibility setting. (a) Transformer based Frontend (b) FastSpeech2 based Acoustic Model (c) WaveRNN based Vocoder Figure 1 : Neural Components in the Baseline Architecture In this section, we describe an industry standard neural Text-To-Speech system and for the purpose of comparison, we call this our baseline system. The baseline generates audio given an unnormalized text sequence. The system consists of three neural components placed sequentially: a text processing frontend, an acoustic model and a vocoder, illustrated in Figure 1 . Each of these components is conventionally trained independently. 1.1 Text Processing Frontend The frontend (FE) is the first stage of speech synthesis, and is responsible for text tokenization, verbalization of non-standard words [ 9 ] , grapheme-to-phoneme conversion [ 10 ] , homograph disambiguation [ 11 ] , and post-lexical analysis [ 12 ] . The most important aspect of the FE is Grapheme-to-Phoneme (G2P) conversion. G2P converts input text from a language specific orthographic system to a language agnostic phonemic sequence. To simplify and unify all of the various processing steps in FE, a Transformer based encoder-decoder architecture, as illustrated in Figure 1(a) , is used for the baseline’s frontend. The autoregressive phoneme sequence prediction strategy is described in [ 13 ] . Traditionally, G2P modeling has been done at the word level, however, modeling at the sentence level enables contextual disambiguation. This modification improves performance on ambiguities in the input text, for example with homograph disambiguation. 1.2 Acoustic Model The acoustic model in a text-to-speech system converts the phonemes predicted by the frontend into acoustic representations, typically in the form of mel-spectrograms. FastSpeech2 (FS2) [ 14 ] is a popular acoustic model typically used for this stage. In addition to predicting mel spectrograms, FS2 also learns to model acoustic features like duration, pitch and energy which help in predicting more contextually accurate spectrograms. This model’s ability to generate high-quality speech with fast and stable training has made it a popular choice for low-latency applications. We use FS2 as our baseline acoustic model for the purpose of this work as shown in Figure 1(b) . 1.3 Vocoder The vocoder is tasked with transforming these acoustic representations into raw waveforms. WaveRNN, illustrated in Figure 1(c) , a powerful neural vocoder, achieves high-fidelity speech synthesis by using a recurrent neural network architecture optimized for efficiency without sacrificing quality [ 15 ] . WaveRNN can produce natural-sounding speech in real-time, making it suitable for deployment in resource-constrained environments. 2 Optimizations In this section, we focus on optimizations of each neural component that enabled us to successfully employ the overall system on low-power devices under strict resource and latency constraints for use in accessibility settings. 2.1 Text Processing Front End We used a combination of techniques to achieve the optimization of the baseline frontend without major regression in performance. Inspired by studies on Machine Translation tasks, we used a deep encoder and shallow decoder setting [ 16 ] . Such an encoder-favored parameterization ensures that the features learned by the encoder, which are later used in the decoder’s cross attention, generalize better and retain sentence level information well. Further, having a smaller decoder which is used multiple times during autoregressive inference is desirable. In addition, redundant computations in the autoregressive decoding are reduced using key-value caching. We used parameter sharing motivated by studies on Transformers [ 17 ] to reduce the frontend’s disk footprint. In particular, we used strategies suggested in the paper [ 18 ] . Interleaved sharing of attention weights between encoder and decoder, sharing of feed forward network weights among the encoder layers and load balanced parameterization across the encoder and decoder worked well when applied to baseline frontend. With such scale of sharing involved, we allowed the bias terms to be independent across layers for possible layer adaptation as discussed in paper [ 19 ] . Figure 2 illustrates the parameter sharing. Figure

Read the complete text