WhisperFlow: speech foundation models in real time Conference: The 23rd Annual International Conference on Mobile Systems, Applications and Services; June 23–27, 2025; Anaheim, CA, USA The 23rd Annual International Conference on Mobile Systems, Applications and Services (MobiSys ’25), June 23–27, 20

Rongxiang Wang, Zhiming Xu, Felix Xiaozhu Lin · 2024

Read WhisperFlow: speech foundation models in real time Conference: The 23rd Annual International Conference on Mobile Systems, Applications and Services; June 23–27, 2025; Anaheim, CA, USA The 23rd Annual International Conference on Mobile Systems, Applications and Services (MobiSys ’25), June 23–27, 20 by Rongxiang Wang, Zhiming Xu, Felix Xiaozhu Lin (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-11272.txt.

Description

Rongxiang Wang, Zhiming Xu, and Felix Xiaozhu Lin present WhisperFlow in a 2024 paper on live speech software. Speech models like Whisper read recorded audio with high accuracy, yet they handle live streams poorly. They expect long fixed clips, encode many tokens per clip, and search hard for each output word, which makes live use costly on small devices.

WhisperFlow adds three fixes. A short learned sound clip tells the model to stop cleanly at the end of each chunk. A pruning step reuses earlier decoding work as new audio arrives. A pipeline also splits work between the two kinds of processor in a phone-class chip. On modest hardware, the system cut the delay per word by up to about five times, with almost no loss in accuracy.

Catalog record

CREATORRongxiang Wang, Zhiming Xu, Felix Xiaozhu Lin
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE64 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-11272

Browse the full library · About the Tower of Babel