Whisfusion: Parallel ASR Decoding via a Diffusion Transformer

Taeyoun Kwon, Junhyuk Ahn, Taegeun Yun, Heeju Jwa, Yoonchae Choi, Siwon Park, Nam-Joon Kim, Jangchan Kim, Hyun Gon Ryu, Hyuk-Jae Lee · 2025

Read Whisfusion: Parallel ASR Decoding via a Diffusion Transformer by Taeyoun Kwon, Junhyuk Ahn, Taegeun Yun, Heeju Jwa, Yoonchae Choi, Siwon Park, Nam-Joon Kim, Jangchan Kim, Hyun Gon Ryu, Hyuk-Jae Lee (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-07048.txt.

Description

Taeyoun Kwon and nine co-workers wrote a 2025 paper on faster speech-to-text with a diffusion decoder. Top transcription models read text out left to right, so wait time grows with length. Fully parallel models are fast but weaker on whole-text flow. Their system, Whisfusion, pairs a frozen speech encoder with a masked diffusion decoder.

The decoder fills in a masked transcript in a few steps, all at once. Training used tens of thousands of hours of speech in many languages. In tests, it beat a leading model on average accuracy across language groups. It also ran several times faster than that model and its turbo variant.

Catalog record

CREATORTaeyoun Kwon, Junhyuk Ahn, Taegeun Yun, Heeju Jwa, Yoonchae Choi, Siwon Park, Nam-Joon Kim, Jangchan Kim, Hyun Gon Ryu, Hyuk-Jae Lee
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-07048

Browse the full library · About the Tower of Babel