Transcript-Prompted Whisper with Dictionary-Enhanced Decoding for Japanese Speech Annotation

Rui, Xiaolong, Jiawang, Shixi, Zhenpeng · 2025

Read Transcript-Prompted Whisper with Dictionary-Enhanced Decoding for Japanese Speech Annotation by Rui, Xiaolong, Jiawang, Shixi, Zhenpeng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-07646.txt.

Description

Rui Hu and four co-authors present a method to label Japanese speech with sound and rhythm marks in a 2025 paper. Such labels are needed to build text to speech data sets, and hand labeling is slow. Their approach fine tunes a large speech recognition model, feeding it the true transcript so it outputs both the text chunks and the labels at once.

A decoding step then fixes sound label errors using dictionary knowledge. In objective tests the method beat older text only and audio only approaches. Speech built from its labels sounded as natural as speech built from hand labels, listeners judged.

Catalog record

CREATORRui, Xiaolong, Jiawang, Shixi, Zhenpeng
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-07646

Browse the full library · About the Tower of Babel