Predicting Compact Phrasal Rewrites with Large Language Models for ASR Post Editing

Hao Zhang, Felix Stahlberg, Shankar Kumar · 2025

Read Predicting Compact Phrasal Rewrites with Large Language Models for ASR Post Editing by Hao Zhang, Felix Stahlberg, Shankar Kumar (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-13831.txt.

Introduction

Hao Zhang and co-authors speed up speech transcript editing by predicting compact rewrites, in a 2025 paper. Automatic speech recognition often needs a cleanup pass. Large language models rewrite well, but they pay full decoding cost for every output word. Input and output mostly overlap, so much of that work is wasted.

Earlier work compressed rewrites into edit spans, cutting output length sharply. This paper proposes edit phrase forms borrowed from phrase-based machine translation. The authors compare the two schemes across rewriting tasks. The goal is the same accuracy with far fewer generated tokens. Faster post-editing matters for live captioning and voice tools. The study gives builders a clear map of the trade-offs.

Catalog record

CREATORHao Zhang, Felix Stahlberg, Shankar Kumar
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE18 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-13831

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Predicting Compact Phrasal Rewrites with Large Language Models for ASR Post Editing Hao Zhang, Felix Stahlberg, Shankar Kumar Abstract Large Language Models (LLMs) excel at rewriting tasks such as text style transfer and grammatical error correction. While there is considerable overlap between the inputs and outputs in these tasks, the decoding cost still increases with output length, regardless of the amount of overlap. By leveraging the overlap between the input and the output, Kaneko and Okazaki [ 1 ] proposed model-agnostic edit span representations to compress the rewrites to save computation. They reported an output length reduction rate of nearly 80% with minimal accuracy impact in four rewriting tasks. In this paper, we propose alternative edit phrase representations inspired by phrase-based statistical machine translation. We systematically compare our phrasal representations with their span representations. We apply the LLM rewriting model to the task of Automatic Speech Recognition (ASR) post editing and show that our target-phrase-only edit representation has the best efficiency-accuracy trade-off. On the LibriSpeech test set, our method closes 50-60% of the WER gap between the edit span model and the full rewrite model while losing only 10-20% of the length reduction rate of the edit span model. I Introduction Large Language Models pretrained on vast amounts of texts and then fine-tuned, instruction-tuned, or prompted for generation tasks have obtained remarkable success in the past few years [ 2 , 3 , 4 , 5 , 6 ] . These models excel at text rewriting tasks, and text style transfer [ 7 ] and grammatical error correction [ 8 , 9 ] in particular. However the superior quality of these models comes along with a steep increase in the cost of computation. To enable broad deployment for a large user base, it is crucial to reduce the computational cost while maintaining the accuracy. One of the common characteristics of the above-mentioned rewriting tasks is that their output often repeats spans of text in the input. Exploiting the common sub-strings between the input and the output can result in more compact output representations. LLMs can be fine-tuned on examples that map inputs to their compact rewrite representations instead of plain rewrites. At inference time, such a decoding output needs to be composed with the input to expand into complete rewrites. Kaneko and Okazaki [ 1 ] proposed one such representation, which is a numerical span indexing into the input sequence followed by a target phrase that will substitute the source phrase in the given span. We propose two new alternative representations. The first one uses a source-target phrase pair to represent each rewrite pattern, analogous to phrase pairs used by phrase-based statistical machine translation [ 10 ] . The second representation only uses a target phrase along with left and right context words that appear in the input. We call the new representations phrase representations to distinguish them from the span representation [ 1 ] . The clear advantage of compact representations over complete rewrites is that the number of decoding steps, and hence the computational cost of inference, is reduced. [ 1 ] reported an output length reduction rate of 80%. The disadvantage of such representations is that decoding errors can lead to inconsistency with the input sequence in the expansion stage, causing an error propagation effect. For example, using the numerical span representation, if the left index or the right index is off by one, the source phrase to be substituted will also be off by one. The concatenation of the context and the substitution can therefore become disfluent. With phrase representations, context words are provided before and after substitution phrases, which can alleviate the problem of disfluency upon substitution. However, phrase representations have their own problems too. The predicted context phrase may not match the input, which makes it necessary to discard the subsequent rewrite. The main focus of the paper is to evaluate the efficiency-accuracy trade-off of the different representations. We choose Automatic Speech Recognition (ASR) post editing as the task for applying LLM-based rewrite models and report word error rates (WER) and output length reduction rates using the span representation and the full rewrite models as the baselines. Our contributions include the following: • We propose a compact edit string representation with superior efficiency-accuracy trade-off than [ 1 ] . • We apply edit representation based rewriting LLMs to the task of ASR output correction. To the best of our knowledge, our work is the first to combine compact rewriting with generative LLMs to achieve substantial ASR WER reduction with manageable decoding cost. TABLE I: Example output under various representations. We underline the numerical spans or substrings to be matched against the source text in the expansion stage. source text Since we do not to bring cash to pay for the transportation fee , enormous time have been saved target text Since we do not need to bring cash to pay for the transportation fee , enormous time has been saved span 4 4 need, 16 17 has phrase pair rewrite: not to , not need to, rewrite: time have been , time has been target only rewrite: not need to , rewrite: time has been Fig. 1: Visualization of the span, phrase pair, and target phrase representations. II Rewrite Representations Mathematically speaking, for rewrite examples (\mathbf{x},\mathbf{y}) , where \mathbf{x} is the input string and \mathbf{y} is the output string, there is a compression function \mathrm{C} and an expansion function \mathrm{E} satisfying \mathrm{E}(\mathbf{x},\mathrm{C}(\mathbf{x},\mathbf{y}))=\mathbf{y} (1) with the constraint that |\mathrm{C}(\mathbf{x},\mathbf{y})| has a much smaller average value than |\mathbf{y}| . At training time, examples are converted to (\mathbf{x},\hat{\mathbf{y}}=\mathrm{C}(\mathbf{x},\mathbf{y})) , where \hat{\mathbf{y}} is the reference output. At inference time, the final output is obtained by applying the expansion function \mathrm{E}(\mathbf{x},\mathbf{y^{\prime}}) where \mathbf{y^{\prime}} is the decoding output for \mathbf{x} . The edit representations in this section differ in the choice of the function pair \mathrm{C} and \mathrm{E} . II-A Edit Span Representation The span representation [ 1 ] is derived from a word alignment graph \mathbf{a} between \mathbf{x} and \mathbf{y} . Given a bipartite alignment graph between the input sequence and the output sequence, we can identify pairs of word spans between the two sides. Each span pair is a local rewrite instance indicating that the source span is substituted by the target span. In practice, the alignment is derived from the Levenshtein distance algorithm with the guarantee that the alignment links are monotonically ordered. It is always feasible to represent the entire rewrite as a sequence of local rewrite spans. For LLMs to predict the rewrites, we need a string representation of the span pairs. In their paper, the span representation is specified as (i,j,\;\mathbf{y}_{\mathbf{a}(i...j)}) , where \mathbf{a}(i...j) is the corresponding target span of a source span i...j and \mathbf{y}_{\mathbf{a}(i...j)} is the target phrase in this span. Under this representation, \mathrm{C} is the concatenation of the ordered span representations: \mathrm{C}=\oplus_{(i,j)\in\mathbf{a}}{(i,j,\;\mathbf{y}_{\mathbf{a}(i...j)})}. (2) \mathrm{E} is the program of applying the ordered local rewrites to the input sequence. II-B Phrase Pair Representation The representation in Equation 2 is concise. It uses a pair of integers to represent a source span. However, this implies that LLMs have to count source tokens and generate indexing integer tokens interleaved with content tokens following the

Read the complete text