Fabricator or dynamic translator?
Lisa Vasileva, Karin Sim · 2026
Read Fabricator or dynamic translator? by Lisa Vasileva, Karin Sim (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-15165.txt.
Introduction
Vasileva and Sim sort the extra text language models add when translating, asking when it fabricates and when it genuinely helps. Overgeneration ranges from model self-explanations to risky confabulations to apt explanations of the kind a human translator might offer. Telling these apart is hard, yet it matters for readers who need to grasp the target text. The paper details strategies tried in commercial work and reports what each revealed.
Catalog record
| CREATOR | Lisa Vasileva, Karin Sim |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38205 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.15165
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Fabricator or dynamic translator? Lisa Vasileva, Karin Sim Abstract LLMs are proving to be adept at machine translation although due to their generative nature they may at times overgenerate in various ways. These overgenerations are different from the neurobabble seen in NMT and range from LLM self-explanations, to risky confabulations, to appropriate explanations, where the LLM is able to act as a human translator would —enabling greater comprehension for the target audience. Detecting and determining the exact nature of the overgenerations is a challenging task. We detail different strategies we have explored for our work in a commercial setting, and present our results. 1 Introduction Prior to Large Language Models (LLMs), machine translation was most recently via encoder-decoder models which encoded the source sequentially and decoded into the target language. LLMsare generative models, and as a result of their very nature we see different types of issues from those seen in Neural Machine Translation (NMT). Analysing these issues with Machine Translation (MT)output from LLMs, it has become apparent that there are a range of overgenerations which occur. In NMT, additional content in the target was often in the format of hallucinations, where we could detect repetitions of single words, or neurobabble. For LLMsthere may be hallucinations which are detached from the source to varying degrees, meaning that they are not grounded in the source text. We find that these overgenerations themselves differ in the type of content which they contain. There may be instructions or explanations from the Large Language Model (LLM), or they may be totally related and coherent content, simply not in the source text but rather hallucinations, or more appropriately confabulations ( 14 ) . We research the question: what types of overgenerations do we find in the real world of commercial MTwith LLMs, and what type of content do they contain? We have found that working with our Machine Translation Quality Estimation (MTQE)models in a commercial setting, we could see that sometimes the predicted MTQElabel was one of Good where in fact the gold label was Adequate. In analysing the discrepancies and getting additional confirmation from human translators, it became apparent that our MTQEmodels were not in fact detecting all the overgenerations which were now occurring in LLMMToutput. These overgenerations are often fluent and therefore much more subtle than the neurobabble encountered in NMToutput, which was much easier to detect. They can range from additions consisting of explanations on the part of the LLM, to confabulations which may be hard to detect, depending on how long they are and how much they differ from the surrounding context. Our second research question is whether we can detect these new types of overgenerations. We report our investigations and insights from work as a Language Services Provider, and examine how well we can detect the different types of overgeneration, deploying two different strategies to do so. After a brief survey of related work (Section 2 ) we detail our categorization of overgenerations (Section 3 ). We then detail the various data sets we have used in Section 4 , before moving on to describing our strategies in Section 5 . We analyse the results (Section 6 ) and offer some insights in Section 7 , before offering some conclusions and future work in Section 8 . 2 Previous work There is much research on hallucination issues previously encountered in NMTwhich exhibited repetitions and neurobabble ( 13 ) . As mentioned, there are various types of overgeneration in LLMoutput, so after reviewing previous literature, we use that term here that as a more generic term than hallucination. 8 focus on NMThallucinations, consider a range of detection methods, and develop DeHallucinator, a method for alleviating hallucinations at test time. They consider 3 categories: oscillatory, strongly detached and fully detached hallucinations, where the target is detached from the source sequence. They find that the TNG (Top n-gram count) method devised for oscillatory actually performs worst, that reference-free metrics can handle oscillatory but fail on detached type, and that sequence log probabilities are the best means of hallucination detection overall. Strongly detached hallucinations are harder to detect, as evidenced from their scores this category (approx 50%) Note that LLMsexhibit detached overgenerations, which are harder to detect than the oscillatory type, particularly since the ones in LLMoutput may only be partially detached, making the task harder still. In fact, as we detail below, MToutput with LLMsalso results in partially and minimally detached overgenerations which are perfectly fluent. These much harder to reliably detect. Again for NMT, 6 create ALTI+, which is the adaptation of encoder-based ALTI (Aggregation of Layer-wise Tokens Interactions) method ( 7 ) to the encoder-decoder Transformer. ALTI+ measures token to token interactions and evaluates the relative contributions of both source and target tokens to model predictions. As hallucinations are translations detached from the source sequence, ALTI+ aims to detect them by identifying sentences with minimal source contribution. There is interesting research on an Optimal transport method by 10 whereby they leverage the fact that hallucinations are detached from source content and exhibit different cross-attention patterns that are statistically different from high quality translations. They frame the problem with an optimal transport formulation and propose a fully unsupervised, plug-in detector that can be used with an attention-based NMTmodel. This work, together with the notion of cross-attention contributions from ALTI+ inspired the work in Section 5.2 to detect hallucinations. In their exploration of how well GPT models translate in comparison to NMTones, 11 investigate unaligned source or target words amoung other criteria for comparing quality. As pointed out in the survey by 9 who cover NMTand LLMoutputs from 100 translation directions and report on the prevalence, properties, and mitigation of hallucinations, LLMsproduce hallucinations that are “qualitatively different from those of conventional translation models, mostly consisting of off-target translations, overgeneration, and even failed attempts to translate” ( 9 ) . These are what we need to address. Their work focuses on exploring prevalence in different types of models. 2 explore detecting hallucinations in MTfor High-Resource Languages (HRL)and Low-Resource Languages (LRL), using LLMsfor detection, and comparing to Blaser-QE, which they regard as the SOTA. They find that even without being trained on the MTtask, LLMscan achieve superior performance to Blaser-QE for HRL, and comparable for LRL. They use the HalOmi dataset ( 3 ) which consists of NMToutput and is notable for being a collection of naturally occurring hallucinations, rather than generated artificially. There are previous shared tasks on general LLM hallucinations, including the SHROOM Shared Task Series on Hallucinations and Related Observable Overgeneration Mistakes in 2024 ( 12 ) and 2025 ( 15 ) . These include train and test data for the task participants to detect multilingual hallucinations in languages. These are NLG type hallucinations emitted by models in various scenarios, and do not include hallucinations which occur during the translation task. As such the task is different from our models which use the source text to determine if there is a hallucination in the target text. 3 Categorization of overgenerations While the term hallucinations has been used in past research as exemplified above, overgenerations covers a broader range of categories, and confabulations is actually a more appropriate term for the specific phenomena