A New NMT Model for Translating Clinical Texts from English to Spanish
Rumeng Li, Xun Wang, Hong Yu · 2025
Read A New NMT Model for Translating Clinical Texts from English to Spanish by Rumeng Li, Xun Wang, Hong Yu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-18607.txt.
Introduction
Rumeng Li, Xun Wang, and Hong Yu built a translation model for clinical texts from English to Spanish in two thousand twenty-five. Health record narratives are important to translate, but aligned training text is scarce and unknown words are common. Standard neural translation struggles on both counts.
Their system, NOOV, needs little in-domain aligned data. It adds a bilingual lexicon learned automatically from aligned corpora, plus a phrase lookup table drawn from a large biomedical knowledge source. Those additions ease the unknown word problem and a word-repeat flaw in neural output. Evaluation showed better accuracy and fluency on health record translation than comparison systems. The authors present it as a practical route for clinical language access.
Catalog record
| CREATOR | Rumeng Li, Xun Wang, Hong Yu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 20 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-18607
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A New NMT Model for Translating Clinical Texts from English to Spanish Rumeng Li, Xun Wang, Hong Yu Abstract Translating electronic health record (EHR) narratives from English to Spanish is a clinically important yet challenging task due to the lack of a parallel-aligned corpus and the abundant unknown words contained. To address such challenges, we propose NOOV (for No OOV), a new neural machine translation (NMT) system that requires little in-domain parallel-aligned corpus for training. NOOV integrates a bilingual lexicon automatically learned from parallel-aligned corpora and a phrase look-up table extracted from a large biomedical knowledge resource, to alleviate both the unknown word problem and the word-repeat challenge in NMT, enhancing better phrase generation of NMT systems. Evaluation shows that NOOV is able to generate better translation of EHR with improvement in both accuracy and fluency. 1 Introduction Providing language-appropriate access to EHR narratives is important for patients, as better understanding of their medical conditions could promote communication between physicians and patients and improve patients’ medication adherence and self-managed care. In the Unites States, most EHRs are written in English. However, there are over 37.6 million people who speak Spanish at home, of which 16.5 million report speaking English less than very well ryan2013language . Hospitals face challenges in providing translated EHR narratives due to the time, work burden, expenses of translating service and privacy concerns, thus the development of an efficient and secure machine translation system is needed for the US hospitals. In contrast to the active research of MT in the open-domain, relatively few MT systems have been developed in the biomedical domain with early work mostly on SMT systems eck2004improving ; liu2006babelmesh ; merabti2011translating ; zeng2010can ; liu2015translating ; bojar2014findings and later on NMT methods wolk2015neural ; yepes2017findings . Neural machine translation (NMT) methods have recently achieved promising results kalchbrenner2013recurrent ; cho2014learning ; bahdanau2014neural ; vaswani2017attention ; gehring2017convolutional . A major drawback of NMT systems is their inability to correctly translate rare, out-of-vocabulary (OOV) or unknown words. Most NMT systems limit the vocabulary to frequent words and convert rare words into a single unk (unknown words) symbol to reduce computational complexity. Such NMT models fail to learn the translation of unknown words. Many existing work also reported that sentences with many rare words tend to be translated poorly sutskever2014sequence ; bahdanau2014neural ; li2016towards . Various approaches have been proposed to address the unknown word problem in NMT including building representations for unknown words based on characters and other sub-word units belinkov2017synthetic ; ling2015character ; luong2016achieving ; lee2016fully ; zhao2016deep ; sennrich2015neural . Lexicon/dictionary information is also used in existing work arthur2016incorporating ; luong2014addressing . Another approach aims to replace unknown words by copying relevance source words through alignment tools luong2014addressing or attention scores hashimoto2016domain ; jean2015montreal , or replace the rare words with similar in-vocabulary words li2016towards . The rare word problem is even more serious for the clinical domain. The clinical text is a sub-language, and it has a rich domain-specific vocabulary including complex disease and drug names, medical terms, shortened forms, and other domain-specific jargon. For example, in our EHR corpus, unknown words are as high as 25.6%. Furthermore, in part due to the high-cost and HIPAA regulation, there is no parallel-aligned EHR corpus available, making it harder to develop an NMT system. Existing parallel corpora for biomedical machine translation are generally literature data automatically extracted from medical articles liu2015translating ; yepes2017findings , which are quite different from the EHR data. To solve the above problems, we first created a benchmark parallel-aligned English-Spanish EHR corpus. Using this corpus as the gold standard, we evaluate a new NMT system that integrates a bilingual lexicon automatically learned from open-domain parallel corpora and a phrase look-up table extracted from a biomedical knowledge resource. Unknown words due to its infrequency makes the NMT system difficult to learn sufficient information for proper translation. The lexicon, on the other hand, can remedy this problem by assigning probability to every aligned occurrence. The incorporation of the lexicon serves as an extra guide for the decoder. The incorporation of the phrase look-up table comes in the process of decoding, instead of usual ways of post-processing. In addition to improving phrase translation, the incorporation of the phrase look-up table helps guide the decoder in the following step, without damaging the decoder’s language model, hence reduce the word repetition problem in NMT feng2016implicit ; lin2018decoding . We further conduct experiments to verify the effectiveness of our NOOV model in addressing the unknown words problem. 2 Methods The proposed model adopts the encoder-decoder scheme and uses attention mechanism. Encoding Same with the seq2seq model, we use a biLSTM to scan the input s=(w^{s}_{1},w^{s}_{2},...,w^{s}_{m}) to generate its vector representations: o^{f}_{i},h^{f}_{i}=\text{biLSTM}(w_{i},h^{f}_{i-1});\;o^{b}_{i},h^{b}_{i}=\text{biLSTM}(w_{i},h^{b}_{i-1});\;h_{i}=[h^{f}_{i}:h^{b}_{i}] (1) Lexicon and phrase look-up table assisted decoding Then the decoder generates the target sentence word by word based on the encoded vector. Attention mechanism enables the decoder to look at the input again and chooses the most relevant part to attend to at each step in translation. \displaystyle u^{\prime}_{i} \displaystyle=\sum_{j}att_{j}^{i}*h_{j};att_{j}^{i}=g(h_{*},h^{\prime}_{i-1},w^{\prime}_{i-1}) (2) \displaystyle prob^{i}(tok_{k}^{t}) \displaystyle=\frac{exp^{pr^{i}(tok_{k}^{t})}}{\sum_{k}(exp^{pr^{i}(tok_{k}^{t})})};pr^{i}(tok_{k}^{t})=\sum_{tok_{j}^{s}\in V^{s}}(att^{i}_{j}*p_{k}^{j}) \displaystyle w^{\prime}_{i} \displaystyle=\arg\max((1-\alpha)*v(o^{\prime}_{i})+\alpha*prob^{i});o^{\prime}_{i},h^{\prime}_{i}=LSTM(w^{\prime}_{i-1},u^{\prime}_{i}) During decoding, at the i^{th} step, the hidden state h^{\prime}_{i} of the decoder and the hidden states of the encoder \{h_{0},h_{1},...,h_{i},...,h_{m}\} are leveraged to calculate the attention weights. att^{i}_{j} is the weight assigned to h_{j} in the i^{th} step. It is also regarded as the probability of word w_{j}^{s} in the source sentence aligned to the word w_{i}^{t} in the target sentence. We leverage these att_{*}^{i} to encode lexicon information for better translation. The lexicon \mathcal{L} we built contains alignments between source words and target words. lex(tok_{j}^{s})=\{tok_{k}^{t}:p_{k}^{j},tok_{l}^{t}:p_{l}^{j},...\} . lex(tok_{j}^{s}) is a set of candidate target words associated with probabilities. p_{k}^{j} is the probability of tok_{j}^{s} in the source language mapping to tok^{t}_{k} in the target language. One word in the source language can be aligned to several words in the target language. We collect lex(w_{j}^{s}) for each word w_{j}^{s} in the source sentence. The details of collecting lex(w_{j}^{s}) are explained below. At the i_{th} step, a probability prob^{i} over all words in the target language is built using lex(w_{*}^{s}) and a^{i}_{*} . V^{s}=\{tok_{k}^{s}\} and V^{t}=\{tok_{k}^{t}\} are the source vocabulary and the target vocabulary. For each word in the target vocabulary, its probability is prob^{i}(tok_{k}^{t}) . p^{j}_{k} is the probability that w_{j}^{s} is mapped to tok_{k}^{t} . Meanwhile, the decoder outputs a probability over the target vocabulary, which is written as v(o^{\prime}_{i}) . The final output at the i^{th} step is decided according to output_{i}=\alpha*prob^{i}(V^{t})+(1-\alpha)v(o^{\prime}_{i}) . \alpha is a parameter which balances the impact of lexicon information and that of the decoder’s outputs. This parameter is tuned in the following experiments. We enhance the output from the seq-to-seq model by the use of prob(V^{t}) . As a weighted sum of mapping between source words and target words, prob(V^{t}) is more robust in dealing with rare words. The infrequency of unknown words makes seq-to-seq model difficult to learn its corresponding language model at decoding and correct translation generation, the prob(V^{t}) acts as an extra supervision to remedy such insufficiency. At decoding, the attention-based seq-to-seq model sometimes tends to repeatedly output the same word when the attention mechanism consecutively focuses on the same position in the source sentence. Such repetition problem