Dictionary Insertion Prompting for Multilingual Reasoning on Multilingual Large Language Models
Hongyuan Lu, Zixuan Li, Wai Lam ♡ \heartsuit The Chinese University of Hong Kong · 2024
Read Dictionary Insertion Prompting for Multilingual Reasoning on Multilingual Large Language Models by Hongyuan Lu, Zixuan Li, Wai Lam ♡ \heartsuit The Chinese University of Hong Kong (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-01141.txt.
Description
Abstract There are two shortages in the current Large Language Models (LLMs) era. The first is short of multilingual models, where most LLMs are English-centric and performance is limited on multilingual reasoning. The second is the place of external knowledge to be used, where most retrieved knowledge is prepended to the user queries (maybe sub-optimal).
Catalog record
| CREATOR | Hongyuan Lu, Zixuan Li, Wai Lam ♡ \heartsuit The Chinese University of Hong Kong |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2411-01141
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Dictionary Insertion Prompting for Multilingual Reasoning on Multilingual Large Language Models Hongyuan Lu, Zixuan Li, Wai Lam ♡ \heartsuit The Chinese University of Hong Kong Abstract There are two shortages in the current Large Language Models (LLMs) era. The first is short of multilingual models, where most LLMs are English-centric and performance is limited on multilingual reasoning. The second is the place of external knowledge to be used, where most retrieved knowledge is prepended to the user queries (maybe sub-optimal). This paper presents a novel and simple yet effective method called D ictionary I nsertion P rompting ( DIP ). When providing a non-English prompt, DIP looks up a word dictionary and inserts words’ English counterparts into the middle of the prompt for LLMs. It then enables better translation into English and better English model thinking steps which leads to obviously better results. We experiment with 10 to 200 languages from FLORES-200. 1 1 1 The number of languages varies on the datasets, and we experiment with 200 languages on GSM8K as in Appendix Since there are no adequate datasets, we use the NLLB translator to create synthetic multilingual benchmarks from the existing 4 English reasoning benchmarks such as GSM8K and AQuA. The synthetic benchmarks are translated back into English for quality assurance with manual annotation. Interestingly, the place for injecting the dictionary plays an important factor in the performance gains, and we found that interleaving the dictionary with the original words gives a better performance compared to prepending/appending the dictionary, under the same dictionary constructed. Equal Contribution. 1 Introduction In the quick development with large language models (LLMs), there have been quite many popular research areas such as chain-of-thought reasoning ( Wang et al., 2023a ; Wei et al., 2024 ; Yang et al., 2024 ) , machine translation ( Lu et al., 2023b ; Zhu et al., 2024a ; Zhu et al., 2024b ) , code generation ( Li et al., 2023 ; Zhang et al., 2023 ; Hou et al., 2025 ) , dialogue generation ( Li et al., 2022 ; Lu et al., 2022 ; Lu and Lam, 2023 ; Yang et al., 2025 ; Yang et al., 2026 ) , and even spatial understanding ( Hu et al., 2024 ) . Among these, an important research area is multilingual large language models (MLLMs), which consider not only the tasks of machine translation but also reasoning tasks represented in different languages ( Huang et al., 2023 ) . This scales the horizon of English-centric LLMs such as popular ChatGPT and enables them to be used by people who mainly speak low-resourced languages. Yet, current methods are usually training-based ( Lu et al., 2024c ; Lim et al., 2024 ) , which usually requires many GPU/TPU computational resources to update model weights from LLMs. Figure 1: An illustrated comparison of the GSM8K dataset made up in Buginese. Compared to the standard prompting baseline, DIP inserts additional dictionary knowledge written in knowledge in an interleaving manner. This leads to a better intermediate English translation and succeeding English thought. Finally, DIP produces promising results, surpassing several strong baselines. In contrast, this paper investigates how to incorporate a dictionary as an auxiliary knowledge into prompting. In comparison, this is lightweight and flexible, where the dictionary can be customized and replaced in a plug-and-play manner without model training. While the dictionary-based method has been studied on the task of traditional machine translation ( Arthur et al., 2016 ) , how to incorporate them into reasoning tasks on MLLMs has been under-studied. This is yet important and needs to be empirically justified whether, and how dictionary-based methods can be better used to improve multilingual reasoning tasks, which obviously enhances LLMs’ usefulness in our daily life. To this end, we propose a novel method called D ictionary I nsertion P rompting ( DIP ). We present the overall algorithm as in Figure 1 . By providing a customized dictionary that maps words represented in low-resourced languages into English, DIP inserts the English representation into the original input. This then helps LLMs in pivoting the original input into a complete English representation. This improves the succeeding chain of thought reasoning, which results in a better final output. By looking deeply into the experimental analysis, we found that the place of insertion is an important factor in improving the performance of DIP. While the prior dictionary-based method usually presents the dictionary in front of the prompt, we found it sub-optimal, and placing the dictionary in an interleaving manner is better. We postulate that such a design puts words with their English counterparts closer, and makes it easy to be understood by LLMs. Experiments also show that better translation quality and thought quality are the key factors to the usefulness of DIP. We benchmark DIP on about 200 languages from FLORES-200 ( NLLB-Team, 2022 ) . Since there are no adequate datasets for covering those languages, we use a high-quality SOTA translator NLLB 3.3B 2 2 2 https://huggingface.co/spaces/Narrativaai/NLLB-Translator to translate existing arithmetic reasoning benchmarks such as GSM8K ( Cobbe et al., 2021 ) and commonsense reasoning benchmarks such as Date ( Srivastava et al., 2022 ) . In general, we make three key contributions. • We propose a simple, novel, yet computationally lightweight approach called DIP for better reasoning on multilingual tasks on LLMs. • Extensive strong results across several benchmarks on ChatGPT and Llama LLMs verify the effectiveness of DIP. • We investigated further why DIP is useful and found that better intermediate translation and thinking steps are the key. 2 Dictionary Insertion Prompting Large language models show their promising translation performance when sufficiently pre-trained ( Wang et al., 2023b ; Lu et al., 2023a ; Tang et al., 2024 ; Lu et al., 2024a ; Lu et al., 2024b ; Lu et al., 2025 ; Lu et al., 2026 ) . Yet, such translation ability usually diminishes in low-resourced languages and it is still an under-studied topic for reasoning in those low-resourced languages such as the ones from FLORES-200 ( NLLB-Team, 2022 ) . This paper proposes a novel, yet simple and effective framework called DIP ( D ictionary I nsertion P rompting) to address these difficulties by integrating the dictionary knowledge into the reasoning process. DIP first looks up a customized dictionary from the original non-English prompt to place its English counterpart accordingly. This is followed by pivoting into English, which succeedingly invokes a better English thinking process. The final result is then obviously improved by DIP. Therefore, DIP is as illustrated in Figure 1 : (1) Solve the question. The question is made up of the <language> language. (2) First translate the following question from <language> into English: (3) <question> (4) Note: Your output format is as follows: 1. Translated English sentences 2. The derivation process 3.The final numerical answer , where <language> denotes the language of the non-English question prompts 3 3 3 We conduct our experiments on non-English languages with English-centric LLMs, and we leave other settings to future work. that are written in, and <question> denotes the actual question that is written in those non-English languages. The reason for choosing English as the pivoting language for DIP on LLMs