Moral Reasoning Across Languages: The Critical Role of Low-Resource Languages in LLMs

Huichi Zhou, Zehao Xu, Munan Zhao, Kaihong Li, Yiqiang Li, Hongtao Wang · 2025

Read Moral Reasoning Across Languages: The Critical Role of Low-Resource Languages in LLMs by Huichi Zhou, Zehao Xu, Munan Zhao, Kaihong Li, Yiqiang Li, Hongtao Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-19759.txt.

Introduction

Huichi Zhou and five co-authors built a test of moral sense for big language models that runs across five tongues. Moral calls shift with tongue and with how much text wraps the case. Their test, the MMRB, grades cases at three depths: a lone line, a full passage, and a whole file. Tongues with thin web lore, such as Vietnamese, sit at the weak end of model skill.

Scores fell as depth grew, and fell most for the thin-lore tongues. The team then tuned an open model on one-tongue data, both clean and spiked. In a turn that caught them off guard, thin-lore tongues swayed the mixed-tongue scores more than rich ones did. Weak tongues are thus not a side case: they steer how fair a model can be for all.

Catalog record

CREATORHuichi Zhou, Zehao Xu, Munan Zhao, Kaihong Li, Yiqiang Li, Hongtao Wang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE18 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-19759

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Moral Reasoning Across Languages: The Critical Role of Low-Resource Languages in LLMs Huichi Zhou, Zehao Xu, Munan Zhao, Kaihong Li, Yiqiang Li, Hongtao Wang Abstract In this paper, we introduce the Multilingual Moral Reasoning Benchmark (MMRB) 1 1 1 https://anonymous.4open.science/r/ethics_scenarios-913B to evaluate the moral reasoning abilities of large language models (LLMs) across five typologically diverse languages and three levels of contextual complexity: sentence, paragraph, and document. Our results show moral reasoning performance degrades with increasing context complexity, particularly for low-resource languages such as Vietnamese. We further fine-tune the open-source LLaMA-3-8B model using curated monolingual data for alignment and poisoning. Surprisingly, low-resource languages have a stronger impact on multilingual reasoning than high-resource ones, highlighting their critical role in multilingual NLP. 1 Introduction The advancement of Large Language Models (LLMs), such as ChatGPT ( OpenAI, 2023 ) , has garnered significant attention in various downstream applications. Recently, many researchers have focused on the moral reasoning capabilities of LLMs and on preventing them from generating toxic or harmful content ( Zhou et al., 2023 ; Rao et al., 2023a ) . Unlike traditional language models ( e.g. , BERT ( Devlin et al., 2019 ) ), which tend to make binary moral judgments and are often criticized for their lack of interpretability ( Rao et al., 2023a ; Talat et al., 2022 ) , LLMs are capable of performing inference from a reasoning perspective ( e.g. , chain-of-thought ( Wei et al., 2023 ) ). However, prior research on moral reasoning has mostly focused on monolingual datasets (primarily English), overlooking the importance of evaluating LLMs’ multilingual moral reasoning capabilities. The lack of robustness in multilingual moral reasoning may undermine the fairness of LLM-based systems, potentially affecting user experiences across different language backgrounds. Therefore, it is crucial to assess both the consistency and the key influencing factors of LLM performance in multilingual moral reasoning. Do LLMs behave consistently on moral reasoning across different languages? To answer this question, we propose MMRB , the Multilingual Moral Reasoning Benchmark , spanning five languages. Specifically, we collect well-established moral reasoning datasets, including ETHICS BASE ( Hendrycks et al., 2020 ) , ETHICS PRO ( Ma et al., 2023 ) , and curate ETHICS MAX using the Moral Stories dataset ( Scherrer et al., 2024 ) and the moral dilemma framework ( Rao et al., 2023b ) to create more complex moral reasoning scenarios. These datasets are then translated into five typologically diverse languages (English, Chinese, Russian, Vietnamese, and Indonesian). Based on MMRB , we evaluate the multilingual moral reasoning performance of five prominent LLMs, including both closed-source (e.g., GPT-4 ( OpenAI, 2023 ) ) and open-source (e.g., LLaMA ( AI@Meta, 2024 ) ) models. Our experiments reveal significant inconsistencies in moral reasoning across different languages. To further explore the impact of language on model performance, we fine-tune LLaMA-3-8B on MMRB to analyze cross-linguistic effects in alignment and poisoning settings. Our contributions are: (1) We introduce MMRB , a multilingual moral reasoning dataset comprising 2,170 moral scenarios across five languages. (2) We benchmark five popular LLMs using MMRB , revealing that their moral reasoning performance is inconsistent across languages. (3) We demonstrate that fine-tuning on low-resource languages has a disproportionately strong impact (both positive and negative) on multilingual reasoning performance. 2 Related Work Monolingual Moral Reasoning. Most prior research on moral reasoning focuses on English. Hendrycks et al. (2020) proposed the ETHICS dataset, which covers concepts such as justice, well-being, duties, virtues, and commonsense morality, and evaluated models like BERT ( Devlin et al., 2019 ) . Ziems et al. (2022) introduced the MIC dataset to study whether chatbots can generate utterances aligned with Rules of Thumb (RoT) in moral contexts. The Moral Stories dataset ( Emelin et al., 2021 ) examines whether language models can produce moral responses in social scenarios. To analyze LLMs’ moral reasoning in ambiguous cases, Scherrer et al. (2024) created datasets with 680 high-ambiguity and 687 low-ambiguity scenarios. However, these methods largely do not leverage LLMs’ advanced reasoning capabilities, such as chain-of-thought (CoT) prompting ( Wei et al., 2022 ) , which can improve the stability and accuracy of moral inference. Moreover, they often overlook the generative nature of LLMs, which allows for explicit reasoning steps and nuanced decision-making beyond binary classification. Multilingual Moral Reasoning. According to the LLaMA-3 technical report, 2 2 2 https://ai.meta.com/blog/meta-llama-3/ LLMs are trained on different quantities of data per language, with over 5% of the training data comprising high-quality non-English content covering more than 30 languages. As a result, performance is typically better in English. Haemmerl et al. (2023) trained monolingual Sentence-BERT models for five languages to assess multilingual moral reasoning, but their approach focused on short, unambiguous sentences and high-resource languages. Khandelwal et al. (2024) used GPT-4, ChatGPT, and LLaMA2-70B-Chat to evaluate moral dilemmas across six languages, finding weaker performance in Hindi and Swahili compared to Spanish, Russian, Chinese, and English. 3 Experiment 3.1 Setup Dataset. We conduct multilingual alignment experiments using three datasets: (1) ETHICS BASE : A sentence-level binary classification dataset covering five moral dimensions (e.g., commonsense, justice). (2) ETHICS PRO : A paragraph-level dataset enhanced from ETHICS BASE using LLM-generated moral reasoning (e.g., ChatGPT), incorporating richer context and implicit ethical cues. (3) ETHICS MAX : A document-level dataset created by expanding paragraph-level entries into structured moral dilemmas. Each scenario involves three ethical branches (Virtue, Deontological, Consequentialist), each tied to six normative principles and three answer choices. To support multilingual evaluation, we translate all datasets into five typologically diverse languages: English, Chinese, Russian, Vietnamese, and Indonesian. We use the DeepSeek API for initial translation, followed by manual verification by bilingual annotators. For quality assurance, we apply cross-validation using Google Translate to identify and correct translation inconsistencies or cultural mismatches. We also filter out scenarios with conflicting or ambiguous moral content, ensuring linguistic and conceptual fidelity. The resulting corpus, MMRB , contains 2,170 high-quality scenarios across three complexity levels. Model Selection. We evaluate five strong multilingual LLMs: GPT-4 3 3 3 platform.openai.com/docs/models/gpt-4-turbo-and-gpt-4 , GPT-3.5 4 4 4 platform.openai.com/docs/models/gpt-3-5-turbo , LLaMA3-70B-Instruct 5 5 5 replicate.com/meta/meta-llama-3-70b-instruct/api , LLaMA3-8B-Instruct 6 6 6 replicate.com/meta/meta-llama-3-8b-instruct/api , and Mixtral-8x7B-Instruct-v0.1 7 7 7 replicate.com/mistralai/mixtral-8x7b-instruct-v0.1/api . Inference is conducted using greedy decoding (temperature = 0 ) for consistency. Fine-tuning Strategy. To investigate how monolingual data quality affects multilingual performance, we fine-tune the open-source LLaMA3-8B model ( AI@Meta, 2024 ) using LLaMA-Factory ( Zheng et al., 2024 ) . We design two fine-tuning settings: Monolingual Alignment: Fine-tuning on verified, high-quality data in a single language (e.g., Indonesian) to test cross-lingual generalization. Monolingual Poisoning: Fine-tuning on label-corrupted data in one language to examine its negative impact on other languages. 3.2 Results 3.2.1 Evaluation GPT-4 GPT-3.5 Llama3-70b Llama3-8b Mixtral ETHICS BASE-EN 96.5 86.5 92.9 88.3 90.8 ETHICS BASE-ZH 88.1 78.4 84.3 79.7 81.2 ETHICS BASE-RU 91.7 75.3 84.9 76.2 84.2 ETHICS BASE-VI 86.7 67.3 77.1 65.2 73.4 ETHICS BASE-ID 90.3 75.6 79.2 63.0 72.5 ETHICS BASE-AVG 90.6 76.6 83.7 74.5 80.4 ETHICS PRO-EN 84.8 72.0 75.7 72.1 79.6 ETHICS PRO-ZH 81.8 65.9 72.5 69.0 71.9 ETHICS PRO-RU 82.9 64.5 74.7 64.6 74.5 ETHICS PRO-VI 80.6 66.3 71.5 58.2 68.7 ETHICS PRO-ID 82.9 68.9 67.1 57.1 71.1 ETHICS PRO-AVG 82.6 67.5 72.3 64.2 73.2 ETHICS MAX-EN 91.1 78.5

Read the complete text