Simorgh at SemEval-2026 task 7: Region-Aware Hybrid Retrieval for Low-Resource Cultural Reasoning in Multilingual Question Answering
Hadi Bayrami Asl Tekanlou, Mahdi Bakhtiyarzadeh, Jafar Razmara · 2026
Read Simorgh at SemEval-2026 task 7: Region-Aware Hybrid Retrieval for Low-Resource Cultural Reasoning in Multilingual Question Answering by Hadi Bayrami Asl Tekanlou, Mahdi Bakhtiyarzadeh, Jafar Razmara (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-27636.txt.
Introduction
Hadi Bayrami Asl Tekanlou, Mahdi Bakhtiyarzadeh, and Jafar Razmara tackle culture-based question answering for low-resource languages. Their region-aware hybrid retrieval mixes word matching and dense meaning links with local weights, feeding a set prompt to a quantized Qwen3-14B model. Hybrid retrieval improves cross-language steadiness over model-only answers, though gaps between well- and poorly-resourced languages persist.
Catalog record
| CREATOR | Hadi Bayrami Asl Tekanlou, Mahdi Bakhtiyarzadeh, Jafar Razmara |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 16518 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.27636
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Simorgh at SemEval-2026 task 7: Region-Aware Hybrid Retrieval for Low-Resource Cultural Reasoning in Multilingual Question Answering Hadi Bayrami Asl Tekanlou, Mahdi Bakhtiyarzadeh, Jafar Razmara Abstract Although Large Language Models (LLMs) demonstrate excellent capabilities and performance for general reasoning tasks within the general public domain, they may face challenges with culturally grounded knowledge within languages with limited digital and textual data. In this paper, we investigate culturally grounded multiple-choice question answering with the BLEnD benchmark, which consists of a multilingual corpus of 30 languages and covers various socio-cultural domains, such as cuisine, sports, family, etc. We propose a region-aware hybrid retrieval approach that combines BM25 lexical matching and dense semantic similarity with regional weighting heuristics to improve the relevance of the answer. The retrieved documents are used to construct a structured prompt for the Qwen3-14B quantized model with logit-based deterministic answer selection. The experimental results show improvements to cross-lingual stability with the hybrid retrieval approach over pure parametric inference for culturally grounded question answering. However, there are still notable performance gaps between languages with more and less training data. This shows that the limitations of the retrieval augmentation approach are not entirely overcome by the training data imbalance problem. 1 Introduction Large Language Models (LLMs) have shown impressive capabilities in delivering outstanding results across a broad range of natural language processing tasks 12 ; 1 . When asked to answer questions based on Western culture, their answers are highly accurate. However, if they are asked to answer questions related to everyday cultural knowledge from an Eastern background or other less-represented regions, their performance is not as satisfactory 14 ; 7 . If these models are asked questions related to everyday cultural knowledge, their limitations become quite apparent. For example, if they are asked to answer questions like “What do Australians need to do to prevent fires in summer?” their limitations become quite apparent. The reason behind this is that these models are trained on large-scale corpora like Wikipedia and Common Crawl, which are dominated by information from specific regions, languages, and perspectives. The distribution of culturally specific everyday knowledge in these models is inherently unbalanced 8 ; 7 . As a result, these models are likely to produce answers that are incomplete, too general, or even incorrect 4 . The unbalanced distribution of culturally specific everyday knowledge in these models can result in hallucinations, leading to stereotypical answers 5 . The limitations of these models in question-answering systems need to be evaluated in relation to everyday knowledge. This research focuses on the answering of multiple-choice questions culturally based with large language models. The following are the main items we will accomplish: • Create a hybrid retrieval method that incorporates BM25 (what the BM25 algorithm does is attempts to match relevant documents to the query) matched to the query lexically and matched semantically . • Implement a region-based enhancement on the retrieval process (where we prioritize evidence that is culturally relevant to the question). • Utilize retrieval based augmentation to add culturally appropriate evidence retrieved to the structured prompt for culturally situated reasoning. • Apply a logit-based approach for determining the answer choice (A-D) so we can provide deterministic and efficient selection of multiple-choice answers. 2 Related Work 2.1 Cultural Commonsense Knowledge in Large Language Models Even though Large Language Models (LLMs) perform well on general types of commonsense reasoning, they do not perform as well when evaluated based on culture-specific commonsense reasoning. The differences between cultures often result in very different performance on the general commonsense tasks provided to LLM’s. For example, while both general commonsense and cultural context influence performance of LLMs on commonsense reasoning, the language used in the query modifies the accuracy of the LLM on tasks related to only culture 13 . These differences in performance indicate that LLM’s exhibit built-in biases towards a particular culture, resulting from an unbalanced amount of training data that is heavily weighted toward dominant cultural representations as well as completely monolingual, English-language datasets 8 . The same types of biases exist with respect to moral values. For example, monolingual English LLMs do not adequately account for the fine-grained differences between countries with respect to moral concepts such as homosexuality or divorce, nor do they have a good grasp of the overall patterns of global moral diversity found in datasets such as the World Values Survey or PEW Surveys. The fine-tuning of LLMs using representative data will increase the accuracy of moral inferences across multiple countries but decrease accuracy with respect to norms within the United States 10 . 2.2 Performance on Eastern and Region-Specific Cultural Knowledge Evaluations of LLMs (Large Language Models) confirm that they perform less well with everyday knowledge of Eastern cultures. In a case study, the Korean medical licensing examination (Korean National Licensing Examination for Korean Medical Doctors, K-NLEKMD) had a 66.18% successful score for GPT-4. While this was above the cutoff of 60%, LLMs lost accuracy on localized topics (e.g. public health law, internal medicine, acupuncture) when compared to non-localized topics (compared TKM-specialized versus non-TKM-specialized).Improvements in prompting (e.g., annotating the prompts with Chinese terms, self-consistency) were an important factor in the LLM’s successful score; reflecting how LLMs continue to encounter challenges when adapting to other languages and cultures 3 . n the context of Mandarin Chinese, ChatGPT-3.5 had a passing score of 153.5/300 in the Postgraduate Examination for Clinical Medicine (passed at the 20th percentile), but ChatGPT-3.5’s performance on the open-ended exam questions (31.5% accuracy) failed; results were 42% in common, 37% in multi-choice, and 17% in case analysis, although there was a 90% average agreement and high insight from the generated results. These results suggest that the recall/diagnosis performance of the LLM was better than that of the intervention-based performance and suggest that the LLM faces challenges adapting to other languages 17 . Research comparing English to Chinese has shown vast differences between these two languages as well as how LLM’s are challenged greatly with Chinese language formality and structure, or grammar, due to cultural differences 16 . Figure 1: Overview of the proposed system architecture. 2.3 Bias Origins and Mitigation Strategies selection bias of the training data, I.e., it may reflect a variety of social biases (e.g., those related to gender, age, ethnicity, religion, and language) 8 . An imbalanced corpus can also contribute to under-representation of low-resource languages and cultures. Examples of initiatives intended to mitigate this problem include NusaWrites, which creates a culturally relevant corpus using paragraphs written by native speakers in each of the 12 low-resource languages spoken in Indonesia, showing more lexical diversity than scraping or translation methods of obtaining data 2 . LLMs present opportunities for researchers in the humanities to use low-resource languages for research; however, researchers face difficulties due to the scarcity of data, adaptability, and cultural sensitivity of the data collected. Researchers require a customized model and collaboration with interdisciplinary teams in order to overcome these obstacles 18 . 3 Methodology We designed a two step pipeline for the BLEnD dataset consisting of: • a Hybrid Evidence Ranker • a Quantized LLM Inference Engine to address both the cultural rationale of the dataset and the Multiple Choice Question (MCQ) objectives. The overall system architecture is illustrated in