Injecting Knowledge from Social Science Journals to Improve Indonesian Cultural Understanding by LLMs
Adimulya Kartiyasa, Bao Gia Cao, Boyang Li · 2026
Read Injecting Knowledge from Social Science Journals to Improve Indonesian Cultural Understanding by LLMs by Adimulya Kartiyasa, Bao Gia Cao, Boyang Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-12921.txt.
Introduction
Adimulya Kartiyasa, Bao Gia Cao, and Boyang Li improve how language models understand Indonesian cultures. Efforts to teach models Indonesian cultural knowledge have intensified of late. One attractive source has been largely overlooked: local social science journals. Such journals likely contain substantial cultural studies written from a native perspective.
The team inject knowledge from those journals into language models. Native scholarship can correct the outsider framing that web data often carries. Better cultural understanding should make models safer and more useful for Indonesian users. The approach may generalize to other cultures with strong local journals. It treats regional academia as a first class data source.
Catalog record
| CREATOR | Adimulya Kartiyasa, Bao Gia Cao, Boyang Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-12921
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Injecting Knowledge from Social Science Journals to Improve Indonesian Cultural Understanding by LLMs Adimulya Kartiyasa, Bao Gia Cao, Boyang Li Abstract Recently there have been intensifying efforts to improve the understanding of Indonesian cultures by large language models (LLMs). An attractive source of cultural knowledge that has been largely overlooked is local journals of social science, which likely contain substantial cultural studies from a native perspective. We present a novel text dataset of journal article passages, created from 151 open-source Indonesian social science journals, called IndoSoSci. We demonstrate an effective recipe for injecting Indonesian cultural knowledge therein into LLMs: extracting the facts related to Indonesian culture, and apply retrieval-augmented generation (RAG) with LLM-generated hypothetical documents as queries during retrieval. The proposed recipe yields strong performance gains over several strong baselines on the IndoCulture benchmark. Additionally, by combining IndoSoSci with Indonesian Wikipedia, we set a new state-of-the-art accuracy on the IndoCulture benchmark. Dataset and code will be made available at a later date. 1 Introduction Most large language models today are trained with text in predominantly Western languages, which may have created a Western bias in these models ( 7 ; 1 ; 30 ; 34 ) . When interacting with users from underrepresented regions such as South-East Asia (SEA), the LLMs may generate responses that are insensitive, irrelevant, or otherwise premised on Western cultural norms. Additionally, the Western bias presents the risk of flattening global cultural diversity. Therefore, improving the cultural awareness and understanding of LLMs has gained increasing research attention. As the fourth populous country in the world, Indonesia has been historically under-represented in NLP research ( 2 ) . Indonesia is also one of the most ethnically and culturally diverse countries, with 600 to 1200 ethnic groups in the country, depending on the classification method ( 5 ) . In recent years, there has been an intensifying effort to improve the availability of NLP resources for Indonesia. This includes development of benchmarks, such as 19 and 20 , as well as a consolidation and standardization of disparate Indonesian datasets, as part of the SEACrowd initiative to facilitate usage of the datasets in research ( 30 ) . One attractive source of cultural knowledge that has been largely overlooked is social science 1 1 1 For brevity, in this paper ”social science” refers to both humanities and social sciences. publications produced locally, which likely contain studies into local cultures from a native perspective. We present a novel text dataset of journal article passages, IndoSoSci, which is created from Indonesian social science journals indexed in the Directory of Open Access Journals 2 2 2 https://doaj.org/ , and demonstrate its effectiveness on the Indonesian cultural benchmark, IndoCulture ( 20 ) . On top of the dataset, the present study devises an effective technique to inject the cultural knowledge into LLMs. Inspired by previous research that retrieval may be more suitable for injecting specialized knowledge into LLMs than finetuning ( 33 ; 36 ) , we propose to employ IndoSoSci in retrieval-augmented generation (RAG). First, we extract the factual statements regarding Indonesian culture in the journal articles. This is to prevent other types of text in the articles from interfering with the RAG process. During retrieval, following 13 , for each question the LLM is prompted to generate a hypothetical answer, which is then used as an informative key for retrieval. We evaluate on the IndoCulture benchmark ( 20 ) , which covers diverse cultures in eleven Indonesian provinces. The proposed recipe results in strong performance gains over baselines. The best model achieves an accuracy of 79.3%, 2.9 percentage points higher than the previous SOTA. We carefully verify the effectiveness of all components of the proposed recipe using ablation experiments. The contributions of this paper are as follows: 1. We present a novel text dataset created from carefully parsed Indonesian social science publications, IndoSoSci, which contains Indonesian cultural knowledge. 2. We demonstrate a technique for injecting Indonesian cultural knowledge from the journal articles into the LLMs. Facts extracted from the papers serves as values to be retrieved whereas hypothetical documents generated from the target question serve as the query. 3. We set a new SOTA of 81.4% on IndoCulture by using RAG with a mixed corpus of Indonesian Wikipedia and extracted facts from journals, highlighting the potential of our corpus in complementing more common sources of knowledge. 2 Related Work NLP Corpora of Academic Papers. Academic publications in science and engineering fields have been included in NLP corpora. For example, papers from ArXiv and PubMed have been included in open-source datasets such as The Pile ( 12 ) . Beyond the STEM fields, S2ORC ( 29 ) covered more academic disciplines by collecting papers from Semantic Scholar. A small minority of papers contained in S2ORC are from social studies subjects such as Sociology, History, and Art; however, the dataset is limited to English-language papers. OpenMSD ( 14 ) is a multilingual dataset of scientific papers, including papers from Southeast Asia (SEA). Nevertheless, the dataset is primarily intended for scientific document similarity measurement, and the data distribution is still heavily skewed towards STEM subjects. To our knowledge, there have been no previous corpora that focus on social science journal articles designed for the task of cultural understanding in Southeast Asia. Retrieval-Augmented Generation. Pioneered by 24 , retrieval-augmented generation (RAG) is a technique for augmenting the internal knowledge of an LLM by retrieving documents from an external database and placing the retrieved documents in the LLM decision context. Current implementation of RAG generally involves three steps: indexing, retrieval, and generation ( 15 ) . In the indexing step, text chunks are encoded into vector representations using an embedding model, and the vector representations are collected into a vector database. In the retrieval step, the same embedding model is used to encode a user query, and the similarity scores between the query vector and the vectors in the database are calculated. A predefined number D of documents with the highest similarity scores is subsequently added to the prompt that will be given to the LLM, as context to the user query. In the generation step, the LLM is instructed to answer the new prompt containing both the context and the original query. Much research has been conducted on RAG since the technique’s introduction. There has been research into improving the retrieval stage ( 13 ; 45 ; 22 ) , instruction finetuning of the LLM to make more effective use of the retrieved documents ( 28 ; 4 ) , and developing new evaluation frameworks ( 44 ) . Other works investigate interleaving RAG with multi-step reasoning to improve the LLM’s reasoning capability ( 25 ; 17 ) , and applying RAG to improve LLM performance in various domains ( 26 ; 40 ) . Application of RAG for LLM Cultural Understanding. There have been few studies into the use of RAG in cultural context. In the review of 34 regarding prior works on LLM cultural awareness, it is reported that training-free methods to improve LLM cultural awareness have historically focused on prompting techniques. For example, 9 prompted LLMs to generate personas of various demographic groups to investigate stereotypes contained in the LLMs; 3 showed