Konkani LLM: Multi-Script Instruction Tuning and Evaluation for a Low-Resource Indian Language
Reuben Chagas Fernandes, Gaurang S. Patkar · 2026
Read Konkani LLM: Multi-Script Instruction Tuning and Evaluation for a Low-Resource Indian Language by Reuben Chagas Fernandes, Gaurang S. Patkar (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-23529.txt.
Introduction
Fernandes and Patkar build tools for Konkani, a low-resource Indian language. Language models stumble on Konkani for two reasons: scarce training text and three writing scripts. The pair create Konkani-Instruct-100k, a synthetic teaching set made with Gemini 3. They benchmark open models like Llama 3.1 and Gemma 3 against closed rivals. Their fine-tuned Konkani LLM lifts machine translation over its base models. In several settings it matches or beats the closed systems. A multi-script benchmark is also in the works.
Catalog record
| CREATOR | Reuben Chagas Fernandes, Gaurang S. Patkar |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54723 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.23529
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Konkani LLM: Multi-Script Instruction Tuning and Evaluation for a Low-Resource Indian Language Reuben Chagas Fernandes, Gaurang S. Patkar Abstract Large Language Models (LLMs) consistently under perform in low-resource linguistic contexts such as Konkani. This performance deficit stems from acute training data scarcity compounded by high script diversity across Devanagari, Romi and Kannada orthographies. To address this gap, we introduce Konkani-Instruct-100k, a comprehensive synthetic instruction-tuning dataset generated through Gemini 3. We establish rigorous baseline benchmarks by evaluating leading open-weights architectures including Llama 3.1, Qwen2.5 and Gemma 3 alongside proprietary closed-source models. Our primary contribution involves the development of Konkani LLM, a series of fine-tuned models optimized for regional nuances. Furthermore, we are developing the Multi-Script Konkani Benchmark to facilitate cross-script linguistic evaluation. In machine translation, Konkani LLM delivers consistent gains over the corresponding base models and is competitive with and in several settings surpasses proprietary baselines (Table 4 ). [ BoldFont = tex-gyre-termes.bold.otf, BoldItalicFont = tex-gyre-termes.bold-italic.otf, ] [Script=Devanagari] [Script=Kannada] \setTransitionsFor Devanagari \devanagarifont \setTransitionsFor Kannada \kannadafont \setTransitionTo Latin 1 Introduction 1.1 Background Konkani is an Indo-Aryan language distinct to the Goa and Konkan regions of India. It presents substantial challenges for Natural Language Processing (NLP) due to its multiscript nature. The language uses Devanagari, Romi (Latin) and Kannada scripts officially. Konkani retains low-resource status despite its recognition as a scheduled language of India. High-quality parallel corpora remain scarce. Existing datasets suffer from fragmentation across these diverging scripts which impedes the training of unified language models. 1.2 The Gap: Language Contamination Large language models (LLMs) such as Deepseek V3.2, Claude sonnet 4.6 and GPT-5.2 achieve strong multilingual performance but Konkani remains challenging. While some base models may have encountered Konkani during pre-training, they typically lack sufficient instruction-tuning data to reliably treat Konkani as a distinct target language. 1.3 Proposed Solution We adopt a pipeline centered on Synthetic Data Generation and parameter-efficient fine-tuning to mitigate these issues. We used Gemini 3 to synthesize Konkani-Instruct-100k in the absence of native instruction-tuning data. The dataset balances Devanagari, Romi and Kannada scripts. We then fine-tune open-weight architectures, including Llama 3.1 8B, Gemma 3 4B, Gemma 3 12B, Gemma 3 27B, Qwen2.5 14B and Qwen2.5 1.5B on this synthetic corpus. Our goal is to show that compact models trained on linguistically controlled data can outperform general-purpose counterparts in script fidelity and grammatical accuracy for Konkani. 1.4 Contributions This study delivers the following contributions: Konkani-Instruct-100k: The first large-scale multiscript instruction-tuning dataset generated via distillation. Konkani LLM: A suite of fine-tuned models optimized for Konkani. Konkani-Bench: A 200-pair benchmark spanning Romi, Devanagari and Kannada scripts alongside English. 2 Related Work Research on large language models for Indian languages has accelerated, yet coverage remains skewed towards high-resource languages such as Hindi and Tamil, with comparatively limited attention to Konkani. OpenHathi extends Llama 2 7B with an expanded tokenizer and bilingual training to support Hindi, English and Hinglish and reports performance comparable to GPT-3.5 on Hindi tasks in both Devanagari and Roman scripts [ 1 ] . Sarvam-1 targets ten Indic languages and reports strong results on benchmarks such as MMLU, ARC-Challenge, IndicGenBench and Flores translation and includes support for Konkani; however, it does not explicitly address Konkani in Roman (Romi) and Kannada scripts [ 2 ] . Tamil-LLaMA augments LLaMA 2 with additional Tamil tokens and instruction tuning on translated instruction datasets to improve monolingual Tamil understanding and generation, but the approach is not extended to minority Goan languages [ 3 ] . Existing Indic LLM efforts therefore provide strong baselines for Hindi and Tamil but leave a gap for low-resource languages like Konkani, especially in code-mixed, multi-script settings. Synthetic instruction data has emerged as a standard strategy to compensate for limited supervised corpora and to boost downstream performance. Self-Instruct style pipelines use strong teacher models to generate diverse instructions and responses, followed by automatic filtering and curriculum design for student model fine-tuning [ 4 ] . CoT-Self-Instruct extends this paradigm by enforcing chain-of-thought planning before synthetic example generation and reports higher accuracy on reasoning tasks compared to vanilla Self-Instruct [ 5 ] . These methods indicate that synthetic data generation is a viable approach for building instruction-tuned models in settings where high-quality human-annotated data for Konkani is sparse or absent. Work on script handling for Indian languages covers both transliteration and parallel data construction across scripts. Tamil-LLaMA relies on a standardized Tamil script and improved subword segmentation rather than explicit multi-script modeling, which limits transfer to languages like Konkani that routinely appear in Devanagari, Roman and Kannada scripts [ 3 ] . For Konkani specifically, existing NLP research focuses on foundational tasks such as POS tagging and sentiment analysis, often by leveraging transfer from resource-rich Marathi through parallel corpora and zero-shot models, not through dedicated multi-script LLMs. Recent efforts on Indic LLMs and resources highlight the scarcity of digital resources for Konkani and motivate dedicated multi-script evaluation and modeling [ 6 ] . 3 Konkani-Instruct-100k 3.1 Dataset Creation 3.1.1 Preliminary Experiments Initial resource scarcity necessitated reliance on sparse open-source repositories. We utilized a 15,000-sample subset from Nagvekar [ 9 ] , consisting of Wikipedia-based edit/fix tasks rather than instruction-following pairs. We augmented these data with a Devanagari Alpaca dataset and employed Aksharamukha for transliteration into Romi and Kannada scripts to address script scarcity. Pilot fine-tuning revealed limited generalization: models produced poorly formed sentences and struggled with open-ended queries. We attribute this to two factors. First, the closed-style nature of the training data did not promote conversational fluency. Second, automated transliteration introduced phonological errors that conflicted with dialectal nuances. 3.1.2 Synthetic Methodology A synthetic generation pipeline addressed these structural weaknesses. Based on human evaluation, we selected Gemini 2.5 Pro and Gemini 3 Flash as the most reliable teacher models for Konkani. We also tested Grok 4.1, which performed competitively but tended to produce less appropriate lexical choices and occasionally generated NSFW content, making it unsuitable for data generation. We implemented a ”Tutor-Style” pedagogical framework rather than standard QA pairs. This approach teaches core linguistic mechanics via 18 foundational topics (Table 1 ) to ensure basic competence in everyday Konkani. 3.1.3 Data Structure and Morphology Morphological analysis tables constitute a distinguishing feature of this dataset. All 100% of samples provide the answer alongside a structural breakdown including part-of-speech tags and base-modifier morphology. Over 3,000 samples explicitly detail grammatical rules such as gender suffix modifications. Alternative phrasings for identical intents mitigate overfitting. 3.1.4 Prompting and Domain Diversity Beyond these foundational lessons, we expand coverage by varying prompt domains (e.g., politics, science and technology) to increase contextual diversity. We employ a human-in-the-loop process in which the teacher model populates reusable instruction skeletons with synthetic entities, yielding prompts that better reflect natural variation. ID Foundational topic 1 Food 2 Family relationships 3 Numbers 4 Time 5 Body parts 6 Colors 7 Nature 8 Common words 9 Animals 10 Adjectives 11 Pronouns 12 Tenses 13 Question formation 14 Gender 15 Locations 16 Names 17 Greetings 18 Oblique cases and active voice Table 1: Foundational topics used to generate tutor-style lessons for basic Konkani competence. Script Train Validation Devanagari 26,006 4,820 Romi (Latin) 38,382 4,008 Kannada 26,845 3,917 English 1,000 1,000 Total 92,233 13,745 Table 2: Distribution of Konkani-Instruct-100k across scripts and splits. 3.1.5 Dataset Statistics The