Minimal-Edit Instruction Tuning for Low-Resource Indic GEC
Akhil Rajeev P · 2025
Read Minimal-Edit Instruction Tuning for Low-Resource Indic GEC by Akhil Rajeev P (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-00219.txt.
Introduction
Correcting grammar in Indic languages is hard because training data is scarce, scripts vary, and word forms are rich. Akhil Rajeev presents a minimal-edit approach to the problem in a 2025 study. The setup uses instruction-tuned large language models with conservative decoding.
A mid-size Gemma model is instruction-tuned efficiently, using four-bit precision and parameter-efficient fine-tuning with Alpaca-style formatting. The augmentation-free design avoids inventing extra training data. Minimal edits matter because a correction tool should fix errors without rewriting a writer's voice. That restraint matters most for low-resource languages with few digital tools. Indic languages serve hundreds of millions of speakers, so the gap in tooling is significant.
Catalog record
| CREATOR | Akhil Rajeev P |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-00219
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Minimal-Edit Instruction Tuning for Low-Resource Indic GEC Akhil Rajeev P Abstract Grammatical error correction for Indic languages faces limited supervision, diverse scripts, and rich morphology. We propose an augmentation-free setup that uses instruction-tuned large language models and conservative decoding. A 12B Gemma 3 model is instruction-tuned in bnb 4-bit precision with parameter-efficient fine-tuning (PEFT) and Alpaca-style formatting. Decoding follows a deterministic, constraint-aware procedure with a lightweight normaliser that encourages minimal, meaning-preserving edits. We operationalise inference, subsequent to instruction fine-tuning (IFT), via a fixed, language-specific prompt directly synthesised from a deterministic error classifier’s taxonomy, label distributions, and precedence ordering computed on the training corpus. Under the official untuned GLEU evaluation, the system scores 92.41 on Malayalam, sixth overall, and 81.44 on Hindi, third overall. These results indicate that classifier-informed prompt design, adapter-based instruction tuning, and deterministic decoding provide a reproducible and computationally efficient alternative to augmentation-centred pipelines for Indic GEC. The approach also motivates future work on stronger morphosyntactic constraints and human centered evaluation of conservative edits. 1 Introduction Grammatical error correction for Indic languages remains limited by scarce supervision, complex morphology, and script diversity. Many recent systems improve performance through large synthetic corpora and augmentation-based training of sequence-to-sequence models. While these approaches are effective in high-resource environments, they are costly to reproduce for languages such as Hindi and Malayalam and tend to be brittle when the available supervision falls below a thousand examples per language ( 12 ; 16 ; 18 ) . Complementary work by 4 introduces a Bangla GEC pipeline that defines a twelve-class error taxonomy, collects native speaker data, and applies rule-based noise injection to generate erroneous sentences from clean references. The resulting dataset, “Vaiyakarana” ( 3 ) , demonstrates that linguistically motivated error inventories combined with targeted synthetic generation can bootstrap meaningful supervision and support effective LLM-based correction. In contrast, our study focuses on Hindi and Malayalam under strict data limits and develops an augmentation-free approach emphasizing minimal-edit instruction fine-tuning and deterministic decoding. Rather than expanding the corpus, we use a deterministic error classifier to analyze existing data and to guide prompt design. This work adopts a metric-driven, augmentation-free design suited to the BHASHA IndicGEC benchmark, where systems are ranked by the GLEU metric. 1 1 1 We report the “GLEU without tuning” variant ( 14 ) for consistency with the shared task. Instead of creating pseudo-parallel pairs, we cast GEC as an instruction-following problem and adapt a general-purpose model using instruction fine-tuning and prompt optimization. The system employs Alpaca-style supervision formatting 2 2 2 Alpaca is a documented instruction-tuning framework derived from LLaMA and trained on 52k instruction–response pairs using the Self-Instruct method ( 20 ; 19 ) . and parameter-efficient adapters through Unsloth, 3 3 3 Unsloth is an open-source fine-tuning framework optimized for low-VRAM LoRA and QLoRA training ( 23 ) . trained on fewer than one thousand human-annotated examples per language. Decoding and post-processing are designed to produce conservative, meaning-preserving edits that maximize n-gram alignment with reference sentences. The overall design emphasizes: (i) simplicity —a single-stage instruction-tuning setup using concise prompts instead of multi-step augmentation; (ii) adaptability —instruction-following behavior improves resilience to mixed-script and domain variation common in Indic text; and (iii) efficiency —adapter-based training and compact prompts reduce memory and compute requirements. We evaluate this setup on Hindi and Malayalam datasets, analyzing where instruction-based adaptation narrows or maintains the gap with high-resource or multilingual-transfer baselines ( 12 ; 16 ) . Evaluation protocol (GLEU). Consistent with IndicGEC evaluation, corpus-level GLEU is used as the primary metric, applying the “without tuning” variant ( 14 ) . To align modeling with the metric, the system (a) limits edits to preserve reference n-grams, (b) normalizes punctuation and script-specific conventions such as danda and whitespace, and (c) calibrates decoding on development data to prevent overcorrection or paraphrastic drift that reduces GLEU. ( 16 ) . Contributions. • A GenAI-based, augmentation-free framework for Hindi and Malayalam GEC optimized for GLEU under sub-thousand supervision. • Instruction-tuned prompts and adapter strategies that favor minimal, meaning-preserving edits consistent with reference overlap objectives. • A disciplined evaluation setup using the official GLEU metric with systematic comparison against multilingual and augmentation-based baselines. 2 Dataset The official Hindi and Malayalam grammatical error correction (GEC) datasets released by the AACL–IJCNLP 2025 BHASHA Workshop serve as the primary supervision source for the IndicGEC shared task. 4 4 4 Workshop site: https://bhasha-workshop.github.io/ . Shared task page: https://bhasha-workshop.github.io/sharedtask.html . Repository: https://github.com/BHASHA-Workshop/IndicGEC2025/ . The task specifies sentence-level GEC with single-reference gold outputs and evaluates systems using the GLEU metric on held-out test sets ( 14 ) . The shared task documentation defines GLEU as the official scoring metric and provides language-specific data directories containing train.csv and dev.csv , while test-only inputs are released subsequently for final leaderboard evaluation ( 10 ; 2 ) . Format and schema. Each split is a CSV with two columns: Input sentence (possibly erroneous) and Output sentence (the corrected reference). This layout supports minimal edit modeling and straightforward metric computation via n gram overlap ( 2 ) . Preprocessing. Identical script-aware normalization is applied to both languages, comprising: (i) elimination of zero-width and other non-visible Unicode artifacts, (ii) normalization of whitespace, (iii) script-specific punctuation and orthographic normalization, including standardized danda treatment, and (iv) removal of null entries and exact duplicate pairs. No oversampling or synthetic augmentation is introduced prior to training, ensuring that the experimental setting remains authentically low-resource ( 10 ; 2 ) . Splits and sizes. We adopt the official splits and report the counts used in our experiments: Language Train Dev Test Hindi 600 107 236 Malayalam 300 50 102 Gold references for the test sets are withheld by the organizers. Leaderboard scoring uses GLEU without tuning as stated on the shared task site ( 14 ; 10 ) . 3 Methodology 3.1 Why Gemma 3 for Indic GEC The Gemma 3 family is employed as the model backbone due to its strong cross-lingual alignment and architectural efficiency, both of which are essential for Indic grammatical error correction. Gemma 3 incorporates a revised tokenizer and post-training stack with expansive coverage over more than 140 languages, enabling robust treatment of scripts such as Devanagari and Malayalam. These scripts exhibit ligatures, vowel diacritics, and script-specific punctuation that complicate n -gram fidelity under GLEU-based evaluation. The refined tokenizer demonstrably mitigates token fragmentation and enhances the accuracy of edit-preserving corrections. Gemma 3 also supports long-context inference (up to 128K tokens, except for the 1B variant) with optimized KV-cache management. This capability allows for batched evaluation, structured prompt scaffolding, and transparent post-hoc analysis without heavy memory costs. Finally, its instruction-tuned checkpoints are released with open weights and standardized chat templates, enabling seamless integration for edit-constrained prompting and reproducible, deterministic experimentation. 3.2 System Overview Our pipeline operates in two coordinated stages. Stage 1 conducts Instruction Fine-Tuning (IFT) on a quantized 12B backbone using Alpaca-style supervision with Unsloth + PEFT/LoRA on the 4-bit checkpoint unsloth/gemma-3-12b-it-unsloth-bnb-4bit 21 ; 9 ; 8 ; 7 ; 25 . Stage 2 performs deterministic inference followed by a light post-processing normalizer. All reported results are obtained from Stage 2 using the frozen inference templates derived from the analysis in § 3.5 . 5 5 5 Final inference prompts and Alpaca prompts