Multilingual Grammatical Error Annotation: Combining Language-Agnostic Framework with Language-Specific Flexibility

Mengyang Qiu Tran Minh Nguyen Zihao Huang Zelong Li Yang Gu, Qingyu Gao Siliang Liu Jungyeul Park · 2025

Read Multilingual Grammatical Error Annotation: Combining Language-Agnostic Framework with Language-Specific Flexibility by Mengyang Qiu Tran Minh Nguyen Zihao Huang Zelong Li Yang Gu, Qingyu Gao Siliang Liu Jungyeul Park (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-07719.txt.

Introduction

Mengyang Qiu and co-authors published a research paper in 2025 on a shared framework for marking grammar errors in many languages at once. Error marking guides tools that correct learners' writing, but the main framework was built for English and strains on unlike languages. Their design pairs one language-blind core with tidy add-ons for each language's quirks.

They rebuilt the classic tool on a modern language engine to widen its reach. Trials on English, German, Czech, Korean, and Chinese show the framework bends from plain marking to fine custom work without breaking the shared rules. The code and tools are public. The authors hope for steadier scoring of writing tools across languages.

Catalog record

CREATORMengyang Qiu Tran Minh Nguyen Zihao Huang Zelong Li Yang Gu, Qingyu Gao Siliang Liu Jungyeul Park
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-07719

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Multilingual Grammatical Error Annotation: Combining Language-Agnostic Framework with Language-Specific Flexibility Mengyang Qiu Tran Minh Nguyen Zihao Huang Zelong Li Yang Gu, Qingyu Gao Siliang Liu Jungyeul Park Abstract Grammatical Error Correction (GEC) relies on accurate error annotation and evaluation, yet existing frameworks, such as errant , face limitations when extended to typologically diverse languages. In this paper, we introduce a standardized, modular framework for multilingual grammatical error annotation. Our approach combines a language-agnostic foundation with structured language-specific extensions, enabling both consistency and flexibility across languages. We reimplement errant using stanza to support broader multilingual coverage, and demonstrate the framework’s adaptability through applications to English, German, Czech, Korean, and Chinese, ranging from general-purpose annotation to more customized linguistic refinements. This work supports scalable and interpretable GEC annotation across languages and promotes more consistent evaluation in multilingual settings. The complete codebase and annotation tools can be accessed at https://github.com/open-writing-evaluation/jp_errant_bea . 1 Introduction Grammatical Error Correction (GEC), which aims to automatically detect and correct errors in written text, has emerged as one of the most important and widely studied tasks in Natural Language Processing (NLP) for educational applications, particularly those supporting language learning and writing improvement. It benefits both native speakers (L1), by enhancing clarity and fluency in their writing, and non-native learners (L2), by providing immediate, structured feedback that reinforces correct grammatical patterns, boosts writing confidence, and, ultimately, supports language development and acquisition ( Marjokorpi 2023 ; Van Beuningen et al. 2012 ) . Over the years, the lion’s share of research has focused on advancing GEC systems—evolving from rule-based and statistical approaches to neural architectures, such as neural machine translation with transformers ( Zhao et al. 2019 ) and, more recently, prompting-based approaches built on large language models ( Zeng et al. 2024 ; for a comprehensive review, see Bryant et al. 2023 ). Yet, automatic error annotation and evaluation play an equally critical role in GEC. Error annotation identifies and categorizes linguistic errors, while evaluation measures how effectively GEC systems correct them. Together, these two components help establish standardized benchmarks, influencing everything from system development to the quality of corrections eventually delivered to users. However, despite their importance, they have historically received less attention and are often treated as ancillary to system development and dataset creation. Among existing tools for automatic error annotation and evaluation (e.g., M 2 , Dahlmeier and Ng 2012 ; GLEU , Napoles et al. 2015 ), errant (ERRor ANnotation Toolkit) has established itself as the de facto framework for English GEC. What makes errant stand out is its detailed linguistic annotations, with a total of 55 possible error types for English ( Bryant et al. 2017 ) . errant ’s significance was solidified in the Building Educational Applications 2019 Shared Task: Grammatical Error Correction (BEA-2019) , where it was used to standardize multiple datasets and served as the official scorer ( Bryant et al. 2019 ) . While this toolkit has proven effective for English, further refinements are needed to improve its versatility and adaptability, especially in multilingual scenarios. Recent years have seen growing interest in multilingual GEC, as demonstrated by initiatives like the MultiGEC-2025 Shared Task , which brought together efforts across twelve typologically diverse European languages ( Masciolini et al. 2025a ; Masciolini et al. 2025b ) . However, this surge in interest has outpaced the development of consistent multilingual annotation resources. As noted by Masciolini et al. 2025a , only three languages in MultiGEC–namely Czech, German, and Greek–have received errant -style annotation. For the remaining languages, the authors acknowledge that, due to limited time and resources, they implemented only coarse-grained alignment between original and corrected texts to support holistic scoring, without access to the kind of detailed error analysis enabled by errant for English. Even in existing adaptations of errant for various languages, implementations vary considerably in their design choices–ranging from annotation label schemes to tokenization and part-of-speech (POS) tagging tools–and differ in the level of granularity applied to language-specific error types. Although differences in orthographies and morphosyntactic structures across languages are unavoidable, greater consistency in annotation practices is highly desirable. To address these challenges, our goal is to develop a consistent and reusable framework for grammatical error annotation that can be readily adapted across typologically diverse languages. Drawing inspiration from the original errant ’s dataset-agnostic design, we extend its core philosophy to multilingual settings by separating the annotation pipeline into two components: a shared architecture that applies across languages, and optional extensions tailored to language-specific features. Even within the language-specific layer, we introduce structured templates for common error types, such as spelling, word order, and word boundary errors, which can be reused or adapted across languages with similar orthographic or syntactic patterns. In addition, our implementation relies on the stanza toolkit for tokenization and POS tagging, which provides standardized processing pipelines for over 70 languages ( Qi et al. 2020 ) , allowing our framework to be readily extended to annotate new GEC datasets of other languages when they become available. The rest of the paper is organized as follows: \lx@sectionsign 2 reviews the original errant framework, discusses challenges in its multilingual adaptations, and motivates the use of stanza for more consistent cross-linguistic preprocessing. \lx@sectionsign 3 introduces our proposed grammatical error typology, which combines a language-agnostic core with structured, language-specific extensions. \lx@sectionsign 4 presents our reimplementation of English errant and demonstrates the framework’s applicability to multiple languages, ranging from generic use in European languages, to minor template refinements for Korean, and deeper customization for Chinese. Finally, \lx@sectionsign 5 summarizes our contributions and emphasizes the framework’s flexibility and extensibility for multilingual GEC. 2 Background and Related Work 2.1 Description of errant errant is a unified framework for error annotation and evaluation in English GEC. It provides a rule-based, dataset-agnostic approach for extracting and categorizing edits between original and corrected sentences, making it a crucial tool for system evaluation and benchmarking ( Bryant et al. 2017 ) . At the core of its annotation pipeline is a linguistically enhanced alignment algorithm that identifies edit boundaries between sentence pairs. This algorithm, originally proposed by Felice et al. 2016 , extends the Damerau-Levenshtein distance with a linguistically informed cost function that considers part-of-speech tags, lemmas, and character similarity. Unlike surface-level edit distance, this method prioritizes alignments between tokens that are syntactically or morphologically related (e.g., meet and meeting ), and handles both one-to-one edits and multi-token reordering. A rule-based merging strategy is then applied to combine adjacent edits where appropriate, based on patterns frequently observed in learner data, such as phrasal verb edits. This alignment process significantly improves the consistency and quality of extracted edits ( Felice et al. 2016 ) . Following alignment, errant applies a rule-based annotation scheme to categorize edits into fine-grained grammatical error types, enabling both comprehensive feedback and error-type evaluation. Specifically, it defines 25 primary error types based on POS and morphological properties obtained from spaCy 1 1 1 https://spacy.io , and further classifies them into three edit operations: M issing, U nnecessary, and R eplacement, resulting in a total of 55 possible error types (e.g., R:VERB:TENSE indicates a replacement error related to verb tense). To store annotations, errant generates output in M2 format, the standard representation for GEC annotations

Read the complete text