Revisiting Classification Taxonomy for Grammatical Errors

Deqing Zou, Jingheng Ye, Yulu Liu, Yu Wu, Zishan Xu, Yinghui Li, Hai-Tao Zheng, Bingxu An, Zhao Wei, Yong Xu · 2025

Read Revisiting Classification Taxonomy for Grammatical Errors by Deqing Zou, Jingheng Ye, Yulu Liu, Yu Wu, Zishan Xu, Yinghui Li, Hai-Tao Zheng, Bingxu An, Zhao Wei, Yong Xu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-11890.txt.

Introduction

Deqing Zou and nine co-authors revisit how grammatical errors should be classified for language learning systems, in a paper from twenty twenty-five. How errors are grouped shapes the feedback learners receive. Existing taxonomies often lack strict validation. The result is mixed, unreliable feedback for students.

The authors present a systematic, qualitative frame for judging a taxonomy. It examines exclusivity, coverage, balance, and usability. They built a high-quality dataset of grammatical errors labeled under multiple taxonomies. Each taxonomy is then scored against the frame. The tests expose clear drawbacks in the past schemes. The authors aim to make error analysis more precise. Better classification, they argue, gives learners feedback they can grasp and act on.

Catalog record

CREATORDeqing Zou, Jingheng Ye, Yulu Liu, Yu Wu, Zishan Xu, Yinghui Li, Hai-Tao Zheng, Bingxu An, Zhao Wei, Yong Xu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE76 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-11890

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Revisiting Classification Taxonomy for Grammatical Errors Deqing Zou, Jingheng Ye, Yulu Liu, Yu Wu, Zishan Xu, Yinghui Li, Hai-Tao Zheng, Bingxu An, Zhao Wei, Yong Xu Abstract Grammatical error classification plays a crucial role in language learning systems, but existing classification taxonomies often lack rigorous validation, leading to inconsistencies and unreliable feedback. In this paper, we revisit previous classification taxonomies for grammatical errors by introducing a systematic and qualitative evaluation framework. Our approach examines four aspects of a taxonomy, i.e., exclusivity, coverage, balance, and usability. Then, we construct a high-quality grammatical error classification dataset annotated with multiple classification taxonomies and evaluate them grounding on our proposed evaluation framework. Our experiments reveal the drawbacks of existing taxonomies. Our contributions aim to improve the precision and effectiveness of error analysis, providing more understandable and actionable feedback for language learners. 1 1 1 All the codes and data will be released after the review. 1 Introduction Errors are an inevitable aspect of language acquisition, serving as critical indicators of learners’ linguistic development and providing valuable insights for educators and intelligent language learning systems ( Dulay and Burt, 1972 ; Dodigovic, 2007 ; Heift and Schulze, 2007 ; Liu et al., 2022 ; Li et al., 2022b ; Li et al., 2023b ; Li et al., 2024b ) . In Error Analysis (EA), the systematic identification, categorization, and interpretation of learner errors play a pivotal role in improving personalized instruction, generating automated feedback, and enabling effective language assessment ( Corder, 1967 ; James, 1998 ; Bialystok et al., 1982 ; Li et al., 2022a ; Li et al., 2023a ) . Central to this process is the use of grammatical error classification taxonomies ( Ma et al., 2022 ; Ye et al., 2024a ; Fei et al., 2023 ) , which organize learner errors based on linguistic or cognitive principles. These taxonomies have been widely adopted in applications such as grammatical error correction (GEC) Li et al. (2024a) ; Ye et al. (2023a) ; Ye et al. (2023b) ; Li et al. (2025) and automated essay scoring (AES), significantly enhancing error detection Huang et al. (2023) ; Zhang et al. (2023a) , correction Ye et al. (2022) , and feedback generation ( Liang et al., 2023 ; Yannakoudakis et al., 2017 ) . A well-designed grammatical error classification taxonomy allows language learners to better understand the nature and causes of their errors, facilitating targeted improvements in their linguistic competence Ye et al. (2025) . However, existing taxonomies are often developed based on empirical assumptions or ad-hoc practices, without rigorous validation ( He et al., 2021 ; Bryant et al., 2017 ) . This lack of systematic evaluation has led to issues such as overlapping categories, insufficient coverage of error types, and limited applicability in real-world educational contexts. To address these challenges, this paper revisits grammatical error classification taxonomies by systematically assessing their quality and utility. Specifically, we introduce a multi-metric evaluation framework that examines four key dimensions of a taxonomy: ❶ Exclusivity ensures that error categories are mutually exclusive, with clearly defined boundaries to minimize overlap and ambiguity; ❷ Coverage evaluates the extent to which the taxonomy captures both common and rare error types, ensuring a comprehensive representation of learner errors. ❸ Balance measures the taxonomy’s ability to balance attention between frequent and infrequent error types, avoiding overemphasis on a narrow subset of errors; ❹ Usability assesses the clarity and practical applicability of the taxonomy. To validate our evaluation framework, we construct a high-quality grammatical error dataset annotated with multiple classification taxonomies. This dataset is created through a collaborative annotation approach that leverages large language models (LLMs) and human annotators, ensuring scalability and annotation reliability. Using this dataset, we systematically evaluate four widely-used error classification taxonomies: POL73 ( Politzer and Ramirez, 1973 ) , TUC74 ( Tucker et al., 1974 ) , BRY17 ( Bryant et al., 2017 ) , and FEI23 ( Fei et al., 2023 ) . Through performance comparisons and annotator agreement experiments, we evaluate the rationality of these taxonomies. An ablation study on error type merging further reveals that classification taxonomies should not rely solely on empirical intuition but require systematic validation. These findings underscore the necessity of a rigorous evaluation for grammatical error taxonomy. In summary, our contributions are as follows: \bullet We propose a novel multi-metric evaluation framework for grammatical error classification taxonomies, incorporating dimensions of exclusivity, coverage, balance, and usability. \bullet We construct a high-quality grammatical error dataset annotated with multiple taxonomies, leveraging a collaborative annotation process involving LLMs and human experts. \bullet We conduct a comprehensive evaluation of four widely used taxonomies, providing insights into their strengths, limitations, and practical implications for error analysis in language learning. 2 Related Work Existing studies propose various error classification methods based on different perspectives, such as linguistic structure, cognitive processes, and communicative impact. Bialystok et al. (1982) categorize language errors into four primary taxonomies: (1) Linguistic Category Taxonomy ( Politzer and Ramirez, 1973 ; Tucker et al., 1974 ; Ângela Costa et al., 2015 ) classifies errors based on language components or linguistic constituents; (2) Surface Strategy Taxonomy ( Özkayran and Yılmaz, 2020 ; Rixha et al., 2021 ; Suhono, 2017 ) focuses on structural modifications made by learners; (3) Comparative Taxonomy ( Kafipour and Laleh, 2012 ; Dulay and Burt, 1974 ; White and Ontario Institute for Studies in Education, 1977 ) classifies errors by comparing L2 learner errors to L1 acquisition errors or native language structures; (4) Communicative Effect Taxonomy ( Burt, 1975 ; Waruwu and Harefa, 2024 ) classify errors based on their communicative impact. In addition to the above systematic classification taxonomies, other studies ( CHAN, 2010 ; Dahlmeier et al., 2013 ; Nicholls, 2003 ; Fei et al., 2023 ) have introduced other taxonomies, such as the simplification strategy taxonomy ( Bertkua, 1974 ) and the rule-based annotation toolkit ( Bryant et al., 2017 ; Ye et al., 2023c ) . 3 Methodology Given a dataset D of English learner texts, our goal is to systematically and quantitatively evaluate the rationality of n error classification taxonomies F={\{F_{1},F_{2},...,F_{n}\}} . Each taxonomy F_{i} consists of m predefined error types, denoted as F_{i}=\{ET_{1},ET_{2},...,ET_{m}\} , where ET_{j} represents the j -th error type in the taxonomy F_{i} . All notations are detailed in the Appendix A . We evaluate the rationality of each taxonomy along four dimensions: Exclusivity , Coverage , Balance , and Usability . Exclusivity and Usability are based on the generated results by the LLM, while Balance and Coverage are computed obtained directly from the manually annotated dataset through statistical analysis. 3.1 Exclusivity The error types in the taxonomy should be mutually exclusive, meaning that each error instance belongs to a single distinct category. Overlapping error types introduce instability and inconsistencies in error analysis, reducing the reliability of classification results. If a model frequently assigns high confidence to multiple categories for the same error, it suggests that the taxonomy lacks clear boundaries between certain error types. To quantify this issue, we assess exclusivity by analyzing the confidence scores of an LLM’s predictions. Confidence estimation plays a crucial role in this evaluation, we incorporate three established methods ( Xiong et al., 2024 ) for improved reliability

Read the complete text