Mitigating Bias in Automated Grading Systems for ESL Learners: A Contrastive Learning Approach

Kevin Fan, Eric Yun · 2026

Read Mitigating Bias in Automated Grading Systems for ESL Learners: A Contrastive Learning Approach by Kevin Fan, Eric Yun (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-16724.txt.

Introduction

Kevin Fan and Eric Yun study bias in machines that grade essays. Schools now use such tools for tests with high stakes. Tools trained on native speech can grade harshly. Students who learn English as a new tongue may lose marks. Their style traits get read as flaws. Their 2026 preprint seeks a fairer way to grade.

The team uses a learn-by-contrast trick in training. The trick pulls the tool's eye to real skill. It pushes the eye off surface traits of new learners. Fair grades guard life paths like school seats and jobs. This work shows one path to tame such bias in code.

Catalog record

CREATORKevin Fan, Eric Yun
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE13 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-16724

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Mitigating Bias in Automated Grading Systems for ESL Learners: A Contrastive Learning Approach Kevin Fan, Eric Yun Abstract As Automated Essay Scoring (AES) systems are increasingly used in high-stakes educational settings, concerns regarding algorithmic bias against English as a Second Language (ESL) learners have increased. Current Transformer-based regression models trained primarily on native-speaker corpora often learn spurious correlations between surface-level L2 linguistic features and essay quality. In this study, we conduct a bias study of a fine-tuned DeBERTa-v3 model using the ASAP 2.0 and ELLIPSE datasets, revealing a constrained score scaling for high-proficiency ESL writing where high-proficiency ESL essays receive scores 10.3% lower than Native speaker essays of identical human-rated quality. To mitigate this, we propose applying contrastive learning with a triplet construction strategy: Contrastive Learning with Matched Essay Pairs . We constructed a dataset of 17,161 matched essay pairs and fine-tuned the model using Triplet Margin Loss to align the latent representations of ESL and Native writing. Our approach reduced the high-proficiency scoring disparity by 39.9% (to a 6.2% gap) while maintaining a Quadratic Weighted Kappa (QWK) of 0.76 . Post-hoc linguistic analysis suggests the model successfully disentangled sentence complexity from grammatical error, preventing the penalization of valid L2 syntactic structures. 1 Introduction Automated Essay Scoring (AES) has transitioned from feature-engineering approaches to transformer-based deep learning encoder models, specifically large language models (LLMs) such as BERT and DeBERTa [ 4 , 6 ] . While these models achieve high correlation with human raters, they are prone to “shortcut learning” [ 5 ] , where the model relies on easy-to-learn surface heuristics rather than complex semantic reasoning. In the context of ESL writing, transformer attention heads often disproportionately attend to distinct L2 markers such as prepositional misuse or specific sentence structures as proxies for predicting lower scores, ignoring the semantic vector. For ESL students, this results in a large penalty. They face the cognitive load of writing in a second language, and subsequently AES systems interpret valid L2 (Second Language) linguistic markers such as distinct determiners or tense inconsistencies as evidence of poor reasoning. Recent investigations indicate that standard AES models penalize ESL students more compared to native speakers with identical human ratings, even when the model has high overall accuracy [ 10 ] . State-of-the-art (SOTA) AES systems utilize pre-trained transformer models like BERT or RoBERTa fine-tuned with a regression head [ 4 ] . While these models achieve high aggregate correlation with human scores, they fail to generalize fairly across native language groups. Existing mitigation strategies, such as adversarial debiasing or data augmentation, often result in trade-offs where they reduce bias but degrade overall scoring accuracy [ 8 ] . Furthermore, simple class re-weighting is often ineffective because it does not teach the model why an ESL essay is equivalent to a native one, as it simply forces the model to use more attention on the ESL class without correcting the underlying representation [ 1 ] . There is currently no widely adopted framework that targets the representation gap between ESL and Native writing without sacrificing performance. Input Sequence x w_{1} w_{2} \dots w_{n} [CLS] Transformer Encoder f_{\theta}(x) (DeBERTa Multi-Head Attention) Latent Embedding h_{\texttt{[CLS]}}\in\mathbb{R}^{768} Pooling \hat{y} Final Score Linear Projection \hat{y}=W^{T}h+b Shortcut Learning: Encoding surface features (syntax) instead of semantics Figure 1: Vector-Based Scoring Architecture. The model f_{\theta} maps the input tokens to a latent vector h . The regression head projects h to a score. Bias occurs when f_{\theta} learns to encode surface-level heuristics (shortcuts) into h rather than true semantic quality. Our initial testing of the DeBERTa model, a state-of-the-art AES system, revealed a non-linear bias distribution. While the model graded low-proficiency essays fairly, it imposed a constrained scoring effect on high-proficiency ESL students. When an ESL student produced a high-quality essay (human score >0.8 normalized), the model systematically predicted a lower score compared to a Native peer. We propose a domain-specific intervention: Contrastive Learning with Matched Essay Pairs . By explicitly optimizing the embedding space to cluster essays based on quality rather than linguistic origin , we aim to remove demographic information from the decision boundary without losing the semantic signals required for accurate grading. 2 Methodology 2.1 Data Preparation We constructed a unified fairness benchmark by merging two major datasets: 1. ASAP 2.0 [ 3 ] : A large-scale corpus of argumentative essays. We utilized the ell_status metadata to distinguish Native speakers ( N\approx 17,000 ) from ESL speakers ( N\approx 2,600 ). 2. ELLIPSE [ 2 ] : A corpus of \approx 6,000 essays specifically from English Language Learners. Since ASAP 2.0 utilizes a 1–6 scale and ELLIPSE utilizes a 1–5 scale, all scores were normalized to a floating-point range of [0,1] . The combined dataset was split into training (80%) and testing (20%), stratified by group status to ensure adequate representation of ESL examples in the evaluation set. 2.2 Baseline Model We fine-tuned microsoft/deberta-v3-base using Low-Rank Adaptation (LoRA) [ 7 ] with rank r=16 . The model was trained with a regression head using Mean Squared Error (MSE) loss. This represents the current state-of-the-art (SOTA) approach for resource-constrained AES deployment [ 9 ] . 2.3 Fairness Triplet Construction To implement contrastive learning, we algorithmically generated a dataset of “Fairness Triplets” (A,P,N) . For every Native essay in the training set (Anchor A ) with normalized score S , we selected: • Positive ( P ) : An ESL essay with a human score in the range [S-0.02,S+0.02] . • Negative ( N ) : An essay (Native or ESL) with a score in the range [S\pm 0.20] . This resulted in 17,161 triplets . This curation forces the model to view an ESL essay and a Native essay of the same score as “quality twins,” mathematically requiring their embeddings to be closer to each other than to an essay of a different score. We selected the \pm 0.02 threshold for positives to ensure high-confidence quality matching (essays within \approx 1\% of score range) while the \pm 0.20 threshold for negatives creates sufficient margin to distinguish quality levels. These values were chosen based on the score distribution in our training set. 2.4 Contrastive Training We removed the regression head of the DeBERTa model and fine-tuned the backbone using Triplet Margin Loss : L=\max(0,d(A,P)-d(A,N)+\alpha) (1) where d is the Euclidean distance and \alpha is the margin. We utilized \alpha=1.0 based on preliminary ablation studies. The model was trained for 2 epochs using LoRA to update only the query and value projection matrices, preserving the pre-trained knowledge of the base model while aligning the latent space. Anchor Negative Positive LEARNING Anchor Positive Negative Figure 2: Triplet Loss Visualization. The loss function minimizes the distance between an anchor and a positive (same identity/score) while maximizing the distance between the anchor and a negative (different identity/score). 2.5 Validation and Scoring After contrastive alignment, the backbone was frozen to preserve the fair embedding space. A lightweight linear regression head was trained on top of the frozen embeddings for 5 epochs to map the vector representations back to scalar scores. This was the basis of the contrastive model. 3 Results 3.1 Phase 1: Baseline Testing The baseline DeBERTa model achieved a Quadratic Weighted Kappa (QWK) of 0.79. However, error

Read the complete text