REC-CBM: Rubric-Aware Error-Correction Concept Bottleneck Models for Trustworthy Open-Ended Grading

Chengshuai Zhao, Fan Zhang, Kumar Satvik Chaudhary, Yiwen Li, Lo Pang-Yun Ting, Ying-Chih Chen, Huan Liu · 2026

Read REC-CBM: Rubric-Aware Error-Correction Concept Bottleneck Models for Trustworthy Open-Ended Grading by Chengshuai Zhao, Fan Zhang, Kumar Satvik Chaudhary, Yiwen Li, Lo Pang-Yun Ting, Ying-Chih Chen, Huan Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-27402.txt.

Introduction

Chengshuai Zhao and colleagues make automated grading of open-ended answers trustworthy. Their REC-CBM model routes scores through rubric concepts, calibrates ranking across rubric dimensions, and corrects noisy concept predictions before grading. Tests show better grading performance and more faithful concept-level reasoning than strong baselines, in realistic classroom settings.

Catalog record

CREATORChengshuai Zhao, Fan Zhang, Kumar Satvik Chaudhary, Yiwen Li, Lo Pang-Yun Ting, Ying-Chih Chen, Huan Liu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE93512
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.27402

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

REC-CBM: Rubric-Aware Error-Correction Concept Bottleneck Models for Trustworthy Open-Ended Grading Chengshuai Zhao, Fan Zhang, Kumar Satvik Chaudhary, Yiwen Li, Lo Pang-Yun Ting, Ying-Chih Chen, Huan Liu Abstract Open-ended grading is central to equitable and personalized education, yet manual grading remains time-consuming and costly, underscoring the need for automated grading systems. Although recent neural and large language model (LLM) based systems have demonstrated superior performance, they are typically black-box models whose scoring processes and rationales are difficult for educators to verify and trust. Concept bottleneck models (CBMs) have emerged as a promising approach by routing predictions through human-interpretable concepts, providing a mechanistic guarantee of transparency. However, standard CBMs are not tailored to open-ended grading: they do not explicitly model fine-grained rubric dimensions, inadequately capture the ordinal semantics of scoring scales, and neglect inherent reliability issues in human concept annotations. To address these limitations, we propose REC-CBM , a rubric-aware error-correction concept bottleneck model for trustworthy open-ended grading. REC-CBM introduces a rubric-aware concept encoder that learns concept-specific representations over responses and an ordinal pairwise calibration objective that preserves ranking structure among rubric dimensions. It further incorporates a latent concept error-correction module that denoises concept predictions before final grade prediction while preserving interpretability. Comprehensive experiments on publicly available datasets show that REC-CBM consistently improves grading performance and produces more faithful concept-level reasoning than both state-of-the-art black-box models and transparent baselines. Further analyses validate the contribution of each component and demonstrate the applicability in realistic educational settings. Overall, this work provides a practical, interpretable grading solution that enables educators to inspect, intervene in, and trust automated decisions, advancing more transparent and trustworthy education. Code is available: https://github.com/scott-f-zhang/REC-CBM . Data are available: https://huggingface.co/datasets/scott-f-zhang/REC-CBM 1 Introduction. Open-ended grading evaluates students’ written responses to open questions, which plays a fundamental role in education across subjects and levels. At the same time, evaluating free-text responses at scale is labor-intensive and expensive, especially in settings that require timely feedback across large numbers of students [ 7 ] . This tension has made automated grading an important topic for educational data mining and AI for education communities [ 1 ] . Figure 1: Comparison of open-ended grading systems. Recent neural and large language model based grading systems have substantially improved predictive performance [ 32 , 10 ] , making automated scoring increasingly viable in practice. However, educational assessment is a high-stakes domain where raw accuracy alone is insufficient. When a system assigns a score to a student’s response, instructors need to understand why that score was produced, whether it aligns with the intended rubric, and how to intervene when the model makes a questionable judgment. The black-box nature of these models makes it difficult for educators to verify the reasoning behind a grade, contest, or audit specific decisions, and provide feedback that is pedagogically useful [ 33 , 6 , 39 ] . This lack of transparency can undermine trust and limit the potential of automated grading systems in reliable educational contexts [ 35 ] . Concept bottleneck models (CBMs) offer a promising middle ground between predictive strength and user-centric transparency [ 19 ] . By constraining predictions to pass through human-interpretable intermediate concepts, CBMs can expose task-specific evidence behind a final grade rather than leaving the decision process hidden inside an end-to-end encoder, which makes them particularly appealing for educational assessment. Despite this promise, existing CBMs are not yet well-suited to open-ended grading. Firstly, standard CBMs assume all concepts reside in a shared latent space, which is suboptimal for fine-grained grading settings where concepts capture different rubric dimensions and attend to different aspects of responses. In addition, they treat concept scores as evenly spaced categories, which discards the natural ranking information in the rubric annotations. Furthermore, rubric concept labels in real world are inherently noisy and subjective due to annotator disagreement, overlapping rubric dimensions, and ambiguity in open-ended responses [ 40 ] . Existing CBMs generally assume concept labels are reliable, which can degrade grading performance and undermine trust in the concept-level explanations. To address these limitations, we propose REC-CBM , a rubric-aware error-correction concept bottleneck model for trustworthy open-ended grading. REC-CBM first performs rubric-aware concept extraction so that each grading dimension is grounded in evidence relevant to that aspect of the response. It then applies ordinal concept calibration to preserve ranking structure among rubric levels [ 4 ] , thus capturing the score semantics. Finally, it incorporates latent concept error correction to denoise intermediate concept predictions while preserving the interpretable bottleneck. Through comprehensive experiments, REC-CBM demonstrates superior grading performance with faithful concept-level reasoning than both state-of-the-art black-box models and transparent baselines. Further analyses validate the contribution of each component and demonstrate the applicability in realistic educational settings. Our contributions are fourfold: • Problem formulation: We approach trustworthy open-ended grading through the lens of transparent decision-making, grounding the grading process on human-interpretable concepts, enabling educators to inspect, verify, and intervene on reasoning. • Data resources: We curate and annotate public open-ended grading benchmarks with rubric-aligned concept labels, creating a valuable resource for future research on interpretable educational assessment. • Methodological innovation: We identify key limitations of standard CBMs in the grading context and propose REC-CBM , a novel framework that integrates rubric-aware concept encoding, ordinal pairwise calibration, and latent concept error correction to address these limitations. • Empirical validation: We conduct extensive evaluations and experiments, offering insights into the performance, interpretability, and practical utility of REC-CBM in real-world grading scenarios. 2 Related Work. In this section, we review three lines of work relevant to our work. 2.1 Automated Grading Systems. Automated grading has evolved from feature-engineered systems such as e-rater [ 1 ] to neural scoring models [ 7 ] . Pre-trained language models further improved accuracy [ 32 ] , and recent LLM-based approaches incorporate rubric descriptions through prompting [ 10 ] , yet their internal reasoning remains opaque. A growing line of work seeks to address this: Wang et al. [ 33 ] and Chu et al. [ 6 ] align scoring with human rationales, while Schaller et al. [ 26 ] audit fairness in essay scoring. However, these approaches surface post-hoc explanations without mechanistically guaranteeing that the explanation determines the assigned grade. 2.2 Concept Bottleneck Models. Concept Bottleneck Models [ 19 ] improve transparency by routing predictions through human-interpretable concepts. Extensions address prediction uncertainty via probabilistic [ 18 ] , stochastic [ 31 ] , and post-hoc [ 36 ] concept bottlenecks, though none model measurement error from a psychometric perspective or account for ordinal concept structure. CBMs have also been adapted beyond vision: Tan et al. [ 29 ] interpret pretrained language models via concept bottlenecks, Ismail et al. [ 15 ] target protein design, and Espinosa Zarlenga et al. [ 37 ] extend CBMs to tabular data. These works demonstrate the versatility of the CBM approaches in different domains, but the application to open-ended grading remains underexplored. 2.3 Trustworthy AI in Education. Trustworthy AI in education, spanning fairness, transparency, accountability, and reliability, has become a growing concern as AI systems are deployed at scale [ 13 , 22 ] . Research has documented algorithmic bias across demographic groups in student modeling and admissions [ 2 ] , while explainable AI methods have been adapted

Read the complete text