Comparing Few-Shot Prompting of GPT-4 LLMs with BERT Classifiers for Open-Response Assessment in Tutor Equity Training

Sanjit Kakarla, Conrad Borchers, Danielle Thomas, Shambhavi Bhushan, Kenneth R. Koedinger · 2025

Read Comparing Few-Shot Prompting of GPT-4 LLMs with BERT Classifiers for Open-Response Assessment in Tutor Equity Training by Sanjit Kakarla, Conrad Borchers, Danielle Thomas, Shambhavi Bhushan, Kenneth R. Koedinger (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-06658.txt.

Description

Sanjit Kakarla, Conrad Borchers, and colleagues compare two AI graders for open-ended answers in tutor training in a 2025 education study. Trainee tutors answer equity scenarios in free text, and judging those answers needs sensitivity to context. A large commercial model brings broad knowledge but cannot be fine-tuned freely, while an older encoder model learns the local rubric yet knows less about the world.

Few-shot prompting of the large model is tested head to head against fine-tuned smaller classifiers. The comparison tells training programs which grader to trust for nuanced, values-laden responses where a careless score can teach the wrong lesson.

Catalog record

CREATORSanjit Kakarla, Conrad Borchers, Danielle Thomas, Shambhavi Bhushan, Kenneth R. Koedinger
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE19 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-06658

Browse the full library · About the Tower of Babel