Context-Aware Prediction of Student Quiz Performance with Multimodal Textbook Features

Samin Khan · 2026

Read Context-Aware Prediction of Student Quiz Performance with Multimodal Textbook Features by Samin Khan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-24770.txt.

Introduction

Samin Khan asks whether the wording and pictures inside a question help predict how a student will score. Learning platforms usually forecast performance from past interactions alone. Yet questions themselves vary in language and visual complexity. Khan studies chapter review questions from CourseKata, an online textbook platform. Lightweight features are extracted from the question text. Image features come from the textbook visuals attached to the same chapters.

The dataset holds 4,742 student chapter observations drawn from 562 class student pairs. Adding content features improves quiz score prediction by about nine percent over the prior performance baseline. That test groups students into five folds. A stricter test holds out whole chapters instead. There, text features still reduce prediction error. Models using image features, however, do worse than the baseline. Khan concludes that question text carries useful signal about difficulty, while the visual features tried here do not.

Catalog record

CREATORSamin Khan
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE15 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.24770

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Context-Aware Prediction of Student Quiz Performance with Multimodal Textbook Features Samin Khan Abstract . Educational platforms often predict student performance from prior interactions, but the assessment content itself also varies in linguistic and visual complexity. This paper studies whether lightweight content features extracted from CourseKata chapter-review questions improve prediction of end-of-chapter quiz scores beyond a student’s average prior exercise performance. The study combines 2023 CourseKata student response data with chapter-level text features from review-question wording and image features from textbook visuals. Across 4,742 student-chapter observations from 562 class-student IDs, adding content features improves student-grouped five-fold quiz prediction performance by 9.1% relative to a prior-performance baseline. In leave-chapter-out validation, text features reduce prediction error relative to the baseline, while image-containing models have higher error. This paper suggests that a context-aware model adds useful signal about the text and visual features of questions to better predict student quiz performance compared with using past student performance alone. 1. Introduction Predicting student performance is a central task in educational data mining because accurate forecasts can support adaptive practice, formative feedback, and earlier identification of students who may need help. Classical knowledge tracing models estimate evolving student mastery from interaction histories ( 3 ) , and modern neural variants have shown that flexible sequence models can capture richer patterns in student records ( 5 ) . However, a student’s future score is not only a function of prior performance. The content of the target assessment can also vary in vocabulary, length, statistical topic, representational form, and visual complexity. This distinction matters for online textbooks such as CourseKata, where students work through interactive statistics and data science materials before taking end-of-chapter quizzes. CourseKata provides large-scale anonymized data from college courses using the ABC and ABCD versions of Statistics and Data Science: A Modeling Approach ( 7 ) , whose pedagogical design emphasizes statistical modeling and connected conceptual understanding ( 6 ) . If assessment content varies in difficulty-relevant ways, then models that use only prior exercise scores may leave predictable variance unexplained. The broad research question is therefore: Do text and image features from chapter-review content improve prediction of student quiz performance beyond prior exercise performance? This question has two practically distinct forms. First, for student modeling in an existing course, the relevant question is whether content features improve prediction for held-out students when the same chapters are represented in the training data. In this setting, content features supplement prior student performance with information about how other students performed on similar chapter content. Second, for new-curriculum use cases, the relevant question is whether relationships learned from existing chapters transfer to a chapter withheld during training. This setting is closer to predicting quiz difficulty or expected performance for newly designed instructional content before large-scale student data are available. Accordingly, this paper addresses two research questions: (1) RQ1: Held-out student prediction. Do text and image features improve prediction of quiz scores for students not seen during training, when the target chapters are represented by other students in the training data? (2) RQ2: Held-out chapter prediction. Do text and image features improve prediction of quiz scores for a chapter not seen during training? This paper answers these questions with an interpretable pipeline: aggregate prior exercise scores at the student-chapter level, extract lightweight text and image features from review questions, compare linear and regularized models, and evaluate both held-out-student and held-out-chapter generalization. 2. Related Work Student performance prediction has increasingly moved from simple historical summaries toward models that incorporate richer exercise and item information. Exercise-aware knowledge tracing explicitly studies how exercise content can improve prediction and interpretability beyond interaction records alone ( 4 ) . Recent multimodal work similarly argues that question semantics and student-question structure should be modeled together, including approaches that combine graph neural networks with language-model-derived question representations ( 10 ) . This paper is also related to automatic item difficulty prediction. A systematic review of text-based question difficulty prediction found that linguistic features are frequently important and that automatic approaches can address the cost and scalability limits of expert calibration and pretesting ( 1 ) . Recent empirical work comparing text-feature models for item difficulty likewise shows that lexical and wording features can be useful predictors in assessment settings ( 9 ) . The present study differs by predicting individual student quiz scores, not only item difficulty, while asking whether assessment content adds signal after controlling for each student’s prior chapter performance. 3. Data All data presented here were collected and shared with the author by CourseKata ( https://coursekata.org ). The analysis uses a processed split derived from the 2023 CourseKata college dataset. The end-of-chapter file contains student quiz scores for chapters 1–9, and the response file contains exercise-level attempts, points earned, and points possible. A student is represented as a unique class-student pair, consistent with CourseKata’s public data documentation. After merging student exercise aggregates, quiz scores, and content features, the analytic dataset contains 4,742 student-chapter observations from 562 class-student IDs across five classes and two textbooks: ABC and ABCD. The outcome is the end-of-chapter quiz score, scaled from 0 to 1. The baseline predictor is avg_exercise_scores , the mean proportion correct across prior exercises attempted by the same student in the same chapter. 4. Methods 4.1. Content Feature Extraction Chapter-review pages were scraped from CourseKata preview pages using Selenium because the review items are dynamically rendered. The cleaned public code replaces the original notebook’s unsafe eval parsing with ast.literal_eval and separates scraping, feature construction, and modeling into reusable scripts. Text features were computed with wordfreq ( 8 ) : average word length, mean word rarity defined as 7-\mathrm{ZipfFrequency}(w) , number of complex words with Zipf frequency below 5, and total word count. These features approximate surface-level linguistic complexity, verbosity, and vocabulary rarity. Image features were computed with OpenCV ( 2 ) . For each review-page image, Canny edges were extracted, Hough line segments were counted, and external contours were counted as a rough proxy for visual regions. Chapter-level image features are the average number of lines per image, average number of regions per image, and total image count. Figure 1 shows an example of this processing pipeline. Figure 1 . Example image associated with a CourseKata quiz question processed using Canny edge detection. 532 line segments and 58 visual regions were detected. A processed textbook image showing edge-detected line and contour structure used to derive image-complexity features. 4.2. Models and Evaluation The analysis compares four feature sets: (1) Base : prior exercise performance only. (2) Base + Image : base feature plus image features. (3) Base + Text : base feature plus text features. (4) Base + Image + Text : all features. For comparability with the original notebook, ordinary least squares models are first fit in sample. The primary evaluation for RQ1 is five-fold grouped cross-validation where all chapters from the same class-student ID are kept in the same fold. In each split, the model is trained on four folds of students and tested on the held-out fold of students. The chapters are represented in both train and test folds, but no class-student ID appears in both. This evaluates generalization to new students in known course content. The evaluation for RQ2 is leave-chapter-out validation.

Read the complete text