Surfacing Isolated Learners with Outcome-Independent Mediation of Feedback between Teachers and Students Using AI

Junsoo Park, Youssef Medhat, Htet Phyo Wai, Ploy Thajchayapong, Ashok K. Goel · 2026

Read Surfacing Isolated Learners with Outcome-Independent Mediation of Feedback between Teachers and Students Using AI by Junsoo Park, Youssef Medhat, Htet Phyo Wai, Ploy Thajchayapong, Ashok K. Goel (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-29240.txt.

Introduction

Junsoo Park and colleagues propose a clear decision layer for AI-augmented classrooms. It ranks course topics that need attention using three signals, without waiting for grades. The signals are how widespread student difficulty is, gaps between what learners report and what is observed, and worries raised by teachers. Each ranked topic comes with a short record that explains its place. In one graduate computing course, the ranked topics matched teacher concerns well and surfaced struggling learners missed by any single signal alone.

Catalog record

CREATORJunsoo Park, Youssef Medhat, Htet Phyo Wai, Ploy Thajchayapong, Ashok K. Goel
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26438
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.29240

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Surfacing Isolated Learners with Outcome-Independent Mediation of Feedback between Teachers and Students Using AI Junsoo Park, Youssef Medhat, Htet Phyo Wai, Ploy Thajchayapong, Ashok K. Goel Abstract AI-augmented classrooms generate rich teacher and student feedback before graded outcomes become available, yet these signals can be difficult to translate into timely instructional decisions. We propose an interpretable decision layer : a transparent mechanism that ranks course topics requiring attention without using grades or post-hoc outcome labels. The approach combines three signals: student learning difficulty prevalence, disagreement between learner self-reports and observed difficulties, and unresolved teacher concerns. The output is a ranked set of topic priorities with per-topic decision records explaining each ranking. In one graduate CS course offering ( n = 5 n=5 instructor interviews; n = 279 n=279 survey responses), prioritized topics aligned with instructor concerns (top-5 overlap 3/5; Spearman ρ = 0.80 \rho=0.80 ) and student-reported topic difficulty ( ρ = 0.46 \rho=0.46 , p = .048 p=.048 ). Multi-signal integration also surfaced learners not identified through individual signal sources alone (AUC = 0.96 =0.96 vs. 0.91 0.91 for gap prevalence alone). Reflective thinking, help-seeking, and self-efficacy provided additional evidence that student behavioral signals align with learning-related constructs. While preliminary, these findings suggest that transparent coordination mechanisms may help support human–AI co-agency when feedback is incomplete. [ orcid=0009-0003-4450-0103, [email protected], ] [ [email protected], ] [ [email protected], ] [ orcid=0009-0000-5993-1094, [email protected], ] [ orcid=0000-0003-4043-0614, [email protected], ] keywords Human-AI Co-Agency ,Bidirectional Feedback Loops ,Process-Based Assessment ,Interpretable Decision Layer ,AI-Augmented Classrooms † † copyrightyear: 2026 † † copyright: Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). † † venue: HAI-Agency: Workshop on Orchestrating Human and AI Agency for Proactive and Reflective Learning, co-located with the 27th International Conference on Artificial Intelligence in Education (AIED 2026), Seoul, Korea, June 27 - July 3, 2026. † † corresp: Corresponding author. † † address: Georgia Institute Of Technology, North Avenue, Atlanta, GA 30332, USA 1 Introduction AI teaching assistants are increasingly used in real classrooms. Teachers and learners generate rich sources of information: interviews, surveys, and interaction logs, but these signals are difficult to combine into clear instructional actions 9 ; 20 ; 3 ; 17 . Graded outcomes often arrive too late to support decisions during learning. We therefore ask: how can teacher and learner feedback be coordinated when outcome data are unavailable? We propose an interpretable decision layer : a transparent mechanism that combines teacher and student feedback to rank course topics needing attention. Rather than using grades or outcome labels, the approach relies on three signals: i) how common a learning difficulty is among students (gap prevalence), ii) disagreement between learner self-reports and observed difficulties, and iii) unresolved teacher concerns. These signals are combined into ranked topic priorities together with a decision record that explains why a topic received its rank. The system organizes teacher, student, and researcher roles around this shared decision process. In one graduate CS course offering ( n\!=\!5 instructor interviews; n\!=\!279 survey responses), prioritized topics aligned with instructor concerns and student-reported difficulty (Section 5 ). Reflective thinking, help-seeking, and self-efficacy aligned with topic understanding as complementary evidence of learning-related behavior 13 ; 1 , while combining multiple signals surfaced learners missed by single-source approaches. Definitions. Topic. A knowledge-graph node representing a course concept (e.g., planning, analogical reasoning); this offering contains 54 topics and 47 prerequisite links (Appendix B ). Gap prevalence ( R_{t} ). The fraction of students identified as having difficulty with topic t . Survey disagreement ( D_{t} ). Difference between observed topic difficulty and student self-reported difficulty: |R_{t}-s_{t}| when survey difficulty s_{t} is available (Eq. 1 ). Teacher friction ( F ). A summary measure of unresolved instructor concerns derived from coded interview themes (Appendix F ). Mediation layer. Combines (R_{t},D_{t},F) into a topic priority score P_{t} and ranking (Eq. 2 ). Decision record. Per-topic inputs, score, rank, and diagnostics. Figure 1 uses the synonymous label “Decision Trace.” Isolated learner. A learner not identified by any single signal source, but surfaced when multiple signals are considered together. Research question and hypotheses. RQ: Can a transparent decision layer combine teacher and student feedback to identify useful support priorities without relying on grades or other outcome labels? H1. The mediation layer produces interpretable topic priorities that remain similar across reasonable weight choices. H2. Student behavioral signals such as reflective thinking and help seeking are associated with learning-related measures at the cohort level. H3. Combining multiple signals identifies learners who would not be identified by any individual signal alone. 2 Related Work Teacher orchestration and reflection. Prior work highlights the need for teacher orchestration in AI-enhanced classrooms and for interfaces that help instructors interpret learner activity and make instructional decisions 11 ; 10 ; 5 . Open and inspectable learner models further emphasize the importance of transparency in educational AI systems 4 . Our work builds on these ideas by focusing on an interpretable coordination process in which topic priorities can be traced back to their contributing signals. Unlike approaches centered on post-hoc analytics or dashboard reflection alone, we consider how teacher and student perspectives can be combined during instruction when outcome measures are not yet available. Student modeling: reflection and help seeking. Student reflection and help-seeking behavior are widely associated with learning processes and self-regulated learning. We draw on the Reflective Thinking Questionnaire (RTQ) 13 , based on Mezirow’s distinction between reflection and critical reflection 14 , together with discussion-board help-seeking behavior and self-efficacy measures 1 . In this work, these constructs serve as complementary evidence that student signals capture learning-relevant behavior rather than directly determining mediation priorities. Knowledge-graph learner modeling. Knowledge-graph approaches represent relationships among concepts and learner understanding at the topic level. Prior work in A4L extracts curriculum knowledge graphs and topic-level learner representations 8 . We use topic-level gap prevalence derived from this pipeline as one source of evidence in the mediation process. Human–AI co-agency and feedback coordination. AI teaching assistants such as Jill Watson demonstrate how AI systems can participate in classroom interactions 7 ; 18 ; 12 . More recent work on bidirectional feedback and human–AI collaboration suggests that classroom support can benefit from integrating perspectives from multiple participants rather than relying on a single source of information 2 ; 15 ; 16 . Our work explores how teacher, student, and researcher perspectives may be coordinated through a shared, transparent decision layer. 3 Approach Our approach organizes teacher, student, and researcher roles around a shared mediation layer that coordinates feedback across the classroom process (Figure 1 ). Teacher signals derive from instructor use of a deployed Jill Watson analytics dashboard (Appendix A ): dashboard observations inform semi-structured interviews, which are coded into themes and summarized as teacher friction F . Student signals derive from Jill Watson interaction traces and a mid-course survey, yielding gap prevalence R_{t} , self-reported difficulty s_{t} , and survey disagreement D_{t} . Researcher-defined weights \mathbf{w} determine how these signals contribute to topic priorities and decision records. Figure 1: Bidirectional feedback mediator. Teacher and student subsystems provide signals to a mediation layer that produces topic priorities and decision records. Researcher weights are specified rather than learned in the current

Read the complete text