Learning in Blocks: A Multi Agent Debate Assisted Personalized Adaptive Learning Framework for Language Learning
Nicy Scaria, Silvester John Joseph Kennedy *, Deepak Subramani · 2026
Read Learning in Blocks: A Multi Agent Debate Assisted Personalized Adaptive Learning Framework for Language Learning by Nicy Scaria, Silvester John Joseph Kennedy *, Deepak Subramani (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-22770.txt.
Introduction
Scaria, Kennedy, and Subramani swap quiz-led language courses, where learners move up despite gaps in live talk, for Learning in Blocks. The plan ties progress to shown skill in talk. Expert agents debate grammar, words, and live talk scores before a judge merges them into one view, and tips point at weak skills. Moving up needs 70 percent mastery, with spaced review fighting skill loss.
Catalog record
| CREATOR | Nicy Scaria, Silvester John Joseph Kennedy *, Deepak Subramani |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32165 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.22770
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Learning in Blocks: A Multi Agent Debate Assisted Personalized Adaptive Learning Framework for Language Learning Nicy Scaria, Silvester John Joseph Kennedy *, Deepak Subramani Abstract Most digital language learning curricula rely on discrete-item quizzes that test recall rather than applied conversational proficiency. When progression is driven by quiz performance, learners can advance despite persistent gaps in using grammar and vocabulary during interaction. Recent work on LLM-based judging suggests a path toward scoring open-ended conversations, but using interaction evidence to drive progression and review requires scoring protocols that are reliable and validated. We introduce Learning in Blocks , a framework that grounds progression in demonstrated conversational competence evaluated using CEFR-aligned rubrics. The framework employs heterogeneous multi-agent debate (HeteroMAD) in two stages: a scoring stage where role-specialized agents independently evaluate Grammar, Vocabulary, and Interactive Communication, engage in debate to address conflicting judgments, and a judge synthesizes consensus scores; and a recommendation stage that identifies specific grammar skills and vocabulary topics for targeted review. Progression requires demonstrating 70% mastery, and spaced review targets identified weaknesses to counter skill decay. We benchmark four scoring and recommendation methods on CEFR A2 conversations annotated by ESL experts. HeteroMAD achieves a superior score agreement with a 0.23 degree of variation and recommendation acceptability of 90.91%. An 8-week study with 180 CEFR A2 learners demonstrates that combining rubric-aligned scoring and recommendation with spaced review and mastery-based progression produces better learning outcomes than feedback alone. † † This is a preprint. The Version of Record of this contribution will be published in International Conference on Artificial Intelligence in Education (LNAI), and will be available shortly. 1 Introduction Digital language learning curricula typically rely on discrete-item quizzes that test recall of grammar rules or vocabulary definitions, creating a disconnect between assessment formats (isolated recall tasks) and and what constitutes communicative proficiency (applied performance in context). When progression is driven by quiz performance, learners can advance through content while gaps in applying Grammar, Vocabulary, and Interactive Communication persist, producing a Swiss cheese learning [ 17 ] pattern. Large language models have enabled conversational AI applications for language learning [ 16 ] , allowing learners to engage in open-ended dialogue practice. However, using open-ended spoken interaction for assessment and progression requires scoring protocols that are reliable and validated against expert judgment. In the absence of such validation, platforms often default to quiz-based signals that are straightforward to standardize and score at scale, even when those signals do not capture whether learners can apply skills during interaction. Recent work on rubric-based LLM judging [ 31 , 19 ] suggests a path toward scoring open-ended interaction more directly, creating an opportunity to ground progression decisions in demonstrated conversational competence rather than quiz performance. This challenge is especially acute for conversational proficiency, which requires coordinated control of grammatical accuracy, lexical appropriateness, and interactional success across multiple turns [ 13 ] . In language learning contexts, CEFR provides widely adopted descriptors that operationalize these dimensions through rubric-aligned criteria for Grammar, Vocabulary, and Interactive Communication [ 7 ] . Applying such rubrics to open-ended conversations requires context-dependent judgments that weigh multiple aspects of performance simultaneously. At the same time, studies of LLM-as-a-Judge protocols report systematic pitfalls, including position bias, verbosity bias, and overconfident single-model judgments [ 25 ] , which complicate the use of automated scores for instructional decisions. Beyond reliable scoring, operationalizing mastery-based progression for conversational practice raises two additional challenges. First, what constitutes sufficient evidence of mastery when success cannot be reduced to item correctness? Learners may demonstrate knowledge of grammatical rules in isolation, yet struggle to use them accurately in conversation [ 10 ] . Second, how can these systems support skill maintenance over time? Even after learners show strong performance, skills can decay without targeted re-elicitation and reinforcement [ 9 ] . We address these challenges through Learning in Blocks , which links progression and review to evidence from conversations evaluated with CEFR-aligned rubrics. Conversational scoring and recommendations are produced using heterogeneous multi-agent debate (HeteroMAD). For each conversation, the system outputs CEFR-aligned scores for Grammar, Vocabulary, and Interactive Communication, along with feedback and ranked recommendations for Grammar skills and Vocabulary topics to revisit. Recommendations are converted into targeted review lesson groups, and progression to the next Concept Block occurs only after the learner meets a 70% mastery criterion on the block-aligned evaluation. Our contributions are threefold: (1) we benchmark multiple conversation-scoring mechanisms and select HeteroMAD for CEFR-aligned scoring and recommendations; (2) we formalize a curriculum structure that organizes Grammar, Vocabulary, and Conversation into aligned lesson groups, so that each conversation lesson is designed to elicit the grammar and vocabulary taught in the surrounding instruction; and (3) we run an 8-week learner study with three cohorts to isolate the impact of Learning in Blocks framework. 2 Related Work and Background 2.1 Generative AI for Conversational Assessment and Diagnosis LLMs are increasingly used for open-ended language practice and feedback [ 16 ] , but dialogue assessment differs from objective grading because it requires rubric-based judgments of quality, appropriateness, and task success that are context-dependent and span multiple turns. Conversational proficiency is inherently multi-dimensional, involving tradeoffs across grammatical control, lexical appropriateness, and interactional success [ 13 ] . In language learning contexts, CEFR provides widely used descriptors that motivate rubric-aligned dimensions such as Grammar, Vocabulary, and Interactive Communication (IC) [ 7 ] . These challenges have motivated ‘LLM-as-a-Judge’ approaches for scoring open-ended outputs [ 31 , 19 ] , but prior work also reports systematic pitfalls such as position and verbosity biases, underscoring the need for careful protocol design and explicit validation [ 25 ] . Self-Consistency and Self-Refine are common test-time strategies for improving reliability by aggregating multiple generations or iteratively revising a single output [ 28 , 21 ] . Multi-agent debate (MAD) extends this idea by distributing evaluation across multiple agents that propose and critique judgments before a final decision is made [ 8 ] . Previous evidence suggests that debate-style methods do not consistently outperform strong baselines in well-specified objective tasks [ 5 ] , but rubric-based dialog assessment may benefit from multiple perspectives, as it requires judgment across dimensions. Prior work has also explored multi-agent architectures for conversation-based assessment [ 14 ] . However, validated comparisons of diagnostic mechanisms for CEFR-aligned dialogue scoring and the relevance of generated recommendations remain limited. We benchmark Self-Consistency, Self-Refine, and both homogeneous and heterogeneous MAD variants, testing whether role prompting alone or combining complementary model perspectives yields more stable rubric-aligned scores and more actionable recommendations. 2.2 Mastery-Based Progression Mastery-based learning is built on the premise that learning outcomes must remain constant, allowing variable time for individual learners so that learners progress only after demonstrating a clear performance criterion [ 3 , 12 ] . However, many digital language curricula operationalize mastery using discrete-item quizzes that primarily test recall of grammar rules or vocabulary definitions. While such assessments efficiently measure declarative knowledge, they do not necessarily predict whether learners can apply those concepts in sentence building or real-time conversation [ 10 ] , leading to weak transfer where surface progression masks persistent application gaps [ 18 ] . A central challenge for mastery-based progression in open-ended language practice is, therefore, defining mastery evidence when success is