LearnMate 2 : Design and Evaluation of an LLM-powered Personalized and Adaptive Support System for Online Learning Conference: Designing Interactive Systems Conference; June 13–17, 2026; Singapore, Singapore Designing Interactive Systems Conference (DIS ’26), June 13–17, 2026, Singapore, Singapore DOI: 10.1145/3800645.3812972 ISBN: 979-8-4007-2563-0/2026/06 CCS: Human-centered computing HCI design and evaluation methods CCS: Computing methodologies Natural language processing
Xinyu Jessica Wang, Christine P Lee, Bilge Mutlu · 2026
Read LearnMate 2 : Design and Evaluation of an LLM-powered Personalized and Adaptive Support System for Online Learning Conference: Designing Interactive Systems Conference; June 13–17, 2026; Singapore, Singapore Designing Interactive Systems Conference (DIS ’26), June 13–17, 2026, Singapore, Singapore DOI: 10.1145/3800645.3812972 ISBN: 979-8-4007-2563-0/2026/06 CCS: Human-centered computing HCI design and evaluation methods CCS: Computing methodologies Natural language processing by Xinyu Jessica Wang, Christine P Lee, Bilge Mutlu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-06257.txt.
Introduction
Xinyu Jessica Wang, Christine Lee, and Bilge Mutlu design and evaluate an AI support system for online learning. Online courses reach everyone but give little personal guidance, a gap large language models might help close. The team iteratively built their system to provide personalized study plans, real time contextual help, and adaptive learning activities for each student.
A preliminary study with 24 users assessed effectiveness and usability and shaped refinements to the design. The refined system was then evaluated with 16 users against a leading online learning platform paired with a general language model. Results indicate the system improved both learning outcomes and user experience over those existing tools. The authors discuss what the work shows about the design space of AI driven educational support.
Catalog record
| CREATOR | Xinyu Jessica Wang, Christine P Lee, Bilge Mutlu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 89 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.06257
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LearnMate 2 : Design and Evaluation of an LLM-powered Personalized and Adaptive Support System for Online Learning Conference: Designing Interactive Systems Conference; June 13–17, 2026; Singapore, Singapore Designing Interactive Systems Conference (DIS ’26), June 13–17, 2026, Singapore, Singapore DOI: 10.1145/3800645.3812972 ISBN: 979-8-4007-2563-0/2026/06 CCS: Human-centered computing HCI design and evaluation methods CCS: Computing methodologies Natural language processing Xinyu Jessica Wang, Christine P Lee, Bilge Mutlu Abstract . Personalization is crucial for effective learning, yet online learning, designed for widespread availability and open access, lacks personalized guidance. Recent advancements in large language models (LLMs) offer opportunities to bridge this gap. We explore how LLM-driven tools may be designed to support personalized and adaptive learning and examine how they shape user experience and learning outcomes. We iteratively designed LearnMate 2 to support online learning by providing personalized study plans, real-time contextual assistance, and adaptive learning activities. A preliminary study ( n = 24 n=24 ) assessed the effectiveness and usability of LearnMate 2 and informed refinements in our system, which we then evaluated ( n = 16 n=16 ) against a combination of a state-of-the-art online learning platform and an LLM for learning support. Results indicate that LearnMate 2 advances AI pedagogy by improving both learning outcomes and user experience compared to existing online learning and support tools. This work advances our understanding of the design space of personalized, AI-driven educational tools and their potential impact on user experience. Figure 1 . From fixed online learning workflows to personalized and adaptive support — The figure contrasts a typical online learning workflow (left), where learners often struggle with cold start problems ( A ), limited timely support ( B ), and fixed learning plans ( C ), with the LearnMate 2 workflow (right). LearnMate 2 addresses these challenges through PlanMate for personalized study planning ( A ), StudyMate for real-time contextual assistance ( B ), and AdaptMate for adaptive learning activities based on quiz performance and progress ( C ). LearnMate² system overview A two-panel diagram. The left panel, labeled "Current Online Learning Workflow," shows three challenges: (A) Cold Start Problem, illustrated by a learner overwhelmed by 50+ lessons not knowing where to start; (B) No Timely and Effective Help, where a learner cannot get a contextual answer about the Neolithic Revolution; and (C) Fixed Learning Plans, where a learner who failed a quiz has no way to adjust their plan. The right panel, labeled "Online Learning Workflow with LearnMate²," shows the three corresponding solutions: (A) PlanMate generates a personalized learning schedule based on goals, time, path, and pace; (B) StudyMate provides real-time contextual assistance alongside course video; and (C) AdaptMate generates an adapted learning schedule based on quiz performance. 1. Introduction Online education has transformed access to learning by making educational content available across geographic, economic, and scheduling constraints ( Kumar et al. 2019 ; Money and Dean 2019 ; Iniesto and Rodrigo 2024 ; Badge et al. 2008 ; Veletsianos and Houlden 2019 ; Black et al. 2019 ) . Online platforms now support millions of learners, from students in underserved areas to working professionals pursuing continuing education, through resources such as video lectures, tutorials, digital textbooks, and curated articles ( Pardamean et al. 2022 ) . As a result, online learning has become an essential part of modern education. Despite these benefits, current online learning platforms often provide limited personalized support. Many still adopt a one-size-fits-all model, presenting standardized content without adequately accounting for differences in learners’ goals, pace, progress, or support needs ( Gillett-Swan 2017 ) . This limitation is reflected in persistent challenges such as low completion rates and reduced satisfaction in large-scale online courses, where learners often receive little individualized guidance ( Daniel 2012 ) . Prior work points to three recurring gaps in online learning environments. First, learners often struggle to plan and regulate their study process without instructor guidance, leading to difficulties with time management and goal setting ( Zimmerman 2002 ; Bernacki et al. 2021 ) . Second, learners often lack timely instructional support while actively studying, making it difficult to receive clarification or scaffolding when confusion arises ( Taber 2018 ; Spouse 1998 ; Hung and Nguyen 2022 ) . Third, many systems collect performance data but do not effectively translate it into adaptive interventions that respond to learners’ ongoing progress ( Maier and Klotz 2022 ; Murtaza et al. 2022 ) . Recent advances in artificial intelligence (AI), particularly large language models (LLMs), create new opportunities to address these gaps. Emerging AI-based educational systems have shown promise in areas such as adaptive feedback, personalized content generation, and in-situ learning support ( Kazemitabaar et al. 2024 ; Chen et al. 2025 ; Shochcho et al. 2025 ) . However, it remains unclear how such capabilities should be integrated into online learning workflows as a coordinated form of personalization rather than as isolated features. In particular, the effects of different LLM-supported personalization mechanisms on learner performance and user experience remain underexplored. Recent work has begun to explore LLM-based personalization in online learning ( Wang et al. 2025 ; Park et al. 2024 ; Wen et al. 2024 ) , providing a foundation for studying how these mechanisms can be designed and evaluated more systematically. In this paper, we extend the personalization framework of Wang et al. 2025 through LearnMate 2 , an LLM-powered personalized and adaptive learning support system for online learning. LearnMate 2 integrates three components that correspond to the gaps above: (1) PlanMate , which generates personalized study plans based on four dimensions—goals, time, pace, and path; (2) StudyMate , which provides real-time contextual assistance during learning sessions through progressive disclosure, follow-up explanations, practice questions, and external resources; and (3) AdaptMate , which adjusts learning activities and study plans based on quiz performance and learning progress. Together, these components form a closed-loop learning workflow in which planning, studying, and adaptation continuously inform one another. At a high level, we investigate, what are the overall effects of personalization on learning outcomes in online learning? More specifically, this paper addresses the following research questions: (1) RQ1 : How do personalized study plans support learners’ planning and learning experience? (2) RQ2 : How does real-time contextual assistance during learning sessions support learners’ performance and learning experience? (3) RQ3 : How do adaptive learning activities affect learners’ progress and learning experience? (4) RQ4 : How does a personalization workflow that integrates these components affect the learner’s learning outcomes and experience? To answer these research questions, we designed and developed LearnMate 2 and conducted two studies. Across these studies, we examine learning performance primarily through quiz performance and learning experience through participants’ reported usability and satisfaction. We first conducted a preliminary study ( n=24 ) to assess how well the system supported personalized learning and to identify usability issues. We then refined the system and conducted a final user study ( n=16 ) comparing LearnMate 2 against a workflow combining a state-of-the-art online learning platform and an LLM used for learning support. Our results show that LearnMate 2 improves quiz performance and user experience compared to existing online learning and support tools. Our work makes the following contributions: (1) System contribution: We present LearnMate 2 , an LLM-based