EDM-ARS: A Domain-Specific Multi-Agent System for Automated Educational Data Mining Research

Chenguang Pan, Zhou Zhang, Weixuan Xiao, Chengyuan Yao · 2026

Read EDM-ARS: A Domain-Specific Multi-Agent System for Automated Educational Data Mining Research by Chenguang Pan, Zhou Zhang, Weixuan Xiao, Chengyuan Yao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-18273.txt.

Introduction

Pan and co-authors present EDM-ARS, a multi-agent system that automates educational data mining research. Five LLM agents cover problem framing, data work, analysis, critique, and writing. A state-machine coordinator supports revision loops, checkpoint recovery, and sandboxed code runs. From a prompt and dataset, it produces a full LaTeX paper with real citations and peer review. Current limits include single-dataset scope and formulaic output; causal and multi-dataset work is planned.

Catalog record

CREATORChenguang Pan, Zhou Zhang, Weixuan Xiao, Chengyuan Yao
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48878
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.18273

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

EDM-ARS: A Domain-Specific Multi-Agent System for Automated Educational Data Mining Research Chenguang Pan, Zhou Zhang, Weixuan Xiao, Chengyuan Yao Abstract In this technical report, we present the Educational Data Mining Automated Research System (EDM-ARS), a domain-specific multi-agent pipeline that automates end-to-end educational data mining (EDM) research. We conceptualize EDM-ARS as a general framework for domain-aware automated research pipelines, where educational expertise is embedded into each stage of the research lifecycle. As a first instantiation of this framework, we focus on predictive modeling tasks. Within this scope, EDM-ARS orchestrates five specialized LLM-powered agents (ProblemFormulator, DataEngineer, Analyst, Critic, and Writer) through a state-machine coordinator that supports revision loops, checkpoint-based recovery, and sandboxed code execution. Given a research prompt and a dataset, EDM-ARS produces a complete LaTeX manuscript with real Semantic Scholar citations, validated machine learning analyses, and automated methodological peer review. We also provide a detailed description of the system architecture, the three-tier data registry design that encodes educational domain expertise, the specification of each agent, the inter-agent communication protocol, and mechanisms for error-handling and self-correction. Finally, we discuss current limitations, including single-dataset scope and formulaic paper output, and outline a phased roadmap toward causal inference, transfer learning, psychometric, and multi-dataset generalization. EDM-ARS is released as an open-source project to support the educational research community. 1 1 1 This document is a technical report for EDM-ARS rather than a research paper. For the latest updates, please visit our website: https://edmars.ai/ . The source code is available at https://github.com/cgpan/edm-ars-public . Keywords: Educational Data Mining, Automated Research, Multi-Agent Systems, Large Language Models, HSLS:09 1 Introduction Automated scientific research powered by large language models (LLMs) has become an active area of study. Systems such as The AI Scientist [ 4 ] , its successor AI Scientist v2 [ 9 ] , CycleResearcher [ 7 ] , DeepScientist [ 8 ] , AI-Researcher [ 6 ] , and Aletheia [ 3 ] have shown that LLM-based agents can perform literature review, formulate hypotheses, write and execute code, analyze results, and draft manuscripts with minimal human intervention. These systems represent an emerging paradigm in which the scientific discovery loop, from idea generation to paper writing, is largely automated. Several concerns arise when examining this landscape. A general, domain-agnostic system may aim for generality across scientific fields but consequently lacks the domain knowledge that experienced human researchers bring to specialized areas. For example, a general system that does not know that an educational data survey uses negative sentinel codes for missing data, or that temporal leakage between survey waves invalidates predictive models, will silently produce methodologically flawed research. The computational cost of these systems is also high; general-purpose agentic search strategies (e.g., the tree-search approach of AI Scientist v2) require many LLM calls, and the token consumption per paper can be impractical for researchers without institutional API budgets. The quality of generated papers remains debated, with many outputs exhibiting formulaic structure, shallow interpretation, and hallucinated citations. And despite the large and growing body of educational data mining (EDM) and learning analytics (LA) research [ 5 ] , virtually no automated research system has targeted the educational research community. To address these gaps, we present EDM-ARS (Educational Data Mining Automated Research System), a domain-specific multi-agent pipeline that automates end-to-end EDM research, beginning with prediction tasks as its first fully supported paradigm. EDM-ARS is built on three premises. A portion of EDM research follows standardized, well-defined methodological workflows: problem formulation from a dataset codebook, data cleaning with survey-specific conventions, model training with interpretability analysis, fairness-aware evaluation, and structured academic writing. These workflows are sufficiently regular to be captured by a pipeline of specialized agents. Domain knowledge is treated as a first-class design element rather than an afterthought: the system’s data registry, validation rules, and agent prompts encode accumulated expertise in working with large-scale educational survey data. And by constraining the system’s scope to a well-defined research paradigm (prediction on a single representative dataset for the current version), we aim to achieve higher reliability and output quality than a general-purpose system attempting to cover all of science. As a first instantiation of this pipeline, EDM-ARS currently focuses on prediction tasks using the High School Longitudinal Study of 2009 (HSLS:09), a nationally representative longitudinal survey conducted by the National Center for Education Statistics (NCES) tracking approximately 23,500 students from 9th grade (2009) through postsecondary education (2016). Beyond this first instantiation, we plan to expand EDM-ARS to support other educational data mining tasks in future versions includes causal inference (quasi-experimental designs, heterogeneous treatment-effect analysis, optimal treatment regimes), transfer learning (cross-domain and cross-population generalization), and psychometric applications. This project began as a hobby project, but we treat it with the rigor of a production system. The remainder of this report is organized as follows. Section 2 surveys the current landscape of automated research systems and positions EDM-ARS within it. Section 3 describes the pipeline architecture, the orchestrator state machine, design rationale, and technology stack. Section 4 details the data registry, including the three-tier variable strategy, domain knowledge encoding, and leakage prevention mechanisms. Section 5 specifies each of the five agents. Section 6 describes inter-agent communication protocols. Section 7 outlines the evaluation framework. Section 8 discusses current limitations and future work. Section 9 provides deployment and usage instructions. 2 Related Work: Automated Research Systems The landscape of LLM-powered automated research has evolved rapidly since 2024. We organize the discussion around general-purpose automated research systems, domain-specific automation, and educational data mining. 2.1 General-Purpose Automated Research Systems The AI Scientist [ 4 ] was among the first end-to-end systems to automate the full research lifecycle: idea generation, experiment execution, manuscript writing, and simulated peer review. Operating within predefined code templates (e.g., NanoGPT, 2D diffusion), the system used an LLM to iteratively modify experimental code, collect results, and write a paper. Its main limitation was the reliance on human-authored code scaffolds, which constrained the system to domains where such templates existed. The AI Scientist v2 [ 9 ] addressed this constraint through an agentic tree search architecture. Rather than following a single linear pipeline, v2 explores multiple experimental branches in parallel, managed by a dedicated Experiment Manager agent. Different branches produce different manuscripts, and the best is selected via automated review. The system also eliminated reliance on human code templates by generating experimental code from scratch, and introduced a Vision-Language Model (VLM) feedback loop for iterative figure quality improvement. AI Scientist v2 demonstrated workshop-level paper quality, though at high computational cost due to the breadth-first exploration strategy. CycleResearcher [ 7 ] introduced an iterative reinforcement learning framework for automated research. CycleResearcher (the policy model) generates papers, while CycleReviewer (the reward model) scores them. Preference pairs are constructed from multiple samples, and the policy is refined via SimPO (Simple Preference Optimization). This feedback loop allows the system to learn from its own review signals, producing progressively better papers. CycleResearcher differs from single-shot pipelines in a fundamental way: rather than relying on a fixed prompt to produce quality, it uses learned optimization to improve output over time. DeepScientist [ 8 ] extended this direction to frontier-level scientific findings. The system maintains a cumulative knowledge base across runs, so each new research question is informed by prior outputs. This addresses a

Read the complete text