Developing a Multi-Agent System to Generate Next Generation Science Assessments with Evidence-Centered Design

Yaxuan Yang, Jongchan Park, Yifan Zhou, Xiaoming Zhai · 2026

Read Developing a Multi-Agent System to Generate Next Generation Science Assessments with Evidence-Centered Design by Yaxuan Yang, Jongchan Park, Yifan Zhou, Xiaoming Zhai (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-18451.txt.

Description

Yaxuan Yang and three co-authors automate the creation of science assessments aligned with modern standards. Their 2026 paper starts from the Next Generation Science Standards, which demand assessments of students' ability to apply science knowledge to problems and design solutions. Performance-based assessments elicit that ability, but Evidence-Centered Design, the framework safeguarding their validity, needs diverse expertise and is costly and labor-intensive.

The team integrates Evidence-Centered Design into a multi-agent system that ensembles several large language models with varying expertise, automating multi-stage item generation workflows once done by human experts. AI-generated items match human-developed items in overall quality on alignment with three-dimensional standards and cognitive demands. They show distinct strength in inclusivity but lag in clarity, brevity, and multimodal design, while both AI and human items share weak spots in evidence collectability and student interest alignment. Human expertise remains essential.

Catalog record

CREATORYaxuan Yang, Jongchan Park, Yifan Zhou, Xiaoming Zhai
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-18451

Browse the full library · About the Tower of Babel