SCALE : Towards Collaborative Content Analysis in Social Science with Large Language Model Agents and Human Intervention

Chengshuai Zhao , Zhen Tan , Chau-Wai Wong , Xinyan Zhao , Tianlong Chen , Huan Liu · 2025

Read SCALE : Towards Collaborative Content Analysis in Social Science with Large Language Model Agents and Human Intervention by Chengshuai Zhao , Zhen Tan , Chau-Wai Wong , Xinyan Zhao , Tianlong Chen , Huan Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-10937.txt.

Description

Chengshuai Zhao and co-authors set language model agents to do the slow scholarly work of content analysis. Social scientists often read thousands of texts and sort them into theory-based categories, a grind of coding, debate, and rule revision. Their framework, SCALE, casts agents in those very roles, coding texts, arguing in simulated discussion, and growing the codebook as they learn.

Human experts can step in at several points to correct or steer the agents. On real research datasets spanning several hard tasks, the system came close to human-level agreement. The authors see a partnership model: machines supply the stamina and first drafts, while scholars keep judgment and theory in the loop. Such teamwork could make large text studies far cheaper to run well.

Catalog record

CREATORChengshuai Zhao , Zhen Tan , Chau-Wai Wong , Xinyan Zhao , Tianlong Chen , Huan Liu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE106 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-10937

Browse the full library · About the Tower of Babel