Toward Trustworthy Evaluation of Sustainability Rating Methodologies: A Human–AI Collaborative Framework for Benchmark Dataset Construction

Xiaoran Cai, Wang Yang, Xiyu Ren, Chekun Law, Rohit Sharma, Peng Qi · 2026

Read Toward Trustworthy Evaluation of Sustainability Rating Methodologies: A Human–AI Collaborative Framework for Benchmark Dataset Construction by Xiaoran Cai, Wang Yang, Xiyu Ren, Chekun Law, Rohit Sharma, Peng Qi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-17106.txt.

Description

Xiaoran Cai led a six-author team tackling a credibility problem in sustainability ratings. Agencies score firms on environmental, social, and governance performance from disclosures and outside data, yet different agencies rate the same company very differently, which weakens comparability and trust. The paper proposes a shared human-AI collaboration framework for building benchmark datasets against which rating methods can be judged.

The framework has two parts: STRIDE, a principled scoring system that guides construction of firm-level benchmarks with large language models, and SR-Delta, a discrepancy-analysis procedure that surfaces where adjustments may be needed. Together they promise scalable, comparable evaluation of rating methods. The authors urge the wider AI community to adopt such approaches to strengthen ratings that back urgent sustainability goals.

Catalog record

CREATORXiaoran Cai, Wang Yang, Xiyu Ren, Chekun Law, Rohit Sharma, Peng Qi
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE71 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-17106

Browse the full library · About the Tower of Babel