Toward Trustworthy Evaluation of Sustainability Rating Methodologies: A Human–AI Collaborative Framework for Benchmark Dataset Construction
Xiaoran Cai, Wang Yang, Xiyu Ren, Chekun Law, Rohit Sharma, Peng Qi · 2026
Read Toward Trustworthy Evaluation of Sustainability Rating Methodologies: A Human–AI Collaborative Framework for Benchmark Dataset Construction by Xiaoran Cai, Wang Yang, Xiyu Ren, Chekun Law, Rohit Sharma, Peng Qi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-17106.txt.
Description
Xiaoran Cai led a six-author team tackling a credibility problem in sustainability ratings. Agencies score firms on environmental, social, and governance performance from disclosures and outside data, yet different agencies rate the same company very differently, which weakens comparability and trust. The paper proposes a shared human-AI collaboration framework for building benchmark datasets against which rating methods can be judged.
The framework has two parts: STRIDE, a principled scoring system that guides construction of firm-level benchmarks with large language models, and SR-Delta, a discrepancy-analysis procedure that surfaces where adjustments may be needed. Together they promise scalable, comparable evaluation of rating methods. The authors urge the wider AI community to adopt such approaches to strengthen ratings that back urgent sustainability goals.
Catalog record
| CREATOR | Xiaoran Cai, Wang Yang, Xiyu Ren, Chekun Law, Rohit Sharma, Peng Qi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 71 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-17106