ESGenius: Benchmarking LLMs on Environmental, Social, and Governance (ESG) and Sustainability Knowledge

Chaoyue He, Xin Zhou, Yi Wu, Xinjia Yu, Yan Zhang, Lei Zhang, Di Wang, Shengfei Lyu, Hong Xu, Xiaoqiao Wang, Wei Liu, Chunyan Miao · 2025

Read ESGenius: Benchmarking LLMs on Environmental, Social, and Governance (ESG) and Sustainability Knowledge by Chaoyue He, Xin Zhou, Yi Wu, Xinjia Yu, Yan Zhang, Lei Zhang, Di Wang, Shengfei Lyu, Hong Xu, Xiaoqiao Wang, Wei Liu, Chunyan Miao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-01646.txt.

Introduction

Chaoyue He and co-authors built ESGenius in 2025, a test set for language models on green and social duty topics. Firms face growing pressure on their environmental, social, and governance record, and AI tools get asked about these rules. The set checks how well models know this field.

The question part holds 1,136 multiple choice questions, made by models and checked by experts, each tied to its source text. A paired library holds 231 base documents from seven trusted sources. The team tested 50 models, from small to very large, both cold and with source lookup. Cold scores sat near 55 to 70 percent. Lookup raised scores a lot, most of all for small models. A clear knowledge gap shows in this field.

Catalog record

CREATORChaoyue He, Xin Zhou, Yi Wu, Xinjia Yu, Yan Zhang, Lei Zhang, Di Wang, Shengfei Lyu, Hong Xu, Xiaoqiao Wang, Wei Liu, Chunyan Miao
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE96 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-01646

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ESGenius: Benchmarking LLMs on Environmental, Social, and Governance (ESG) and Sustainability Knowledge Chaoyue He, Xin Zhou, Yi Wu, Xinjia Yu, Yan Zhang, Lei Zhang, Di Wang, Shengfei Lyu, Hong Xu, Xiaoqiao Wang, Wei Liu, Chunyan Miao Abstract We introduce ESGenius , a comprehensive benchmark for evaluating and enhancing the proficiency of Large Language Models (LLMs) in Environmental, Social and Governance (ESG) and sustainability-focused question answering. ESGenius comprises two key components: (i) ESGenius-QA , a collection of 1,136 MCQs generated by LLMs and rigorously validated by domain experts, covering a broad range of ESG pillars and sustainability topics. Each question is systematically linked to its corresponding source text, enabling transparent evaluation and supporting Retrieval-Augmented Generation (RAG) methods; and (ii) ESGenius-Corpus , a meticulously curated repository of 231 foundational frameworks, standards, reports, and recommendation documents from 7 authoritative sources. Moreover, to fully assess the capabilities and adaptation potential of the model, we implement a rigorous two-stage evaluation protocol— Zero-Shot and RAG . Extensive experiments across 50 LLMs (ranging from 0.5B to 671B parameters) demonstrate that state-of-the-art models achieve only moderate performance in zero-shot settings, with accuracies typically around 55-70%, highlighting ESGenius’s challenging nature for LLMs in interdisciplinary contexts. However, models employing RAG demonstrate significant performance improvements, particularly for smaller models. For example,“DeepSeek-R1-Distill-Qwen 14B” improves from 63.82% (zero-shot) to 80.46% with RAG. These results demonstrate the necessity of grounding responses in authoritative sources for enhanced ESG understanding. To the best of our knowledge, ESGenius is the first benchmark curated for LLMs and the relevant enhancement technologies, focusing on ESG and sustainability topics. 1 Introduction Figure 1: Relationship between model size and zero-shot accuracy across 50 LLMs evaluated on the ESGenius benchmark. Model sizes are plotted on a log 10 scale (in billions of parameters), with accuracy shown as percentages. There is a moderate positive correlation between model size and performance, suggesting larger models generally perform better. Dot colours denote 5 model families (DeepSeek, Gemma, GPT, Meta-Llama, Qwen). For proprietary API models, parameter counts are based on industry estimates (see Table 2 for details). Environmental, Social, and Governance (ESG) knowledge encompasses a vast domain of sustainability and corporate responsibility information that LLMs must effectively process to serve emerging business needs. This field spans across critical agendas from climate change and emissions tracking ( IPCC 2023 ) , to workplace safety and human rights ( Global Reporting Initiative 2023 ) . Such knowledge is codified in numerous technical frameworks and standards—major ones including GRI ( Initiative 2023 ) , SASB ( Board 2023 ) , TCFD ( Board 2017 ) , ISSB ( International Sustainability Standards Board 2023 ) , and CDP ( CDP Worldwide 2023 ) —which are constantly evolving. While LLMs show promise in processing complex ESG documents and providing relevant answers to user queries, their capabilities in this interdisciplinary domain remain largely unevaluated . Considering the high stakes involved, where incorrect responses about ESG requirements or metrics could lead to serious compliance violations or misguided sustainability initiatives, this lack of assessment is particularly problematic. However, there exists no comprehensive question-answering (QA) benchmark specifically designed to evaluate how well LLMs understand and reason about ESG concepts . Most existing QA datasets either completely omit ESG topics or address them superficially. This gap leaves researchers and practitioners without a reliable way to measure and improve LLMs’ ESG knowledge comprehension and question-answering abilities. To close this gap, we present ESGenius , a curated benchmark that targets MCQ-QA as the core evaluation task for ESG and sustainability knowledge. Our contributions are fourfold: (1) ESGenius Benchmark : A comprehensive evaluation framework comprising two integrated components: (i) ESGenius-QA , a collection of 1,136 MCQs across various ESG pillars and sustainability topics, generated through SOTA LLM approaches and validated by domain experts. Each question is explicitly mapped to supporting evidence from authoritative source texts, enabling transparent evaluation and facilitating RAG applications; and (ii) ESGenius-Corpus , a collection of 231 ESG documents and frameworks enabling efficient knowledge retrieval from 7 major authoritative sources. (2) Evaluation Protocol : We implement a comprehensive two-stage evaluation protocol consisting of zero-shot testing and RAG to systematically assess LLM capabilities. This assessment framework provides valuable insights into the current limitations and future potential of LLMs in understanding ESG.(3) Evaluation Analysis : Testing across a diverse set of 50 LLMs (ranging from 0.5B to 671B parameters) reveals significant performance gaps in zero-shot settings, with most models achieving 55-70% accuracy and the best model (o3) achieves a top score of 72.54%, as shown in Figure 1 . However, models demonstrate substantial potential for improvement through RAG approaches, with “DeepSeek-R1-Distill-Qwen 14B” improving from 63.82% (zero-shot) to 80.46%. (4) Open Source Initiative : To foster community engagement and collaborative advancement, we have made our complete benchmark suite publicly available at https://github.com/ANGEL-NTU/ESGenius . This includes comprehensive documentation, evaluation code, model implementations, and the full ESGenius dataset. We also maintain an interactive web portal at https://angel-ntu.github.io/ESGenius/ featuring a real-time leaderboard and detailed performance visualizations through heatmaps (see Appendix Figures 7 and 8 ), enabling researchers to track progress and identify areas for improvement in ESG-focused language models. The remainder of this paper is organized as follows: Section 2 surveys prior efforts on ESG knowledge resources, question-answering benchmarks, and retrieval-augmented generation; Section 3 details the construction of the ESGenius benchmark, describing both the ESGenius-QA dataset and the ESGenius-Corpus ; Section 4 outlines our experimental protocols—including zero-shot and RAG settings—and presents a comprehensive evaluation of 50 LLMs; Section 5 discusses the benchmark’s current limitations and avenues for future extension; and Section 6 concludes the paper and highlights directions for subsequent research. 2 Related Work This section reviews relevant literature across three key areas in using LLMs for ESG benchmarking, including knowledge sources, QA benchmarks and RAG. 2.1 ESG & Sustainability Knowledge Resources ESG data have traditionally been guided by voluntary reporting frameworks such as the Global Reporting Initiative (GRI) and the Sustainability Accounting Standards Board (SASB). These frameworks define taxonomies of ESG topics (e.g., emissions, labor practices, board governance) and indicators, but the unstructured text in corporate sustainability reports poses significant challenges for computational use. To impose structure, researchers have proposed ontologies and knowledge bases for ESG. For instance, Zhou and Perzylo 2023 introduced OntoSustain , aligning GRI and EU standards to capture key sustainability indicators, though broad coverage and automated population remain difficult. Several ESG-focused text corpora have emerged to support NLP research in this domain. One recent example is SusGen-30K , a dataset of 30k instances spanning multiple financial NLP tasks (e.g., sentiment, Q&A) and an ESG report generation task Wu et al. 2024 . Meanwhile, Chang et al. 2024 built a benchmark labeling sentences from corporate sustainability reports covering 36 ESG topics. These datasets remain relatively restricted, especially compared to general-domain corpora, and focus primarily on classification or generation tasks rather than MCQ question answering, thus motivating the need for broader coverage like ESGenius . Prior NLP work on sustainability texts has largely centered around information extraction and text classification. Early methods relied on topic models or lexicon-based approaches Raghupathi and Raghupathi 2020 , but these often fell short in capturing nuanced ESG terminology. More recent efforts leverage LLMs. For instance, Bronzini et al. 2023 employed a GPT-based pipeline to

Read the complete text