Bridging AI and Carbon Capture: A Dataset for LLMs in Ionic Liquids and CBE Research
Gaurab Sarkar, Sougata Saha · 2025
Read Bridging AI and Carbon Capture: A Dataset for LLMs in Ionic Liquids and CBE Research by Gaurab Sarkar, Sougata Saha (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-06964.txt.
Introduction
Gaurab Sarkar and Sougata Saha benchmark language models on chemical engineering questions about carbon capture, in a 2025 paper. General models shine at everyday knowledge, but narrow scientific fields lack good tests. The pair built an expert-curated dataset of 5,920 examples centered on ionic liquids, salt-like solvents that can trap carbon dioxide. Questions vary in difficulty, balancing language complexity against depth of domain knowledge.
Three open models under ten billion parameters took the test. They showed basic familiarity with ionic liquids but lacked the specialized reasoning advanced work demands. The authors discuss ways to sharpen such models for carbon capture research. They note the irony that the models themselves carry a carbon footprint, and frame progress in both fields as a joint push toward carbon neutrality by 2050.
Catalog record
| CREATOR | Gaurab Sarkar, Sougata Saha |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-06964
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Bridging AI and Carbon Capture: A Dataset for LLMs in Ionic Liquids and CBE Research Gaurab Sarkar, Sougata Saha Abstract Large Language Models (LLMs) have demonstrated exceptional performance in general knowledge and reasoning tasks across various domains. However, their effectiveness in specialized scientific fields like Chemical and Biological Engineering (CBE) remains underexplored. Addressing this gap requires robust evaluation benchmarks that assess both knowledge and reasoning capabilities in these niche areas, which are currently lacking. To bridge this divide, we present a comprehensive empirical analysis of LLM reasoning capabilities in CBE, with a focus on Ionic Liquids (ILs) for carbon sequestration—an emerging solution for mitigating global warming. We develop and release an expert-curated dataset of 5,920 examples designed to benchmark LLMs’ reasoning in this domain. The dataset incorporates varying levels of difficulty, balancing linguistic complexity and domain-specific knowledge. Using this dataset, we evaluate three open-source LLMs with fewer than 10 billion parameters. Our findings reveal that while smaller general-purpose LLMs exhibit basic knowledge of ILs, they lack the specialized reasoning skills necessary for advanced applications. Building on these results, we discuss strategies to enhance the utility of LLMs for carbon capture research, particularly using ILs. Given the significant carbon footprint of LLMs, aligning their development with IL research presents a unique opportunity to foster mutual progress in both fields and advance global efforts toward achieving carbon neutrality by 2050. Dataset link: https://github.com/sougata-ub/llms_for_ionic_liquids 1 Introduction Despite notable advancements in modeling and simulation methods van Gunsteren and Mark (1998) ; van Gunsteren et al. (2018) ; Frenkel and Smit (2023) , fundamental research in CBE continues to rely heavily on experimental results. As computational models Zhao et al. (2023) , LLMs are predominantly advantageous in computation-intensive fields, making their precise role in enabling progress within experiment-driven domains like CBE unclear. Nonetheless, recent breakthroughs in material discovery Lu et al. (2023) ; Luu and Buehler (2024) ; Lu et al. (2024) ; Buehler (2023b) and protein engineering Jumper et al. (2021) ; Liu et al. (2022) ; Yu et al. (2022b) ; Yu et al. (2022a) ; Hu and Buehler (2022) ; Khare et al. (2022) demonstrate the potential of AI technologies to contribute meaningfully to such fields. To unlock the potential applications of LLMs in CBE, it is critical to assess their knowledge and reasoning capabilities. However, this requires robust, domain-specific evaluation benchmarks, which are currently lacking in CBE. While evaluation frameworks exist in related fields, they predominantly rely on cloze-style tasks to assess LLMs’ knowledge capacity or focus on narrow, task-specific evaluations Zhao et al. (2024) ; Murakumo et al. (2023) ; Zhang et al. (2024) ; Guo et al. (2023) ; Bran et al. (2023) . Such approaches are often insufficiently general and may not adequately capture the complexities of CBE. Given that LLMs have been trained on a vast corpus of publicly available online data Villalobos et al. (2022) ; Villalobos et al. (2024) , studies Chu et al. (2025) have shown that these models can easily memorize and regurgitate information during cloze-style factual assessments. This limitation provides only a superficial understanding of LLM capabilities across domains. Furthermore, the concept of knowledge extends beyond factual recall to include its application ( p-knowledge ) Fierro et al. (2024) . Therefore, evaluating knowledge capacity alone fails to capture reasoning ability, hindering the practical deployment of LLMs, particularly in fields like CBE, where their utility remains uncertain. To address this gap, we introduce a reasoning evaluation test-bed designed to more effectively estimate LLMs’ applicability in such domains. Global warming caused by greenhouse gas emissions remains a critical challenge Wang et al. (2016) ; Sanz-Pérez et al. (2016) , necessitating accelerated research into effective carbon capture solutions Sheridan et al. (2018) . Meeting the ambitious carbon-neutral target of the 2015 Paris Agreement by 2050 Rhodes (2016) requires not only reducing carbon emissions but also investing in technologies to remove CO 2 from the atmosphere. Among potential solutions, Ionic Liquids (ILs) Zanco et al. (2021) stand out as promising candidates for CO 2 separation processes due to their non-volatile, non-toxic nature ( "green solvents" ), ease of regeneration, and high CO 2 absorption efficiency. However, experimentation with ILs and achieving industrial scalability are resource-intensive and costly, a challenge that AI technologies like LLMs could help address. In this paper, we take a foundational step toward exploring the role of LLMs in supporting carbon capture research using ILs. Specifically, we assess the potential of general-purpose LLMs in domain-specific scenarios by constructing a test bed of 5,920 expert-curated examples, spanning varying levels of difficulty, to evaluate the factual knowledge and reasoning capabilities of these models in the context of ILs. We benchmark three open-weight LLMs—Llama 3.1-8B Dubey et al. (2024) , Mistral-7B Jiang et al. (2023) , and Gemma-9B Team et al. (2024) —on this dataset. Given the absence of prior research in this area, our work represents a critical step toward identifying the potential applications of LLMs in IL research. Furthermore, leveraging LLMs for CO 2 capture research offers an opportunity to indirectly address concerns about their environmental impact Patterson et al. (2021) ; Strubell et al. (2019) ; Faiz et al. (2024) ; Li et al. (2023) ; Rillig et al. (2023) by aligning their use with climate solutions. Our contributions are as follows: • Dataset Creation: Using ILs for carbon capture as a use case, we construct and publicly share 1 1 1 Dataset available at: https://github.com/sougata-ub/llms_for_ionic_liquids a textual entailment test bed containing 5,920 expert-curated samples designed to evaluate LLM reasoning capabilities in CBE. • Benchmarking: We systematically benchmark three open-weight LLMs—Llama 3.1-8B, Mistral-7B, and Gemma-9B—on the test bed and share the resulting insights. • Analysis: We discuss the implications of our results and the broader potential for LLMs to advance IL research and CO 2 capture technologies. 2 Related Work 2.1 Ionic Liquids for Carbon Capture COP21 showed that amongst 196 participating countries, China, the United States, and India comprise the top three nations by share of worldwide CO 2 emissions. While the United States has pledged to reach “net-zero” by 2050, the deadlines set by China and India (the two most populous countries) are 2060 and 2070 respectively Rhodes (2016) ; Guiot and Cramer (2016) ; Dimitrov (2016) ; Robbins (2016) . In an attempt to offset the rising atmospheric carbon dioxide levels, carbon sequestration has emerged as an effective field of research and the timely development of materials and methods is pivotal for the efficient capture of CO 2 Wang et al. (2016) ; Sanz-Pérez et al. (2016) . Ionic Liquids have presented themselves as an excellent solution for CO 2 capture due to their environmentally friendly nature Blanchard et al. (1999) ; Blanchard et al. (2001) ; Pérez-Salado Kamps et al. (2003) ; Anthony et al. (2002) ; Zeng et al. (2017) ; Husson-Borg et al. (2003) ; Aghaie et al. (2018) ; Ramdin et al. (2012) . Thorough experimentation, with ILs, to provide a practical solution is time-conducive and entails high cost Sheridan et al. (2018) ; Maginn (2009) . In that regard, various machine learning methods have found use to alleviate dependence on experiments Cao et al. (2018) ; Baskin et al. (2022) ; Dhakal and Shah (2022)