Can Agents Price a Reaction? Evaluating LLMs on Chemical Cost Reasoning
Yuyang Wu Yue Huang Shuaike Shen Xujian Wang, Shuhao Zhang Qiyao Xue Weichen Liu Runtian Gao, Jian Ma Xiangliang Zhang Olexandr Isayev · 2026
Read Can Agents Price a Reaction? Evaluating LLMs on Chemical Cost Reasoning by Yuyang Wu Yue Huang Shuaike Shen Xujian Wang, Shuhao Zhang Qiyao Xue Weichen Liu Runtian Gao, Jian Ma Xiangliang Zhang Olexandr Isayev (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-07251.txt.
Description
Yuyang Wu and ten co-authors test whether AI agents can price out a chemical reaction from supplier catalogs. Agents must identify chemicals, fetch quotes, pick valid purchasable pack sizes, normalize amounts, and compute a total. Their ChemCost benchmark grounds more than fourteen hundred reactions in a frozen snapshot holding thousands of chemicals and vast numbers of quotes.
Scoring needs no judge model, and each stage of the task can be diagnosed separately, from parsing to arithmetic. Noise-injected versions perturb names, quantities, and formatting. Even the strongest agents reached only about half accuracy on clean inputs, within a generous error margin. They fell further with realistic noise, exposing brittle tool use.
Catalog record
| CREATOR | Yuyang Wu Yue Huang Shuaike Shen Xujian Wang, Shuhao Zhang Qiyao Xue Weichen Liu Runtian Gao, Jian Ma Xiangliang Zhang Olexandr Isayev |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 59 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.07251