How Well Does Agent Development Reflect Real-World Work?
Zora Z. Wang Sanidhya Vijayvargiya Aspen Chen Hanmo Zhang Venu Arvind Arangarajan Jett Chen Valerie Chen Diyi Yang Daniel Fried Graham Neubig Carnegie Mellon University {}^{\raisebox{-1.42262pt}{\includegraphics[width]{./logo/s.png}}}\penalty\ Stanford University [email protected] https://zorazrw.github.io/ai4work/ · 2026
Read How Well Does Agent Development Reflect Real-World Work? by Zora Z. Wang Sanidhya Vijayvargiya Aspen Chen Hanmo Zhang Venu Arvind Arangarajan Jett Chen Valerie Chen Diyi Yang Daniel Fried Graham Neubig Carnegie Mellon University {}^{\raisebox{-1.42262pt}{\includegraphics[width]{./logo/s.png}}}\penalty\ Stanford University [email protected] https://zorazrw.github.io/ai4work/ (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-01203.txt.
Description
Wang and nine colleagues measure how well AI agent benchmarks reflect real human work by mapping benchmark tasks to jobs and skills. Across 43 benchmarks and over 72,000 tasks against all 1,016 United States jobs, they find agent work skews code-heavy. Human labor and economic value sit in other fields. Within aimed areas they grade how freely agents act for hands-on guidance. They propose breadth, realism, and fine scoring as rules for benchmarks that capture key social work and hard tech work.
Catalog record
| CREATOR | Zora Z. Wang Sanidhya Vijayvargiya Aspen Chen Hanmo Zhang Venu Arvind Arangarajan Jett Chen Valerie Chen Diyi Yang Daniel Fried Graham Neubig Carnegie Mellon University {}^{\raisebox{-1.42262pt}{\includegraphics[width]{./logo/s.png}}}\penalty\ Stanford University [email protected] https://zorazrw.github.io/ai4work/ |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52155 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.01203