EcoGym: Evaluating LLMs for Long-Horizon Plan-and-Execute in Interactive Economies

2026

Read EcoGym: Evaluating LLMs for Long-Horizon Plan-and-Execute in Interactive Economies (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-09514.txt.

Description

Self-run agents need long-horizon planning, yet most evaluations run short episodes in single domains without persistent economic dynamics. Xavier Hu and fifteen co-authors introduce EcoGym, a 2026 benchmark for continuous plan-and-execute decisions in interactive economies. Three worlds, vending, freelance work, and ops, share one decision process with standard plugs and budgeted actions over nearly unbounded horizons of a thousand steps or more.

Scoring follows business outcomes such as net worth, income, and daily active users, testing plan sense and toughness under limited view and chance. Trials across eleven leading language models reveal a steady tension: no model leads in all three settings, and each falters either in high-level strategy or in efficient action execution. Released as an open testbed, EcoGym aims to make long-horizon agent evaluation transparent and to probe control tradeoffs.

Catalog record

YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-09514

Browse the full library · About the Tower of Babel