OPT-BENCH: Evaluating the Iterative Self-Optimization of LLM Agents in Large-Scale Search Spaces
Xiaozhe Li, Jixuan Chen, Xinyu Fang, Shengyuan Ding, Haodong Duan, Qingwen Liu, Kai Chen, Tongji University, Shanghai AI Lab, Fudan University, Zhejiang University, University of California San Diego · 2026
Read OPT-BENCH: Evaluating the Iterative Self-Optimization of LLM Agents in Large-Scale Search Spaces by Xiaozhe Li, Jixuan Chen, Xinyu Fang, Shengyuan Ding, Haodong Duan, Qingwen Liu, Kai Chen, Tongji University, Shanghai AI Lab, Fudan University, Zhejiang University, University of California San Diego (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-08904.txt.
Introduction
Xiaozhe Li and six coauthors test whether language model agents can keep improving their own solutions inside huge search spaces, the way human experts refine through feedback. Their OPT-Bench combines twenty machine learning tasks with ten classic hard combinatorial problems, demanding adaptation through reflection rather than memorized patterns. Their OPT-Agent answers with a loop of perception, memory, and reasoning that revises solutions as environment feedback arrives.
Across nineteen models from seven families, stronger models exploit feedback better, yet adaptability stays capped by base capacity and even the best models trail human experts. Self improvement research gains a rigorous arena separating genuine refinement skill from rote tool use.
Catalog record
| CREATOR | Xiaozhe Li, Jixuan Chen, Xinyu Fang, Shengyuan Ding, Haodong Duan, Qingwen Liu, Kai Chen, Tongji University, Shanghai AI Lab, Fudan University, Zhejiang University, University of California San Diego |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.08904
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
OPT-BENCH: Evaluating the Iterative Self-Optimization of LLM Agents in Large-Scale Search Spaces Xiaozhe Li, Jixuan Chen, Xinyu Fang, Shengyuan Ding, Haodong Duan, Qingwen Liu, Kai Chen, Tongji University, Shanghai AI Lab, Fudan University, Zhejiang University, University of California San Diego Abstract Large Language Models (LLMs) have demonstrated remarkable capabilities in reasoning and tool use. However, the fundamental cognitive faculties essential for problem-solving—perception, reasoning, and memory—remain the stable core of intelligence. Unlike memorizing specific patterns, humans succeed in novel environments by applying these intrinsic faculties to adapt and optimize. Yet, whether LLMs possess this essential capacity—namely, the ability to continuously refine solutions in response to dynamic environmental feedback—remains underexplored. To address this challenge, we introduce OPT-BENCH , a benchmark for evaluating self-improvement capabilities in large-scale search spaces. By combining 20 machine learning tasks with 10 classic NP-hard problems, OPT-BENCH provides a rigorous setting to assess whether agents can adapt through intrinsic self-reflection rather than rote tool application. We further propose OPT-Agent , a framework that emulates human-like cognitive adaptation. It operates via a general perception–memory–reasoning loop, iteratively refining solutions based on environmental feedback. Through extensive experiments on 19 LLMs from 7 model families, including reasoning models, general models, and open-source models ranging from 3B to 235B parameters, we demonstrate stronger models are more effective at leveraging feedback signals for self-improvement. However, this upper-bound adaptability remains fundamentally constrained by the models’ base capacity, and even the most advanced LLMs still fall short of human expert performance. Figure 1: OPT-BENCH framework. This framework evaluates Iterative Self-Optimization by integrating two distinct reasoning modalities: Continuous Parametric Optimization for machine learning tasks (Top), and Discrete Combinatorial Reasoning for NP-hard problems (Bottom). In both settings, the agent leverages Environmental Feedback to guide its trajectory of improvement and debugging, progressively bridging the gap between initial hypotheses and optimal solutions through intrinsic reasoning. 1 1 footnotetext: † represents the corresponding author, email: [email protected]. 1 Introduction The advent of Large Language Models (LLMs) 20 ; 9 ; 10 has revolutionized artificial intelligence, demonstrating exceptional performance across a wide range of tasks 3 ; 21 ; 1 ; 5 ; 26 ; 8 . While specific interaction tools evolve rapidly, the fundamental cognitive faculties required for problem-solving— perception , reasoning , and memory —remain the stable core of intelligence. Humans succeed in novel environments not by memorizing specific interfaces, but by applying these intrinsic principles to adapt and optimize. However, determining whether LLMs possess this essential capacity for iterative self-optimization —continuously refining solutions in response to dynamic environmental feedback —remains an underexplored frontier. Current benchmarks primarily measure a model’s ability to generate correct responses in a single pass 7 ; 16 , overlooking the essential human capability of learning from experience. Humans routinely refine reasoning by integrating feedback—scientists adjust hypotheses, students revise strategies, and chess players improve tactics based on past outcomes. Yet, existing evaluations lack a unified framework to assess this trajectory of self-improvement, leaving a critical gap in understanding how agents navigate complex search spaces through intrinsic reflection rather than rote memorization. To address this limitation, we introduce OPT-BENCH , a comprehensive benchmark designed to probe the limits of LLM agents in large-scale search space self-optimization driven by environmental signals. OPT-BENCH juxtaposes two distinct feedback landscapes: (1) 20 Real-world Machine Learning (ML) tasks (sourced from Kaggle), representing continuous optimization spaces where feedback provides noisy but directional gradients (e.g., improving validation accuracy); and (2) 10 Classical NP-hard problems , representing discrete combinatorial spaces characterized by brittle constraints and local optima. This deliberate combination challenges agents to demonstrate versatile cognitive adaptability: employing inductive reasoning to tune hyperparameters in ML tasks, while switching to deductive logic to construct valid structures in NP problems. To evaluate performance relative to human cognition, we curate gold-standard solutions from Kaggle leaderboards for ML tasks and implement Human-expert Heuristics for NP tasks, establishing a baseline rooted in expert reasoning patterns rather than brute-force computation. To facilitate rigorous evaluation, we present OPT-Agent , a framework designed to emulate human-like cognitive adaptation . Unlike complex agent architectures that rely heavily on engineered prompts or external solvers, OPT-Agent employs a perception–memory–reasoning loop . It generates solutions, validates them against the environment, and iteratively refines them by retrieving and analyzing accumulated environmental signals . Using this benchmark, we conduct extensive experiments on 19 LLMs from 7 model families. Our analysis uncovers a "Scaling Law of Self-Improvement": stronger base models are significantly more effective at leveraging historical feedback to optimize solutions. However, we reveal a critical cognitive divergence : while historical context effectively drives optimization in continuous ML domains, its benefit is often constrained in discrete NP tasks. Even frontier models (e.g., GPT-4o) struggle with incremental refinement in combinatorial spaces, frequently resetting solutions rather than repairing them—highlighting a persistent gap compared to human expert performance. In summary, our contributions are: • We propose OPT-BENCH , a benchmark that juxtaposes continuous (ML) and discrete (NP) optimization tasks to evaluate the intrinsic self-optimization capabilities of LLMs, moving beyond rote tool application to assess dynamic adaptability. • We introduce OPT-Agent , an evaluation framework that emulates the human perception-memory-reasoning loop, enabling a rigorous assessment of how models leverage historical environmental feedback to refine solutions iteratively. • We provide a comprehensive analysis of 19 LLMs, revealing that while strong models excel at inductive optimization in continuous spaces, they face fundamental reasoning barriers in discrete combinatorial refinement, marking a key direction for future AGI research. Figure 2: Overview of the OPT-BENCH dataset and OPT-Agent Framework. The left panel illustrates the data structure of OPT-Bench, encompassing ML and NP problems. Each module includes problem definitions, dataset files, validation script (NP), evaluation metrics, and submission formats, integrating human-verified initial solutions and LLM-assisted refinement. The right panel details the evaluation workflow, where solutions are iteratively generated and validated in steps. Valid solutions proceed to metric calculation, while buggy solutions trigger error analysis via LLM or rule-based diagnostics, followed by iterative optimization. This framework enables systematic and automated assessment of LLMs’ optimization capabilities across diverse problem domains. 2 OPT-BENCH To rigorously evaluate adaptive intelligence across diverse feedback landscapes, OPT-BENCH integrates 30 distinct tasks: 20 machine learning (ML) challenges and 10 classical NP-hard problems. This composition is intentionally designed to juxtaposition two fundamental modes of reasoning: (1) Continuous Inductive Optimization (ML tasks): utilizing noisy, directional feedback (e.g., accuracy metrics) to tune continuous hyperparameters in applications like sales forecasting and sentiment analysis. (2) Discrete Deductive Reasoning (NP tasks): navigating brittle, combinatorial search spaces with strict constraints, covering graph theory and resource allocation problems such as Hamiltonian cycle and TSP. These problems are selected for their computational intractability at scale, forcing agents to rely on heuristic planning rather than brute-force memorization. See Appendix A for the complete task list. 2.1 Dataset Curation and Analysis As illustrated in Figure 2 , the preparation of ML tasks ensures a rigorous testbed for real-world engineering capabilities. We collect task descriptions from Kaggle, refine them via GPT-4o, and verify them through human experts to ensure ambiguity-free instructions. Evaluation metrics are strictly defined to provide objective feedback signals. To simulate a realistic optimization starting point, we generate an initial solution using the AIDE agent 14 , followed by refinement from four PhD-level experts. This ensures the initial state is