Opt-Verifier: Unleashing the Power of LLMs for Optimization Modeling via Dual-Side Verification
Haoyang Liu, Jie Wang, Boxuan Niu, Xiongwei Han, Yian Xu, Mingxuan Ye, Zijie Geng, Fangzhou Zhu, Tao Zhong, Mingxuan Yuan, Jianye Hao · 2026
Read Opt-Verifier: Unleashing the Power of LLMs for Optimization Modeling via Dual-Side Verification by Haoyang Liu, Jie Wang, Boxuan Niu, Xiongwei Han, Yian Xu, Mingxuan Ye, Zijie Geng, Fangzhou Zhu, Tao Zhong, Mingxuan Yuan, Jianye Hao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-29556.txt.
Introduction
Haoyang Liu and ten colleagues present Opt-Verifier, a system that checks language-model-built optimization models from two sides. Building such models matters in operations research but demands human skill, and auto methods often skip checks on constraint rationality and answer validity, hurting fixes and scores. Structure-side checks align the model's structure with the first problem text, holding constraints and needs. Answer-side checks read and grade answers, proving the models sound in logic and math. Tests on popular benchmarks show score gains above twenty percent.
Catalog record
| CREATOR | Haoyang Liu, Jie Wang, Boxuan Niu, Xiongwei Han, Yian Xu, Mingxuan Ye, Zijie Geng, Fangzhou Zhu, Tao Zhong, Mingxuan Yuan, Jianye Hao |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 100077 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.29556
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Opt-Verifier: Unleashing the Power of LLMs for Optimization Modeling via Dual-Side Verification Haoyang Liu, Jie Wang, Boxuan Niu, Xiongwei Han, Yian Xu, Mingxuan Ye, Zijie Geng, Fangzhou Zhu, Tao Zhong, Mingxuan Yuan, Jianye Hao Abstract Building mathematical optimization models is critical in operations research (OR), while it requires substantial human expertise. Recent advancements have utilized large language models (LLMs) to automate this modeling process. However, existing works often struggle to verify the correctness of the generated optimization models, without checking the rationality of the constraints and variables or the validity of solutions to the generated models. This hampers the subsequent verification and correction steps, and thus it severely hurts the modeling accuracy. To address this challenge, we propose a novel LLM-based framework with Dual-side Verification (Opt-Verifier) from both structure and solution perspectives, thereby improving the modeling accuracy. The structure-side verification ensures that the modeling structure of the generated optimization models aligns with the original problem description, accurately capturing the problem’s constraints and requirements. Meanwhile, the solution-side verification interprets and evaluates the solutions’ validity, confirming that the optimization models are logically and mathematically sound. Experiments on popular benchmarks demonstrate that our approach achieves over 20% improvement in accuracy. † † affiliationnotice: This work was completed while Haoyang Liu <[email protected]> was an intern at Huawei Technologies. 1 Introduction Figure 1 : Opt-Verifier outperforms other baselines in solving accuracy (SA) across the benchmarks. Optimization problems are foundational to operations research (OR), with wide applications in manufacturing ( 16 ) , transportation ( 42 ) , and service industries ( 5 ) . In practice, OR problem statements are typically specified in natural language. Practitioners must therefore (i) translate these descriptions into an appropriate mathematical optimization model (defining objectives, decision variables, and constraints) and (ii) implement the solver code (e.g., SCIP ( 1 ) , Gurobi ( 11 ) , or Pyomo ( 6 ; 12 ) ) to obtain solutions. This workflow is labor-intensive, demands substantial domain expertise in problem context, mathematical modeling, and code-level implementation or debugging, and is consequently costly and time-consuming ( 2 ) . Given the impressive capabilities of large language models (LLMs) in natural-language understanding and domain knowledge acquisition, a growing number of works have employed LLMs to automate the processes of modeling, programming, and debugging. Existing approaches can be broadly grouped into two categories. The first category, prompt-based methods, relies on pre-trained LLMs (e.g., GPT-4 ( 29 ) and GPT-4o ( 30 ) ), which are prompted to construct mathematical models incrementally. In practice, these methods are often implemented into a carefully designed framework, such as multi-agent cooperation ( 40 ; 2 ) and Monte Carlo tree search ( 3 ) . The second category enhances modeling capabilities through fine-tuning, which involves constructing a large, labeled dataset for LLM training ( 14 ; 38 ; 17 ; 8 ; 25 ) . Beyond these two approaches, recent research has explored self-correction strategies to improve the modeling performance ( 17 ; 40 ; 2 ) . These methods trigger correction primarily from error messages produced during code execution. However, such strategies confine self-correction to code-level issues, and the underlying model can remain flawed even when the code runs without failure. To develop more effective model verification methods, we characterizes incorrect models by the following features. First, incorrect models often overlook indispensable constraints that the textual problem description does not explicitly state. For example, when formulating a maximum flow problem, an LLM might omit flow-balance constraints at intermediate nodes simply because they are not explicitly mentioned, yielding a structurally incomplete model. Second, incorrect models can yield solutions that are infeasible or violate basic logical principles, even though the solver executes successfully and reports an improved objective. To address these challenges, we propose a novel multi-agent framework with Dual-side Verification (Opt-Verifier) from both the modeling structure and solution perspectives, improving the modeling accuracy. This approach moves beyond simple code-execution signals by translating both the optimization model and its resulting solutions into natural language and evaluating their semantic correctness. To the end, Opt-Verifier introduces two novel evaluation metrics for self-correction: modeling structure consistency and solution validity. (1) Consistency in the modeling structure ensures that the mathematical formulation (its variables, constraints, and objective) is a complete and faithful translation of the original problem description. To evaluate this metric, one LLM agent performs a back-translation that abstracts the generated model into a compact, multi-level description of its components. A second agent then aligns this abstraction with the structure derived from the original specification to reveal omissions or mismatches. (2) Solution validity assesses whether Othe solution of the produced model is logically and contextually sound for the real-world task. To assess it, one agent interprets the numeric solution in natural language, explaining its meaning in the context of the real-world problem. Another agent then critiques that interpretation to expose logical absurdities or mathematical violations that code-execution checks miss. Finally, we use the verification feedback for model refinement. As illustrated in Figure 1 , extensive experiments on five popular benchmarks showcase that our approach significantly outperforms the state-of-the-art, achieving an average improvement of approximately 10% in solving accuracy. Notably, Opt-Verifier is designed as a plug-and-play framework, capable of effectively verifying and refining optimization models generated by any existing pre-trained or fine-tuned OR LLMs. 2 Related Work Automated Optimization modeling Machine Learning for solving Operations Research (OR) has drawn much attention in recent years ( 4 ; 32 ; 23 ) . In practice, the OR problems often arise from real-world situations, which are typically described in natural language. Consequently, automated optimization modeling has emerged as a critical area aimed at reducing the labor and time costs associated with the modeling process ( 7 ; 19 ) . Notable early efforts in this field include the NL4Opt competition ( 34 ) . Since then, several benchmarks have been introduced to evaluate performance, such as ComplexOR ( 40 ) , NLP4LP ( 2 ) , Mamo ( 15 ) , IndustryOR ( 14 ) and Optibench ( 41 ; 36 ) . Recent research primarily falls into two categories: prompt-based methods and fine-tuned methods. Prompt-based approaches utilize pre-trained large language models (LLMs) with carefully crafted prompts to iteratively construct models. The early works utilize the multi-agent techniques ( 28 ; 45 ; 43 ; 27 ) . For instance, Chain-of-Experts ( 40 ) and OptiMUS ( 2 ) frameworks employ multi-agent cooperation, while some other methods ( 3 ) leverage Monte Carlo tree search techniques to explore potential models. OptiTree ( 22 ) uses tree search to retrieve the LLM-collected modeling knowledge for improved performance, and Opt-Miner ( 21 ) leverages web search to bridge the knowledge gap in semantic understanding and modeling techniques. To further enhance the modeling capabilities of LLMs, researchers also work on fine-tuning these models with extensive OR and modeling knowledge ( 14 ; 38 ; 17 ; 8 ; 24 ) . For example, LLaMoCo ( 26 ) utilizes an instruction tuning framework to adapt LLMs for solving optimization problems in a code-to-code manner. ORLM ( 14 ) trains open-source LLMs specifically designed for optimization modeling and solver code development. Additionally, advanced techniques such as DPO ( 33 ; 10 ; 20