A Survey of Optimization Modeling Meets LLMs: Progress and Future Directions

Ziyang Xiao, Jingrong Xie, Lilin Xu, Shisi Guan, Jingyan Zhu, Xiongwei Han, Xiaojin Fu, WingYin Yu, Han Wu, Wei Shi, Qingcan Kang, Jiahui Duan, Tao Zhong, Mingxuan Yuan, Jia Zeng, Yuan Wang, Gang Chen, Dongxiang Zhang · 2025

Read A Survey of Optimization Modeling Meets LLMs: Progress and Future Directions by Ziyang Xiao, Jingrong Xie, Lilin Xu, Shisi Guan, Jingyan Zhu, Xiongwei Han, Xiaojin Fu, WingYin Yu, Han Wu, Wei Shi, Qingcan Kang, Jiahui Duan, Tao Zhong, Mingxuan Yuan, Jia Zeng, Yuan Wang, Gang Chen, Dongxiang Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-10047.txt.

Introduction

Ziyang Xiao and co-authors survey in 2025 the fast-growing craft of letting language models build optimization models. Turning a word problem into math that a solver can chew takes scarce expert time. The survey covers the whole stack: data synthesis, fine-tuning, inference frameworks, benchmarks, and scoring.

One finding stings: the benchmark data sets carry a surprisingly high error rate. The team cleaned them, rebuilt a fair leaderboard, and opened a portal with the data, code, and papers. They close by marking where current methods fall short and where the field should push next.

Catalog record

CREATORZiyang Xiao, Jingrong Xie, Lilin Xu, Shisi Guan, Jingyan Zhu, Xiongwei Han, Xiaojin Fu, WingYin Yu, Han Wu, Wei Shi, Qingcan Kang, Jiahui Duan, Tao Zhong, Mingxuan Yuan, Jia Zeng, Yuan Wang, Gang Chen, Dongxiang Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-10047

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Survey of Optimization Modeling Meets LLMs: Progress and Future Directions Ziyang Xiao, Jingrong Xie, Lilin Xu, Shisi Guan, Jingyan Zhu, Xiongwei Han, Xiaojin Fu, WingYin Yu, Han Wu, Wei Shi, Qingcan Kang, Jiahui Duan, Tao Zhong, Mingxuan Yuan, Jia Zeng, Yuan Wang, Gang Chen, Dongxiang Zhang Abstract By virtue of its great utility in solving real-world problems, optimization modeling has been widely employed for optimal decision-making across various sectors, but it requires substantial expertise from operations research professionals. With the advent of large language models (LLMs), new opportunities have emerged to automate the procedure of mathematical modeling. This survey presents a comprehensive and timely review of recent advancements that cover the entire technical stack, including data synthesis and fine-tuning for the base model, inference frameworks, benchmark datasets, and performance evaluation. In addition, we conducted an in-depth analysis on the quality of benchmark datasets, which was found to have a surprisingly high error rate. We cleaned the datasets and constructed a new leaderboard with fair performance evaluation in terms of base LLM model and datasets. We also build an online portal that integrates resources of cleaned datasets, code and paper repository to benefit the community. Finally, we identify limitations in current methodologies and outline future research opportunities. 1 Introduction Optimization modeling aims to mathematically model complex decision-making problems that arise from a wide range of industry sectors, including supply chain Cuthbertson 1998 , healthcare Delgado et al. 2022 , air traffic flow management de Matos and Ormerod 2000 ; Zhang et al. 2023 and geometry problems Xiao and Zhang 2023 . Despite its potential to enhance operational efficiency, there exists an expertise barrier that limits the broader adoption of modeling tools. According to a survey of Gurobi users, 81\% of them hold advanced degrees, with nearly half specializing in operations research Gurobi Optimization 2023 . Figure 1: An example of an optimization modeling task. The orange text in the problem description implies domain-specific terminology, and the green text denotes implicit constraints. To automate the procedure and reduce the dependence on domain-specific modeling experts, NL4Opt (Natural Language for Optimization) Ramamonjison et al. 2023 has emerged as an attractive but challenging NLP task. Its objective is to translate the text description of an OR problem into math formulations for optimization solvers. Figure 1 illustrates an instance of NL4Opt task. It transforms an input problem text into a formal mathematical model, including variables, constraints, and objective function. The problem is challenging because the optimization problems often require a large amount of domain-specific knowledge to understand terminologies, such as “megawatt hour” and “startup cost”, highlighted in orange text. Moreover, these descriptions may contain numerous implicit constraints that need to be inferred by human experts. Automatic optimization modeling can enhance time and cost efficiency while enabling access for users without deep optimization expertise. Recently, large language models (LLMs) offer a promising way to make optimization more accessible. They can understand the complicated text descriptions — identity the optimization objective and extract the decision variables and constraints. Consequently, they automatically build the mathematical model and generate the code. Numerous works have been proposed in this rapidly expanding field: • Domain-specific LLM . Representative works such as ORLM Tang et al. 2024 and LLMOPT Jiang et al. 2024 take advantage of data synthesis and instruction tuning to enhance the capability of base model for optimization modeling. • Advanced Inference Framework : Various reasoning frameworks have emerged, include multi-agent systems (e.g. Chain-of-Experts Xiao et al. 2024b and OptiMUS AhmadiTeshnizi et al. 2024 ) and chain-of-thought variants (e.g. Tree of Thoughts Yao et al. 2023 , Autoformulation Astorga et al. 2024 ). • Benchmark Datasets and Evaluation. There have been multiple benchmark datasets released, such as IndustryOR Tang et al. 2024 , NL4Opt Ramamonjison et al. 2023 and MAMO Huang et al. 2024 . However, these datasets vary significantly in quality, and evaluation methods lack standardization across different studies. Thus, it is of high necessity to present a just-in-time survey to summarize the progress and indicate possible future research directions. In this paper, we propose the first systematic review of optimization modeling in the era of LLMs. As shown in Figure 3 , we present a detailed taxonomy of the various methodologies employed to harness the power of LLMs for optimization modeling. Besides, we noticed that existing benchmark datasets are associated with high error rates and performed data cleaning to enhance quality. We constructed a new leader-board with fair comparison in terms of base model and benchmark datasets, and deliver new insights of performance evaluation. To benefit the community, these datasets and implementation code are accessible from our online portal 1 1 1 https://llm4or.github.io/LLM4OR . 2 Background 2.1 Problem Definition Optimization modeling transforms a problem description in natural language \mathcal{P} into a model \mathcal{M} . Mathematically, an optimization model is defined by an objective and a set of constraints, as shown in Equation 1 . \displaystyle\begin{split}\mathop{\text{minimize}}_{\mathbf{x}}&\quad f(\mathbf{x})\\ \text{subject to}&\quad g_{i}(\mathbf{x})\leq 0,\quad i=1,...,m\\ &\quad h_{j}(\mathbf{x})=0,\quad j=1,...,p\end{split} (1) Here, \mathbf{x} is the vector of decision variables, f(\mathbf{x}) denotes the objective function, g_{i}(\mathbf{x}) and h_{j}(\mathbf{x}) represent the inequality and equality constraints respectively. Figure 2: Comparison between concrete and abstract models. The right part illustrates a linear programming formulation example. 2.2 Abstract Model and Concrete Model In practice, optimization models can be categorized into two types: abstract models and concrete models . A model whose parameters are denoted by mathematical symbols called a abstract model, while a model whose parameters are specified by numerical values is called a concrete model. Correspondingly, optimization modeling can be divided into two types: concrete modeling and abstract modeling , as illustrated in Figure 2 . Concrete modeling directly translates a problem description containing numerical parameters into a concrete model. In contrast, abstract modeling follows a model-data separation approach where the problem description only contains the model structure, with parameters are provided separately at a later stage. Figure 3: Left: Taxonomy of LLMs-based optimization modeling, organized according to the LLMs’ technical stack. Right: Representative works for each category are presented in chronological order. The dashed arrows indicates where later works build upon techniques proposed in earlier studies. 3 Technical Stack of Optimization Modeling This section presents a typical technical stack for applying LLMs to optimization modeling. The pipeline consists of four key steps: (1) data preparation and LLM fine-tuning; (2) inference; (3) benchmarking; and (4) evaluation. Figure 3 shows the representative works in each step of this pipeline. 3.1 Data Synthesis and Fine-tuning 3.1.1 Data Synthesis Methods It is a common practice to fine-tune language models for specialized domains such as optimization modeling. However, fine-tuning requires a substantial amount of high-quality training data. In the field of optimization modeling, data availability is limited due to the scarcity of problem sources and the high cost of problem annotation. To address this challenge, current approaches employ data synthesis to generate training datasets. Formally, the data synthesis process can be defined as seed\rightarrow\{\mathcal{P}^{\prime},\mathcal{M}^{\prime}\} , where \mathcal{P}^{\prime} represents the generated problem description, \mathcal{M}^{\prime} denotes the corresponding modeling and seed is the seed data of generation process. Depending on the primary focus of the generation process, existing works can be divided into two approaches: problem-centric and model-centric. Problem-centric The problem-centric approach involves two

Read the complete text