How to Auto-optimize Prompts for Domain Tasks? Adaptive Prompting and Reasoning through Evolutionary Domain Knowledge Adaptation

Yang Zhao, Pu Wang, Hao Frank Yang · 2025

Read How to Auto-optimize Prompts for Domain Tasks? Adaptive Prompting and Reasoning through Evolutionary Domain Knowledge Adaptation by Yang Zhao, Pu Wang, Hao Frank Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-21148.txt.

Introduction

Yang Zhao leads two co-authors on EGO-Prompt, an evolving framework that auto tunes prompts and reasoning steps for domain tasks with language models. Hand crafting prompts for niche fields is slow, and weaving in domain knowledge well is an open craft.

Starting from a general prompt with fault tolerant initialization, evolving graph optimization refines wording, reasoning steps, and causal knowledge integration together. The system can even return refined knowledge hints to domain experts. Automated prompt design at this level could standardize how professional applications, from medicine to engineering, get reliable model behavior.

Catalog record

CREATORYang Zhao, Pu Wang, Hao Frank Yang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE72 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-21148

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

How to Auto-optimize Prompts for Domain Tasks? Adaptive Prompting and Reasoning through Evolutionary Domain Knowledge Adaptation Yang Zhao, Pu Wang, Hao Frank Yang Abstract Designing optimal prompts and reasoning processes for large language models (LLMs) on domain-specific tasks is both necessary and challenging in real-world applications. Determining how to integrate domain knowledge, enhance reasoning efficiency, and even provide domain experts with refined knowledge integration hints are particularly crucial yet unresolved tasks. In this research, we propose Evolutionary Graph Optimization for Prompting (EGO-Prompt), an automated framework to designing better prompts, efficient reasoning processes and providing enhanced causal-informed process. EGO-Prompt begins with a general prompt and fault-tolerant initial Semantic Causal Graph (SCG) descriptions, constructed by human experts, which is then automatically refined and optimized to guide LLM reasoning. Recognizing that expert-defined SCGs may be partial or imperfect and that their optimal integration varies across LLMs, EGO-Prompt integrates a novel causal-guided textual gradient process in two steps: first, generating nearly deterministic reasoning guidance from the SCG for each instance, and second, adapting the LLM to effectively utilize the guidance alongside the original input. The iterative optimization algorithm further refines both the SCG and the reasoning mechanism using textual gradients with ground-truth. We tested the framework on real-world public health, transportation and human behavior tasks. EGO-Prompt achieves 7.32%–12.61% higher F1 than cutting-edge methods, and allows small models to reach the performence of larger models at under 20% of the original cost. It also outputs a refined, domain-specific SCG that improves interpretability. 1 Introduction Foundation models, particularly Large Language Models (LLMs), are increasingly being adapted for domain-specific tasks, providing reasoning and decision support across real-world applications, such as public health [ 1 , 2 , 3 ] , transportation [ 4 , 5 , 6 , 7 ] , medical treatment [ 8 , 9 ] , and robotics [ 10 , 11 , 12 ] . For these models to be used effectively in particular domains, additional task adaptations are usually necessary. Prompt engineering has emerged as the primary, flexible, and cost-effective method for this adaptation [ 4 , 13 , 14 , 15 , 16 ] . In this process, domain experts generally incorporate specialized knowledge and priors into prompt design by structuring the prompts and excluding irrelevant information to enhance the reasoning process. However, experts can also inadvertently introduce assumptions, mechanisms, even biases in prompt engineering [ 17 , 18 ] . Therefore, critical questions arise – how can we optimize prompts and reasoning procedures, and discover better combinations that integrate structured domain knowledge for domain-specific tasks? Furthermore, how can we automate this prompt-and-reasoning optimization? Seeking the potentially better prompt and reasoning procedure for domain-specific tasks without fine-tuning involves three challenges: 1) Domain-knowledge Adaptation involves organizing extensive domain knowledge in textual form to maximize task performance while minimizing bias [ 19 , 20 ] . For example, in traffic crash modeling [ 4 , 21 ] , LLMs must consider a broad range of textual inputs, including driver attributes, vehicle characteristics, infrastructure conditions, environmental factors, and driving behaviors, etc. Although these elements are represented in text, factors such as word choice, linguistic description, level of detail, and paragraph organization can influence how effectively the model learns and reasons about the domain. 2) Optimal Domain-adaptive Reasoning recognizes that even with high-quality textual descriptions, physical priors still play a crucial role. Domain-specific conditional distributions and causal graphs strongly influence the reasoning process and performance [ 22 ] . Leveraging these priors can further guide the reasoning procedure, resulting in a more efficient overall process [ 23 ] . Without explicit guidance, LLMs may overlook key domain relationships [ 19 ] , leading to hallucinations or outputs lacking verifiable evidence. To bridge the gap between a model’s general reasoning abilities and the domain-specific priors needed for robust inference, it is crucial to incorporate causal structures and other form of domain representations. 3) Task Evolutionary then considers whether LLMs can continue refining their reasoning once domain knowledge and causal relations are effectively integrated. For instance, can active knowledge or ground truth data help a domain-adaptive prompt match or surpass state-of-the-art legacy models? If so, the resulting improved domain knowledge and causal relations can further support evolving domain knowledge and uncover hidden factors, thereby reinvigorating expert-driven research. In response, a promising approach is to leverage graph-based structural domain knowledge to guide both prompt design and the reasoning process [ 24 ] . Graph data offers explicit representations of entities and their conditional relationships, capturing key domain features correlations, whether factual (e.g., alcohol involvement increases crash severity regardless of consumption level) or causal (e.g., how alcohol consumption, road-surface conditions, and other factors jointly affect crash severity). By integrating these explicit relational structures, LLMs can better connect textual descriptions to domain-specific causal dependencies, reducing the likelihood of overlooked relationships or unsupported conclusions. Recent work has explored knowledge and causal graphs to enhance LLM reasoning [ 24 , 25 , 26 ] , often via Retrieval-Augmented Generation (RAG) [ 27 , 28 , 29 ] , which retrieves relevant subgraphs or facts to guide the model during inference [ 24 , 25 ] . While this line of research is promising, there remain critical challenges for adaptive reasoning: 1. Limited and Incomplete Domain Knowledge and Graphs. In real-world settings, domain experts often have access only to partial knowledge or graphs for prompt design and reasoning [ 30 ] . However, many methods assume the availability of fully curated knowledge or causal graphs [ 26 , 31 ] , which is an unrealistic and task-inflexible requirement (e.g., in RAG scenarios where no relevant knowledge graphs exist). 2. Interpretability and Knowledge Refinement. Information in external graph databases may be inaccurate, yet existing methods often treat it as a ground-truth reference, potentially leading to performance degradation [ 18 , 32 , 33 ] . 3. Fixed External Priors. Many current methods depend on fixed, external graph databases that can lack coverage or fail to actively evolve with the domain, limiting their adaptability to emerging contexts or newly available information [ 24 , 25 , 26 ] . 4. Automated Evolution. In existing methods, the interaction is typically one-way: external knowledge is used to strengthen LLM reasoning, but the model does not feed back corrections or enhancements to the experts. As a result, experts must invest additional effort to extract new insights from the model’s outputs [ 24 , 25 , 26 ] . Given these constraints, effectively adapting LLMs to novel tasks often requires an active process of graph integration and evolution for improved prompt design and reasoning. In this research, we propose EGO-Prompt (Evolutionary Graph Optimization for Prompt) for evolutionarily incorporating domain structural knowledge into more effective reasoning in LLMs. As shown in Figure 1 , we begin by representing expert knowledge as an active, expert-constructed Semantic Causal Graph (SCG) \mathcal{G} . EGO-Prompt then decomposes the graph-guided reasoning process into two stages: 1) Generating instance-specific reasoning guidance z^{\!*}(x,\mathcal{G}) derived from the SCG \mathcal{G} and the input data x , subject to output constraints, and 2) Performing reasoning conditioned on this guidance. This process can be abstracted as the following equation: \displaystyle p(y\mid x,\mathcal{G})\;=\;p\bigl(y\mid x,\overbrace{z^{\!*}(x,\mathcal{G})}^{\mathclap{\text{Instance-specific Reasoning Guidance}}}) (1) where y is the target

Read the complete text