MODP: Multi Objective Directional Prompting
Aashutosh Nema *, Samaksh Gulati *, Evangelos Giakoumakis *, Bipana Thapaliya * · 2025
Read MODP: Multi Objective Directional Prompting by Aashutosh Nema *, Samaksh Gulati *, Evangelos Giakoumakis *, Bipana Thapaliya * (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-18722.txt.
Introduction
Aashutosh Nema and co-authors present a framework for writing better prompts for language models in this 2025 paper. Prompt design is often based on guesswork and personal feel. Most work tunes a prompt for one task, while ignoring how the model itself behaves during that tuning.
Their framework, called MODP, adds two ideas. One is to treat the model's own behavior as a second goal beside task success. The other is a metrics-driven way to steer prompt changes toward robust, exact prompts. On a summary task with made-up data, the method gained 26 percent over the first prompts. It was also used at Dell for a support tool now used by more than 10,000 agents.
Catalog record
| CREATOR | Aashutosh Nema *, Samaksh Gulati *, Evangelos Giakoumakis *, Bipana Thapaliya * |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-18722
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MODP: Multi Objective Directional Prompting DOI: XXXXXXX.XXXXXXX Conference: ; August 03–07, 2025; Toronto, Canada ISBN: 978-1-4503-XXXX-X/2018/06 Thanks: * All authors contributed equally to this research. CCS: Computing methodologies Natural language processing CCS: Computing methodologies Artificial intelligence CCS: Information systems Information retrieval CCS: Machine learning Neural networks Aashutosh Nema *, Samaksh Gulati *, Evangelos Giakoumakis *, Bipana Thapaliya * Abstract . Recent advances in large language models (LLMs) have led to their popularity across multiple use-cases. However prompt engineering, the process for optimally utilizing such models, remains approximation-driven and subjective. Most of the current research on prompt engineering focuses on task specific optimization, while neglecting the behavior of LLM under consideration during prompt development. This paper introduces MODP – Multi Objective Directional Prompting, a framework on two key concepts: 1) multi-objectivity: the importance of considering a LLM’s intrinsic behavior as additional objective in prompt development and 2) directional prompting: a metrics-driven method for prompt engineering to ensure development of robust and high-precision prompts. We demonstrate the effectiveness of our proposed ideas on a summarization task, using a synthetically created dataset, achieving a 26% performance gain over initial prompts. Finally, we apply MODP to develop prompts for Dell’s Next Best Action support tool, which is now in production and is used by more than 10,000 internal support agents and serving millions of customers worldwide. 1. Introduction Recent advances in large language models (LLMs) have transformed the field of natural language processing (NLP). By improving tasks such as sentiment analysis and chatbot development, these models have found broad acceptance in both research and industry. However, the behavior of any pre-trained LLM is highly dependent on the data and tasks they are trained on, commonly referred to as the LLM prior. When adapting LLMs for practical applications in enterprises, the dependency on pre-trained data presents challenges. LLM fine-tuning and prompt engineering are the primary techniques used for adapting pre-trained LLMs to real-world applications and achieving optimized performance ( Brown et al. 2020 ) . Although fine-tuning is effective, it can get expensive computationally and it generally requires substantial infrastructure ( Qi et al. 2023 ) . Furthermore, gathering high-quality data and securing the necessary permissions can be particularly challenging in the fast-evolving LLM landscape, especially for teams with limited resources. This limitation underscores the importance of prompt engineering , a more accessible and efficient method for leveraging the prior knowledge embedded in LLMs. Careful prompt engineering avoids the costs and complexities of a weight-based learning approach, while extracting optimal in-context performance. However, there is no denying that prompt engineering can be very time-consuming and inefficient if not guided systematically. Recently, various methods of prompt engineering have been identified that extend the role of prompts beyond asking a simple question ( Sahoo et al. 2024 ) ( Vatsal and Dubey 2024 ) . These approaches focus on guiding the LLMs to generate contextually relevant outputs and logically solve problems. While prompt engineering methods can be effective and often serve as guiding examples, crafting prompts still remains a subjective process that relies heavily on trial and error. This subjectivity is amplified by individual users’ variations in creating and interpreting prompts. The absence of a standardized method leads to inconsistencies, particularly when attempting to scale applications across varied tasks and domains. Additionally, the choice of LLMs and the design of prompts are frequently based on anecdotal evidence or heuristic methods, which may not consistently deliver optimal outcomes. With the increasing demand for reliable and scalable LLM solutions, especially for enterprise teams, there is a need for an objective and systematic approach to prompt engineering and model selection. To address the need, this paper proposes a novel framework - MODP , aimed to enhance the process of prompt engineering while reducing individual biases and accounting for LLMs behavior. MODP treats prompt engineering as a multi-objective problem, consisting of task-specific as well as LLM-specific objectives as shown in Figure 1 . Task-specific objectives are related to performance requirements for a given task, while LLM-specific objectives arise from the model’s intrinsic behavior - influenced by its prior and tasks the model is trained on. Once these objectives are identified, the MODP framework treats prompt engineering as an optimization problem. Aiming to directionally improve subsequent prompt iterations across all parameters that influence its task performance. Furthermore, the proposed framework is versatile and can be used to compare different LLMs and prompting methods. Figure 1. Prompt Objectives Figure 2. Methodology 2. Related Work Prompt engineering emerged as a pivotal technique for extending the capabilities of LLMs without modifying core model parameters. Since the introduction of prompt-based learning in language models (Brown et al. 2020), several efforts have been suggested for engineering a prompt (Sahoo et al. 2024). Advancements in research have been majorly around either prompting approaches or automatic prompt generation techniques. A few majorly adopted prompting approaches include few-shot prompting (Brown et al. 2020), Chain of thought prompting (Wei et al. 2023), Self-consistency (Wang et al. 2023) and Tree of thoughts (Yao et al. 2023). However, the suggested approaches remain primarily heuristic and provide a thought process around ”how to design” prompts to achieve the best outcome given a task, but do not provide any improvement guidance or an empirical way to evaluate whether iterative modifications yield any tangible results. When implementing these techniques, the general question that comes to mind is - ”Is the prompt improving”, ”Is the performance better”, and ”What to change”. Additionally, there is limited understanding of how these strategies perform, especially in scenarios where one method might be more effective. 2.1. Challenges in Prompt Development 2.1.1. LLM Hallucinations LLM hallucination is one the most prominent issues in prompt engineering ( Xu et al. 2024 ) which refers to the tendency of LLMs to generate outputs that are factually or logically inconsistent with the input or the task at hand. Hallucinations are particularly problematic in prompting approaches which require decision-making, logical reasoning, or the generation of multi-step solutions. In such cases, a single hallucination could trigger a “domino effect” leading to a response that deviates from the desired outcome. Accounting for hallucinations becomes more necessary in applications where LLMs are the decision makers choosing the answer to a query from a selected context e.g., RAGs. 2.1.2. Instruction loss: The second challenge comes from the under-performance of transformer-based models in processing long texts. While capable of handling large token counts, their attention mechanism tends to prioritize the information at the beginning and end of the input sequence, leading to a loss of information retention from the middle ( Liu et al. 2024 ) . As a result, important details or contextual prompt clues embedded in the middle of a long prompt are given less weightage, leading the instruction to get "lost in the middle". This problem is more prominent when crafting prompts that require sustained attention to detail over long spans of text, such as legal documents, technical instructions, or multi-step reasoning tasks ( Turpin et al. 2023 ) . In such cases, the model may produce incomplete or erroneous responses, as key information from the prompt is effectively ignored. This issue highlights the trade-off between providing detailed, context-rich prompts and ensuring that the model retains and processes