Automatic Prompt Engineering with No Task Cues and No Tuning

Faisal Chowdhury 1 , Nandana Mihindukulasooriya 1 , Niharika S. D’Souza 1 , Horst Samulowitz 1 , Neeru Gupta 2 , Tomasz Hanusiak 2 , Michal Kapitonow 2 · 2026

Read Automatic Prompt Engineering with No Task Cues and No Tuning by Faisal Chowdhury 1 , Nandana Mihindukulasooriya 1 , Niharika S. D’Souza 1 , Horst Samulowitz 1 , Neeru Gupta 2 , Tomasz Hanusiak 2 , Michal Kapitonow 2 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-03130.txt.

Introduction

Faisal Chowdhury and six co-authors present a deliberately simple system for automatic prompt engineering, the craft of finding prompts that make language models perform well. Their system needs no tuning and no explicit clues about the task, yet matches the effectiveness of more elaborate approaches. They test it on cryptic column name expansion, the task of turning terse database column names into full descriptions.

That task matters for searching, accessing, and understanding tabular data, but it has drawn little research attention until now. The team evaluates on datasets in both English and German. They report the first application of automatic prompt engineering to column name expansion, and, to their knowledge, the first such application to a language other than English.

Catalog record

CREATORFaisal Chowdhury 1 , Nandana Mihindukulasooriya 1 , Niharika S. D’Souza 1 , Horst Samulowitz 1 , Neeru Gupta 2 , Tomasz Hanusiak 2 , Michal Kapitonow 2
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE16 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-03130

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Automatic Prompt Engineering with No Task Cues and No Tuning Faisal Chowdhury 1 , Nandana Mihindukulasooriya 1 , Niharika S. D’Souza 1 , Horst Samulowitz 1 , Neeru Gupta 2 , Tomasz Hanusiak 2 , Michal Kapitonow 2 Abstract This paper presents a system for automatic prompt engineering that is much simpler in both design and application and yet as effective as the existing approaches. It requires no tuning and no explicit clues about the task. We evaluated our approach on cryptic column name expansion (CNE) in database tables, a task which is critical for tabular data search, access, and understanding and yet there has been very little existing work. We evaluated on datasets in two languages, English and German. This is the first work to report on the application of automatic prompt engineering for the CNE task. To the best of our knowledge, this is also the first work on the application of automatic prompt engineering for a language other than English. I Introduction The advent of LLMs, especially after the impressive demonstrations of ChatGPT, led to a naive belief in many customers/companies that a new dawn of business intelligence is on the horizon which would require very limited effort to exploit the LLMs on their usecases. It turned out that, more often than not, the downstream performance of LLMs is heavily coupled with the quality of the prompt used to instruct the model. Studies have shown that large language models (LLMs) are quite sensitive to minor variations in prompt phrasing. For example, the addition, removal, or reordering of just a few tokens can lead to significant differences in task performance [ 9 ] . Generic prompts do not typically produce good responses and the most effective prompts are almost always handcrafted by humans. This makes prompt curation a labour-intensive iterative process involving a substantial amount of manual experimentation. This process of optimizing the prompt language to elicit the best possible performance is referred to as “prompt engineering”. Given the human effort involved in the practice, prompt engineering techniques are often brittle, non-transferable, and suffer from scalability issues [ 4 ] . Currently, prompt engineering is more of an art than a science, as a delicate balance is required in the design process to ensure clarity and specificity in the prompts, avoid ambiguity, and steer appropriate behavior to obtain the desired output. More importantly, the human labor involved needs to be repeated whenever the underlying LLM is changed, since previously optimized prompts may no longer yield optimal results. For example, this applies when we switch to a different model, to a different parameter-size variant of the same model, or upgrade to new versions of the LLM trained using different strategies/additional data. This is further complicated when the prompt has to take into account domain specificity or a different language (direct translations of prompts often produce poor results [ 10 ] ). Automatic prompt engineering and optimization are recent trends. The core idea is that an LLM, not a human, is tasked with generating task-specific prompts where the task is presented via output demonstrations (with some given examples). The corresponding LLM generates several instruction candidates, either via direct inference or a recursive process driven by scoring metrics. The LLM executes these instructions, and the best instruction improving the scoring metric is retained. Automatic prompt engineering is envisaged to be applied to any task that is solved by prompting LLMs. However, existing approaches often require an initial human written task specific prompt. They also need extensive number of expensive LLM calls during the optimization process. In addition, they are not easily adoptable due to implementation complexities. Also, all of them require evaluation data for their scoring metrics to work to pick the best possible prompt. In this paper, we propose a system that is based on a new simple yet effective approach for automatic prompt engineering that uses only a few examples and controlled randomized sampling to generate the best possible prompt. To be more precise, the advantages of our approach with respect to known solutions are as follows: • Simple approach that can be easily adopted (even for non-English languages) and works with only a few given examples. • No need of separate training, validation and test data. • No need of initial seed prompt or task specific cues. • No additional LLM calls for scoring or ranking. We evaluated our approach on Cryptic column name expansion (CNE) in database tables. We used three evaluation datasets. Two in English and one in German. To the best of our knowledge, this is the first work on the application of automatic prompt engineering for a language other than English. Furthermore, as part of this work, we are going to make available a dataset in German for the CNE task. We demonstrate that our system is as good as or better than the other existing system but much easier to adopt and much simpler in design. II Task: Cryptic Column Name Expansion (CNE) in Tables Tables in real world customer database often have Short or cryptic column names such as following: (English example) {"table": "Customer Account", "columns": ["DISCOUNT_PCT_APPLIC", "CURRENT_BAL_AMT"]} (German example) {"table": "vbuk", "columns": ["mandt", "vbeln", "gbstk", "vbtyp", "aedat"]} The goal of the CNE task is to expand such column names and produce an output like following: (English example) {"table": "Customer Account", "columns": {"DISCOUNT_PCT_APPLIC": "Discount Percentage Applicable", "CURRENT_BAL_AMT": "Current Balance Amount"}} (German example) {"table": "vbuk", "columns": {"mandt": "Mandant", "vbeln": "Vertriebsbelegnummer eines CAS-Kontaktes", "gbstk": "Gesamtbearbeitungsstatus des Vertriebsbeleges", "vbtyp": "Vertriebsbelegtyp", "aedat": "Datum der letzten Änderung"}} Column names are often not abbreviated in isolation but in the context of the table and other column names in the corresponding table. [ 17 ] showed that abbreviated column names makes it challenging for end users to search and retrieve relevant data for many table-related tasks. One of the widely used human-labeled text2SQL benchmark, the Spider dataset [ 18 ] , contains 6.6% of abbreviated column names. [ 17 ] observed over ten percentage points in performance degradation on the Spider dataset due to simple changes in abbreviated column names. Abbreviated column names also effect table question answering (QA) [ 19 ] . To the best of our knowledge, [ 16 ] is the only existing peer-reviewed work so far for this task that provided experimental results. As they noted, expanding column names has other beneficial aspects such as increased readability of tables (especially when complex or technical data is present), disambiguating between tables with similar column names but different meanings, improved the efficacy of keyword based searches for discovering related tables, etc. As a solution for this task, they semi-automatically generated a large training dataset in English and then fined-tuned an LLM. In contrast, our proposed system requires no training. III Related Work on Automatic Prompt Optimization and Engineering Prompt optimization aims to refine and tune of an existing or original prompt to improve performance across multiple runs or datasets. Whereas the goal of prompt engineering is to design a prompt structure from scratch, often by using techniques like few-shot prompting. [ 2 ] recently provided a through evaluation of of various automatic prompt engineering and optimization approaches for the task of triple extraction from text. The works

Read the complete text