Tuning LLM-based Code Optimization via Meta-Prompting: An Industrial Perspective

Jingzhi Gong 5 2 , Rafail Giavrimis 3 2 , Paul Brookes 2 , Vardan Voskanyan 2 , Fan Wu 2 , Mari Ashiga 4 2 , Matthew Truscott 2 , Mike Basios 2 , Leslie Kanthan 2 , Jie Xu 5 , and Zheng Wang 5 ∗ · 2025

Read Tuning LLM-based Code Optimization via Meta-Prompting: An Industrial Perspective by Jingzhi Gong 5 2 , Rafail Giavrimis 3 2 , Paul Brookes 2 , Vardan Voskanyan 2 , Fan Wu 2 , Mari Ashiga 4 2 , Matthew Truscott 2 , Mike Basios 2 , Leslie Kanthan 2 , Jie Xu 5 , and Zheng Wang 5 ∗ (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-01443.txt.

Introduction

Jingzhi Gong and co-authors present a framework for tuning code improvement prompts across models in this 2025 paper. Companies that run several large language models face a snag: a prompt tuned for one model often fails on another. Retuning by hand for each model is costly and slows real use.

Their framework writes task-specific prompts on its own for each model. It joins project facts, task needs, and model-specific context into the prompt. Tests ran on five real codebases with 366 hours of benchmark time. Gains reached about 19 percent over baseline methods, with the best overall rank. Almost all top gains came from edits that truly changed the code, the authors report.

Catalog record

CREATORJingzhi Gong 5 2 , Rafail Giavrimis 3 2 , Paul Brookes 2 , Vardan Voskanyan 2 , Fan Wu 2 , Mari Ashiga 4 2 , Matthew Truscott 2 , Mike Basios 2 , Leslie Kanthan 2 , Jie Xu 5 , and Zheng Wang 5 ∗
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-01443

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Tuning LLM-based Code Optimization via Meta-Prompting: An Industrial Perspective ∗ Corresponding author: Zheng Wang ([email protected]). Jingzhi Gong 5 2 , Rafail Giavrimis 3 2 , Paul Brookes 2 , Vardan Voskanyan 2 , Fan Wu 2 , Mari Ashiga 4 2 , Matthew Truscott 2 , Mike Basios 2 , Leslie Kanthan 2 , Jie Xu 5 , and Zheng Wang 5 ∗ Abstract There is a growing interest in leveraging large language models (LLMs) for automated code optimization. However, industrial platforms deploying multiple LLMs face a critical challenge: prompts optimized for one LLM often fail with others, requiring expensive model-specific prompt engineering. This cross-model prompt engineering bottleneck severely limits the practical deployment of multi-LLM optimization systems in production environments. To address this, we introduce Meta-Prompted Code Optimization ( MPCO ), a framework that automatically generates high-quality, task-specific prompts across diverse LLMs while maintaining industrial efficiency requirements. MPCO leverages meta-prompting to dynamically synthesize context-aware optimization prompts by integrating project metadata, task requirements, and LLM-specific contexts, and it seamlessly deploys on the Artemis industrial platform for automated validation and scaling. Our comprehensive evaluation on five real-world codebases with 366 hours of runtime benchmarking demonstrates MPCO ’s effectiveness: it achieves overall performance improvements up to 19.06% with the best statistical rank across all systems compared to baseline methods. Analysis shows that 96% of the top-performing optimizations stem from meaningful edits. Through systematic ablation studies and meta-prompter sensitivity analysis, we identify that comprehensive context integration is essential for effective meta-prompting, and that all three major LLMs can serve effectively as meta-prompters, providing actionable insights for industrial practitioners. I Introduction Large language models (LLMs) have demonstrated remarkable capabilities in many aspects across the software development lifecycle, including code completion [ 1 ] , natural‑language documentation [ 2 ] , test-case generation [ 3 ] , automated debugging [ 4 ] , and more [ 5 ] . Beyond these capabilities, recent research studies have demonstrated that LLMs can systematically analyze code patterns to uncover optimization opportunities and produce revised implementations—ranging from memory‑management enhancements and algorithmic refinements to automated parallelization and vectorization—thereby enabling high‑performance, resource‑efficient software [ 6 , 7 , 8 ] . Fig. 1 : Artemis ’ multi-LLM code optimization workflow. The success of LLM-based code optimization has led to the development of specialized industrial platforms that integrate these capabilities into production workflows. These platforms provide enterprise-grade infrastructure for deploying LLM optimization at scale, serving diverse client needs from high-frequency trading systems requiring microsecond latencies to scientific computing applications demanding maximum throughput [ 9 , 10 , 11 , 12 , 13 ] . Among others, TurinTech AI’s Artemis platform [ 14 ] represents the cutting edge of AI‑driven software performance engineering, which deploys multiple LLMs at scale to search for optimal optimizations of critical code snippets with real‑time validation, as illustrated in Figure 1 . It now processes tens of thousands of requests daily, and offers automated performance‑engineering solutions that reduce development costs and accelerate time‑to‑market for performance‑critical applications [ 15 ] . However, in its production deployment, Artemis revealed a fundamental obstacle in multi-LLM code optimization, which we term the “cross-model prompt engineering challenge”—a prompt that produces excellent optimizations with one LLM may yield poor results with others [ 16 , 17 ] . Consequently, platforms need to develop and maintain model-specific prompts for each LLM-task combination; furthermore, industrial platforms often face strict efficiency constraints that limit the feasibility of advanced prompt optimization methods that require extensive trial-and-error or computation [ 17 ] . This raises the critical question for this work: How can we automatically generate high‑quality, task‑specific prompts across diverse code optimization LLMs while maintaining industrial efficiency requirements? Existing solutions cannot adequately address these requirements—conventional prompting studies like Chain-of-Thought (CoT) focus primarily on general-purpose tasks and single-model scenarios, lacking the specialized contextual awareness and cross-model adaptability required for industrial code optimization [ 17 ] ; evolutionary prompt optimization methods require pre-defined evaluation functions for each project and multi-round processing, causing scalability problems [ 18 , 19 ] ; advanced prompt tuning agents, while more flexible and powerful, demand extensive computing resources, impractical for production environments requiring rapid processing [ 20 , 21 ] . To address this, we introduce Meta-Prompted Code Optimization ( MPCO ), an end‑to‑end framework that automates prompt engineering while maintaining industrial requirements for flexibility and efficiency across multiple LLMs. In summary, our key contributions are: 1. We present a novel prompt engineering framework that feeds a meta‑prompting LLM rich contextual information to efficiently generate model‑adaptive prompts, thereby addressing the cross-model prompting challenge that limits industrial platforms. 2. We demonstrate MPCO ’s effectiveness through comprehensive evaluation across multiple LLMs on the Artemis platform using five real-world codebases and three major LLMs. Each optimization is verified through 10 repeated benchmarking, with total validation time exceeding 336 hours. 3. Through detailed ablation studies and meta-prompter sensitivity analysis, we identify that comprehensive context integration is essential for effective meta-prompting, and that all three major LLMs can serve effectively as meta-prompters, providing actionable recommendations for industrial deployment. The remainder of this paper is structured as follows. Section II provides background and related works. Section III details the MPCO framework architecture and its integration with industrial platforms. Sections IV and V describe our comprehensive evaluation on real-world industrial codebases. We discuss the practical implications and limitations of MPCO in Section VI and conclude in Section VII . TABLE I : Comparison of prompting approaches. ✓ = Full support, \sim = Partial support, ✗ = No support Approach Context-Awareness End-to-End Automation Deployment Simplicity Low Overhead Cross-Metric Adaptability Cross-Model Adaptability Cross-Project Adaptability Manual Prompting \sim ✗ ✓ ✗ ✗ ✗ ✗ Baseline Prompting ✗ ✓ ✓ ✓ ✗ ✗ ✗ Evolutionary Prompting ✗ ✓ \sim ✗ \sim \sim \sim Agentic Systems ✓ ✓ ✗ ✗ ✓ ✓ ✓ MPCO (Our Approach) ✓ ✓ ✓ ✓ ✓ ✓ ✓ II Preliminaries II-A Industrial Multi-LLM Code Optimization Code optimization—the process of improving software performance via algorithmic and low‑level code transformations—has traditionally relied on handcrafted algorithmic tweaks, compiler‑driven pre‑defined transformations (e.g., loop unrolling, inlining, and dead‑code elimination), and machine learning–based autotuning systems [ 22 ] . Yet, these methods often struggle to capture domain-specific performance patterns, generalize across diverse codebases, and incur high search overheads [ 22 , 23 ] . Recent advances in LLMs enable automated code optimization systems that learn from vast code corpora and leverage both static knowledge base and environment interaction to generate adaptive, context‑aware optimization suggestions, without expert intervention or limitation on pre-defined operations [ 6 ] . Today, industrial platforms integrate these LLM‑driven optimizations into production pipelines, offering low‑latency inference and high‑throughput processing [ 9 , 14 ] . However, single-LLM approaches cannot meet these diverse industrial requirements effectively, as different models exhibit complementary strengths that are needed for comprehensive optimization coverage [ 11 ] . Consequently, industrial platforms like Artemis deploy multiple LLMs simultaneously to leverage their diverse capabilities [ 11 , 15 ] . Specifically, Figure 1 illustrates the Artemis workflow, where the codebase is first analyzed to identify optimization targets, then sent to a diverse set of LLMs for improvement suggestions, and finally filtered through an intelligent search to find the optimal solution. Fig. 2 : Cross-model

Read the complete text