Does Teaming-Up LLMs Improve Secure Code Generation? A Comprehensive Evaluation with Multi-LLMSecCodeEval

Bushra Sabir, Shigang Liu, Seung Ick Jang, Sharif Abuadbba, Yansong Gao, Kristen Moore, SangCheol Kim, Hyoungshick Kim, Surya Nepal, [0.8em] CSIRO’s Data61, Australia · 2026

Read Does Teaming-Up LLMs Improve Secure Code Generation? A Comprehensive Evaluation with Multi-LLMSecCodeEval by Bushra Sabir, Shigang Liu, Seung Ick Jang, Sharif Abuadbba, Yansong Gao, Kristen Moore, SangCheol Kim, Hyoungshick Kim, Surya Nepal, [0.8em] CSIRO’s Data61, Australia (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-22717.txt.

Introduction

Sabir and co-authors test whether teams of language models write safer code than single models. They present a framework that mixes ensembles, static analysis, and structured teamwork. Ten pipelines are benchmarked on two security test sets. Ensembles with static analysis beat single models by up to 47.3 percent on one set. Pure teamwork pipelines gain less. Hybrid pipelines that also detect and patch flaws do best of all. Ablation tests show model scale alone does not ensure safety. Smaller, well-structured ensembles beat large single models.

Catalog record

CREATORBushra Sabir, Shigang Liu, Seung Ick Jang, Sharif Abuadbba, Yansong Gao, Kristen Moore, SangCheol Kim, Hyoungshick Kim, Surya Nepal, [0.8em] CSIRO’s Data61, Australia
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53087
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.22717

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Does Teaming-Up LLMs Improve Secure Code Generation? A Comprehensive Evaluation with Multi-LLMSecCodeEval Bushra Sabir, Shigang Liu, Seung Ick Jang, Sharif Abuadbba, Yansong Gao, Kristen Moore, SangCheol Kim, Hyoungshick Kim, Surya Nepal, [0.8em] CSIRO’s Data61, Australia Abstract Automatically generating source code from natural language using large language models (LLMs) is becoming common, yet security vulnerabilities persist despite advances in fine-tuning and prompting. In this work, we systematically evaluate whether multi-LLM ensembles and collaborative strategies can meaningfully improve secure code generation. We present MULTI-LLMSecCodeEval , a framework for assessing and enhancing security across the vulnerability management lifecycle by combining multiple LLMs with static analysis and structured collaboration. Using SecLLMEval and SecLLMHolmes, we benchmark ten pipelines spanning single-model, ensemble, collaborative, and hybrid designs. Our results show that ensemble pipelines augmented with static analysis improve secure code generation over single-LLM baselines by up to ↑ \uparrow 47.3% on SecLLMEval and ↑ \uparrow 19.3% on SecLLMHolmes, while purely LLM-based collaborative pipelines yield smaller gains of ↑ \uparrow 8.9-22.3%. Hybrid pipelines that integrate ensembling, detection, and patching achieve the strongest security performance, outperforming the best ensemble baseline by ↑ \uparrow 1.78-4.72% and collaborative baselines by ↑ \uparrow 19.81-26.78%. Ablation studies reveal that model scale alone does not ensure security: smaller, structured multi-model ensembles consistently outperform large monolithic LLMs. Overall, our findings demonstrate that secure code doesn’t emerge from scale; it arises from carefully orchestrated multi-model system design. 1 Introduction LLMs are increasingly adopted by developers for security-critical software engineering tasks, including automated code generation, vulnerability detection, and security patching. Prominent models such as OpenAI’s GPT-4, Google PaLM2, and Meta CodeLLaMA show strong performance on general programming tasks. However, recent benchmarks, including SecLLMHolmes [ 30 ] , indicate that LLMs remain unreliable in security-sensitive settings: even minor edits, such as renaming variables or changing whitespace, produce incorrect vulnerability assessments in 17–26% of cases. LLMs also miss subtle semantic defects and often fail to generalize from synthetic benchmarks to real-world projects, undermining their dependability for vulnerability management. These limitations motivate our core research question: Can the shortcomings of individual LLMs be mitigated by combining multiple models in ensemble or collaborative protocols? This question matters because vulnerability management is a high-stakes task. Undetected or incorrectly assessed flaws can lead to data breaches, financial losses, operational downtime, and risks in safety-critical domains [ 40 ] . Empirical studies show that individual LLMs remain error-prone on security tasks, producing both false negatives that allow vulnerabilities to pass and false positives that erode developer trust and increase remediation costs [ 30 ] . Ensembling and multi-model collaboration offer a principled way to address the shortcomings of individual LLMs in vulnerability management. Ensembles can reduce variance and aggregate complementary capabilities, while collaborative protocols allow models to cross-check outputs, expose inconsistent reasoning, and provide aggregated rationales that are easier for humans to audit [ 40 , 16 , 39 , 1 ] . Prior work has explored components of this idea: EnStack [ 24 ] combines multiple code-specialized LLMs via meta-classifiers to detect vulnerabilities that single models may miss, APPatch [ 19 ] generates and cross-validates multiple candidate patches using several LLMs to improve patch reliability, and EnsLLM [ 16 ] selects the most consistent code outputs across multiple LLMs based on similarity measures. EASE [ 33 ] demonstrates that explanation-aware soft ensembling can improve reasoning quality, while VulMaster [ 39 ] shows that multi-LLM collaboration with richer code and structural representations enhances vulnerability repair. Despite these advances, to the best of our knowledge, no prior work systematically evaluates multi-LLM protocols across the full vulnerability management lifecycle, covering code generation, detection, and patching [ 40 ] . To address this gap, we introduce MULTI-LLMSecCodeEval , a unified framework for the systematic evaluation of multi-LLM protocols in secure software engineering. Our key novelty lies not in proposing new coordination mechanisms, but in their integrated orchestration into a reproducible, end-to-end benchmark spanning secure code generation, vulnerability detection, and automated patching. The framework structures the vulnerability management lifecycle into three automated stages: (1) code generation from natural language prompts, (2) vulnerability detection combining static analysis tools and LLM reasoning, and (3) automated patching followed by re-evaluation. We enforce consistent prompt design and strict output formats to ensure reproducibility and reliable assessment, while supporting hybrid evaluation modes that leverage both static analysis and LLM-based reasoning. To study multi-model coordination in practice, we implement ten pipelines spanning single-model, ensemble, collaborative, and hybrid designs (Fig. 3 ). These pipelines incorporate mechanisms such as majority voting, cross-model unanimous checks, ensemble patching, chain-of-debate, and sequential fixing until security criteria are met. Our empirical study evaluates whether structured multi-LLM orchestration improves robustness and reasoning quality over single-model baselines. Extensive ablation studies show that pipeline orchestration, particularly the combination of static-analysis filtering and cross-LLM verification, contributes more to security gains than model scale alone. Leveraging this deterministic orchestration, our hybrid pipelines reduce unresolved vulnerabilities by over 90% relative to naive baselines and systematically quantify how tool choice and prompt consistency affect real-world generalization. Contributions. This paper makes the following contributions: • We introduce Multi-LLMSecCodeEval , a fully automated framework for evaluating LLMs across secure code generation, vulnerability detection, and automated patching, supporting single-model, ensemble, and hybrid pipelines with static analysis integration. • We conduct a comprehensive empirical study of four state-of-the-art LLMs across two benchmark datasets, popular programming languages, CWE categories, task complexities, and prompting strategies, showing that no single LLM provides consistent security guarantees. • We show that ensembling LLMs with static analysis substantially improves secure code generation, achieving up to 47.3% improvement over single-model baselines and consistently outperforming LLM-only collaborative approaches. • We propose a hybrid pipeline that combines ensemble generation, static analysis, and cross-LLM verification, achieving the highest security rates among the evaluated pipelines (up to 97.32% and 99.06%). • We analyze failure modes, unresolved CWEs, cost trade-offs, and tool dependence, providing practical insights for building scalable and cost-effective secure code generation systems. • We publicly release the Multi-LLMSecCodeEval framework, datasets, and evaluation artifacts to support reproducibility and future benchmarking 1 1 1 https://anonymous.4open.science/r/Multi-LLMSecCodeEval-E26D . 2 Motivation Example To illustrate the necessity of MULTI-LLMSecCodeEval , we tasked four state-of-the-art LLMs with generating a C function for file operations involving environment variables and user-controlled inputs. As shown in Fig. 1 , all single-model baselines produce insecure code, exhibiting distinct and model-specific failure modes. CodeLLaMA (Fig. 1 a) naively concatenates an environment variable and filename using strcat without validation, enabling path injection vulnerabilities (CWE-22). Meta-LLaMA (Fig. 1 b) defaults to permissive write behavior and omits path validation, resulting in world-writable file risks (CWE-732) and unsafe file access patterns. Mistral (Fig. 1 c) neglects resource lifecycle management by failing to close file handles (CWE-775), while GPT-3.5 (Fig. 1 d) similarly enables path traversal through unchecked environment-derived inputs (CWE-22). In contrast, Listings 1 and 2 show a secure implementation in which inputs are validated, file paths are constructed safely, and resources are properly released. Static analysis using CodeQL confirms the presence of vulnerabilities in all baseline outputs. These failures motivate our hybrid pipeline design, which employs static-analysis filtering and cross-LLM verification to deterministically mitigate model-specific blind spots. ⬇ 1 char * dir = getenv ( "dir" ); 2 char * path = malloc ( strlen

Read the complete text