Structured Safety Auditing for Balancing Code Correctness and Content Safety in LLM-Generated Code Conference: ; ; CCS: Software and its engineering Source code generation CCS: Software and its engineering Software safety CCS: Computing methodologies Natural language generation
Honghao Tan, Haibo Wang, Shin Hwei Tan · 2026
Read Structured Safety Auditing for Balancing Code Correctness and Content Safety in LLM-Generated Code Conference: ; ; CCS: Software and its engineering Source code generation CCS: Software and its engineering Software safety CCS: Computing methodologies Natural language generation by Honghao Tan, Haibo Wang, Shin Hwei Tan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-12088.txt.
Introduction
Tan, Wang, and Tan study safety in code generation, where models are judged on correctness. Code models reproduce offensive identifiers from injected renaming instructions without warning. Viewing code as machine execution and human communication, they propose a duality score unifying utility, safety, and warnings. They add Dual Reasoning, a safety audit and code review before generation. On six models and two benchmarks with keyword injections, Dual Reasoning tops the duality score, improving scores by 1.32 to 3.42 times. Chain-of-thought gives negligible gains; a safety-aware prompt gives partial improvement.
Catalog record
| CREATOR | Honghao Tan, Haibo Wang, Shin Hwei Tan |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 57494 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.12088
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Structured Safety Auditing for Balancing Code Correctness and Content Safety in LLM-Generated Code Conference: ; ; CCS: Software and its engineering Source code generation CCS: Software and its engineering Software safety CCS: Computing methodologies Natural language generation Honghao Tan, Haibo Wang, Shin Hwei Tan Abstract . Content Warning: This paper contains examples of harmful keywords used solely for evaluation purposes and does not promote harmful behavior. Large language models (LLMs) for code generation are typically evaluated on functional correctness alone, overlooking whether generated code propagates harmful content embedded in the prompt. Prior work has shown that most Code LLMs reproduce offensive identifiers from injected renaming instructions without warning, yet existing approaches focus on detecting harmful content, neglecting functional correctness. Grounded in the Theory of Dual Channel Constraints (which states that code is a dual-channel medium combining an algorithmic (AL) channel for machine execution and a natural language (NL) channel for human communication, creating a unique safety-utility trade-off where a model must balance functional execution with responsible communication), we propose NLSafety-Utility Duality Score (SUDS), a metric that unifies code utility, safety adherence, and warning awareness into a single score across 12 ranked response scenarios, and Dual Reasoning (DR), a structured inference-time technique that requires an explicit safety audit and task-grounded code review before code generation. Evaluated on five LLMs across two benchmarks augmented with harmful keyword injections (820 and 2,135 samples), DR consistently achieves the highest SUDS across all models, improving mean SUDS by 1.32 × \times to 3.42 × \times over the baseline, while chain-of-thought prompting yields negligible safety gains and a safety-aware prompt provides only partial improvement. Further analysis reveals that DR’s effectiveness scales with model capacity, that the one-shot exemplar primarily stabilizes output format for smaller models, and that structured reasoning cannot compensate for models with limited safety vocabularies. 1. Introduction With the recent advancement of large language models (LLMs), LLMs such as CodeLlama ( Roziere et al. 2023 ) and DeepSeek-Coder ( Guo et al. 2024 ) have shown promising results in reducing the developers’ effort required to write and transform existing code. The traditional evaluation paradigm for these models mainly focuses on functional correctness (i.e., whether the generated code compiles and passes a given test suite (as measured by pass@ k ( Chen et al. 2021a ) ) on benchmarks such as HumanEval ( Chen et al. 2021a ) and MBPP-sanitized ( Austin et al. 2021 ) ). Although functional correctness is a necessary criterion for adoption, it captures only one dimension (i.e., code quality) in deployed code generation systems. Recent studies have demonstrated that LLM-based code generation models readily produce biased ( Liu et al. 2023a ; Al-Kaswan et al. 2025a ) or harmful content when prompted to do so ( Tan et al. 2025 ) . Particularly, prior study of LLM code transformations ( Tan et al. 2025 ) shows that injecting an explicit renaming instruction into an otherwise benign coding prompt (e.g., “Rename the first parameter to [ keyword ]”) causes most Code LLMs to reproduce the harmful identifier verbatim in generated outputs. This active injection threat may arise in a real development workflow in which a realistic threat model involves untrusted or insensitive users (i.e., attackers) who misuse LLMs to produce code with harmful content and submit it to open-source projects ( Tan et al. 2025 ) . Despite this risk, these studies focus mainly on detecting whether harmful content exists in generated code, without addressing how to mitigate it while preserving functional correctness . Our work is conceptually grounded in the Theory of Dual Channel Constraints ( Casalnuovo et al. 2020 ) , which posits that code is a dual-channel medium combining an algorithmic (AL) channel for machine execution and a natural language (NL) channel for human communication. While the AL channel is consumed by machines, which determines the program behavior, the NL channel—comprising identifier names and comments—is essential for developers to understand the purpose and context of the execution. A model that generates functionally correct code with toxic identifier names fails the NL channel, whereas a model that refuses every harmful prompt fails the AL channel. Pass@ k (used for checking for functional correctness) is blind to this duality: it evaluates the AL channel in isolation and cannot quantify the trade-off that safety-aware deployment requires. Meanwhile, the output damage ( Tan et al. 2025 ) that measures the harms caused by toxic generated code only evaluates the NL channel, ignoring the AL channel. This duality creates a unique NLSafety-utility 1 1 1 Throughout this paper, we use NLSafety to refer specifically to the absence of harmful or offensive content in the natural language channel of generated code (e.g., identifier names), distinguishing it from code security, which concerns syntactic vulnerabilities. trade-off: the model must balance functional execution (AL) with responsible communication (NL) . To resolve this duality, we present NLSafety–Utility Duality Score (SUDS) , a new evaluation paradigm that unifies safety and utility into a single metric. Unlike prior approaches that evaluate correctness and safety separately, SUDS captures their interaction, enabling systematic differentiation between behaviors (e.g., unsafe correctness, safe refusal, and balanced generation). This reframes evaluation from a single-objective optimization problem into a dual-objective one, where the overarching goal involves jointly improving both dimensions. To enhance the capability of code generation models in generating code to resolve the duality, we propose Dual Reasoning (DR) , a new technique that directly targets the root cause of the duality. Rather than relying on implicit or post hoc safety mechanisms, DR enforces a structured think-then-act process: the model must first perform an explicit safety audit over the NL channel before generating code in the AL channel. This explicit separation ensures that safety is treated as a prerequisite to generation, not an afterthought. Our approach operates entirely at inference time, requiring no retraining, and is therefore immediately applicable to existing Code LLMs. To encourage future evaluation of the NLSafety–utility trade-off, we construct harmful-content-augmented variants of HumanEval ( Chen et al. 2021a ) and MBPP-sanitized ( Austin et al. 2021 ) , enabling the first systematic evaluation of code generation models under adversarial safety–utility conditions. Our results reveal that existing techniques (including standard prompting, chain-of-thought reasoning, and safety-biased prompts) fail to resolve the duality, often improving one dimension at the expense of the other. In contrast, dual reasoning consistently achieves superior performance under SUDS, demonstrating that structured reasoning is key to enhancing safety and utility. In summary, this paper makes the following contributions: • New Evaluation Paradigm: We introduce SUDS, the first metric to jointly model safety and utility, providing a principled and fine-grained evaluation of code generation behavior. • New Benchmark Setting: We extend HumanEval and MBPP with injected harmful content, enabling systematic analysis of the NLSafety–utility trade-off. • New Prompting Strategy: We propose dual reasoning, a structured inference-time approach that enforces explicit safety auditing prior to code generation. • Empirical Evaluation: Our evaluation on both benchmarks shows that existing techniques fail to address the NLSafety–utility duality, while our approach consistently improves both dimensions across models. Ethical Considerations. Our study involves the deliberate use of harmful and offensive keywords within code generation benchmarks to evaluate model safety behavior. While necessary for rigorous evaluation, we ensured