FLARE: Fine-Grained Diagnostic Feedback for LLM Code Refinement

Yinsheng Yao, Hongxiang Zhang, Weixi Tong, Tianyi Zhang · 2026

Read FLARE: Fine-Grained Diagnostic Feedback for LLM Code Refinement by Yinsheng Yao, Hongxiang Zhang, Weixi Tong, Tianyi Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-03852.txt.

Introduction

Yao and co-authors refine buggy code with FLARE, a framework that points at suspect lines. Language models write code with bugs, and feedback loops fix it. Test failures and self-critiques are coarse signals, though. They rarely say where the bug sits. FLARE adds a lightweight diagnostic model to the loop. It predicts line-level suspiciousness for each draft.

Predictions carry doubt, so FLARE searches the top candidate regions. Execution outcomes pick the best fix among them. On LiveCodeBench and BigCodeBench, five base models all improve. Even without search, gains over the strongest baseline run from about two to seven points. Searching ten candidates adds about nine points on average over no search. The diagnostic model alone beats recent fault localization methods. Coding assistants get fine-grained guidance instead of vague failure notices.

Catalog record

CREATORYinsheng Yao, Hongxiang Zhang, Weixi Tong, Tianyi Zhang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.03852

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FLARE: Fine-Grained Diagnostic Feedback for LLM Code Refinement Yinsheng Yao, Hongxiang Zhang, Weixi Tong, Tianyi Zhang Abstract Large language models often generate code with bugs. Existing methods rely on feedback signals such as test failures and self-critiques to iteratively refine the generated code. Such signals are either too coarse-grained or too high-level, which is not sufficient to inform the model where to fix the bug. In this work, we present Flare , an iterative framework with a lightweight diagnostic model that predicts line-level suspiciousness signals for bug localization and code refinement. Given the inherent uncertainty of diagnostic predictions, Flare searches over the top-k suspicious regions and selects the best candidate according to execution outcomes. Experiments on LiveCodeBench and BigCodeBench with five base LLMs show that, even without candidate search ( k k =1), Flare outperforms the strongest baseline with an absolute improvement from 1.72% to 7.42%. Furthermore, searching over 10 candidates yields an average improvement of 8.50% compared with no candidate search. When evaluated in isolation, our lightweight diagnostic model achieves the best performance compared with recent fault localization methods, demonstrating that it can provide reliable fine-grained guidance for code refinement. 1 Introduction Large Language Models (LLMs) have demonstrated remarkable capabilities in coding ( 11 ; 2 ; 31 ) . Despite this progress, LLMs remain prone to generating code with bugs ( 10 ; 22 ) . Given the challenge of generating correct code in one pass, existing methods often perform iterative refinement, where a model generates an initial program, receives feedback, and then refines the program over multiple rounds ( 4 ; 14 ; 28 ; 29 ) . However, these methods usually rely on feedback signals such as test failures and self-critiques to iteratively refine the generated code. Such signals are either too coarse-grained or too high-level, which is not sufficient to inform the model where to fix the bug. To address this limitation, we propose Flare , an iterative framework with a lightweight diagnostic model that predicts line-level suspiciousness for targeted code refinement. Given an initial code solution, Flare collects the language model’s token-level probabilities and aligns them from subword tokens to lexical units such as identifiers, operators, and keywords. The diagnostic model scores these lexical units and aggregates the scores into line-level suspiciousness predictions. During refinement, Flare combines the ranked suspicious lines with execution feedback to generate targeted candidate refinements. Given the inherent uncertainty of fine-grained feedback predictions, Flare is further equipped with a search procedure over the top- k plausible fault regions at each iteration and selects the next candidate according to execution outcomes. Figure 1: Overview of Flare . The framework aligns LLMs’ probability signals to predict line-level suspiciousness, combines this diagnostic signal with execution feedback, and searches over top- k candidate refinements. We evaluate Flare on two code generation benchmarks, LiveCodeBench ( 9 ) and BigCodeBench ( 32 ) , with five base LLMs and three existing methods, including NL-Debugging ( 29 ) , Self-Debugging ( 4 ) , and Self-Refine ( 14 ) . Even without candidate search ( k=1 ), Flare outperforms the most competitive baseline, NL-Debugging, in 9 of 10 settings with an absolute improvement from 1.72% to 7.42%. Furthermore, searching over 10 candidates ( k=10 ) yields an absolute improvement of 8.50% compared with no candidate search. Ablation studies further show that both fine-grained diagnostic feedback and candidate search contribute significantly to the overall gains. When evaluated in isolation, our diagnostic model achieves 67% Top-1 and 89% Top-10 localization accuracy, outperforming recent fault localization methods, including FlexFL ( 25 ) , LLMAO ( 26 ) , and BAP ( 19 ) . 2 Methodology Figure 1 provides an overview of Flare . Given an initial candidate program, Flare predicts suspicious lines with a lightweight diagnostic model trained on LLM internal signals (Section 2.1 ). To account for diagnostic uncertainty, Flare searches over the top- k suspicious regions and selects the best candidate according to execution outcomes (Section 2.2 ). The process continues until the program passes all tests or reaches a maximum iteration budget. 2.1 Fine-grained Diagnostic Feedback Existing self-refinement methods often rely on coarse-grained or high-level feedback, which provides limited guidance about where the underlying issue resides. Flare introduces a fine-grained diagnostic model that ranks suspicious lines for targeted refinement. 1 1 1 We evaluate the diagnostic model and ablate its components in Section 3.5 . Formally, given a generated program \mathcal{C} segmented into L lines \mathcal{C}=[c_{1},\dots,c_{L}] , we train a diagnostic model f_{\theta} that outputs line-level suspiciousness scores: \mathbf{r}=f_{\theta}(\mathcal{C})\in\mathbb{R}^{L},\qquad\mathbf{r}=[r_{1},\dots,r_{L}], (1) where r_{i} denotes the suspiciousness score assigned to line c_{i} . Training Data Construction. We train the diagnostic model on the training split of Collu-Bench ( 12 ) , which contains 10504 buggy code solutions generated by 11 diverse LLMs on five datasets. In Collu-Bench , each buggy solution is paired with token-level probabilities from the generator LLM and ground-truth bug locations. Previous empirical studies reveal two key insights ( 12 ; 18 ) : LLMs tend to exhibit lower confidence when generating buggy code, and their errors are concentrated in certain token categories, such as Keyword , Identifier , and Type . Motivated by these findings, we use the LLM’s confidence signals, token embeddings, and syntactic token types as inputs to the diagnostic model. To construct these inputs, we first align the LLM’s confidence signals with the code’s lexical-unit representation. While the LLM assigns probabilities to subword tokens, program semantics are organized around syntactic units such as identifiers, operators, and statements. This mismatch is particularly problematic for code, as standard Byte-Pair Encoding (BPE) tokenization often splits a single lexical unit into multiple subword pieces, e.g., max_retries \rightarrow \{ max , _ , retries \} . Specifically, we parse \mathcal{C} and extract the AST-aligned lexical sequence \mathcal{T}(\mathcal{C})=\{t_{1},\dots,t_{M}\}, (2) where t_{j} corresponds to a lexical unit. Let \{p_{n}\}_{n=1}^{N} denote the probabilities of the N subword tokens. Given a lexical unit t_{j} , let \mathcal{S}(j)\subseteq\{1,\dots,N\} denote the subword token indices aligned with t_{j} . Then, we compute the confidence of t_{j} as the probability of its aligned subword sequence: p^{\prime}_{j}=\prod_{n\in\mathcal{S}(j)}p_{n}. (3) For each lexical unit t_{j} in the program \mathcal{C} , we represent it by the tuple \mathbf{x}_{j}=\bigl(t_{j},\;\mathrm{Type}(t_{j}),\;p^{\prime}_{j}\bigr), (4) where \mathrm{Type}(t_{j}) denotes the syntactic category of t_{j} , such as Keyword , Identifier , or Type . 2 2 2 We list type details in Appendix B . Training Objective. Diagnosing bugs in generated code often requires reasoning beyond the local token or line. For example, an incorrect variable update or branch condition may only become suspicious when compared with definitions, uses, or constraints that appear elsewhere in the program. To model these dependencies, we encode the AST-aligned sequence with a bidirectional transformer encoder, which allows each code unit to attend to the full program context ( 7 ) . Given the program \mathcal{C} , we first embed each lexical unit t_{j} in \mathcal{C} into an input vector by concatenating its token embedding, syntactic type, and projected confidence: \displaystyle\mathbf{e}_{j} \displaystyle=\bigl[\mathrm{Emb}_{tok}(t_{j});\,\mathrm{Emb}_{type}(\mathrm{Type}(t_{j})); (5) \displaystyle\mathrm{Proj}(p^{\prime}_{j})\bigr], where \mathrm{Emb}_{tok} and \mathrm{Emb}_{type} are learned embedding layers, and \mathrm{Proj} is a learned linear projection from the scalar p^{\prime}_{j} to a vector representation. 3 3 3 The implementation details are provided in Appendix A.3 We add

Read the complete text