Fine-Tuning LLMs to Analyze Multiple Dimensions of Code Review: A Maximum Entropy Regulated Long Chain-of-Thought Approach DOI: XXXXXXX.XXXXXXX Journal: JACM Volume: 37 4 111 8 CCS: Software and its engineering Software creation and management
Yongda Yu, Guohao Shi, Xianwei Wu, Haochuan He, XueMing Gu, Qianqian Zhao, Kui Liu, Qiushi Wang, Zhao Tian, Haifeng Shen, Guoping Rong · 2025
Read Fine-Tuning LLMs to Analyze Multiple Dimensions of Code Review: A Maximum Entropy Regulated Long Chain-of-Thought Approach DOI: XXXXXXX.XXXXXXX Journal: JACM Volume: 37 4 111 8 CCS: Software and its engineering Software creation and management by Yongda Yu, Guohao Shi, Xianwei Wu, Haochuan He, XueMing Gu, Qianqian Zhao, Kui Liu, Qiushi Wang, Zhao Tian, Haifeng Shen, Guoping Rong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-21170.txt.
Introduction
Human code reviewers weigh correctness, style, security, and more in a single pass; automated reviewers usually check one dimension at a time. Yongda Yu and ten co-authors present MelcotCR in a 2025 paper on fine-tuning language models for code review. Their method teaches models to analyze several dimensions together.
The approach builds long chains of thought for review comments, regulated by a maximum entropy principle to keep reasoning diverse but grounded. Fine-tuning uses richer information than earlier datasets, which the authors blame for vague model feedback. The trained model examines a change from multiple angles before judging it, closer to how senior reviewers work. At twenty-two pages, the paper details the data construction and training. Better automated review could catch defects earlier in large software projects.
Catalog record
| CREATOR | Yongda Yu, Guohao Shi, Xianwei Wu, Haochuan He, XueMing Gu, Qianqian Zhao, Kui Liu, Qiushi Wang, Zhao Tian, Haifeng Shen, Guoping Rong |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-21170
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Fine-Tuning LLMs to Analyze Multiple Dimensions of Code Review: A Maximum Entropy Regulated Long Chain-of-Thought Approach DOI: XXXXXXX.XXXXXXX Journal: JACM Volume: 37 4 111 8 CCS: Software and its engineering Software creation and management Yongda Yu, Guohao Shi, Xianwei Wu, Haochuan He, XueMing Gu, Qianqian Zhao, Kui Liu, Qiushi Wang, Zhao Tian, Haifeng Shen, Guoping Rong Abstract . Large Language Models (LLMs) have shown great potential in supporting automated code review due to their impressive capabilities in context understanding and reasoning. However, these capabilities are still limited compared to human-level cognition because they are heavily influenced by the training data. Recent research has demonstrated significantly improved performance through fine-tuning LLMs with code review data. However, compared to human reviewers who often simultaneously analyze multiple dimensions of code review to better identify issues, the full potential of these methods is hampered by the limited or vague information used to fine-tune the models. This paper contributes MelcotCR , a chain-of-thought (COT) fine-tuning approach that trains LLMs with an impressive reasoning ability to analyze multiple dimensions of code review by harnessing long COT techniques to provide rich structured information. To address context loss and reasoning logic loss issues that frequently occur when LLMs process long COT prompts, we propose a solution that combines the Maximum Entropy (ME) modeling principle with pre-defined reasoning pathways in MelcotCR to enable more effective utilization of in-context knowledge within long COT prompts while strengthening the logical tightness of the reasoning process. Empirical evaluations on our curated MelcotCR dataset and the public CodeReviewer dataset reveal that a low-parameter base model, such as 14B Qwen2.5, fine-tuned with MelcotCR can surpass state-of-the-art methods in terms of the accuracy of detecting and describing code issues, with its performance remarkably on par with that of the 671B DeepSeek-R1 model. 1. Introduction As a cornerstone practice in software quality assurance, code review plays an irreplaceable role in modern software development ( Gousios et al. 2014 ) . Traditional manual review processes, which heavily rely on line-by-line scrutiny of code by developers, are not only time-consuming and labor-intensive but also heavily dependent on individual reviewers’ expertise, a limitation that has driven the innovation and advancement of Automated Code Review (ACR) technologies ( Li et al. 2022a ; Tufano et al. 2021 ; Lu et al. 2023 ; Yu et al. 2024a ; Gupta and Sundaresan 2018 ) . Recent breakthroughs in AI-driven code analysis based on Large Language Models (LLMs) have naturally attracted significant research attention to ACR, resulting in multiple LLM-based ACR systems ( Lu et al. 2023 ; Yu et al. 2024a ) . As an LLM’s capability in context understanding and reasoning is highly influenced by its training data, a common strategy is to fine-tune base LLMs with code-review-specific data so that they can perform better in ACR tasks. In particular, Liu et al. ( Lu et al. 2023 ) fine-tuned Llama to create Llama-Reviewer, which is able to generate review comments through the direct learning of a raw code review dataset from a previous study ( Li et al. 2022b ) to enhance the LLM’s commenting capability. It should be noted that the information used in fine-tuning is only a direct code-to-comment mapping that lacks analytical depth. In contrast, Yu et al. developed Carllm ( Yu et al. 2024a ) that used chain-of-thought (COT) to fine-tune the base LLMs by decomposing the review process into three progressive stages: defect localization, review comment generation, and solution proposal. The COT approach provides richer information for fine-tuning, resulting in improved ACR performance compared to Llama-Reviewer. However, compared to human reviewers who often simultaneously consider multiple dimensions of code review, e.g., code intent, boundary conditions, and invocation relationships, when analyzing code issues ( Bosu and Carver 2013 ; Kononenko et al. 2016 ) , the full potential of LLMs for ACR is hampered by the limited or vague information used to fine-tune them. Specifically, Llama-Reviewer ( Lu et al. 2023 ) merely instructs the LLM to perform ACR via vague prompts, while in Carllm ( Yu et al. 2024a ) , each reasoning step is constrained in such a way that it only requires the LLM to identify issues without providing much guiding information. Some studies have revealed a trend that employing COT to prompt LLMs can significantly enhance their performance in specific tasks ( Wei et al. 2022 ; Feng et al. 2023 ; Zhang et al. 2022 ) , while there exists a positive correlation between COT length and model performance gain ( Chen et al. 2025b ; Xia et al. 2024 ) . In this paper, we contribute MelcotCR , a fine-tuning approach that trains LLMs with an impressive structured reasoning ability to analyze multiple dimensions of code review by harnessing long COT techniques ( Wu et al. 2025 ; Chen et al. 2025b ; Yeo et al. 2025a ) . It systematically decomposes a code review task into multiple fine-grained sub-tasks including code functionality summarization, core logic analysis, change impact analysis, and inspection of multiple concrete issues, thereby stimulating an LLM to generate coherent reasoning chains. However, a known challenge with long COT is its inherent context loss and reasoning logic loss issues, as the length of COT increases ( Chen et al. 2025b ; Xia et al. 2024 ) . To address this challenge, we propose a solution that combines the Maximum Entropy (ME) modeling principle with pre-defined reasoning pathways in MelcotCR to enable more effective utilization of in-context knowledge within long COT prompts while strengthening the logical tightness of the reasoning process. This solution expands each ground truth answer into multiple semantically equivalent but syntactically distinct expressions, enabling the model to learn bias-invariant essential knowledge. The main contributions of this work are summarized as follows: • We propose MelcotCR , a novel fine-tuning approach that trains LLMs with the ability to analyze multiple dimensions in code review by combining the maximum entropy principle with long COT techniques to cultivate diversified problem-solving capabilities. • We conduct empirical evaluations to demonstrate that a 14B low-parameter base model fine-tuned with MelcotCR can outperform state-of-the-art methods, with its ACR performance remarkably on par with that of Deepseek-R1 671B. • We perform experiments to investigate the impacts of fine-tuning methods and reasoning steps on MelcotCR ’s ACR performance. The findings not only validate the proposed approach but also suggest future research directions. The rest of the paper is organized as follows. Section 2 introduces some related work. Section 3 describes the research methodology, followed by the evaluation process and results in Section 4 . Section 5 discusses the implications and validity risks. Section 6 concludes the paper with a summary of contributions and future work. 2. Related work In this section, we review the related work that forms the foundation of this work, including automated code review, chain of thought, and maximum entropy. 2.1. Automated Code Review (ACR) ACR is an essential aspect of improving software development efficiency, which aims to reduce the manual effort and time required for code assessment. The main focus of an ACR system is to detect potential code defects and suggest or generate relevant review comments. It typically comprises two components: defect detection and review comment recommendation/generation. Defect detection