Operationalizing Large Language Models with Design-Aware Contexts for Code Comment Generation Conference: International Conference on Software Engineering; April 12–18, 2026; Rio de Janeiro, Brazil
Aritra Mitra, Srijoni Majumdar, Anamitra Mukhopadhyay, Partha Pratim Das, Paul D Clough, Partha Pratim Chakrabarti · 2025
Read Operationalizing Large Language Models with Design-Aware Contexts for Code Comment Generation Conference: International Conference on Software Engineering; April 12–18, 2026; Rio de Janeiro, Brazil by Aritra Mitra, Srijoni Majumdar, Anamitra Mukhopadhyay, Partha Pratim Das, Paul D Clough, Partha Pratim Chakrabarti (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-22338.txt.
Introduction
Aritra Mitra and five co-authors, Srijoni Majumdar, Anamitra Mukhopadhyay, Partha Pratim Das, Paul D Clough. Partha Pratim Chakrabarti, feed design documents to language models so the comments they write for code actually help. Comments guide everyone who maintains a program later, but novice programmers, now producing more code than ever, often write comments that are useless or worse. Maintainers already fall back on design documents when comments fail.
The study tests whether giving the model that same design context, the intent and structure behind the code, produces more useful comments than code alone could support. The feasibility question is deliberately narrow and practical: does design-aware context beat the naive approach on real codebases. If it does, comment generation stops being a paraphrase of the code and starts carrying the reasoning a future maintainer cannot see in the code itself.
Catalog record
| CREATOR | Aritra Mitra, Srijoni Majumdar, Anamitra Mukhopadhyay, Partha Pratim Das, Paul D Clough, Partha Pratim Chakrabarti |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 25 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-22338
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Operationalizing Large Language Models with Design-Aware Contexts for Code Comment Generation Conference: International Conference on Software Engineering; April 12–18, 2026; Rio de Janeiro, Brazil Aritra Mitra, Srijoni Majumdar, Anamitra Mukhopadhyay, Partha Pratim Das, Paul D Clough, Partha Pratim Chakrabarti Abstract . Comments are very useful to the flow of code development. With the increasing commonality of code, novice coders have been creating a significant amount of codebases. Due to lack of commenting standards, their comments are often useless, and increase the time taken to further maintain codes. This study intends to find the usefulness of large language models (LLMs) in these cases to generate potentially better comments. This study focuses on the feasibility of design documents as a context for the LLMs to generate more useful comments, as design documents are often used by maintainers to understand code when comments do not suffice. 1. Introduction Large Language Models (LLMs) have proved to be very useful in the world of natural language processing. These models are primarily tested for predict-the-next token, however, they can perform tasks like code summarization, test generation, code generation etc. after some fine-tuning ( Chen et al., 2021 ) . In this study, we focus on the task of comment generation. Code comments are the first layer of support for understanding code, offering file- and method-level context, whereas inline comments capture design choices. Discrepancies between comments and code can show bugs and aid developers. Hence, developers should write succinct, accurate, and updated comments to guide future maintainers and users. But, comments are often neglected while writing codes ( Fluri et al., 2007 ; Shmerlin et al., 2015 ) . A good comment reduces maintenance cost while not bloating the codebase ( Sridhara et al., 2010 ; Aggarwal et al., 2002 ; Tenny, 1988 ) , while a bad comment provides little to no help, increasing the bloat. This is especially important when the majority of the time spent by developers is to understand a codebase ( Xia et al., 2018 ; Chatterjee et al., 2023 ) . This task has previously been performed with the code as context, with some additional information. However, we use the design documents as project-specific context. Design documents are used commonly to reference a codebase, as they explain the design choices, intended functionality, and any other common information, provide information on the architecture of the codebase, requirements of the system, contain links to any relevant sources, etc. Hence, for a maintainer, design documents provide useful information while working on the codebase. It is also important for the maintainer to keep these documents accurate, so it does not become erroneous for other maintainers, as well as the end-users. To tie everything together, we focus on the following research questions: RQ1:: Can design documents improve the quality of comments generated by LLMs? RQ2:: Can Abstract Syntax Trees (ASTs) be used as structural cues to generate better comments? We use the paradigm of Retrieval-Augmented Generation (RAG) ( Lewis et al., 2021 ) to generate comments from the code, with the design document as a source for the retrieval. RAG models have been shown to provide substantial benefit over vanilla LLMs for knowledge-intensive tasks ( Gao et al., 2024 ; Lewis et al., 2021 ) . We have performed automated testing of the comments generated in this fashion using metrics such as ROUGE ( Lin, 2004 ) , BLEU ( Papineni et al., 2002 ) , CodeBERTScore ( Cui et al., 2022 ) and GPTScore ( Fu et al., 2023 ) , as well as conducting a study with novice coders, providing them with bug-fixing and codebase enhancement tasks. We have also classified the generated comments by the kind of information they provide. Code Comments ⬇ void check_escaped ( const char * original , const char * escaped , const bool ensure_ascii ) { std :: stringstream ss ; json :: serializer s ( nlohmann :: detail :: output_adapter < char >( ss ), ’ ␣ ’ ); s . dump_escaped ( original , ensure_ascii ); CHECK ( ss . str () == escaped ); } ⬇ Comment generated : Few - Shot Prompting /** * @brief Checks if a string is properly escaped as per JSON standards. * @param original The original string to be escaped. * @param escaped The expected escaped string. * @param ensure_ascii If true, ensures ASCII encoding. */ ⬇ Comment generated : RAG with Design Document [ API Documentation ] /** ... * Use Case: Test in json/test. * Method used: * - dump_escaped, defined at json.hpp:8514 */ ⬇ static int timer_less_than ( const struct heap_node * ha , const struct heap_node * hb ) { const uv_timer_t * a , b ; a = container_of ( ha , uv_timer_t , node . heap ); b = container_of ( hb , uv_timer_t , node . heap ); if ( a -> timeout < b -> timeout ) return 1; if ( b -> timeout < a -> timeout ) return 0; return a -> start_id < b -> start_id ; } ⬇ Comment generated : Few - Shot Prompting /* * Heap comparator function for timers. * Primary: timeout, Secondary: start_id for tie-breaking * Prevents starvation of same-timeout timers */ ⬇ Comment generated : RAG with Design Document [ Implementation Document ] /*-------------------------------------------------------* * static int timer_less_than(const heap_node* a, const heap_node* b) ... * LIMITATION: relies on start_id never wrapping. *-------------------------------------------------------*/ Table 1 . Comparison of comments generated with and without design documents as context. The comment generated with the implementation document detects an exception, and the comment generated with the API documentation correctly locates the definition. We evaluated four generation setups: (i) Few-shot prompting with 30 examples, with at least 10 positive and 10 negative examples, (ii) Few-shot prompting with condensed abstract syntax trees (ASTs) as added context, (iii) Few-shot prompting with RAG on design document, (iv) Combining ASTs and RAG on design document. In the cases with the design document, we have found an increased number of navigational comments: linking to other parts of the code, algorithmic comments: explaining the workings of the code, and exception-based comments: showing edge-cases of the code. This can be seen from the examples in Table 1 , as LLM-generated comments with design documents fare better than the comments without them. We have seen a 35% decrease in bug-fixing time for LLM generated comments with the design document provided. The results for the codebase enhancement task are very similar, except that the decrease in time reduces to 28%. This can be explained due to the time taken to write the code not being affected, and the decrease coming from the time taken in understanding. To the best of our knowledge, this is the first systematic study of comment generation in C using an LLM architecture, with the following three major contributions: • A pipeline with the source code, ASTs, design documents, and RAG to support scalable and informative LLM-based comment generation for C and C++. • An assessment framework for C and C++ comments based on their usefulness, combining LLM-based metrics and human judgment. • Using design documents as context cues to gather project-specific information, and abstract syntax trees to gather system-specific information. 2.