SENAI: Towards Software Engineering Native Generative Artificial Intelligence ISBN: 978-1-4503-XXXX-X/18/06 CCS: Software and its engineering CCS: Software and its engineering Software notations and tools CCS: Computing methodologies Machine learning algorithms
Mootez Saad, José Antonio Hernández López, Boqi Chen, Neil Ernst, Dániel Varró, Tushar Sharma · 2025
Read SENAI: Towards Software Engineering Native Generative Artificial Intelligence ISBN: 978-1-4503-XXXX-X/18/06 CCS: Software and its engineering CCS: Software and its engineering Software notations and tools CCS: Computing methodologies Machine learning algorithms by Mootez Saad, José Antonio Hernández López, Boqi Chen, Neil Ernst, Dániel Varró, Tushar Sharma (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-15282.txt.
Introduction
Mootez Saad leads a team of six on this vision paper from two thousand twenty-five. It argues for a new kind of code model, one bred on software engineering first principles, not just on making code that runs.
Today's models write code that works, yet they skip core habits such as modularity, single duty, and sane ties between parts. The authors want engineering lore baked into the whole pipeline, from training to scoring. They also propose Bloom's taxonomy as the frame for judging how deep a model has soaked up that lore. Such native models, they claim, would suit real-world build work far better than today's. The paper is a call to steer the field.
Catalog record
| CREATOR | Mootez Saad, José Antonio Hernández López, Boqi Chen, Neil Ernst, Dániel Varró, Tushar Sharma |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 23 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-15282
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SENAI: Towards Software Engineering Native Generative Artificial Intelligence ISBN: 978-1-4503-XXXX-X/18/06 CCS: Software and its engineering CCS: Software and its engineering Software notations and tools CCS: Computing methodologies Machine learning algorithms Mootez Saad, José Antonio Hernández López, Boqi Chen, Neil Ernst, Dániel Varró, Tushar Sharma Abstract . Large Language Models have significantly advanced the field of code generation, demonstrating the ability to produce functionally correct code snippets. However, advancements in generative ai for code overlook foundational Software Engineering ( se ) principles such as modularity, and single responsibility, and concepts such as cohesion and coupling which are critical for creating maintainable, scalable, and robust software systems. These concepts are missing in pipelines that start with pre-training and end with the evaluation using benchmarks. This vision paper argues for the integration of se knowledge into llm s to enhance their capability to understand, analyze, and generate code and other se artifacts following established se knowledge. The aim is to propose a new direction where llm s can move beyond mere functional accuracy to perform generative tasks that require adherence to se principles and best practices. In addition, given the interactive nature of these conversational models, we propose using Bloom’s Taxonomy as a framework to assess the extent to which they internalize se knowledge. The proposed evaluation framework offers a sound and more comprehensive evaluation technique compared to existing approaches such as linear probing. Software engineering native generative models will not only overcome the shortcomings present in current models but also pave the way for the next generation of generative models capable of handling real-world software engineering. 1. Introduction Large language models ( llm s) have significantly transformed software development by enabling on-demand code generation and assistance capabilities. These models have demonstrated remarkable capabilities in automating programming tasks, providing code completions, and generating functions or classes based on textual prompts ( Fan et al. 2023 ; Hou et al. 2024 ) . By learning from vast code repositories, llm s have become valuable tools for developers, enhancing productivity and reducing the time required to write boilerplate or routine code. However, while these models are used within a software engineering context, they are not entirely trained on abstract knowledge embodied in the field such as design principles. Indeed, most contemporary models are directly pre-trained on source code, sometimes including other sources such as Jupyter notebooks and GitHub pull requests ( Lozhkov et al. 2024 ) . Furthermore, functional correctness is the main criterion used to evaluate their output. In se , programming is a crucial step in creating concrete, operational systems that address specific problems, use cases, or functionalities from requirements. The evaluation and fitness of written code, however, go beyond mere functional correctness ( ISO/IEC 25010:2011 2011 ) . This is because code is rarely written in isolation; it must conform to many constraints, standards, and best practices that ensure its suitability within a larger context. These constraints stem from the necessity for code to integrate seamlessly with existing systems, be understandable and maintainable by other developers, and remain adaptable to future requirements. Software systems evolve as requirements change or emerge, bugs are fixed, and new features are added; hence, ensuring conformance to these constraints, standards, and best practices is a constant endeavor. More broadly, se is a multifaceted domain, encompassing many development process steps (such as requirement elicitation, design, and testing), attempting to ensure various quality attributes (such as reliability, maintainability, and performance ( ISO/IEC 25010:2011 2011 ) ), while ensuring relevance of the software being developed by appropriate communication and collaboration mechanisms. Current llm development is mostly confined to models that autocomplete functionally correct code. This creates a rift between se expectations and what these models deliver. This deficit in expectation was initially investigated by Pudari and Ernst ( Pudari and Ernst 2023 ) . They created a hierarchy of software abstractions inspired by Koopman’s Autonomous Vehicle Safety Hierarchy of Needs ( Koopman 2022 ) . Using such a hierarchy, they showed that Copilot ( Chen et al. 2021 ) could clear out low levels of abstractions, e.g., autocompleting syntactically correct code, but often struggled with deducing and suggesting the correct design patterns and architectural tactics for a given project. Similarly, Cao et al. ( Cao et al. 2024 ) demonstrated recent models such as WizardCoder ( Luo et al. 2023 ) and DeepSeek-coder 33 B ( AI 2023 ) struggled in programming scenarios involving object-oriented programming concepts, notably inheritance and encapsulation . Given these evidences, it is reasonable to assume that they may produce poorly structured software designs, leading to inflexible, hard-to-main systems. This could result in increased technical debt and higher maintenance costs. In this paper, we first start by providing a brief overview of the steps involved in pre-training and evaluating large language models of code. Then, we lay a vision for how these models can be further aligned with se practices. We propose the integration of higher-level design and architectural insights into the training process, with the aim to equip language models with a deeper understanding of software systems and with better reasoning capabilities. In addition, we propose the incorporation of other evaluation strategies to validate these models that extend beyond software correctness and other techniques for model understanding. 2. The Status Quo This section provides background knowledge by presenting current practices used to pre-train and evaluate llm s for code generation. 2.1. Pre-training of LLMs (of Code) 2.1.1. Pre-training objectives: llm s are typically trained on extensive text-based datasets such as The Pile ( Gao et al. 2020 ) and FineWeb ( Penedo et al. 2024 ) that include source code and question-answer discussions involving code snippets. Language models of code are trained specifically using source code ( Devlin 2018 ) . Pre-training objectives enable them to learn patterns, structures, and semantics of code to predict code-related properties and to generate code effectively. Common pre-training objectives include next-token prediction, used by models such as gpt ( Radford 2018 ) , where the model predicts the next word based on previous words, modeling language in a left-to-right manner. Models such as CodeT 5 ( Wang et al. 2021 ) , specialized for source code, extend these objectives. It consolidates pre-training tasks tailored for code understanding and generation into a single framework. Specifically, it combines masking strategies by masking code spans and identifiers and trains the model to predict them, enhancing its understanding of code syntax and semantics. It also includes dual-generation tasks between code and natural language, training on code-to-text ( e.g., generating documentation from code) and text-to-code ( e.g., generating code from natural language descriptions), to bridge the gap between code and human language. Recently, models such as CodeLlama ( Roziere et al. 2023 ) and StarCoder 2 ( Lozhkov et al. 2024 ) , are pre-trained using the fill-in-the-middle ( fim ) objective. In this objective, the model is trained to generate a missing code segment between a given prefix and suffix. With this, the model becomes adept at understanding bidirectional context and generates code that integrates with surrounding code. This enhances its capability in tasks like code completion, refactoring, and debugging. 2.1.2. Data representation: Another important aspect during pre-training is the input representation fed