From Idea to Implementation: Evaluating the Influence of Large Language Models in Software Development - An Opinion Paper

Sargam Yadav, Asifa Mehmood Qureshi, Abhishek Kaushik, Shubham Sharma, Roisin Loughran, Subramaniam Kazhuparambil, Andrew Shaw, Mohammed Sabry, Niamh St John Lynch, Nikhil Singh, Padraic O’Hara, Pranay Jaiswal, Roshan Chandru, David Lillis · 2025

Read From Idea to Implementation: Evaluating the Influence of Large Language Models in Software Development - An Opinion Paper by Sargam Yadav, Asifa Mehmood Qureshi, Abhishek Kaushik, Shubham Sharma, Roisin Loughran, Subramaniam Kazhuparambil, Andrew Shaw, Mohammed Sabry, Niamh St John Lynch, Nikhil Singh, Padraic O’Hara, Pranay Jaiswal, Roshan Chandru, David Lillis (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-07450.txt.

Introduction

Sargam Yadav and a large team of co-authors present a 2025 opinion paper on large language models in software work. Tools like ChatGPT have moved into coding, debugging, and documentation tasks. The authors gathered views from 11 experts to judge where these models help and where they pose risks.

The experts were broadly positive about the tools. They praised gains in productivity and help across the path from idea to finished code. At the same time, they weighed concerns that come with handing parts of the craft to a model. Their shared views are distilled into guidance for teams. The paper aims to help firms adopt the models in a way that is both useful and responsible.

Catalog record

CREATORSargam Yadav, Asifa Mehmood Qureshi, Abhishek Kaushik, Shubham Sharma, Roisin Loughran, Subramaniam Kazhuparambil, Andrew Shaw, Mohammed Sabry, Niamh St John Lynch, Nikhil Singh, Padraic O’Hara, Pranay Jaiswal, Roshan Chandru, David Lillis
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-07450

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

From Idea to Implementation: Evaluating the Influence of Large Language Models in Software Development - An Opinion Paper Sargam Yadav, Asifa Mehmood Qureshi, Abhishek Kaushik, Shubham Sharma, Roisin Loughran, Subramaniam Kazhuparambil, Andrew Shaw, Mohammed Sabry, Niamh St John Lynch, Nikhil Singh, Padraic O’Hara, Pranay Jaiswal, Roshan Chandru, David Lillis Abstract The introduction of transformer architecture was a turning point in Natural Language Processing (NLP). Models based on the transformer architecture such as Bidirectional Encoder Representations from Transformers (BERT) and Generative Pre-Trained Transformer (GPT) have gained widespread popularity in various applications such as software development and education. The availability of Large Language Models (LLMs) such as ChatGPT and Bard to the general public has showcased the tremendous potential of these models and encouraged their integration into various domains such as software development for tasks such as code generation, debugging, and documentation generation. In this study, opinions from 11 experts regarding their experience with LLMs for software development have been gathered and analysed to draw insights that can guide successful and responsible integration. The overall opinion of the experts is positive, with the experts identifying advantages such as increase in productivity and reduced coding time. Potential concerns and challenges such as risk of over-dependence and ethical considerations have also been highlighted. K eywords Large language models; software development; coding; ChatGPT; transformer; natural language generation 1 Introduction The swift evolution of artificial intelligence in the past decade has allowed advanced Natural Language Processing (NLP) tools to perform tasks such as translation, summarization, and information retrieval. These tools are becoming more easily available and integrate seamlessly with our daily lives in the form of virtual assistants. Large Language Models (LLMs) are large and complex deep learning models with millions or billions of parameters trained on vast amounts of data. Models such as ChatGPT can carry out human-like conversations keeping context in mind, highlighting their advanced Natural Language Understanding (NLU) ( Brown et al. 2020 ) and Natural Language Generation (NLG) ( Dong et al. 2022 ) capabilities. Transformer-based models such as the Bidirectional Encoder Representations from Transformers (BERT) ( Devlin et al. 2018 ) and Generative Pre-Trained Transformer (GPT) ( Radford et al. 2018 ) have demonstrated unparalleled capabilities in tasks such as text generation ( Luo et al. 2022 ) , named entity recognition ( Liu et al. 2021 ) , machine translation ( Clinchant et al. 2019 ) , text classification ( Yadav and Kaushik 2023b ; Kaur et al. 2019 ; Yadav et al. 2022 ; Shah et al. 2020 ) , and more. LLMs are pre-trained on large amounts of data and can gain considerable amount of domain knowledge, which allows them to be a valuable assistant in fields such as software development ( Hörnemalm 2023 ) healthcare ( Biswas 2023 ) education ( Adeshola and Adepoju 2023 ; Kaushik et al. 2025 ) , and agriculture ( Kaushik et al. 2024 ; Yadav et al. 2023 ; Yadav and Kaushik 2023a ; Kovvuri et al. 2023 ) . Since their launch, LLMs such as GPT-4 ( Achiam et al. 2023 ) , Codex ( Chen et al. 2021 ) ,and CodeBERT ( Feng et al. 2020 ) , have been utilised by researchers and practitioners to efficiently perform various tasks in the software development workflow such as code completion ( Raychev et al. 2014 ) , code synthesis from natural language ( Desai et al. 2016 ) , debugging, code review ( Zhao et al. 2023 ) , code comment and documentation generation ( Pu et al. 2023 ) , among others. In the context of software development, LLMs can greatly reduce workload for developers, allowing them to focus on more creative aspects of the coding process ( Eirini Kalliamvakou 2022 ) . They can streamline the development process by ensuring compliance and version control, improving code quality, and promoting collaboration ( Zhang et al. 2024 ) . They can also be leveraged to develop interactive educational tools that can provide tailored feedback to programming students Lyu et al. 2024 Liffiton et al. 2023 . Table 1 displays a list of the commonly used LLMs for software development, along with their parameters, training data, training language, and architecture. Model Parameters Training data Language Architecture CodeParrot 1.5b GitHub Python GPT-2 GPT-Neo 2.7b Pile dataset ( Gao et al. 2020 ) – GPT-2 based GPT-NeoX 20b Pile dataset – GPT-3 GPT-J 6b Pile dataset – GPT-3 Codex 12b GitHub, + Python, + GPT-3 CodeBERT 125M GitHub Multiple BERT CuBERT – GitHub Python BERT PolyCoder 2.7 b GitHub 12 languages GPT-2 Table 1: Large Language Models for Software Development However, there have also been certain risks and limitations associated with using LLMs. LLMs may not fully understand the context of a prompt ( Deng et al. 2023 ) , leading to erroneous responses. Over-reliance on LLMs to perform tasks may inhibit problem solving and critical thinking skills. There are also several ethical considerations of using LLMs for software development such as unintended biases Huang et al. 2023 , security risks Yao et al. 2024 , transparency, and explainability ( Liyanage and Ranaweera 2023 ) . LLMs are trained on vast amounts of data which contains human-like biases ( Tjuatja et al. 2023 ) . The models can reflect these biases in their responses and cause further harm. It is important to mitigate biases and address the concerns to ensure fairness in the model and its related outputs. In this study, a group of software development experts, consisting of researchers and practitioners, was surveyed to understand the potential applications of LLMs in software development. Analysis of opinions obtained through a loosely structured written interview from 11 experts, consisting of researchers and practitioners, has also been performed to draw insights about the potential of LLMs. The rest of the article is structured as follows: In Section 2 , we discuss the motivation behind conducting this survey, as well as define the hypothesis and research questions. Section 3 details the methodology used in the study, Section 4 describes the opinions of experts regarding the integration of LLMs in processes such as code generation, review, debugging, and so on. Section 5 provides an overview and analysis of the expert opinions, as well as a discussion of the research questions. Section 6 concludes the review study. 2 Motivation LLMs such as GPT, CodeBERT, and Codex have demonstrated transformative potential in software development. The hype surrounding LLMs creates ambiguity regarding their potential applications and drawbacks. There is some concern about job security, as it is perceived by some that models will become better at performing software development tasks than professional developers Kuhail et al. 2024 . Although LLMs can be an invaluable tool for learning, students may also use it unfairly and compromise academic integrity Perkins 2023 Pudasaini et al. 2024 . Additionally, as with any new technology, there may be a steep learning curve before it can be seamlessly integrated Kirova et al. 2024 . In order to get a clear picture of the benefits and drawbacks of LLMs in the field of software development, it is important to analyse the opinions of experts working in the field. The motivation behind conducting this study is to analyse

Read the complete text