TVS Sidekick: Challenges and Practical Insights from Deploying Large Language Models in the Enterprise

Paula Reyero Lobo, Kevin Johnson, Bill Buchanan, Matthew Shardlow, Ashley Williams, Samuel Attwood · 2025

Read TVS Sidekick: Challenges and Practical Insights from Deploying Large Language Models in the Enterprise by Paula Reyero Lobo, Kevin Johnson, Bill Buchanan, Matthew Shardlow, Ashley Williams, Samuel Attwood (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-26482.txt.

Introduction

Paula Reyero Lobo and co-authors report lessons from deploying a large language model assistant inside a supply chain firm in a 2025 paper. Many firms are adopting AI to speed internal work, while new rules press them to use it responsibly. Governance frameworks exist on paper, but shared practical infrastructure lags behind, which blocks adoption.

The paper describes the assistant, called TVS Sidekick, built for TVS Supply Chain Solutions. It recounts the ethical, regulatory, and social-technical hurdles met during development and rollout for enterprise use. The account covers how the team handled risk and compliance while shipping a working tool. The authors offer the case as practical guidance for other firms facing the same gap between AI promise and governed practice.

Catalog record

CREATORPaula Reyero Lobo, Kevin Johnson, Bill Buchanan, Matthew Shardlow, Ashley Williams, Samuel Attwood
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-26482

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

TVS Sidekick: Challenges and Practical Insights from Deploying Large Language Models in the Enterprise Paula Reyero Lobo, Kevin Johnson, Bill Buchanan, Matthew Shardlow, Ashley Williams, Samuel Attwood Abstract Many enterprises are increasingly adopting Artificial Intelligence (AI) to make internal processes more competitive and efficient. In response to public concern and new regulations for the ethical and responsible use of AI, implementing AI governance frameworks could help to integrate AI within organisations and mitigate associated risks. However, the rapid technological advances and lack of shared ethical AI infrastructures creates barriers to their practical adoption in businesses. This paper presents a real-world AI application at TVS Supply Chain Solutions, reporting on the experience developing an AI assistant underpinned by large language models and the ethical, regulatory, and sociotechnical challenges in deployment for enterprise use. 1 Introduction Recent developments are driving industry interest in the field of Large Language Models (LLMs). Key developments of note are the abundant availability of commercial language modelling solutions Devlin et al. 2019 ; Brown et al. 2020 ; Thoppilan et al. 2022 and the increased public awareness of the capabilities of LLMs Mialon et al. 2023 ; Qu et al. 2025 . However, to successfully utilise these models, organisations must navigate important societal challenges related to ethics, sustainability, and compliance Hagendorff 2024 ; Laux et al. 2024 . TVS SCS UK is a top-tier third-party logistics (3PL) provider in Europe and the UK, offering comprehensive supply chain solutions. 3PL customers increasingly adopt intelligent technology-led solutions to optimise their supply chain operations and reduce costs Pournader et al. 2021 ; Li et al. 2023 . To stay ahead of the competition, TVS SCS UK are leveraging LLMs to create a competitive advantage and enhance their internal operational efficiency. TVS SCS UK has decided not to use third-party software integrators or product vendors for its solutions, which would negatively impact their agility and innovation. Instead, they have started their journey towards an AI transformation through an in-house AI team. Figure 1: Overview of TVS Sidekick, an AI assistant that leverages LLMs to answer queries with relevant enterprise data using retrieval augmented generation (RAG) via a Microsoft Teams extension. TVS Sidekick is the flagship product of this in-house team. TVS Sidekick is built upon the principles of Retrieval Augmented Generation (RAG) Lewis et al. 2020 . All relevant company documents, as available via their internal cloud-based systems, are vectorised and compared to the input query, with the LLM then performing information extraction for the purposes of question answering with custom prompting Qu et al. 2025 . Users interact with TVS Sidekick via a Microsoft Teams extension (Figure 1 ). As TVS SCS UK advances its AI transformation through the development of Sidekick, it must also navigate a complex legal and regulatory landscape. At the centre of this landscape are the European Union Artificial Intelligence Act (EU AIA) European Commission 2014 and related standards, such as ISO/IEC 42001 for AI Management Systems International Organization for Standardization. 2023 . Furthermore, TVS SCS UK must overcome a range of sociotechnical challenges that accompany the deployment of LLMs, such as issues of fairness, transparency, and accountability Crockett et al. 2023 ; Ojewale et al. 2025 , which limit their practical adoption in enterprise environments. In this paper, we report on TVS SCS UK’s experience developing Sidekick, navigating the relevant legislation and regulations, and overcoming the challenges they have encountered along the way. 1.1 Significance of this Study This study presents practical insights from applied AI research in a real-world business context. To be specific, we contribute to the field in three ways: • Technical Contributions. We describe the design and implementation of Sidekick, an AI assistant underpinned by LLMs that is tailored for enterprise use, including novel approaches to prompt engineering and RAG. • Regulatory Contributions. We present a case study of how a business is aligning its development with emerging legislation and regulations, most notably the EU AIA, by working towards harmonised technical standards (e.g., ISO/IEC 42001). • Sociotechnical Contributions. We explore the sociotechnical challenges that accompany the deployment of LLMs in enterprise environments. We report quantitative statistics relating to the adoption of Sidekick alongside a qualitative analysis of end-user feedback. 1.2 Structure of this Study The remainder of this paper is organized as follows. Section 2 reviews the relevant literature. Section 3 details the technical implementation of Sidekick. Section 4 presents a case study of how TVS SCS UK is aligning its development with emerging legislation and regulations. Section 5 includes a quantitative and qualitative evaluation of the progress to date. Finally, section 6 concludes this paper and describes directions for future work. 2 Related Work 2.1 LLMs in the Enterprise Prominent applications, reviewing strategies to augment LLM capabilities. The transformer architecture enhanced language modelling capabilities and has since sparked great attention in industry Vaswani et al. 2017 . This led to many readily available pre-trained models, which proved their superiority in fine-tuning applications Devlin et al. 2019 . With increased data size and model complexity, decoder-only models like the generative pre-trained transformer (GPT) model series have become more attractive for industry due to their few/zero-shot performance Brown et al. 2020 . This paradigm shift led to methods for aligning to user intent Ouyang et al. 2022 (like reinforcement learning with human feedback) powering popular conversation-focused products like ChatGPT. While these scaled-up models offer business value (e.g. analysing vast data in real-time), issues such as the closed-source nature of existing solutions creates barriers to organisations lacking computational power Yang et al. 2024 . Focus on approaches including RAG (and pipeline parts showing improvement). Recently, the focus has turned into giving more agency to LLMs to become independent problem solvers. For instance, by consulting with external knowledge sources for factual grounding Lewis et al. 2020 ; Thoppilan et al. 2022 . More broadly, a significant step forward is the combination of “tools”, namely tool-augmented LLMs Mialon et al. 2023 , including retrieval-augmented language models for efficiently handling new data. Such approaches generally consist of four stages: task planning (i.e. break down user query into tasks), tool selection, tool calling, and response generation Qu et al. 2025 . Similarly, critical advances require frameworks for enabling LLMs to recall previous interactions Zhang et al. 2024 , allowing for multimodal data processing Sun et al. 2025 ; Song et al. 2025 , or to improve responses based on past interactions Wang et al. 2024 . This paper presents a case study of recent LLM developments in practice, specifically through the technical implementation of an AI assistant that processes heterogeneous enterprise data sources using knowledge augmentation strategies, including novel approaches to prompt engineering and RAG. 2.2 Responsible and Ethical AI Principles Requirements Human oversight & accountability AI to support/augment humans, with humans clearly accountable. Technical robustness and safety AI tools work as expected, minimising potential harms. Transparency Clear notification of AI involvement, clear and traceable outputs. Privacy & data governance Follow existing privacy rules with quality, robust data. Diversity & fairness Output free of bias and does not discriminate or treat unfairly. Social & environmental wellbeing AI is sustainable and beneficial to all. Table 1: Key emerging principles and requirements from global AI regulations British Standards Institution

Read the complete text