AI Engineering Blueprint for On-Premises Retrieval-Augmented Generation Systems

1 st Nicolas Weeger, 2 nd Jakob Winkler, 3 rd Annika Stiehl, 4 th Jóakim von Kistowski, 5 th Christian Uhl, 6 th Stefan Geißelsöder · 2026

Read AI Engineering Blueprint for On-Premises Retrieval-Augmented Generation Systems by 1 st Nicolas Weeger, 2 nd Jakob Winkler, 3 rd Annika Stiehl, 4 th Jóakim von Kistowski, 5 th Christian Uhl, 6 th Stefan Geißelsöder (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-01395.txt.

Introduction

Weeger and colleagues offer a blueprint for on-premises retrieval-augmented generation systems in firms. Strict data rules bar many firms from cloud services, yet most blueprints focus on cloud setups and lack firm-grade parts. Their blueprint gives an end-to-end reference architecture in the 4+1 view model and a reference application for on-premises deployment. It also gives best practices for tooling, development and CI/CD pipelines, all on GitHub. Case studies and expert interviews with industry partners will assess its practical benefits.

Catalog record

CREATOR1 st Nicolas Weeger, 2 nd Jakob Winkler, 3 rd Annika Stiehl, 4 th Jóakim von Kistowski, 5 th Christian Uhl, 6 th Stefan Geißelsöder
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE14519
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.01395

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

AI Engineering Blueprint for On-Premises Retrieval-Augmented Generation Systems 1 st Nicolas Weeger, 2 nd Jakob Winkler, 3 rd Annika Stiehl, 4 th Jóakim von Kistowski, 5 th Christian Uhl, 6 th Stefan Geißelsöder Abstract Retrieval-augmented generation (RAG) systems are gaining traction in enterprise settings, yet stringent data protection regulations prevent many organizations from using cloud-based services, necessitating on-premises deployments. While existing blueprints and reference architectures focus on cloud deployments and lack enterprise-grade components, comprehensive on-premises implementation frameworks remain scarce. This paper aims to address this gap by presenting a comprehensive AI engineering blueprint for scalable on-premises enterprise RAG solutions. It is designed to address common challenges and streamline the integration of RAG into existing enterprise infrastructure. The blueprint provides: (1) an end-to-end reference architecture described using the 4+1 view model, (2) a reference application for on-premises deployment, and (3) best practices for tooling, development, and CI/CD pipelines, all publicly available on GitHub 1 1 1 https://github.com/aiengineeringblueprints/Enterprise_RAG_Blueprint . Ongoing case studies and expert interviews with industry partners will assess its practical benefits. I Introduction Retrieval-Augmented Generation (RAG) has emerged as a powerful technique for enhancing the capabilities of Large Language Models (LLMs) by integrating external knowledge sources [ 1 ] . In addition to the internal regulations and safety issues of companies [ 2 ] , stringent data protection regulations, such as the EU AI Act, the GDPR in Europe, and HIPAA in the healthcare sector, restrict the use of cloud-based LLM services for processing personal or sensitive data, as it involves transferring data outside of the company’s IT infrastructure [ 3 , 4 ] . Research initiatives, including the EU’s OpenEuroLLM [ 5 ] and the Helmholtz Foundation Model Initiative [ 6 ] , highlight the importance of LLMs that comply with these laws. Consequently, many companies opt for on-premises solutions to maintain oversight of their data and adhere to strict data security and compliance standards. Building and deploying RAG systems at an enterprise level poses significant challenges. A recent MIT study from 2025 [ 7 ] highlights this disparity, showing that, while custom AI tools are widely investigated in enterprises, only 5% are successfully deployed in production. A primary reason for this low success rate is the complexity that arises when scaling these systems and integrating them into existing enterprise infrastructure. Several studies emphasize the challenges of building and deploying RAG systems, particularly in enterprise settings [ 8 , 9 , 10 ] . These challenges include the need for specialized expertise in AI and data management, the complexity of integrating RAG systems with existing IT infrastructure, and the difficulty of ensuring data security and compliance with regulations. This research aims to address the challenges associated with building and deploying on-premises RAG systems by providing a simple yet comprehensive, end-to-end blueprint. The main contributions of this paper are: • An end-to-end reference architecture for on-premises enterprise RAG, formally described using the 4+1 view model, with explicit architectural trade-offs and variation points. • A deployable reference application that serves as an adaptable starting point for integrating RAG into existing enterprise infrastructure, including all enterprise-grade components. • Best practices for tooling, development, and CI/CD pipelines tailored to on-premises container deployment. II Related Work RAG systems represent an emerging area of research, with a growing body of literature exploring various aspects of RAG, including its theoretical foundations [ 1 ] , agentic RAG systems [ 11 , 12 ] , and other methods to improve the performance of the RAG system outputs [ 13 ] . The literature also covers evaluation metrics and frameworks [ 14 , 15 , 16 ] or the security of RAG systems [ 17 , 18 ] . The concept of RAGOps [ 19 ] has emerged from LLMOps [ 20 ] by integrating data lifecycle management as a core component. This paradigm characterizes the architecture of RAG applications and outlines the complete lifecycle of these composite systems using the 4+1 architectural view model. RAGOps defines design considerations and provides quality tradeoffs for the various stages of the RAG system lifecycle. In order to understand industry adoption, Brehme et al. [ 2 ] reveal key insights from 13 practitioners. The study indicates that contemporary RAG applications are frequently domain-specific question-answering systems that are predominantly in the prototype stage. The industry’s prevailing requirements place significant emphasis on data protection, security, and quality. Nevertheless, the preprocessing of data remains a considerable challenge, with evaluation being primarily conducted by human analysts. Since practical implementation guidance for RAG systems has not yet been deeply explored in peer-reviewed publications, grey literature provides valuable supplementary insights. Blog posts and white papers discuss, for example, common challenges [ 21 ] , specific RAG architectures [ 22 ] , and implementation patterns and anti-patterns [ 23 ] . However, the majority of these discussions focus on cloud-based RAG systems [ 24 ] , while only few address on-premises deployments [ 25 ] . A number of open-source RAG implementations are available on GitHub, including RAGFlow 2 2 2 RAGFlow, https://github.com/infiniflow/ragflow , accessed: Feb. 9, 2026 , kotaemon 3 3 3 kotaemon, https://github.com/Cinnamon/kotaemon , accessed: Feb. 9, 2026 , and the FELDM RAG Blueprint 4 4 4 RAG Blueprint, https://github.com/feld-m/rag_blueprint , accessed: Feb. 9, 2026 . However, these systems often lack the components necessary for enterprise-level scaleability. Despite this broad theoretical foundation, comprehensive end-to-end reference architectures with code-level implementation guidance remain scarce [ 26 ] — particularly for on-premises deployments, where data security and regulatory compliance are critical concerns. III Proposed Blueprint This blueprint developed in this study builds on the conceptual RAGOps framework [ 19 ] and translates it into a concrete, deployable architecture for on-premises environments. While RAGOps provides a paradigm for operating RAG systems, it remains at the conceptual level without offering implementation artifacts. Existing open-source RAG implementations provide functional applications but lack enterprise-grade components, such as access control, guardrails, and observability, and do not provide structured architectural documentation to support adaptation to existing enterprise infrastructure. This blueprint addresses both of these gaps by combining a formally described reference architecture with a deployable reference application and CI/CD pipelines. The architectural design follows the 4+1 view model by Kruchten [ 27 ] . Detailed views and components can be found in the provided GitHub repository. The accompanying reference application implements all architectural components as a generic, adaptable starting point for enterprise integration, including documentation and a CI/CD pipeline for on-premises container deployment. The blueprint focuses on the RAG application layer. The deployment of LLMs and embedding models, while a pivotal prerequisite, is regarded as a distinct architectural component addressed in separate research [ 20 ] . III-A Requirements The architectural design is informed by the requirements identified in the extant literature. Bruckhaus [ 10 ] emphasizes data protection and security, accuracy and explainability, as well as seamless enterprise integration and scalability. Xu et al. [ 19 ] focus on operational aspects, highlighting monitoring and observability, adaptability, traceability, and reliability. Brehme et al. [ 2 ] confirm and extend these findings through interviews with 13 industry practitioners, identifying additional requirements such as continuous operation and explainability. We adopt the requirements identified by Brehme et al. as our primary basis. However, some requirements, such as usability

Read the complete text