Docker Does Not Guarantee Reproducibility
Julien Malka, Stefano Zacchiroli, Théo Zimmermann · 2026
Read Docker Does Not Guarantee Reproducibility by Julien Malka, Stefano Zacchiroli, Théo Zimmermann (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-12811.txt.
Introduction
Julien Malka, Stefano Zacchiroli, and Theo Zimmermann warn that Docker does not by itself make work reproducible. Teams in software work need to rerun code and get the same result. That need touches joint work, supply chain safety, and science itself. Docker packs a full software setup into an image, a saved copy of a file system that can be shared.
A saved copy feels like proof, but it is not proof. Images can change, lean on live services, or hide steps no one wrote down. The team spell out where the promise of containers breaks. Their point is not to drop Docker, but to stop treating it as a guarantee. Stronger habits must back it up.
Catalog record
| CREATOR | Julien Malka, Stefano Zacchiroli, Théo Zimmermann |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 77 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-12811
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Docker Does Not Guarantee Reproducibility Julien Malka, Stefano Zacchiroli, Théo Zimmermann Abstract . The reproducibility of software environments is a critical concern in modern software engineering, with ramifications ranging from the effectiveness of collaboration workflows to software supply chain security and scientific reproducibility. Containerization technologies like Docker address this problem by encapsulating software environments into shareable filesystem snapshots known as images. While Docker is frequently cited in the literature as a tool that enables reproducibility in theory, the extent of its guarantees and limitations in practice remains under-explored. In this work, we address this gap through two complementary approaches. First, we conduct a systematic literature review to examine how Docker is framed in scientific discourse on reproducibility and to identify documented best practices for writing Dockerfiles enabling reproducible image building. Then, we perform a large-scale empirical study of 5298 Docker builds collected from GitHub workflows. By rebuilding these images and comparing the results with their historical counterparts, we assess the real reproducibility of Docker images and evaluate the effectiveness of the best practices identified in the literature. 1. Introduction The reproducibility of build environments ( 8 ; 23 ) —i.e., the ability to recreate the technical environment needed to develop and deploy a specific piece of software—is a fundamental aspect of modern software engineering, with significant implications for both developers and users. For developers, build environment reproducibility enables reliable recompilation of software from source code—even years after its initial creation—facilitating collaboration, long-term maintenance, and debugging. For end users, the ability to rebuild software from source can provide security guarantees, as it avoids the need of trusting opaque binaries built by third parties ( 21 ) . Yet, the growing complexity of software components and their dependencies ( 9 ) , driven by composability and modular design, makes reproducing exact build environments increasingly challenging. This issue, which contributes to the broader reproducibility crisis ( 16 ; 34 ) issue, is well-documented in scientific computing ( 19 ; 43 ) , where re-running old code to validate or extend prior work is an integral part of the methodology. Yet the problem extends beyond academia and touches all fields of software engineering. Containerization technologies ( 2 ) allow software environments to be encapsulated and packaged as filesystem snapshots known as container images , usually in the OCI standard format ( 31 ) , which can then be shared and executed on other systems. Docker ( 10 ) is a widely adopted open-source container runtime and build tool that popularized the use of build recipes called Dockerfiles to create container images. The straightforward syntax and developer-friendly tooling of Docker contributed to its wide adoption across the open source ecosystem, to the point that a 2020 study mining Dockerfiles from the World of Code archive found that over 1.9 million distinct repositories used them ( 12 ) . Docker supports a broad range of workflows in software engineering—including building, packaging, distributing, and deploying software—by making software environments transportable across systems. Thanks to this portability aspect, Docker is frequently recommended in both industry and academia for improving the reproducibility of computational environments ( 30 ) . However, Docker images are build artifacts, which suffer from common limitations for that type of software artifacts: • they are large, binary artifacts that must be explicitly stored and archived, to avoid becoming unavailable over time ( 33 ) ; • they are typically tied to specific hardware architectures, limiting their portability ( 3 ) ; • they contain arbitrary binary artifacts that are difficult to audit and can represent a security risk ( 32 ; 28 ) ; • modifying them to upgrade contained artifacts or change their behavior is impractical ( 32 ) . To address these limitations, it is essential to be able to reliably rebuild a Docker image from its Dockerfile , ensuring that the rebuilt image matches the original one. Different meanings of “matches” can be considered for this need. When images can be rebuilt in a way that produces bitwise-identical copies (as in the reproducible builds approach ( 21 ) ), trusting third-party distributors like public image registries become easier, strengthening the security of the software supply chain. Even when bitwise reproduction is not achievable, functional equivalence —where the rebuilt image contains the same software components, at the same versions, of the original image—remains an essential requirement. Failing it, discrepancies in image content may lead to observable differences when using the image. Research questions In this work, we study Docker as a tool to enable the reproducibility of software environments, with a particular focus on the gap between theory and practice : what is Docker believed to do in that respect versus what it actually does. Specifically, we address the following research questions: RQ1: What are the claims and recommendations in the scientific literature on the reproducibility of Docker images? RQ2: To what extent can Docker images built in the past be reproduced today from their Dockerfiles, to various degrees of equivalence (bitwise identical, functionally equivalent)? RQ3: Are the recommendations present in the scientific literature about Docker image reproducibility effective? Contributions We make two primary contributions: (1) We perform a systematic literature review (SLR) of scientific literature to identify how Docker is discussed in the context of reproducibility and map documented best practices for writing Dockerfiles that, allegedly, enable image reproducibility (RQ1). (2) We conduct a large-scale empirical study of Docker image reproducibility . Leveraging a dataset of GitHub workflow runs, we reproduce and analyze 5298 historical Docker builds captured in October 2023. By comparing rebuilt images with their historic counterparts, we quantify the level of reproducibility of real-world Docker builds (RQ2). Then we empirically assess how the best practices for Docker reproducibility documented in the literature are used by developers, and whether they lead to improved reproducibility (RQ3). Data availability statement: We make our code required to run our analysis available in the replication package ( 26 ) , which we permanently archive on Software Heritage. 2. Background 2.1. Software reproducibility This article is concerned with the notion of reproducibility in software engineering. Reproducibility can take various meanings, and in this work, we explore several of them. The strictest notion of reproducibility is bitwise reproducibility , a.k.a. reproducible builds . It is defined as the ability to reproduce the exact same binary artifact (bitwise identical) from a fixed source code and build environment. This definition has the advantage of being easy to verify, and it is thus useful both from a software supply chain security perspective ( 21 ) , and from a scientific perspective ( 45 ) , to show that the source code accompanying a scientific publication can be used to reproduce the results of the publication. From a software engineering perspective, however, more relaxed definitions of reproducibility are just as useful. For instance, functional reproducibility focuses on the behavior of the software rather than its exact binary representation. Therefore, when comparing two environments, we will consider them functionally equivalent if they contain the same software components, at the same versions. What to consider as “same versions” can then be discussed. If we want bug-by-bug reproducibility, we should use strictly the same versions. But if we are only interested in compatibility,