A Virtual Laboratory for Managing Computational Experiments Conference: ; ;

Eleni Adamidi, Panayiotis Deligiannis, Nikos Foutris, Thanasis Vergoulis · 2025

Read A Virtual Laboratory for Managing Computational Experiments Conference: ; ; by Eleni Adamidi, Panayiotis Deligiannis, Nikos Foutris, Thanasis Vergoulis (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-01197.txt.

Introduction

Eleni Adamidi and co-authors present a virtual laboratory in 2025 for running and recording computer-based experiments. As such experiments grow in scale, keeping them repeatable gets hard, because settings and scores scatter across files and runs. Their platform, called SCHEMA lab, captures rich records of each run, from settings to performance numbers.

Researchers group single runs into experiments and manage them across their whole life cycle. The paper walks through the platform's design, core features, and build. The authors argue that tidy records of this kind can lift both repeatability and efficiency in computational research, letting other teams rerun work exactly and trust the results more.

Catalog record

CREATOREleni Adamidi, Panayiotis Deligiannis, Nikos Foutris, Thanasis Vergoulis
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-01197

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Virtual Laboratory for Managing Computational Experiments Conference: ; ; Eleni Adamidi, Panayiotis Deligiannis, Nikos Foutris, Thanasis Vergoulis Abstract . Computational experiments have become essential for scientific discovery, allowing researchers to test hypotheses, analyze complex datasets, and validate findings. However, as computational experiments grow in scale and complexity, ensuring reproducibility and managing detailed metadata becomes increasingly challenging, especially when orchestrating complex sequence of computational tasks. To address these challenges we have developed a virtual laboratory called SCHEMA lab, focusing on capturing rich metadata such as experiment configurations and performance metrics, to support computational reproducibility. SCHEMA lab enables researchers to create experiments by grouping together multiple executions and manage them throughout their life cycle. In this demonstration paper, we present the SCHEMA lab architecture, core functionalities, and implementation, emphasizing its potential to significantly enhance reproducibility and efficiency in computational research. 1. Introduction Computational experiments are essential for advancing scientific discovery, enabling researchers to test hypotheses, analyze data, and simulate complex systems. The growing complexity of these experiments, however, presents challenges related to reproducibility and metadata management. Precise documentation and structured execution environments have become essential to ensuring experiments can be consistently reproduced and verified across diverse computing infrastructures. As highlighted by Sandve et al. ( Sandve et al. 2013 ) and Peng ( Peng 2011 ) , computational reproducibility demands meticulous management of workflows, configurations, and performance data. Managing the increasing scale and complexity of scientific workflows requires advanced tools capable of automating and tracking numerous computational tasks simultaneously. Researchers frequently face difficulties in reliably reproducing experiments due to inconsistencies in execution environments, incomplete metadata documentation, and lack of structured management practices. Such challenges significantly hinder scientific transparency, verification, and collaboration. Containerization technologies have fundamentally transformed computational research by offering consistent and portable execution environments. Containerization, particularly through technologies such as Docker ( Boettiger 2015 ) has enabled researchers to encapsulate applications along with their dependencies, thereby simplifying deployment across diverse systems and mitigating issues related to software configuration. Moreover, existing systems like Galaxy ( Afgan et al. 2016 ) and Nextflow ( Di Tommaso et al. 2017 ) have made significant strides in automating scientific workflows. While these advances facilitate reliable execution of computational tasks and workflows, environments that enable researchers to create and holistically manage complex computational experiments-comprising of multiple containerized tasks, rich metadata capture, and reproducibility considerations-are needed. SCHEMA lab 1 1 1 SCHEMA lab: https://github.com/athenarc/schema-lab , addresses this gap by providing an open source virtual laboratory that empowers researchers to create, manage, and monitor containerized computational experiments with ease. Building upon previous work ( Vergoulis et al. 2021 ) , SCHEMA lab allows users to submit and track the execution of both individual tasks and complex workflows while capturing essential metadata for performance analysis and reproducibility. In this demonstration paper, we present the system architecture, core functionalities, and implementation of SCHEMA lab, illustrating its potential for enhancing computational reproducibility and experiment management. 2. Background and Related work Over the past decade, significant advances have been made in container orchestration, workflow management, and reproducibility practices within computational research. Many platforms—such as Galaxy ( Afgan et al. 2016 ) , Snakemake ( Köster and Rahmann 2012 ) , and Nextflow ( Di Tommaso et al. 2017 ) —have been developed to coordinate multi-step scientific workflows with varying degrees of automation and user support. These systems excel in orchestrating interdependent tasks, yet researchers may benefit from additional flexibility in managing diverse computational workloads, particularly when systematically capturing detailed metadata to enhance reproducibility and experiment documentation. SCHEMA lab addresses this need by providing a flexible, lightweight environment for managing containerized computational tasks. It supports the rapid execution of standalone tasks—ideal for quick tests and prototyping—while also enabling users to dynamically group tasks into larger experiments. When tasks are run as part of a grouped experiment, SCHEMA lab aggregates detailed information on experiment configuration, performance metrics, and resource consumption, thereby facilitating easier documentation and reproducibility. In contrast, executing standalone tasks focuses on rapid iteration and debugging. Reproducibility remains a central theme in computational research. Prior studies (e.g., Sandve et al. ( Sandve et al. 2013 ) and Peng ( Peng 2011 ) ) have underscored the importance of meticulously documenting computational processes to enable the exact replication of results. In this context, recent advances in data packaging standards—such as the RO-Crate framework ( Sefton et al. 2020 ) —offer promising strategies for encapsulating experimental data and metadata in a standardized, shareable format. By aligning with established paradigms in containerization, workflow management, and reproducibility, SCHEMA lab is positioned as a tool that not only executes tasks efficiently but also adapts to the evolving needs of researchers—whether they require rapid prototyping or comprehensive experiment management. 3. System Overview The architecture of SCHEMA lab is designed to provide a seamless and flexible environment for managing containerized computational experiments. The system is divided into two primary components: the SCHEMA lab front-end and the SCHEMA api back-end. Together, these components enable researchers to submit, execute, monitor, and manage containerized tasks and computational experiments efficiently. The following subsections describe the overall architecture, the front-end functionalities, the back-end services, and the integration between these components. 3.1. High-Level Architecture Our platform supports the submission and monitoring of containerized task execution requests. Its purpose is to act as a gateway between users and the task execution environment, performing necessary authentication and authorization checks, recording submitted task requests, and scheduling valid, authorized tasks for execution via exposed RESTful endpoints. Under the hood SCHEMA api works over a Kubernetes cluster that runs tasks shipped in Docker containers. Rather than communicating directly with Kubernetes, SCHEMA api schedules task executions through TESK—an implementation of the Task Execution Service (TES) API ( GA4GH 2018 ) developed under the Global Alliance for Genomics and Health (GA4GH) ( Global Alliance for Genomics and Health 2013 ) . Consequently, SCHEMA api requires an operational deployment of TESK ( EMBL-EBI 2019 ) that is accessible via HTTP. On the front-end, SCHEMA lab communicates with SCHEMA api using RESTful calls, ensuring that user actions (such as task or workflow submission and experiment creation) are reliably forwarded to the back-end. The resulting execution status and metadata are then pushed back to SCHEMA lab, enabling real-time monitoring and control of computations. A high-level diagram of this architecture and its dependencies is depicted in Figure 1 . In essence, SCHEMA api preserves the logical abstraction of tasks, contexts, and experiments, while TESK handles the low-level container scheduling on Kubernetes. Figure 1. High-Level Architecture of SCHEMA api. This diagram illustrates how SCHEMA api acts as a gateway between users and the containerized execution environment by performing authentication, scheduling containerized tasks via TESK, and providing monitoring endpoints. A diagram showing the high-level architecture of SCHEMA api, including its interaction with TESK for task scheduling on a Kubernetes cluster. 3.2. Back-End: SCHEMA api SCHEMA api is the engine that powers the admission and scheduling of containerized tasks. At its core, it is designed to orchestrate the execution of containerized software, available in any reachable registry, as well as passing to the execution input files and storing produced output files. Within the SCHEMA api ecosystem, several resources are used to achieve this functionality: • Tasks: single-job containerized executions

Read the complete text