Towards dimensions and granularity in a unified workflow and data provenance framework

Tanja Auge, Sascha Genehr, Meike Klettke, Frank Krüger, Max Schröder · 2025

Read Towards dimensions and granularity in a unified workflow and data provenance framework by Tanja Auge, Sascha Genehr, Meike Klettke, Frank Krüger, Max Schröder (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-11278.txt.

Introduction

Tanja Auge and co-authors work toward a unified framework for workflow and data provenance in a 2025 paper. Provenance records where data and results came from, which is essential for tracing studies and trusting their findings. Two kinds matter: how the workflow ran, and how the data themselves evolved.

Their framework adds dimensions and granularity to both kinds, letting users answer a standard set of provenance questions. A biomedical use case demonstrates the approach, designed to transfer to other fields. Unifying the concepts, the authors argue, supports credibility and reproducibility of research findings.

Catalog record

CREATORTanja Auge, Sascha Genehr, Meike Klettke, Frank Krüger, Max Schröder
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE19 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-11278

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Towards dimensions and granularity in a unified workflow and data provenance framework Tanja Auge, Sascha Genehr, Meike Klettke, Frank Krüger, Max Schröder Abstract Provenance information are essential for the traceability of scientific studies or experiments and thus crucial for ensuring the credibility and reproducibility of research findings. This paper discusses a comprehensive provenance framework combining the two types 1. workflow provenance , and 2. data provenance as well as their dimensions and granularity, which enables the answering of W7+1 provenance questions. We demonstrate the applicability by employing a biomedical research use case, that can be easily transferred into other scientific fields. An integration of these concepts into a unified framework enables credibility and reproducibility of the research findings. \tnotemark [1] \tnotetext [1]This is a preprint of a workshop paper, published at LWDA 2024. keywords data provenance \sep workflow provenance \sep dimensions \sep granularity \sep W7 questions \sep wetlab data 1 Introduction The traceability of scientific studies and experiments is essential for the credibility and reproducibility of their findings. Provenance information provide the essential data for this purpose, e.g. the sequence of activities that resulted in the creation of measurement data, the involved persons or the investigated samples. Thereby, provenance can have a significant impact on the scientific value of the research data. Many approaches and definitions of provenance in different contexts have been proposed that cover some aspects of either the scientific domain or the provenance information itself, e.g. workflow systems [ 15 , 14 ] , jupyter notebooks [ 13 ] , and lab documentation [ 12 , 11 ] . In this paper, we discuss a unification of these definitions along typical biomedical research processes, which can be easily adapted to other scientific domains. Scientific investigations in the biomedical domain can be categorized by their kind into: 1. in-silico , 2. in-vitro , and 3. in-vivo experiments . Often, in-silico studies are performed to simulate real world phenomena based on data obtained from previous in-vitro or in-vivo experiments, i.e. experiments in laboratories respectively living organisms. New findings from the in-silico studies are then again used to validate the effects in in-vitro or in-vivo experiments, leading to a closed loop of data exchange in this scientific process with respect to the experimental kind. For the discussion of the particular provenance concepts, we employ the following use case, that we restrict on the two kinds, in-silico and in-vitro, as the challenges and requirements for provenance tracking in in-vitro and in-vivo are quite similar. Use case. ​ A researcher starts with in-vitro experiments in a wetlab environment measuring the calcium ion mobilization in osteoblasts. In order to measure this real world phenomenon, a series of actions is performed in the laboratory environment. The course of action is standardized in a protocol and documented in a so-called electronic lab notebook (ELN ). In particular, this involves the preparation of cell environments as well as the set-up and measuring of the prepared cells using a microscope in conjunction with a stimulation device. Measurement data are collected as microscopy images that are analysed to extract tabular fluorescence intensity data. In order to exclude environmental and other issues, the experiment is repeated multiple times, resulting in a series of measurements as well as documentation following the same experimental protocol. In a second step, an in-silico study is set up to further elaborate the stimulation parameter settings based on the real world data from the in-vitro experiments. The measurements are used to specify and parameterize a simulation model for this phenomenon. Specifically, the simulation model in combination with the computing environment is run under different simulation settings. The simulation is repeated in order to reduce random influences. New findings are then interpreted from all simulations, which provide insights for the optimization of stimulations. Lastly, these findings are validated in in-vitro experiments again (cf. Figure 1 ). In this entire sequence of experiments, provenance information is needed to support the credibility and reproducibility of the research findings. As such, activities, involved persons and entities (e.g. data, samples, devices and models) are encoded in the provenance information. Contribution. ​ 1. We discuss a unification of workflow and data provenance including the dimension and granularity based on the current state of the art, and 2. we extend the W7 provenance question concept [ 17 ] to W7+1 provenance questions and discuss dimensions and granularity for this unification. Figure 1: Biomedical use case with two experiment types (in-vitro and in-silico) whose research findings are used to validate and optimize the other. Activities are numbered to illustrate the trajectory. 2 Provenance Concepts Provenance is a very broad term with many meanings and definitions. In general, provenance refers to the origin of an object or captures information about the creation or evolution process of an object — in our example these objects are simulation models and measurement data. With respect to the (scientific) workflow , provenance information encode the tracing of the particular research objects resulting in the scientific finding. In the context of a (scientific) database [ 2 ] , provenance includes information about the origin of a data element and details about its scientific processing (parameters, software versions, etc.). Thus, provenance information enable the verification of (scientific) processes as well as database queries [ 6 , 3 , 7 ] . 2.1 Provenance Types Workflow provenance. ​This type refers to the process of a dataset’s derivation in the form of a scientific workflow. As described in [ 6 ] a (scientific) workflow is a directed graph where nodes represent arbitrary functions or modules in general with some input, output, and parameters. Edges model a predefined data or control flow between these modules. It includes information about the workflow’s procedure, deviations from it, and the execution in general. Workflow provenance is often expressed in terms of description logics and derived concepts like ontologies, e.g. the PROV-O ontology [ 8 ] . The PROV-O ontology distinguishes three core concepts: entities , activities , and agents . Entities are data or objects and can be derived from other entities. Activities can generate or use entities. Agents can perform or control activities or produce entities. An example for our biomedical use case in PROV terms is shown in Figure 2 . Figure 2: Biomedical in-vitro experiment with organizations and persons ( agents ), cell cultures and samples as well as data sets ( entities ), and research activities ( activity ) including their relationships in the PROV standard. Note that PROV relationship direction is typically from the result to the origin. Data provenance. ​ Data provenance plays a role in data wrangling, which can often be expressed in terms of relational algebra. It describes the derivation of a piece of data from data sets, typically the result of a database query Q . Considering our use case, the following query encodes the comparison of the fluorescence intensity from two stimulation experiments with respect to their voltage: SELECT voltage_2 FROM R NATURAL JOIN S WHERE intensity_1 < intensity_2 , where two tables R and S are joined on a unique join attribute sample_id , against a source database D (consisting of R and S ). The query result is stored in a table T :

Read the complete text