Africanus IV. The Stimela2 Framework: Scalable And Reproducible Workflows, From Local To Cloud Compute

O. M. Smirnov, S. Makhathini, J. S. Kenyon, H. L. Bester, S. J. Perkins, A. J. T. Ramaila, B. V. Hugo · 2024

Read Africanus IV. The Stimela2 Framework: Scalable And Reproducible Workflows, From Local To Cloud Compute by O. M. Smirnov, S. Makhathini, J. S. Kenyon, H. L. Bester, S. J. Perkins, A. J. T. Ramaila, B. V. Hugo (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-10080.txt.

Description

Oleg Smirnov and co-authors present a workflow framework for radio astronomy in a 2024 paper. Telescope data reduction chains many tools in sequence. Stimela2 writes those chains as short readable recipes in YAML format. Each task is defined by its own schema of inputs and outputs.

Recipes can nest inside other recipes for modular design. Settings stay separate from the recipe logic. Tasks run natively or inside containers for reproducibility. A companion package maintains the open container images. The framework talks to the Slurm scheduler and the Kubernetes cloud system. That lets the same workflow scale from a laptop to a cluster. It is the fourth paper in the Africanus series on astronomy software.

Catalog record

CREATORO. M. Smirnov, S. Makhathini, J. S. Kenyon, H. L. Bester, S. J. Perkins, A. J. T. Ramaila, B. V. Hugo
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE104 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-10080

Browse the full library · About the Tower of Babel