Africanus IV. The Stimela2 Framework: Scalable And Reproducible Workflows, From Local To Cloud Compute
O. M. Smirnov, S. Makhathini, J. S. Kenyon, H. L. Bester, S. J. Perkins, A. J. T. Ramaila, B. V. Hugo · 2024
Read Africanus IV. The Stimela2 Framework: Scalable And Reproducible Workflows, From Local To Cloud Compute by O. M. Smirnov, S. Makhathini, J. S. Kenyon, H. L. Bester, S. J. Perkins, A. J. T. Ramaila, B. V. Hugo (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-10080.txt.
Description
Oleg Smirnov and co-authors present a workflow framework for radio astronomy in a 2024 paper. Telescope data reduction chains many tools in sequence. Stimela2 writes those chains as short readable recipes in YAML format. Each task is defined by its own schema of inputs and outputs.
Recipes can nest inside other recipes for modular design. Settings stay separate from the recipe logic. Tasks run natively or inside containers for reproducibility. A companion package maintains the open container images. The framework talks to the Slurm scheduler and the Kubernetes cloud system. That lets the same workflow scale from a laptop to a cluster. It is the fourth paper in the Africanus series on astronomy software.
Catalog record
| CREATOR | O. M. Smirnov, S. Makhathini, J. S. Kenyon, H. L. Bester, S. J. Perkins, A. J. T. Ramaila, B. V. Hugo |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 104 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-10080