Parsl+CWL: Towards Combining the Python and CWL Ecosystems

Nishchay Karle 1 , Ben Clifford 1 , Yadu Babuji 1 2 , Ryan Chard 2 , Daniel S. Katz 2 and Kyle Chard 1 2 · 2024

Read Parsl+CWL: Towards Combining the Python and CWL Ecosystems by Nishchay Karle 1 , Ben Clifford 1 , Yadu Babuji 1 2 , Ryan Chard 2 , Daniel S. Katz 2 and Kyle Chard 1 2 (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-08062.txt.

Description

Nishchay Karle and co-authors marry in this 2024 paper two workflow worlds that rarely meet. One is the Common Workflow Language, a portable recipe format for science pipelines. The other is Parsl, a Python library that runs work across machines. Their bridge converts CWL tool definitions into Parsl apps that scripts can import directly.

A companion runner executes CWL tools under Parsl, and a proof-of-concept lets Python live inline inside CWL definitions. An image processing workflow shows the gain, benchmarked against other CWL runners. The authors pitch the union as letting scientists reuse shared tool catalogs without leaving Python. Shared tools gain a second engine to run on.

Catalog record

CREATORNishchay Karle 1 , Ben Clifford 1 , Yadu Babuji 1 2 , Ryan Chard 2 , Daniel S. Katz 2 and Kyle Chard 1 2
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-08062

Browse the full library · About the Tower of Babel