ShIOEnv: An Environment for Synthesizing Bash Shell Inputs with Execution Artifacts
\name Jarrod Ragsdale \email [email protected], \name Rajendra Boppana \email [email protected] · 2025
Read ShIOEnv: An Environment for Synthesizing Bash Shell Inputs with Execution Artifacts by \name Jarrod Ragsdale \email [email protected], \name Rajendra Boppana \email [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18374.txt.
Introduction
Jarrod Ragsdale and Rajendra Boppana build a gym for fake shell commands, each tagged with what it truly did when run. Such traces can teach small models to fake a shell for honeypot traps that lure hackers. Old command sets log the text but not the run scars: output, error notes, exit marks, and file shifts. Their rig breeds commands, runs them in a safe box, and logs each scar.
Free breeding bred junk and crash commands by the heap in their tests. Grammar-fenced breeding kept commands valid yet hard to trim, a worth they score by cut tests. One point six million traces span eighty-six shell tools in the set. Small code models trained on it beat rule fakes at call of run acts.
Catalog record
| CREATOR | \name Jarrod Ragsdale \email [email protected], \name Rajendra Boppana \email [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-18374
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ShIOEnv: An Environment for Synthesizing Bash Shell Inputs with Execution Artifacts \name Jarrod Ragsdale \email [email protected], \name Rajendra Boppana \email [email protected] Abstract We introduce ShIOEnv, a Gymnasium-compatible Bash environment for synthesizing execution-annotated shell input traces. Datasets containing such traces are useful for training models to perform execution-free shell simulation, which in turn is useful for deploying honeypots for cyber deception. Existing datasets, such as NL2CMD and other Bash command corpora, do not include complete execution artifacts (stdout, stderr, exit status, and changes to the system’s state) associated with a command’s execution. ShIOEnv synthesizes shell inputs, executes them in a controlled system context, and records stdout, stderr, exit status, and structured filesystem and environment patches. Unconstrained (random) generation of shell inputs (command and arguments) results in multiple instances of arguments that are mostly redundant or result in execution errors. We introduce grammar-constrained synthesis to generate syntactically valid arguments with high input irreducibility, which we estimate through self-ablative contrasts of execution artifacts. We prepare 1.6M samples using grammar-constrained and unconstrained synthesis and collect their execution artifacts for 86 Linux/Bash utilities. Using this dataset, we train small language models (CodeT5) and evaluate their accuracy in predicting the actual execution behavior of attacker-sourced Bash commands. These models outperform commonly used rule-based simulation (Cowrie) and prompted large language models by up to 14.2 pp in output edit similarity (O-ES). We enhance the NL2CMD dataset by constructing 500K samples executable in a grounded system context along with their complete execution artifacts. We empirically demonstrate that models trained on grammar-constrained samples outperform those trained on NL2CMD samples by 22.5 pp in O-ES. ShIOEnv, command grammars, and 2.1M samples used in our primary evaluations are released for use by other researchers. Reviewed on OpenReview: https://openreview.net/forum?id=XXXX Keywords: shell simulation, environment, dataset, bash commands 1 Introduction Execution-free shell simulation is the task of predicting the execution behavior of a shell input, consisting of a command and optional arguments, within a specified system context. The target behavior includes execution artifacts, such as terminal output, exit status, and changes to system state that may affect subsequent inputs. The system context comprises execution-relevant state that affects command behavior, including the filesystem, environment variables, permissions, and user identity. Execution-free simulation may be useful in settings where direct shell input execution is unsafe, costly, or undesirable. Examples include cyber-deception systems that emulate attacker-facing shell interactions and command-line agents that use terminal state or execution feedback when selecting subsequent actions ( Sladić et al., 2024 ; Chakraborti et al., 2017 ; Yang et al., 2024 ) . Preparing models for this task requires a diverse range of shell inputs paired with their execution behavior in a system context. Existing public resources provide only partial support for this supervision regime. Lin et al. (2018) and Fu et al. (2023) provide datasets that primarily support natural-language-to-command translation; cybersecurity training logs provide observed command sequences and metadata; and recent security datasets synthesize command-like activity for detection tasks Švábenskỳ et al. (2021) ; Trizna (2022) . These datasets do not, by themselves, provide complete execution artifacts for commands executed under a shared, reproducible system context To address these gaps, we introduce the shell input-output environment (ShIOEnv), a Gymnasium-compatible data-generation environment for collecting execution behaviors for given commands in grounded system contexts. ShIOEnv casts shell input synthesis as a Markov decision process in which an agent incrementally constructs a shell input through argument-level actions. When construction terminates, ShIOEnv executes the completed input and records the resulting execution behavior. Shell inputs collapse into common execution behavior patterns in which invalid or redundant portions of the input can obscure the effect of individual arguments. As a consequence, syntactic validity alone is insufficient as a data-quality criterion for execution-annotated samples. We define behavioral irreducibility as a measurement of argument redundancy with respect to the final observed execution behavior. We perform this measurement via contrasts of execution behaviors of argument-ablated inputs. An input is considered irreducible if no such ablations produce the same execution artifacts. However, constructing arguments in a command-agnostic manner, which we call unconstrained synthesis (UCS), yields low-irreducibility inputs in which execution behavior is dominated by invalid or redundant arguments. We therefore introduce grammar-constrained synthesis (GCS), in which a command-specific grammar dictates a derivation for producing each argument. This work treats shell-command corpus construction as a data-centric machine-learning problem, with the central question being how the data-generation process and irreducibility criteria affect the utility of execution-annotated shell inputs. We generate 1.6M execution-annotated samples using grammar-constrained and unconstrained synthesis. We also construct 500K execution-annotated samples from NL2CMD commands adapted to a specified system context. Together, these data sources support controlled comparisons among synthesis methods (GCS and UCS) and adapted input-only command corpora from the literature (NL2CMD). We evaluate the resulting datasets by training models for execution-free shell simulation and using model performance as a measure of dataset utility. Models trained on GCS data outperform models trained on unconstrained inputs, adapted NL2CMD inputs, and external execution-free baselines such as Cowrie ( Cowrie, 2014 ) . Moreover, grammar-constrained samples demonstrate greater training efficiency, outperforming models trained on five times as many unconstrained samples. To summarize, we make the following contributions: • We introduce ShIOEnv, a data-generation environment for constructing shell inputs and their execution artifacts for a given system context and for efficiently evaluating argument redundancy. Each sample links a shell input to its execution artifacts. • We introduce grammar-constrained synthesis as a data-generation intervention for shell-command corpora. Command-specific grammars, derived from the man pages, restrict argument generation to context-admissible regions of the input space, reducing degenerate execution behavior and improving the mapping of shell arguments to the resulting execution behaviors. • We curate and release 2.1M execution-annotated samples spanning 86 Linux commands, together with command grammars, execution artifacts, and dataset schema information required for reproducible use. • We evaluate the generated data by assessing its ability to train models to simulate the specified system’s execution behavior, empirically demonstrating improved performance over existing datasets and execution-free approaches. The remainder of the paper is organized as follows. Section 2 introduces preliminaries and related work in shell modeling, synthesis, and executable environments. Section 3 presents ShIOEnv, irreducibility estimation, and grammar-constrained synthesis. Section 4 evaluates estimator fidelity, the distributional effect of grammar constraints, and downstream execution-free modeling utility. Section 5 discusses the implications of the results and remaining limitations. Section 6 summarizes the findings and outlines future work. Appendix A provides supplemental details on the execution environment and generated datasets. 2 Background 2.1 Preliminaries ShIOEnv models shell input synthesis as a sequential decision process in which an agent incrementally constructs a shell input for execution in a controlled system context. A shell input consists of a command string (e.g., find , ls ) followed by optional arguments (e.g., -l , --list ). Our formulation consists of three complementary parts: Markov decision processes to form the basis of argument-level construction, context-free grammars to define a command-admissible argument space, and temporal abstraction to treat variable-length grammar derivations as single argument-level actions. A Markov decision process is a tuple M=\langle\mathcal{S},\mathcal{A},p,r\rangle , where \mathcal{S} is a set of states, \mathcal{A} is a set of actions, p(s^{\prime}\mid s,a) is a transition function, and r(s,a) is a reward function ( Puterman, 2014 ) . In