Analyzing Symbolic Properties for DRL Agents in Systems and Networking Journal: POMACS Volume: 10 2 29 6 DOI: 10.1145/3805627
Mohammad Zangooei, Jannis Weil, Amr Rizk, Mina Tahmasbi Arashloo, Raouf Boutaba · 2026
Read Analyzing Symbolic Properties for DRL Agents in Systems and Networking Journal: POMACS Volume: 10 2 29 6 DOI: 10.1145/3805627 by Mohammad Zangooei, Jannis Weil, Amr Rizk, Mina Tahmasbi Arashloo, Raouf Boutaba (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-04914.txt.
Introduction
Zangooei and four colleagues study how to reason about deep reinforcement learning agents used in systems and networking, from video streaming to wireless control and congestion control. Current checks focus on single fixed input states, giving narrow coverage and demanding heavy manual work to pick relevant cases. The authors instead formulate symbolic properties, rules that must hold across ranges of inputs, with monotonicity and robustness as examples. They show how existing tools can analyze those broader properties, opening safer deployment for agents that face varied real states.
Catalog record
| CREATOR | Mohammad Zangooei, Jannis Weil, Amr Rizk, Mina Tahmasbi Arashloo, Raouf Boutaba |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 93517 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.04914
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Analyzing Symbolic Properties for DRL Agents in Systems and Networking Journal: POMACS Volume: 10 2 29 6 DOI: 10.1145/3805627 Mohammad Zangooei, Jannis Weil, Amr Rizk, Mina Tahmasbi Arashloo, Raouf Boutaba Abstract . Deep reinforcement learning (DRL) has shown remarkable performance on complex control problems in systems and networking, including adaptive video streaming, wireless resource management, and congestion control. For safe deployment, however, it is critical to reason about how agents behave across the range of system states they may encounter in practice. Existing verification-based approaches in this domain primarily focus on point properties – properties defined around fixed input states – which offer limited coverage and require substantial manual effort to identify relevant input-output pairs for analysis. In this paper, we study symbolic properties – properties that specify expected behaviors over ranges of input states – for DRL agents in systems and networking. We present a generic formulation for symbolic properties, with monotonicity and robustness as concrete examples, and show how they can be analyzed using existing DNN verification engines. Our approach encodes symbolic properties as comparisons between related executions of the same policy and decomposes them into practically tractable sub-properties. These techniques serve as practical enablers for applying existing verification tools to symbolic analysis. Using our framework, diffRL, we conduct an extensive empirical study across three representative DRL-based control systems – adaptive video streaming, wireless resource allocation, and congestion control – covering both discrete and continuous action spaces. Through these case studies, we analyze symbolic properties over broad input ranges, examine how property satisfaction evolves during training, study the impact of model size on verifiability, and compare multiple verification backends. Our results show that symbolic properties provide substantially broader coverage than point properties and can uncover non-obvious, operationally meaningful counterexamples, while also revealing practical solver trade-offs and limitations. † † cc-license: by 1. Introduction Deep Reinforcement Learning (DRL) has emerged as a promising approach for control problems in systems and networking, where decisions must be made in complex and highly dynamic environments. In these settings, DRL agents are typically embedded in control loops. The agent observes the system state – represented as a feature vector of performance indicators such as throughput, latency, packet loss, or resource utilization – and selects actions according to its learned policy. These actions correspond to concrete numerical control decisions for resource allocation ( 66 ; 42 ; 44 ; 11 ) , traffic engineering and path selection ( 10 ; 64 ; 18 ) , job scheduling ( 36 ; 34 ; 27 ; 29 ) , and tuning streaming video bitrate ( 35 ; 24 ; 25 ) or congestion window sizes ( 65 ; 52 ; 1 ; 33 ) . To safely and effectively integrate DRL agents into such control loops, it is essential to reason about how “well-behaved” these agents are across the range of inputs they can encounter after deployment. This is particularly important as DRL agents make decisions based on Deep Neural Networks (DNNs) that function as black boxes with opaque input-output relationships from the perspective of human operators. As such, we would like to reason about whether a DRL agent’s selected action a changes in undesirable ways when the observed state x is perturbed to x+s , where s is a small, bounded slack. Such properties are important, as the input to these agents typically comes from real-world measurements that are susceptible to noise caused by variability in measurement timing and limitations in measurement precision ( 14 ) . Moreover, undesirable output deviations in response to small input changes can expose systems to adversarial manipulation, e.g., by inducing disproportionate resource allocations through minor input perturbations ( 32 ) . Previous work ( 15 ; 14 ; 21 ) commonly relies on state-of-the-art DNN verification engines ( 37 ) or Mixed-Integer-Linear-Program (MIP) solvers ( 22 ) to reason about DRL agents in the systems and networking domain. However, these works can only check point properties , where the system state x and the reference action a are fixed to concrete constants (see Fig. 1 , left). For example, consider Pensieve ( 35 ) , a DRL agent for adaptive video streaming. Its input includes the client’s video buffer size and the measured throughput for the past streamed video segment. As its action, Pensieve chooses one of six possible bitrates from 300 Kbps to 4.3 Mbps for the next video segment. The goal is to choose bitrates that enable smooth and high-quality video playback. WhiRL ( 15 ) , which uses the Marabou DNN verification engine ( 37 ) , and similar works ( 14 ; 21 ) , can only analyze Pensieve against point properties such as the following: If the video buffer has zero or one segment and the throughput for the past video segments is as low as a user-specified value (i.e., one concrete point among all possible system states), the DRL agent’s output action should not be 4.3 Mbps (i.e., a concrete action). Figure 1. Point-wise versus symbolic property analysis. Left (Prior Work): Analysis is performed at individual points that humans provide, checking that the property holds under bounded perturbations around one concrete point in the input space. In this example, the property asserts that the output action does not change. Checking properties only at individual points results in limited coverage of the input space and may miss unsafe behavior outside the considered points. Right (This Work): Analysis is performed over an entire operational region using symbolic properties. This provides higher coverage of the input space and enables the detection of unsafe points that are difficult to uncover with point properties. Comparison between point-based and symbolic property verification. Prior work verifies properties on individually provided input points, resulting in low coverage and potential missed unsafe behaviors. In contrast, this work verifies properties over an entire operational region, enabling higher coverage of the input space and improved detection of unsafe points. While checking each individual point property is typically feasible and efficient, relying on point properties to reason about DRL agents operating in complex, dynamic systems and networks has two main drawbacks. First, point properties have low coverage. Each property only provides assurance around a single concrete point in the agent’s operational input space. Second, one must explicitly identify which input points are worth checking. This limitation is often manageable in supervised learning, where a (large) labeled data set explicitly captures the expected output value for a wide range of input points (e.g., AutoSpec ( 26 ) ). In DRL, however, there is no predefined ground truth for the expected behavior at a given state , i.e., the best action is unknown prior to training and must be learned through interactions with the environment ( 50 ) . As such, making point properties effective in this setting requires substantial domain expertise to identify meaningful points, which is difficult to scale to a level that provides sufficient coverage. In this paper, we focus on symbolic properties instead, where the system state x and the reference action a are symbolic (see Fig. 1 , right). This shifts analysis from isolated input points to entire regions of the agent’s operational space, substantially increasing coverage and removing the need for