A Reinforcement Learning Framework for Application-Specific TCP Congestion-Control
Jinming Xing, Muhammad Shahzad · 2025
Read A Reinforcement Learning Framework for Application-Specific TCP Congestion-Control by Jinming Xing, Muhammad Shahzad (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-07042.txt.
Introduction
Jinming Xing and Muhammad Shahzad built a learning framework in two thousand twenty-five for TCP congestion control tuned to each application. Standard control rules follow fixed heuristics that chase top speed, ignoring that many apps care more about delay, loss, or jitter. Learning-based rivals adapt to network change but still ignore app goals.
Their system, ASC, lets an app state any objective mix it wants, then adapts as goals or network conditions shift. A client-server design keeps it scalable as connections come and go, and light on the machines holding the connections. Wide tests, the authors report, show it tracks stated objectives closely across settings.
Catalog record
| CREATOR | Jinming Xing, Muhammad Shahzad |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-07042
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Reinforcement Learning Framework for Application-Specific TCP Congestion-Control Jinming Xing, Muhammad Shahzad Abstract The Congestion Control (CC) module plays a critical role in the Transmission Control Protocol (TCP), ensuring the stability and efficiency of network data transmission. The CC approaches that are commonly used these days employ heuristics-based rules to adjust the sending rate. It is well known that due to their heuristics-based nature, these approaches are not only unable to adapt to changing network conditions but are also agnostic to the diverse requirements that different applications often have. Lately, we have seen several learning-based CC approaches that do not rely on fixed heuristics-based rules and thus can adapt to changing network conditions. Unfortunately, existing learning-based CC approaches have also not been designed to take application requirements into account. Prior heuristics-based as well as learning-based CC approaches focus on achieving a singular objective, which is often to maximize throughput, even though a lot of applications care more about latency, packet losses, jitter, and/or different combinations of various network metrics. Motivated by this, we propose a Deep Reinforcement Learning (DRL) based CC framework, namely ASC RL {}_{\text{RL}} , which allows any application to specify any arbitrary objectives that the network traffic of that application should achieve and is able to swiftly adapt to the changes in the objectives of the applications as well as to the changes in the network conditions. Our ASC RL {}_{\text{RL}} framework further employs a client-server architecture that serves two purposes: 1) it makes ASC RL {}_{\text{RL}} highly scalable in terms of the arrival and departure of TCP connections, and 2) it makes ASC RL {}_{\text{RL}} very lightweight for the nodes maintaining the TCP connections. We implemented and extensively evaluated ASC RL {}_{\text{RL}} in a variety of settings. Our results show that it can not only achieve various objectives but also outperforms prior approaches even in the specific objectives that those approaches were designed to achieve. We will open-source the code and data used in the evaluation of ASC RL {}_{\text{RL}} on the acceptance of this paper. I Introduction Congestion control algorithm’s (which are mostly used by TCP but lately also by other transport and application protocols, such as QUIC and webRTC) are the foundational protocols that enable the Internet to successfully transport the data of billions of devices without getting choked. Unfortunately, the existing approaches to congestion control are not necessarily the most optimal for the networked ecosystems of today, which consist of numerous types of devices and myriads of diverse Internet applications (subsequently referred to as I -Apps) that communicate over the Internet. In particular, there are two fundamental limitations that existing approaches have. But before we describe them, let us first quickly overview the current landscape of congestion control methodologies. Existing congestion control methodologies can be broadly categorized as heuristics-based and learning-based . The heuristics-based approaches use fixed-rules that typically rely on round trip delay and/or packet loss information. The well-known examples of such approaches include New Reno [ 1 ] , Cubic [ 2 ] , DCTCP [ 3 ] , and BBR [ 4 ] . The learning-based approaches, as the name suggests, learn over time the best way to respond to current network conditions to achieve an objective . For all existing learning-based approaches, the objective typically has been to achieve the highest throughput. Examples of existing such approaches include QTCP [ 5 ] , DRL-CC [ 6 ] , TCP-Drinc [ 7 ] , and SmartCC [ 8 ] . Learning-based approaches are newer compared to the heuristics-based approaches and hold the promise to provide tailored services to a diverse set of I -Apps with a diverse set of requirements running on a diverse set of networked devices. I-A Limitations of Prior Art While the work on congestion control has been plenty, prior approaches suffer from either or both of the following two fundamental limitations. Application-agnostic: Different I -Apps have different needs. For example, file transfer needs high throughput [ 9 ] , audio calls need low latency [ 10 ] , cloud streamed games require low latency as well as small jitter [ 11 ] , and so on. Unfortunately, as heuristics-based approaches rely on fixed rules, they cannot take into account such requirements of a diverse set of I -Apps simultaneously. Similarly, all existing learning based approaches fall short as they focus on a singular objective, such as maximizing throughput, and assume that all I -Apps have that same objective. The objectives of many I -Apps of today change during different phases of their operation. For example, during the initial buffering phase, video streaming applications require high throughput to quickly fill the buffer to a sufficient level so that video playback starts right away. During the next playback phase, video streaming requires minimal jitter and packet loss to maintain smooth playback of the stream. During any re-buffering phase, video streaming again requires high throughput. Similarly, during web browsing, whenever a user clicks a link, low latency and high throughput are needed to minimize page load time. Afterwards, as the user interacts with the loaded page, throughput becomes less important and only low latency is needed to facilitate smooth interactions. Yet another example is video conferencing, where signaling/connection setup phase is sensitive to latency and jitter, video/audio streaming phase is additionally sensitive to packet loss, screen/content sharing phase is sensitive to throughput and latency, and remote screen control phase is sensitive to latency. A newer I -App, Cloud-streamed gaming, requires either low latency (to display immediate response when user provides inputs) or high throughput (to transfer video frames from cloud to the user’s device) and frequently switches between these two requirements as the user plays the game. There are numerous other such examples of I -Apps whose objectives change during different phases of their operations. Unfortunately, neither the heuristics-based nor the existing learning-based approaches can handle such changes in objectives. Computational demand: While heuristics based approaches are computationally inexpensive, the learning based approaches are not because they need to be trained and retrained to adapt to the latest network conditions. Existing learning-based approaches are computationally demanding not only for the low-powered devices but also for typical computers/laptops. This is because typical computers have multiple ongoing connections, and this training and retraining often needs to be done separately for each TCP connection. I-B Problem Statement Our goal is to develop a congestion control framework that allows any I -App to specify any arbitrary objectives that the network traffic of that I -App should achieve, such as maximize throughput, minimize latency/jitter, optimize any other network metric, or any combination of these objectives. The framework should further meet the following three requirements: (1) adapt to changes in the objectives of the I -App, (2) require minimal computational capability from the node running the I -App, and (3) be scalable in terms of swiftly handling new and terminating TCP connections. I-C Proposed Approach We present ASC {}_{\text{RL}} , a r einforcement l earning c ongestion control framework that achieves any a pplication- s pecified objectives while meeting the three requirements listed above and addressing the two limitations of prior art. Logically, ASC {}_{\text{RL}} performs two functions: learning and inference. Inference is computationally cheap