CXLAimPod: CXL Memory is all you need in AI era
Yiwei Yang = , Yusheng Zheng = , Yiqi Chen, Zheng Liang, Kexin Chu, Zhe Zhou, Andi Quinn, Wei Zhang · 2025
Read CXLAimPod: CXL Memory is all you need in AI era by Yiwei Yang = , Yusheng Zheng = , Yiqi Chen, Zheng Liang, Kexin Chu, Zhe Zhou, Andi Quinn, Wei Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-15980.txt.
Introduction
Memory hardware is a bottleneck for data-hungry programs like language models and key-value stores. Yiwei Yang and co-authors use CXL memory, a newer link whose channels carry reads and writes at once, unlike the half-duplex design of standard memory. That two-way design should shine under mixed traffic, yet today's software ignores it. Their measurements confirm the prize: CXL beat flat DDR5 bandwidth by over half at balanced read-write mixes.
Their framework, CXLAimPod, closes the gap inside the Linux kernel using eBPF programs. It mixes scheduling strategies and takes hints from process groups about what each program needs, balancing speed, delay, and overhead. Tests showed gains of about seven percent on average for Redis, with far larger jumps on some patterns, and over seventy percent for language model text generation. The work shows that two-way-aware scheduling pays off.
Catalog record
| CREATOR | Yiwei Yang = , Yusheng Zheng = , Yiqi Chen, Zheng Liang, Kexin Chu, Zhe Zhou, Andi Quinn, Wei Zhang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-15980
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CXLAimPod: CXL Memory is all you need in AI era Yiwei Yang = , Yusheng Zheng = , Yiqi Chen, Zheng Liang, Kexin Chu, Zhe Zhou, Andi Quinn, Wei Zhang Abstract . The proliferation of data-intensive applications, ranging from large language models to key-value stores, increasingly stresses memory systems with mixed read-write access patterns. Traditional half-duplex architectures such as DDR5 are ill-suited for such workloads, suffering bus turnaround penalties that reduce their effective bandwidth under mixed read-write patterns. Compute Express Link (CXL) offers a breakthrough with its full-duplex channels, yet this architectural potential remains untapped as existing software stacks are oblivious to this capability. This paper introduces CXLAimPod, an adaptive scheduling framework designed to bridge this software-hardware gap through system support including cgroup-based hints for application-aware optimization. Our characterization quantifies the opportunity, revealing that CXL systems achieve 55-61% bandwidth improvement at balanced read-write ratios compared to flat DDR5 performance, demonstrating the benefits of full-duplex architecture. To realize this potential, the CXLAimPod framework integrates multiple scheduling strategies with a cgroup-based hint mechanism to navigate the trade-offs between throughput, latency, and overhead. Implemented efficiently within the Linux kernel via eBPF, CXLAimPod delivers significant performance improvements over default CXL configurations. Evaluation on diverse workloads shows 7.4% average improvement for Redis (with up to 150% for specific sequential patterns), 71.6% improvement for LLM text generation, and 9.1% for vector databases, demonstrating that duplex-aware scheduling can effectively exploit CXL’s architectural advantages. 1. Introduction Modern applications are becoming increasingly data-hungry. Large language models need hundreds of gigabytes for their parameters ( Brown et al. 2020 ) , while distributed databases manage petabytes of data ( DeCandia et al. 2007 ) . These applications constantly mix reads and writes to memory, making the "memory wall" (the speed gap between processors and memory) worse than ever ( Wulf and McKee 1995 ) . This is a significant problem: studies have shown that recommendation systems can spend over half their time waiting for memory ( Eisenman et al. 2018 ) , and search systems can waste substantial processing cycles on memory delays ( Kanev et al. 2015 ) . These delays cost money: memory systems now consume up to 40% of datacenter power. The root cause is how traditional memory works. For decades, DRAM has used a shared bus that can only handle reads or writes at one time, not both ( Jacob et al. 2018 ) . Even though DDR5 can theoretically deliver 76.8 GB/s per channel ( JEDEC Solid State Technology Association 2020 ) , it struggles with today’s applications that constantly switch between reading and writing data. The problem is the delay when switching directions on the bus ( Hassan et al. 2016 ) . Each switch takes 15-20 cycles (about 11-15 nanoseconds), which seems small but adds up quickly. Measurements show DDR5 performance remains relatively flat across different read-write ratios, varying by only 26% (153-189 GB/s) due to bus turnaround penalties. To work around this, engineers have created complex workarounds: memory controllers batch similar operations together, and programmers rewrite their code to separate reads from writes, making software more complex for marginal gains. The Compute Express Link (CXL) standard offers a potential solution through a different architectural paradigm ( Sharma 2023 ) . Beyond its benefits for coherent memory expansion and pooling ( Li et al. 2023a ) , CXL builds on the PCIe physical layer to provide physically separate, full-duplex channels that enable concurrent bidirectional data transfers ( PCI-SIG 2022 ) . This architectural shift eliminates the half-duplex bottleneck that has limited memory system efficiency for decades. Characterization shows that while DDR5 bandwidth remains relatively flat across read-write ratios, CXL achieves 55-61% bandwidth improvement at balanced read-write ratios, demonstrating significant duplex benefits. However, this potential remains largely untapped. Existing operating systems and their memory schedulers, designed for a half-duplex world, are unaware of CXL’s duplex capabilities. Without intelligent software management, CXL devices cannot fully exploit their architectural advantages. To bridge this software-hardware gap, we present CXLAimPod, an adaptive scheduling framework that helps exploit the performance potential of CXL memory. CXLAimPod intercepts memory requests and intelligently co-schedules them to exploit CXL’s full-duplex link for simultaneous bidirectional transfers, maximizing concurrent utilization while minimizing interference ( Mutlu and Moscibroda 2008 ) . The framework is designed for production deployment, overcoming the challenges of kernel modification through the use of eBPF for safe, efficient in-kernel extension ( Gregg 2019 ) . An important feature of CXLAimPod is the integration of cgroup-based hints that enable application-level guidance of allocation and scheduling decisions ( Menage 2007 ) . This mechanism allows system administrators and applications to communicate their memory access patterns and performance requirements through the familiar cgroup interface, enabling the scheduler to make informed decisions without requiring application modification. The framework incorporates multiple scheduling strategies that explore different trade-offs between performance and overhead, from simple threshold-based approaches to more sophisticated adaptive algorithms ( Åström and Murray 2010 ; Hur and Lin 2007 ; Hashemi et al. 2018 ) , along with cgroup-aware scheduling for multi-tenant optimization ( Verma et al. 2015 ) . This paper makes the following key contributions: • We provide an in-depth characterization of CXL’s full-duplex performance under mixed read-write workloads. • We design and implement CXLAimPod, a duplex-aware scheduling framework for Linux that is both high-performance and safely deployable without kernel modifications using eBPF. • We demonstrate through evaluation with microbenchmarks and real-world applications, including Redis, RocksDB, graph analytics, and a 671B-parameter language model, that CXLAimPod delivers 7.4% average improvement for Redis (up to 150% for sequential patterns), 71.6% improvement for LLM text generation, and 9.1% for vector databases over default CXL configurations. The remainder of this paper is organized as follows. Section provides background on CXL and memory scheduling. Section 3 presents our performance characterization. Section 4 describes the design of our scheduling framework. Section 5 details the CXLAimPod implementation. Section 6 presents our evaluation, and we conclude in Section . 2. Background and Motivation This section examines the limitations of DDR architectures and the opportunities presented by CXL’s full-duplex design, providing context for why existing software fails to exploit modern hardware capabilities. 2.1. The Half-Duplex Legacy of DDR Memory DDR5 achieves 76.8 GB/s per channel through high signaling rates and wide data paths ( JEDEC Solid State Technology Association 2020 ) . However, it retains the half-duplex, shared-bus architecture from earlier DRAM interfaces ( Jacob et al. 2018 ) . The shared command and data buses require explicit coordination when switching between reads and writes ( Hassan et al. 2016 ) . This bus turnaround penalty (15-20 cycles on DDR5 systems) creates idle time where no useful work occurs. At 6400 MT/s, this translates to 11.25-15 nanoseconds of idle time per direction switch. 2.2. CXL: A Full-Duplex Architecture In contrast to DDR’s limitations, CXL introduces a fundamentally different approach. It builds on PCIe 5.0’s physical layer with a protocol stack optimized for memory ( Sharma 2023 ; CXL Consortium 2022 ) . Each PCIe lane operates at 32 GT/s with 128b/130b encoding, yielding 31.5 Gb/s per direction with under 2% overhead ( PCI-SIG 2022 ) . The key distinction lies in CXL’s use of separate transmit (TX) and receive (RX) paths inherited from