Secure IVSHMEM: End-to-End Shared-Memory Protocol with Hypervisor-CA Handshake and In-Kernel Access Control

Hyunwoo Kim, Jaeseong Lee, Sunpyo Hong, Changmin Han · 2025

Read Secure IVSHMEM: End-to-End Shared-Memory Protocol with Hypervisor-CA Handshake and In-Kernel Access Control by Hyunwoo Kim, Jaeseong Lee, Sunpyo Hong, Changmin Han (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-19004.txt.

Introduction

Virtual machines on one host often share memory to talk fast, but the shared region in today's systems has no locks: any program can read or rewrite it. Four computer scientists present Secure IVSHMEM in 2025. Their design splits channels and lets a kernel module police who may touch which part. The hypervisor runs a handshake so the two ends prove their identity to each other. Application hooks keep the extra work light.

Benchmarks show the handshake finishes in under 200 milliseconds, once. After that, round trips stay within 5 percent of the unprotected speed, and bandwidth barely changes. The authors aim the design at cars and other in-host systems where delay and safety both matter.

Catalog record

CREATORHyunwoo Kim, Jaeseong Lee, Sunpyo Hong, Changmin Han
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-19004

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Secure IVSHMEM: End-to-End Shared-Memory Protocol with Hypervisor-CA Handshake and In-Kernel Access Control Hyunwoo Kim, Jaeseong Lee, Sunpyo Hong, Changmin Han Abstract In-host shared memory (IVSHMEM) enables high-throughput, zero-copy communication between virtual machines, but today’s implementations lack any security control, allowing any application to eavesdrop or tamper with the IVSHMEM region. This paper presents Secure IVSHMEM , a protocol that provides end-to-end mutual authentication and fine-grained access enforcement with negligible performance cost. We combine three techniques to ensure security: (1) channel separation and kernel module access control, (2) hypervisor-mediated handshake for end-to-end service authentication, and (3) application-level integration for abstraction and performance mitigation. In microbenchmarks, Secure IVSHMEM completes its one-time handshake in under 200 μ \mu s and sustains data-plane round-trip latencies within 5% of the unmodified baseline, with negligible bandwidth overhead. We believe this design is ideally suited for safety- and latency-critical in-host domains, such as automotive systems, where both performance and security are paramount. I Introduction The automotive industry is rapidly evolving, driven by advances in semiconductor technology that have shifted system architectures from traditional microcontrollers (MCUs) to powerful Systems-on-Chip (SoCs). This evolution not only enhances computational capabilities but also paves the way for Software-Defined Vehicles (SDVs), where flexibility, scalability, and rapid updates are paramount. In SDVs, virtualization technology plays a crucial role by enabling the coexistence of multiple virtual machines (VMs) on a single hardware platform, ensuring isolated yet efficient execution of diverse applications. For example, modern cockpit domain controllers often deploy separate VMs for real-time operations (RTOS) and infotainment systems, which is essential for balancing performance and safety [ 1 ] [ 2 ] . Inter-VM communication in these environments is critical. Traditional approaches, such as TCP/UDP over a network stack or even UART-based messaging, often fall short in terms of speed and resource efficiency [ 3 ] . Alternative solutions like VirtIO offer a para-virtualized communication mechanism through ring buffers (VirtQueues), but they do not fully leverage the benefits of shared physical memory. [ 4 ] IVSHMEM (Inter-VM Shared Memory) addresses these limitations by mapping each VM’s virtualized PCI device to a common physical memory region, allowing rapid data exchange through shared memory [ 5 ] [ 6 ] [ 3 ] . Despite its performance advantages, this method introduces significant security challenges; multiple VMs accessing the same memory space creates vulnerabilities where a compromised or malicious VM could potentially access or modify data belonging to another VM [ 7 ] . This concern is particularly acute in scenarios where critical systems interact with less secure environments. For instance, when an RTOS communicates with an Android-based infotainment VM, there is a tangible risk that a malicious application within Android might tamper with the shared memory region [ 2 ] . Such tampering could result in attacks ranging from man-in-the-middle to eavesdropping, ultimately compromising system stability and safety. In response to these challenges, we propose a secure protocol designed specifically for IVSHMEM communication. Our approach introduces robust security measures on top of the IVSHMEM framework, ensuring data integrity and access control even in an environment with inherent vulnerabilities. While our protocol does introduce some performance overhead, we have implemented techniques to mitigate this impact, ensuring that the overhead remains minimal relative to the performance gains achieved by shared memory communication. In this paper, we provide a detailed analysis of the security threats associated with IVSHMEM, explore the limitations of existing inter-VM communication methods, and describe our protocol’s architecture and mitigation strategies. Through comprehensive evaluation, we demonstrate that our secure protocol successfully balances between robust security and the high-performance demands of IVSHMEM applications—such as those found in modern automotive systems. II Background In this section, we examine the IVSHMEM (Inter-VM Shared Memory) mechanism, outline its challenges, and review recent progress. II-A IVSHMEM: Mechanism and Architecture IVSHMEM is a specialized implementation of shared memory IPC designed for virtualized environments. It emulates a virtual PCI device to expose the shared memory’s base address and size to guest VMs [ 8 ] [ 9 ] . As shown in Figure 1 , the hypervisor exposes the shared-memory region to the guest VM, and the UIO kernel driver maps the emulated PCI device, allowing applications to access that memory directly via /dev/uioX . The design leverages the standardized PCI configuration to facilitate memory mapping and efficient communication. Specifically, IVSHMEM utilizes: • BAR0 (Base Address Register 0): This region (256 bytes of MMIO) holds the device registers, which control the operation of the virtual device. • BAR1: It contains the MSI-X table and Pending Bit Array (PBA), primarily used by the IVSHMEM doorbell mechanism for signaling interrupts. • BAR2: This is mapped to the shared memory object, providing a direct communication channel between VMs. The doorbell interrupt mechanism enabled by this configuration allows VMs to notify one another when new data is available, ensuring efficient core utilization and reducing latency in inter-VM communication [ 10 ] . Fig. 1: The overview of IVSHMEM architecture II-B Security Concerns of Shared Memory While shared memory IPC within a single operating system benefits from well-established security mechanisms—such as file access controls, sandboxing, and enhanced security modules like SELinux or AppArmor—IVSHMEM presents unique challenges [ 11 ] [ 12 ] . In a traditional OS environment, the operating system enforces strict access controls over shared memory regions, ensuring that only authorized processes can read or write data. However, when multiple, potentially untrusted VMs share the same memory space, these protections are significantly diminished. A compromised or malicious VM could easily access or tamper with data in the shared region, leading to unauthorized data disclosure, corruption, or even system instability [ 7 ] . II-C Secure Communication Over Insecure Channels The challenge of ensuring secure communication in IVSHMEM environments is analogous to securing communication over the Internet, where multiple parties exchange information over an inherently insecure channel. In network communications, protocols such as TLS rely on key exchange mechanisms, mutual authentication, and end-to-end encryption to safeguard data integrity and confidentiality [ 13 ] [ 14 ] . Similarly, secure multi-party communication techniques—such as Diffie-Hellman key exchange and advanced encryption standards—are employed to establish trust even when the channel is compromised [ 15 ] [ 16 ] . In the context of IVSHMEM, the situation is even more complex because multiple services must share the same restricted memory space as a communication channel. This necessitates the design of a secure protocol that not only ensures confidentiality and integrity—akin to TLS or other network security protocols—but also accommodates the shared nature of the memory resource. Our research addresses these challenges by proposing a secure protocol that protect the data transmitted via IVSHMEM, while also mitigating the performance overhead typically associated with such security measures. II-D Recent Progress of IVSHMEM Communication Recent research and developments in IVSHMEM communication have focused on balancing performance and security, with various approaches having distinct trade-offs. The SIVSHM project introduces a segmentation approach to IVSHMEM, enhancing security by isolating shared memory regions among VMs. However, this strict isolation introduces notable overheads from reduced buffer size. [ 7 ] . Performance-centric solutions, such as XenLoop and MemPipe, have been proposed to optimize IVSHMEM by improving transparency and reducing latency [ 17 ] [ 3 ] [ 18

Read the complete text