Fray: An Efficient General-Purpose Concurrency Testing Platform for the JVM

Ao Li, Byeongjee Kang, Vasudev Vikram · 2025

Read Fray: An Efficient General-Purpose Concurrency Testing Platform for the JVM by Ao Li, Byeongjee Kang, Vasudev Vikram (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-12618.txt.

Introduction

Ao Li and co-authors present, in a 2025 paper, a platform that systematically hunts concurrency bugs in Java programs. Races between threads are notoriously hard to reproduce. Earlier test platforms forced trade-offs among power, reach, and upkeep.

Their platform, Fray, steers thread scheduling directly without swapping out the program's own concurrency tools. That design keeps real-world code testable as written. Push-button operation is the goal: developers run it without becoming scheduling experts. The team maps the design space and argues their point in it balances the trade-offs best. Reliable race hunting matters because such bugs slip into production and fail rarely, confusingly, and at the worst times.

Catalog record

CREATORAo Li, Byeongjee Kang, Vasudev Vikram
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-12618

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Fray: An Efficient General-Purpose Concurrency Testing Platform for the JVM Ao Li, Byeongjee Kang, Vasudev Vikram Abstract Concurrency bugs are hard to discover and reproduce. Prior work has developed sophisticated algorithms to search for concurrency bugs, such as partial order sampling (POS); however, fundamental limitations with existing platforms for concurrency control hinder effective testing of real-world software. We observe that the design space for concurrency control on managed code involves complex trade-offs between expressibility, applicability, and maintainability on the one hand, and bug-finding efficiency on the other hand. This paper presents Fray , a platform for performing push-button concurrency testing of data-race-free JVM programs. The key insight behind Fray is that effective controlled concurrency testing requires orchestrating thread interleavings without replacing existing concurrency primitives, while encoding their semantics for faithfully expressing the set of all possible program behaviors. Fray incorporates a novel concurrency control mechanism called shadow locking , designed to make controlled concurrency testing practical and efficient for JVM programs. In an empirical evaluation on 53 benchmark programs with known bugs (SCTBench and JaConTeBe), Fray with random search finds 70% more bugs than JPF and 77% more bugs than RR’s chaos mode. We also demonstrate Fray’s push-button applicability on 2,655 tests from Apache Kafka, Lucene, and Google Guava. In these mature projects, Fray successfully discovered 18 real-world concurrency bugs that can cause 363 tests to fail reproducibly. 1 Introduction Software testing is the predominant form of validating correctness for large real-world programs due to its simplicity, wide-spread applicability, efficiency, and reproducibility. However, testing multi-threaded programs remains challenging in practice, despite the fact that concurrency bugs are among the most difficult to detect and diagnose [ 34 , 37 ] . Take Java, which by several metrics is the most popular programming language with native support for concurrent multi-threading [ 52 , 14 , 55 ] . Let’s assume that programmers write test cases to validate the correctness of concurrent programs via assertions (e.g., that a parallel-sort operation produces a sorted list). How can we check such properties? In practice, a common approach appears to be simply re-running concurrent tests multiple times [ 2 ] to check whether some assertion fails. This approach is neither effective nor reproducible. Figure 1 : Design trade-off space for JVM controlled concurrency testing platforms. Y-axis : Effectiveness ( bugs/time ) is run-time performance ( execs/time ) divided by size of search space ( execs/bug , given random sampling of interleavings). X-axis : Viability of running the tool depends on expressibility of interleavings, applicability to arbitrary targets, and maintainability of the platform itself. Fray identifies a sweet spot. An ideal concurrency testing system would (a) provide an efficient mechanism for controlled concurrency [ 54 ] ; that is, to deterministically execute a multi-threaded program along a fixed schedule (i.e., sequence of thread interleavings), and (b) provide support for systematically or randomly exploring thread schedules using state-of-the-art search strategies (e.g., partial order sampling [ 63 ] ) to uncover hard-to-find concurrency bugs. To maximize impact and usability, such a system should also be able to target any multi-threaded program without requiring any manual effort. Yet, no such general testing framework for the JVM currently exists, despite decades of research on algorithms for concurrency testing—why is that? We observe that the design space for performing concurrency control on the JVM is complex with subtle trade-offs. In prior work, concurrency has been controlled by intercepting OS-level thread-scheduling decisions (as in rr [ 38 ] , the record-and-replay tool for Linux), by emulating the JVM completely (as in Java Path Finder (JPF) [ 57 ] ), by mocking concurrency primitives in the JDK (as done by Lincheck [ 28 ] , for testing concurrent data structures), or by pausing individual threads to defer their scheduling (as in CalFuzzer [ 26 ] ). As shown in Fig. 1 (and detailed in Section 3 ), these design choices have impacts on (i) the viability of running the tool on real-world programs, due to its applicability (i.e., the kinds of programs that it targets), the expressibility of the search space (i.e., whether all interleavings can be deliberately and faithfully exercised), and the maintainbility of the tool itself (i.e., how easily it can keep up with evolving Java versions); as well as (ii) its bug-finding effectiveness , which for a given search algorithm (e.g., random sampling) depends on maximizing the run-time performance of executions and minimizing the search space of which interleavings need to be considered. Our key insight is that to make controlled concurrency testing effective and practically viable for managed code, we need to (i) orchestrate thread interleavings without replacing existing concurrency primitives with mocks and (ii) encode the semantics of concurrent primitives for faithfully expressing the set of all possible program behaviors. In this paper, we present Fray , a new concurrency testing platform for the JVM. Fray’s objective is to find concurrency-induced assertion violations, run-time exceptions, as well as deadlocks, in programs that are free of data races. 1 1 1 Data-race freedom is a common assumption made by concurrency testing tools [ 37 , 41 , 8 ] , as there exist many good data-race detectors [ 10 , 11 , 48 , 39 , 50 ] and as many real-world concurrency bugs are found in well-synchronized programs [ 34 , 35 ] ; Section 2 discusses the scope in greater detail. To perform concurrency control, Fray introduces a unique mechanism called shadow locking , which mediates access to shared resources in a specified order (the “thread schedule”) with extra locks whose semantics are coupled to concurrency primitives used in the original program. With this design, Fray identifies a sweet spot on the trade-off space (Fig. 1 ). Fray works on off-the-shelf JVM programs (compiled from Java, Scala, Kotlin, etc.), requiring no manual annotation or source rewriting. Fray efficiently performs deterministic concurrency control over the space of application-thread interleavings where context switches occur only at synchronization points. Fray’s search space is sound and complete in the absence of data races and other sources of non-determinism (e.g., timers); that is, every concurrency bug discovered by Fray will be a true positive, and for every concurrency bug that can manifest in the original program there is a corresponding interleaving that Fray can execute to uncover it. In order to search for concurrency bugs, Fray can run various well-known algorithms such as random walk, probabilistic concurrency testing (PCT) [ 4 ] , and partial-order sampling (POS) [ 63 ] . For debugging purposes, any saved interleaving can be deterministically replayed. We empirically show that, as a platform, Fray outperforms currently available alternatives for performing a controlled random search— rr and JPF—in bug-finding effectiveness on 53 programs from independently developed benchmark suites JaConTeBe [ 31 ] and SCTBench [ 54 ] (the latter ported to Java). We also demonstrate Fray’s push-button applicability to 2,655 concurrent test cases from mature software projects such as Apache Kafka [ 13 ] , Apache Lucene [ 12 ] , and Google Guava [ 16 ] —we believe this is the largest evaluation of controlled concurrency testing on real-world software . Fray has successfully identified 18 distinct concurrency bugs across these projects (11 confirmed and 7 fixed

Read the complete text