Gleaner: A Semantically-Rich and Efficient Online Sampler for Microservice Diagnostics DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS: Do Not Use This Code Generate the Correct Terms for Your Paper
Yifan Yang, Aoyang Fang, Songhan Zhang, Pinjia He · 2026
Read Gleaner: A Semantically-Rich and Efficient Online Sampler for Microservice Diagnostics DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS: Do Not Use This Code Generate the Correct Terms for Your Paper by Yifan Yang, Aoyang Fang, Songhan Zhang, Pinjia He (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-16810.txt.
Introduction
Yang and colleagues propose Gleaner, an online tail-sampling framework for microservice diagnostics. Tail-based samplers model traces as graphs, but expensive graph analysis blocks online use. Gleaner treats each trace as a bag-of-edges enriched with log semantics, using set operations. An alarm-driven quota and diversity strategy prioritize rare traces for root cause analysis. Gleaner processes traces in zero point seven four milliseconds each, improving pattern coverage by about 129 percent. At a 1 percent sampling rate, root cause accuracy improves by 42 to 107 percent over the next-best sampler.
Catalog record
| CREATOR | Yifan Yang, Aoyang Fang, Songhan Zhang, Pinjia He |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58868 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.16810
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Gleaner: A Semantically-Rich and Efficient Online Sampler for Microservice Diagnostics DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS: Do Not Use This Code Generate the Correct Terms for Your Paper Yifan Yang, Aoyang Fang, Songhan Zhang, Pinjia He Abstract . Distributed tracing in microservices is critical for diagnostics but generates overwhelming data volumes, necessitating intelligent sampling. To maximize fidelity, state-of-the-art (SOTA) tail-based samplers analyze complete (or even log-enriched) traces by modeling them as graphs. However, this reliance on computationally expensive graph analysis creates a performance bottleneck that prohibits their use in online settings. To this end, we propose Gleaner, an online tail-sampling framework that breaks this trade-off. It is founded on the key insight that explicit graph structures are unnecessary for high-fidelity trace grouping. Instead, Gleaner represents each trace as a "bag-of-edges" augmented with log semantics, replacing slow graph algorithms with highly efficient set-based operations. It also employs an alarm-driven quota and a diversity-preserving strategy to prioritize anomalous and rare traces for downstream Root Cause Analysis (RCA). Experimentally, Gleaner processes traces at 0.74ms each, improving Trace Pattern Coverage by up to 128.7% and Shannon Entropy by up to 32.9% over baselines. At just a 1% sampling rate, Gleaner improves RCA accuracy by 42%-107% over the next-best sampler. Moreover, RCA on Gleaner’s sampled data is more accurate than with the entire, unsampled dataset. This result reframes intelligent sampling from a data reduction technique to a powerful signal enhancement paradigm for automated operations. 1. Introduction Microservice architectures, while enabling rapid development and deployment, introduce profound operational complexity ( 49 ; 52 ; 24 ; 8 ) . A single user request can trigger a cascade of interactions across hundreds of services, making system behavior difficult to comprehend ( 51 ) . Distributed tracing has emerged as an indispensable tool for observability, providing visibility into these complex request flows ( 43 ; 1 ; 2 ; 11 ) . However, the sheer volume of data—often terabytes daily ( 31 ) —makes comprehensive storage and analysis infeasible. Furthermore, trace data typically follows a long-tail distribution, where a few common execution paths dominate. Consequently, intelligent trace sampling is not just a practical necessity but a critical component of modern observability pipelines. The most straightforward approach, head-based random sampling , is widely considered inadequate. By making sampling decisions before the execution outcome is known, it disproportionately retains common, healthy traces while being highly likely to discard rare patterns ( 15 ) . Since system anomalies and critical failures often manifest as these rare traces, head-based methods risk discarding the very signals that engineers and SREs need most for diagnostics. This fundamental limitation has driven the community towards more sophisticated tail-based sampling techniques ( 21 ; 22 ; 18 ; 15 ; 17 ; 39 ) . By waiting until a trace is complete, these methods can leverage holistic signals such as full call structure, end-to-end latency, and span status, and can bias the budget toward rare or abnormal traces, significantly improving the quality of the sampled data. However, existing tail-based samplers suffer from a critical limitation: they rely almost exclusively on coarse, span-level information . They evaluate a trace based on its call structure (e.g., service A called B) and the final status of its constituent spans, overlooking the rich semantic details within each span’s execution. This is a significant blind spot. The diagnostic value of a trace is often determined not by its high-level structure but by fine-grained events—such as error logs, retry attempts, or custom application events—that occur during a span’s lifetime ( 45 ; 26 ; 20 ) . A sampler that is blind to this information might discard a trace that appears structurally normal (e.g., low latency, success status) but contains a critical error log, silently dropping the most vital signal for root cause analysis (RCA). This disconnect is increasingly problematic as both industry and academia move towards multi-modal diagnostics. Open-source standards like OpenTelemetry are actively standardizing trace-log correlation and incorporating inner-span events into sampling strategies ( 28 ; 29 ) . Concurrently, advanced RCA algorithms increasingly leverage the semantic information in logs to improve diagnostic accuracy ( 44 ; 10 ; 48 ; 23 ; 35 ; 33 ; 36 ; 32 ; 37 ; 5 ; 12 ; 50 ; 13 ; 40 ; 7 ) . Recognizing this, a handful of recent studies have attempted to incorporate logs into the sampling process ( 34 ) . However, these pioneering methods rely on computationally expensive models like Heterogeneous Graph Neural Networks (HGNNs), which require extensive training and are too slow for online, real-time processing in high-throughput environments ( 39 ; 9 ) . This reveals a critical fidelity-performance trade-off in modern trace sampling: methods that are computationally efficient remain blind to semantically rich internal signals, while methods fine-grained enough to capture these signals are too slow for practical deployment. The central challenge, therefore, is to design a sampling mechanism that is both semantically rich and computationally tractable. To bridge this gap, we propose Gleaner , a novel online tail-sampling framework that breaks this trade-off. Our key insight is that explicit, computationally expensive graph structures are not necessary for high-fidelity, semantically-aware sampling. Instead, Gleaner introduces a lightweight event-pair set (EPS) representation, modeling each trace as an efficient set of event pairs (§ 3.2 ). This representation captures both inter-span call dependencies and crucial intra-span events (e.g., logs, status codes) and replaces costly graph algorithms with highly efficient set operations. Building on this, Gleaner incorporates an alarm-driven quota mechanism (§ 3.3 ) to prioritize resources during incidents and a diversity-preserving selector (§ 3.4 ) based on an optimized Determinantal Point Process (DPP) to ensure a rich and varied sample set. Compared to head-based sampling, Gleaner provides vastly superior coverage of rare patterns. Compared to existing trace-only tail-samplers, it captures far richer semantic signals at a comparable or even faster speed. And critically, compared to emerging trace-and-log samplers, Gleaner achieves its semantic richness without the prohibitive overhead of model training and inference, making it suitable for high-volume online deployment. We conducted an extensive evaluation on a new, large-scale benchmark dataset. Our results demonstrate that Gleaner achieves high-throughput processing at 0.74ms per trace, while significantly outperforming state-of-the-art samplers on quality metrics. It achieves 11.6%~128.7% improvement in Trace Pattern Coverage and 2.8%~32.9% gains in Shannon Entropy over baselines. At a mere 1% sampling rate, Gleaner boosts the accuracy of downstream RCA tools by 42%~107% over the next-best sampler. Remarkably, RCA on Gleaner’s sampled data is more accurate than when using the entire, unsampled dataset. This finding reframes intelligent sampling from a mere data reduction technique to a powerful signal enhancement paradigm for modern automated operations. The main contributions of this work are as follows: • We propose a novel method to construct a unified representation from traces and their associated logs, capturing call relationships, internal execution dynamics, and overall anomaly